John DiMarco on IT (and occasionally other things)
I welcome comments by email to jdd at

Thu 09 Mar 2017 12:58

A closer look at's 2017 rankings for Computer Science.

The QS World University Rankings for 2017 are out, including the subject rankings. For the subject "Computer Science & Information Systems", the University of Toronto does very well, placing tenth.

A closer look at the top ten shows some expected leaders (MIT, Stanford, CMU, UC Berkeley) but some less expected ones, such as Oxford and Cambridge. These are superb Universities with good Computer Science programs, but are their CS programs really among the ten best in the world?

A closer look at how the score is computed sheds some light on this question. The Overall Score is a combination of Academic Reputation, Citations per Paper, Employer Reputation, and H-index Citations. Academic Reputation and Employer Reputation are, in essence, the opinions of professors and employers respectively. While (hopefully) they are reasonably well founded opinions, this is a subjective, not an objective, metric. On the other hand, Citations per Paper and H-index Citations are objective. So I looked at Citations per Paper and H-index Citations for the top forty schools on the 2017 QS Computer Science & Information Systems ranking.

By Citations per Paper, top five of those forty are:

  1. Princeton
  2. Stanford
  3. UT Austin
  4. Washington
  5. UC Berkeley

No MIT? This seems off. So lets look at the top five by H-Index Citations:

  1. Stanford
  2. MIT
  3. UC Berkeley
  4. UI Urbana-Champaign
  5. UT Austin

That looks more reasonable. So let's look at the top twenty by H-Index Citations:

  1. Stanford
  2. MIT
  3. UC Berkeley
  4. UI Urbana-Champaign
  5. UT Austin
  6. Georgia IT
  7. CMU
  8. Tsinghua
  9. Nanyang
  10. ETH Zurich
  11. Washington
  12. Princeton
  13. UBC
  14. Toronto
  15. Waterloo
  16. NU Singapore
  17. UC London
  18. Cornell
  19. UCLA
  20. CU Hong Kong

That's a list that makes more sense to me. While it puts my department 14th instead of 10th, I think I have more confidence in the objectivity of this ordering than I do in the QS Overall Score ordering.

/misc permanent link

Wed 26 Oct 2016 10:41

Remembering Kelly Gotlieb

On October 16th, 2016, Kelly Gotlieb, founder of the Department of Computer Science at the University of Toronto, passed away in his 96th year. I had the privilege of knowing him. Kelly was a terrific person: brilliant, kind, and humble. He was always willing to make time for people. He was a great thinker: his insights, particularly in the area of computing and society, were highly influential. I never fully realized how influential he was until we, here at the department of Computer Science, created a blog,, in honour of the 40th anniversary of Social Issues in Computing, the seminal textbook he and Allan Borodin wrote in 1973 in the area of computers and society. I served as editor of the blog, and solicited contributions from the top thinkers in the field. So many of them responded, explaining to me how influential his ideas had been to them, and the blog was filled with insightful articles building in various ways upon the foundation that he and Allan had laid so many years before. I interviewed Kelly for the blog, and he was terrific: even in his nineties, he was full of insights. His mind active and enthusiastic, he was making cogent observations on the latest technologies, ranging from self-driving cars to automated medical diagnosis and treatment.

To me, Kelly epitomized the truth about effective teaching that is all too often missed: teaching is not just about information, teaching is about inspiration. Kelly was a truly inspiring teacher and thinker. He was completely authentic in everything he did, he was full of enthusiasm, and that enthusiasm was infectious. Conversations with Kelly so often left me energized and inspired, thinking along new directions of thought that something he said had triggered, or leaping past obstacles that had previously seems insurmountable. That is true teaching. Information without inspiration is simply fodder for forgetfulness, but teaching that inspires leads to new insights, integration of ideas, genuine understanding, and a better, clearer and sharper window on the world. Kelly inspired so many people for so many years. We are truly blessed that he was among us. He will be remembered.

/misc permanent link

Tue 23 Apr 2013 12:56

Handling Unsolicited Commercial Email

My email address is all over the web: at the time of writing this, a search on google for my email address produces about 15,800 results. So anyone who wants to find my email address can do so easily. Many people or companies who want to sell me something send me email out of the blue. I get a great deal of such unsolicited commercial email, too much to read or pay adequate attention to. I simply delete them. Unfortunately, many sources of such email persist. So for some time now, I've elicited the help of technology. I process my incoming email using procmail, a powerful piece of software that lets me script what happens to my email. When I receive unsolicited commercial email, if it is from a vendor or organization I don't have a relationship with, I will often add a procmail rule to discard, unseen, all future email messages from that vendor. I've got about 400 organizations (mostly vendors) in my discard list so far, and the list slowly grows. Am I still getting unsolicited commercial email from these sources? I am, but I am not seeing it. It's the same effect, really, as manual deletion (i.e. the message is deleted, unread), but it's easier for me, because I am not interrupted. But of course I think it would be better still if the email were not sent at all.

If you are a vendor with whom I do not have a pre-existing relationship, and you want to send me email introducing your products, please don't. I do not accept cold salescalls either. Instead, advertise effectively on the web, so that if I am looking for a product like yours, I can find you. If you must contact me directly, send me something by postal mail, where, unlike email, the communication does not have an interruptive aspect.

/misc permanent link

Thu 29 Nov 2012 00:00

A closer look at the University of Toronto's international ranking in Computer Science.

International rankings of universities seem to be all the rage these days. The interest seems to be fed by three rankings of particular prominence that have emerged in the past decade. These are Shanghai Jiao Tong University's Academic Ranking of World Universities (sometimes known as AWRU, or simply as the "Shanghai Ranking"), Quacquarelli Symonds' QS World University Rankings, and the Times Higher Education World University Rankings. Part of the attractiveness of these rankings is that they can become a way of "keeping score", of seeing how one institution does in comparison to others.

My employer, the University of Toronto, does quite well in these rankings, particularly my department, Computer Science. The subject area of Computer Science is not ranked separately in the Times Higher Education World University Rankings (it's bundled together with Engineering), but in the other two, Toronto has consistently ranked in the top ten in the world each year in Computer Science, with only one exception.

This exception is recent, however, and worth a closer look. In the QS World University Rankings for Computer Science and Information Systems, Toronto dropped from 10th in 2011 to 15th in 2012. This big drop immediately raises all sorts of questions: has the quality of Toronto's Computer Science programme suddenly plummetted? Has the quality of Computer Science programmes at other universities suddenly soared? Or has the QS World University Rankings changed its methodology?

To answer this question, let's look at how other universities have changed from 2011 to 2012 on this ranking. Many (MIT, Stanford, Berkeley, Harvard, Oxford, Cornell, and others) stayed where they were. Others dropped precipitously: Cambridge University dropped from 3rd to 7th, UCLA from 8th to 12th, and Caltech plummetted from 7th to 27th. Some other universities went up: Carnegie Mellon University (CMU) went from 9th to 3rd, ETH Zurich from 11th to 8th, the National University of Singapore (NUS) from 12th to 9th, and the Hong Kong University of Science and Technology (HKUST) soared from 26th to 13th. Surely these curious and significant changes reflect a methodology change? But what?

The QS university rankings website, in the Methodology section, Academic subsection, reveals something of interest:

	NEW FOR 2012 - Direct Subject Responses

	Until 2010, the survey could only infer specific opinion on
	subject strength by aggregating the broad faculty area opinions
	of academics from a specific discipline. From the 2011 survey
	additional questions have been asked to gather specific opinion
	in the respondent's own narrow field of expertise. These responses
	are given a greater emphasis from 2012.
To understand this change, it needs to be recognized that the QS rankings rely highly on the opinions of academics. A large number of academics around the world are surveyed: the QS rankings website indicates that in 2012, 46079 academic responses were received, of which 7.5% addressed Computer Science." The seemingly modest change made in 2012, to weigh more heavily the opinions of academics in a field about their own field, given its impact on the 2012 results for Computer Science, leads one to wonder about the regional distribution of academics in Computer Science in comparison to academics in other disciplines. One significant factor may be China.

In 1999, courses in the fundamentals of computer science became required in most Chinese universities, and by the end of 2007, China had nearly a million undergraduates studying Computer Science. While QS rankings does not indicate regional distribution by discipline for the academics whose opinions it consults, the surge in the number of Chinese computer scientists worldwide in the past decade almost certainly must have an effect on the regional distribution of academics in Computer Science as compared to other disciplines. As such, is it any surprise to see world universities prominent in China that possess strong Computer Science programmes (such as HKUST and NUS) climb significantly in the rankings, and others less prominent in China plummet? But if a world ranking of universities is so affected by regional shifts in those whose opinion is being solicited, how reliable is it as an objective gage of the real quality of a given university?

Perhaps a more reliable gage of quality can be found in the Shanghai ranking, which is not opinion-based, but relies on concrete indicators and metrics. On the Shanghai ranking, the University of Toronto consistently ranks 10th in the world in Computer Science in 2010, 2011, and 2012. But what does this mean, concretely?

To answer these questions, we need to grapple with an important fact: in Computer Science, the US dominates. As a nation, the US has been enormously supportive of Computer Science ever since the field first existed, and as a result, it has become pre-eminent in computing. Nine of the top ten schools in the Shanghai ranking, and twenty of the top twenty-five, are in the US. For the University of Toronto to be one of the handful of universities outside the US to break into the top twenty-five, and the only one to break into the top ten, is a significant accomplishment. A chart is illustrative:

Of course, the University of Toronto is in Canada, so a comparison to other schools in Canada is also illustrative. For Computer Science, on the Shanghai ranking, there seems to be no close Canadian rival. In 2012, UBC comes closest, being a only a few points short of breaking into the top 25, but all other Canadian schools rank well back:

Even compared to other disciplines that have Shanghai rankings (only science, social science, and related disciplines seem to be ranked), Toronto's pre-eminence in Computer Science in Canada is striking:

From a score-keeping perspective, I think we can conclude that the University of Toronto is doing very well in Computer Science with respect to other universities in Canada, and it is one of the few non-US schools that can keep up with the US in this field.

But all this needs to be put into perspective. After all, rankings are not a full picture, they're aggregations of metrics of varying value, they represent a formulaic approach to something (university education) that cannot always be so conveniently summarized, and they reflect methodologies chosen by the producers of the rankings, methodologies that may not always best reflect objective quality. Of course, if the University of Toronto were to climb to fifth, I'd be pleased, and if it were to drop to fifteenth, I'd be disappointed: surely the score-keeper in me can be allowed this much. But in the overall scheme of things, what matters most for Computer Science at Toronto is not our score on a ranking system, but the objective quality of our programme, the learning outcomes of our students, and the impact of our research, and these things, not our score on rankings, must always remain our top priorities.

/misc permanent link

Thu 15 Dec 2011 15:14

Dealing with unsolicited salescalls (cold calls).

For many years, I've been plagued by unsolicited salescalls. It's not very hard to find my phone number, and various people (mostly in the IT realm) call me up out of the blue hoping to sell me something. The interruption is unwelcome, even if the product isn't.

For some years now, my policy is to explain to the caller that I don't accept unsolicited salescalls, sincerely apologize, and end the call. Occasionally, I am then asked how I am to be contacted. I explain that I prefer to do the contacting myself: when I have a need, I am not too shy to contact likely vendors and make inquiries about their products.

Occasionally I run into someone who is offended by my unwillingness to take their unsolicited salescall. I do feel more than a little sympathy for the salesperson when this happens: I imagine they may think I objected to something they did, or to their manner. The fact is, I handle all unsolicited salescalls this way. As for whether it is intrinsicly offensive to reject unsolicited salescalls out of hand, I don't think it is. Indeed, it is natural for a salesperson to want their salescall, even if unsolicited, to be better accepted. But it is unreasonable for any salesperson to expect that unsolicited sales inquiries to strangers will always be welcome. But I do apologize, each time, and in general, when I so quickly end telephone conversations with salespersons who call me out of the blue.

Dear reader, if you are a salesperson, and you are tempted to contact me to sell me something, please do not call. Instead, just advertise generally (and if you must, send me some mail in the post). Trust me to find you when the need arises. I frequently do.

/misc permanent link

Mon 30 May 2011 21:26

Einstein's special relativity isn't as complicated as many people seem to think.

I run into people who think that special relativity is some sort of mysterious thing that only Einstein and physicists can understand. But it's not. It's a bit weird, but it's no weirder than the earth being a globe.

Originally people thought that light moved like any other moving object. Einstein thought about this and wondered: what would happen if you followed some light and sped up until you travelled at the same speed as it. Then light would look to you like it was stopped. But stopped light (light "standing still") didn't (and still doesn't) make sense. So Einstein thought: what if light travels at the same speed no matter how fast you're going? What would this mean?

Well, what does it mean to travel "at the same speed"? It means light covers the same amount of distance in a given amount of time. Or, put another way, light takes the same amount of time to cover a given distance. So if the distance is short, light takes less time to go the distance. If the distance is longer, light takes proportionally more time to cover it.

So Einstein thought: OK, if light travels at the same speed for everyone no matter how fast they're going, what would that mean for someone going very fast? Imagine they're going nearly the speed of light, and are being chased by a beam of light. Clearly the light isn't going to get closer to that person as quickly as it would get closer to someone who was standing still. Ordinarily, you would think that light was moving "slower" for the person who is moving away from it. But if light moves at the same speed for everyone, than something else must be going "slower" for that person. The only possibility is time.

Put it this way: light covers a certain distance in a second. To someone watching, the pursuing light isn't making up the distance quite so fast between it and the moving person, because the person is moving away so fast. But for the moving person, light is moving as fast as it always does, it is the second that takes longer.

This sounds a little bit crazy since we aren't used to thinking of time moving faster for some people and slower for others. But it does. The reason we don't notice is that the speed of light is very fast and we can't easily go at speeds close to it.

It's the same sort of thing as the world being round (i.e. a globe). It looks flat to us, but only because it is so big that we can't see enough of it at once to see it curve. Go high enough and we can see the curve of the earth's surface easily enough.

Similarly with special relativity. Time moves slower for those who move fast. It's not obvious to us because we usually don't move very fast, so at the speeds we move, the time differences are too small to notice. But in 1971, Joseph Hafele and Richard Keating took some very accurate (cesium atomic) clocks abord commercial airliners and flew around the world. They compared their clocks to the very accurate clocks in the US naval observatory: the clocks were indeed different, and showed the results that Einstein had predicted.

What this this mean? Well, if you can wrap your head around the concept of the world being a globe, you can wrap your head around the concept of time moving more slowly for those who move fast. And that's it, right?

Well, not really. There's also general relativity (and it affects Hafele and Keating's results too). But that's a bit more complicated, and I'm not going to get into it now.

/misc permanent link

Fri 04 Sep 2009 14:57

Assessing H1N1 risk
What sort of risk does H1N1 (Swine Flu) present this flu season? To assess this, it might be helpful to estimate some H1N1 risks and then compare it to risks with which we are more familiar.

So let's look at some numbers. The worldwide case fatality rate of H1N1 (the number of people who have died of H1N1, divided by the number of people who have gotten H1N1) has been estimated to be 0.45%. Unlike seasonal flu, roughly 80% of those who have died of H1N1 are less than 65 years old (typically 90% of seasonal flu fatalities are 65 years old or over). If we assume a 15% probability of getting H1N1 this flu season, the likelihood of someone under the age of 65 dying of H1N1 this season is thus 0.15 x 0.0045 x 0.80, i.e 0.054% or 1 in 1852. This is a little less than the one-year general odds of death due to external causes in the US, approximately 1 in 1681.

/misc permanent link


[Valid HTML 4.01 Transitional] [Valid RSS]