Planet DCS@UofT

Serendipitous and Unexpected

2009-11-22T21:14:34+00:00

Most research effort does not produce what is thought of as a traditionally publishable result. That doesn’t mean, however, that nothing was gained by conducting the research. These results, whether they are failures or merely perplexing, can provide valuable insights into open problems and prevent other researchers from duplicating work. We started a journal that focuses on serendipitous (I have no idea why this worked) and unexpected (it seems like this technique should work on this problem but it doesn’t) results. The goal of the journal is to provide a venue where ideas can flow and be debated.

The Journal of Serendipitous and Unexpected Results (JSUR) is an open-access forum for researchers seeking to further scientific discovery by sharing surprising or unexpected results. These results should provide guidance toward the verification (or negation) of extant hypotheses. JSUR has two branches, one focusing on Computational Sciences and the other on the Life Sciences. JSUR submissions include, but are not limited to, short communications of recent research results, full-length papers, review articles, and opinion pieces.

Recently, we launched the beta version of the journal site at http://jsur.org. We would love to get your feedback and even better, a submission for the first issue.

To get the journal started, we’re looking to collect a large number of short (2-4 page) reports. I know you have something to publish. Please help us spread the word and forward this information to interested colleagues.

Sincerely,
The JSUR Editorial Board

Adaptation just as important as Mitigation

2009-11-20T20:33:12+00:00

Brad points out that much of my discussion for a research agenda in climate change informatics focusses heavily on strategies for emissions reduction (aka Mitigation) and neglects the equally important topic of ensuring communities can survive the climate changes that are inevitable (aka Adaptation). Which is an important point. When I talk about the goal [...]

Modeling tools news - Nov. 20 2009

2009-11-20T17:26:34+00:00

Recent releases of new modeling tools (in a few cases, "new" may refer to "new to me", meaning that I've discovered the tool just now but maybe the tool already existed):

New version of the Dresden OCL toolkit for Eclipse (to parse textual OCL expressions). Changelog for the new version .

Will Peak Oil Save us from Climate Change?

2009-11-20T16:51:56+00:00

Yesterday, I posted that the total budget of fossil fuel emissions we can ever emit is 1 trillion tonnes of Carbon. And that we’ve burnt through about half of that since the dawn of industrialization. Today, I read in the Guardian that existing oil reserves may have been deliberately overestimated by the International Energy Agency. [...]

Academic Genealogy

2009-11-20T10:59:49+00:00

Just ran my advisors through these two services and came up with this:

Methods & Tools - a free software development magazine

2009-11-20T02:39:23+00:00

Methods & Tools is a free software development magazine with practical knowledge, news and resources on software development: Software Testing, Project Management, Programming (Java,.NET, Ruby on Rails, Ajax), UML, Agile (eXtreme Programming, Scrum, Test Driven Development), Configuration Management, Databases, RUP, Software Analysis, Software Design, Software Quality Assurance, Software Maintenance, Software Process Improvement (CMMI), Software Development Tools, User Interface, etc.

Animated Database Courseware

2009-11-20T01:14:03+00:00

I've just found an online Animated DataBase Courseware offering an "Interactive Approach for Teaching the Principles of DataBase Concepts", including topics like database design (ER notation, transformation rules for deriving database schemas from ER models, normalizations and functional dependencies concepts,...), sql, transactions and security.

All this is complemented with proposals of exercises (solution is provided) and completely free.

Does Microsoft still believe in model-driven development? - the Oslo move

2009-11-20T00:46:09+00:00

When Microsoft started Oslo (in short, Microsoft strategy for model-driven development consisting in the M modeling language + the Quadrant modeling environment + a model repository), we all became very excited. If Microsoft was keen on investing in MDD it was a good indication that they were convinced that they could make some money out of it.

Engineering the Software for Understanding Climate Change

2009-11-19T21:01:40+00:00

Our paper, Engineering the Software for Understanding Climate Change finally appeared today in IEEE Computing in Science and Engineering. The rest of the issue looks interesting too – a special issue on software engineering in computational science. Kudos to Greg and Andy for pulling it together.

One Trillion Tonnes of Carbon

2009-11-19T15:16:17+00:00

I posted a few times already about Allen et al’s paper on the Trillionth Tonne, ever since I saw Chris Jones present it at the EGU meeting in April. Basically, the work gets to the heart of the global challenge. If we want to hold temperatures below a 2°C rise, the key factor is not [...]

First evaluation result

2009-11-18T19:11:00+00:00

I have a (pretty terrible) result: with the way things stand right now, tracSnap is able to predict 16% of the people who comment on tickets related to Hadley model defects.

How I arrived at this result:

Jon gave me a list of tickets (defects) that he was able to associate with a specific revision of the repository.
For each such ticket, I looked at the files that were involved in its associated revision, and used tracSnap to get the "experts" for those files.
I compared this list of experts with the people who actually helped to fix the defect, and checked to see if they are the same people.

I have a hunch that I can improve this statistic by finding a better way to relate changes in the branches to the files in the trunk. Right now, if you make changes in your branch, you are an 'expert' on only the files in your branch, and not the corresponding file in the trunk. So if you aren't the one to merge your changes to the trunk, you never get associated with the trunk files, and thus will not get suggested as an expert.

As of yet, I've only looked at the prediction rate using the "experts", and not the ticket reporter's "suggested contacts". I suspect that both analyses will produce similar results, since the suggested contacts and experts are not independent (contact suggestions rely partially on the expertise calculations).

16%... lame.

Special Issue

2009-11-18T18:32:06+00:00

A special issue of Computing in Science & Engineering that Andy Lumsdaine and I edited, devoted to software engineering in computational science, is now available. We’d like to thank everyone who contributed:

Report on the Second International Workshop on Software Engineering for CSE, by Jeffrey Carver (University of Alabama)
Managing Chaos: Lessons Learned Developing Software in the Life Sciences, by Sarah Killcoyne and John Boyle (Institute for Systems Biology)
Scientific Computing’s Productivity Gridlock: How Software Engineering Can Help, by Stuart Faulk (University of Oregon), Eugene Loh and Michael L. Van De Vanter (Sun Microsystems), Susan Squires (Tactics), and Lawrence G. Votta, (Brincos)
Mutation Sensitivity Testing, by Daniel Hook (Engineering Seismology Group Solutions) and Diane Kelly (Royal Military College of Canada)
Automated Software Testing for MATLAB, by Steve Eddins (The MathWorks)
The libflame Library for Dense Matrix Computations, by Field G. Van Zee, Ernie Chan, and Robert A. van de Geijn (University of Texas at Austin), and Enrique S. Quintana-Ortí and Gregorio Quintana-Ortí (Universidad Jaime I de Castellón)
Engineering the Software for Understanding Climate Change, by Steve Easterbrook (University of Toronto) and Timothy Johns (Hadley Centre for Climate Prediction and Research)

zuzelvp

2009-11-18T17:16:03+00:00

Today I am readying a paper that describes in detail an interesting study in which 120 computer science students rated 100 small code snippets (7.7 lines on average) from 1 to 5 according to their readability. The dataset and the tool used for the study are available. Some of the results of this study are [...]

What He Said

2009-11-18T13:36:42+00:00

‘Nuff said…

What we actually know about software development by Greg Wilson

2009-11-18T00:27:32+00:00

Today seems to be "the recommendation day". For those of you that do not follow (yet) Greg Wilson's I'd like to recommend Greg's presentation "Bits of evidence: What we actually know about software development and why we believe it's true" where he discusses the low standards for proof in software engineering (too many claims without any kind of empirical validation) and how we can start addressing this situation.

Don't miss Johan den Haan series of posts on basic concepts of MDD

2009-11-17T23:50:11+00:00

Johan den Haan has a very interesting blog on MDE topics. Recently he has been publishing a series of posts on basic model-driven development principles/misperceptions/challenges that I'd like to recommend you. Take a look at my selection of posts (useful specially if you are new in the area of MDE):

First dollar with the Amazon Affiliate program

2009-11-17T23:10:15+00:00

When I started this portal I joined two affiliate programs:

Amazon associates for the UML books (and OCL , modeling and MDD books) I comment on the portal

I’m Just Not Very Creative

2009-11-17T22:56:50+00:00

At least, not by comparison with this.

Evolution in Action

2009-11-17T16:15:08+00:00

It turns out that a human lifetime may in fact be long enough to see a new species emerge. Cool.

Embodied Social Proxies

2009-11-17T03:26:56+00:00

While at Microsoft last week, Gina Venolia introduced me to George. Well, not literally, as he wasn’t there, but I met his proxy. Gina and co have been experimenting with how to make a remote team member feel part of the team, without the frequent travel, in the Embodied Social Proxies project. The current prototype [...]

Evaluation of me as a TA

2009-11-17T01:20:58+00:00

Last year I was a TA for two terms for the same undergrad course (CSC207 Software Design). I just received the student evaluations for my performance as a TA during the winter term, so now I have a clearer view of my performance during the whole year.

There were 9 and 8 evaluation responses during the Fall and Winter terms respectively, and the students gave me a mark from 1 to 7 inclusive, where 1 stands for extremely poor, 4 stands for adequate and 7 for outstanding. Here are my stats:

	Fall 2008		Winter 2009
	mean	std dev	mean	std dev
presentation	2.33	1.41	4.38	1.51
English	3.67	1.41	5.12	1.55
clarity	2.44	1.67	5.00	1.07
enthusiasm	2.78	1.79	6.00	1.07
question handling	2.56	2.07	5.12	1.73
grading fairness	3.44	2.07	4.88	1.25
grading speed	3.67	1.94	4.62	1.69
overall	2.67	2.06	5.38	1.41

I think that the stats kind of tell the story of me coming and adapting to Canada. I came last September and for the first term, I really really sucked as a TA. But then, as I slowly became more and more accustomed to the place, the people, the language and the academic environment, things got much better.

I especially note the “enthusiasm” stat. It’s actually surprisingly accurate: the more I got accustomed to this place, the more I started identifying myself with it and regarding it as a place to which I want to contribute. I just feel sorry for the poor students that had to suffer me during my adaptation period…

Posted in blogging, self-reference, teaching

Why Am I Not Surprised…

2009-11-16T19:10:30+00:00

…that Lego would be a leader in augmented reality?

When I Said “The Last Twenty Years…”

2009-11-16T18:13:27+00:00

Last week, in response to Google’s announcement of a new programming language called Go, I said:

I’m underwhelmed: it’s as if the last 20 years of programming language research hadn’t happened.

Turns out I was being generous: read this post from start to finish, and you’ll see what I mean.

So what should a new programming language do to get my attention? First, just as applications should be designed for testability, languages should be too. That means choosing constructs to make the lives of static and dynamic analysis tools better. Building such tools after the fact is like trying to add security to an app after it has been deployed; I think we’d do better to treat the capabilities of today’s leading-edge program analysis tools as hard (but not unbreakable) constraints on what’s allowed to go into a language, and see how far it gets us. I suspect this will push us toward strongly typed and mostly functional languages.

Second, user testing of language features. The folks at CWI did this with ABC (a precursor of Python); Steven Clarke has done excellent work on API usability at Microsoft (see for example this DDJ article from 2004), and there’s lots of other prior art — hell, I did a little myself nine years ago for Python (see these messages for details). I’m not suggesting design by committee [1], but checking to see how comprehensible or surprising feature XYZ is going to be to the average programmer before it’s put into the language just seems like common sense. I suspect this will push us away from pure functional languages: monads are just plain hard, and while purely functional data structures are possible, they’re hardly intuitive.

Third, a new language should explicitly be designed to make the expression of common design patterns as straightforward as possible. Languages (of all kinds, not just programming languages) evolve by formalizing the common usages of the day: idiomatic uses of goto statements become for loops, structs with function pointers become objects, and so on. There’s a tremendous literature on design patterns at several scales; why not treat them as something akin to use cases?

Of course it’s never too late — if someone has the time and energy, they could apply these three criteria to Go (or any other language) right now. Hm… sounds like an interesting thesis topic…

[1] Which gets an unfairly bad rap — both the American Constitution and the King James version of the Bible were produced by committees.

Speaking at CUSEC 2010

2009-11-16T17:41:14+00:00

As they just announced on their blog, I’ll be speaking at CUSEC 2010 in Montreal in January on evidence-based software engineering (which is a lot more fun than you’d guess from the title). Hope to see some of you there.

Bend It ‘Til It Breaks

2009-11-16T14:16:44+00:00

Want to know how strong a piece of steel is? Bend it ’til it breaks. Want to know how usable a programming system is? Make a few deliberate mistakes and see how comprehensible the error messages are. It’s not the only approach, but it’s the one Zef Hemel took with Ruby on Rails. In his original post, he took a critical look at how helpful Rails is when a developer mistypes something. A lot of people misunderstood what he was doing, which prompted a follow-up post; since then, he has tried the same approach with JBoss Seam and Scala Lift. I think this is pretty cool — so cool, in fact, that I’m wondering if there’s a thesis topic in there somewhere…

Pitfalls of informal modeling languages (cartoon)

2009-11-16T13:41:35+00:00

John Mylopoulos , in his keynote talk at the ER'09 conference , used the following Far Side cartoon to illustrate the misunderstandings thay may occur when using informal modeling languages: there may be a huge difference between what we say and what others understand!!

Three posters at the AGU meeting

2009-11-16T03:21:18+00:00

Our group had three posters accepted for presentation at the upcoming AGU Fall Meeting. As the scientific program doesn’t seem to be amenable to linking, here are the abstracts in full: Poster Session IN11D. Management and Dissemination of Earth and Space Science Models (Monday Dec 14, 2009, 8am – 12:20pm) Fostering Team Awareness in Earth System Modeling [...]

Bad surveys

2009-11-15T17:58:04+00:00

Yesterday, a grad student asked me whether I could answer a survey he was doing for his research. I’ve struggled getting participants in the past, and seeing that the survey would only take a couple of minutes, I accepted.

It was a survey about the urban design of a particular place at the University of Toronto, which was fine with me. Halfway through the survey, though, I realized he was trying to put some answers in my mouth. He asked me whether I liked that place, and when I said I did he replied: “Really? There’s nothing to like there.” I insisted that I liked the landscape around it; he objected, pointing out that it was just a grass field. I kept insisting, and he grudgingly wrote down my answer.

We went through this process several times. The last straw fell when I said the lighting at that place was good and he responded that “it’s pretty dark there right now,” circling the “bad lighting” answer. He only erased that answer after I lost my patience and told him that these were my answers and if he wanted others he should ask someone else.

I know how this survey’s results will look like. Some large percentage of users of this space, it will say, are terribly dissatisfied with it –hence providing support for whatever project this student is designing. What gets me is that results from a survey as poorly and dishonestly executed as this one will carry greater weight than any non-quantitative arguments simply because they produce a percentage number in the end. We’re in love with quantitative evidence, no matter how poorly it is constructed.

As I left the place that evening I looked around with a critical eye. There were definitely some areas that could be improved. Come to think about it, I thought, it was plain to see that lighting was actually pretty bad — and no survey results will convince me otherwise.

My sample Google/Microsoft interview question

2009-11-14T13:38:56+00:00

As in, one I have to deal with, as opposed to being asked.

Let’s say you want to calculate the powerset P of a list of items. For each subset of P you will do a non-zero amount of work, so we would like to make P really small. The size of P is … 2^N. Meaning of course for any useful input, P is enormous, heat-death of the universe type size.

We have some function f(s) that takes a subset member of P (a set) and returns True or False. If True, we want to filter out all the subsets of the subset set. E.g., if f([1,2,3]) = True, we would like to remove [1,2], [1], [2] etc. from P (so we don’t evaluate them as well). So the question is, what is an efficient way to do this? The naïve solution is to generate all the members of P, then iterate over them to remove the members that are subsets of a solution. But of course, for large N, this is infeasible to store and to do. Consider the case where the member we find is the set of all items. In this case we should stop our loop. A good solution will allow us to do this in constant time (and not by using a special case).

Update: I should make it clear that f([1]) = T and f([2]) = T does not mean f([1,2] = T.

Solutions I’ve pondered include using a bit matrix and bit masking, but I can’t see how to do that in constant time. We might also represent the subsets using a tree with related by subset, which would probably be log or sublog time.

One question I have is whether these questions are always checked for solvability. I mean, it is relatively easy to pose a question that is not solvable in deterministic polynomial time, and just as easy to pose one whose answer cannot even be checked for correctness. That would be a rather unfair question, wouldn’t it? Like, find a linear time algorithm for finding a truth assignment to the following formula. If you can solve that you should probably start your own company.

What do we want from climate informatics tools?

2009-11-13T21:28:12+00:00

I gave my talk last night to TorCHI on Usable Climate Science. I think it went down well, especially considering that I hadn’t finished preparing the slides, and had just gotten off the plane from Seattle. I’ll post the slides soon, once I have a chance to tidy them up. But, judging by the questions [...]

The Munk Debates: Climate Change

2009-11-13T18:38:53+00:00

Be it resolved that climate change is mankind's defining crisis, and demands a commensurate response.

On December 1st, the Munk Debates feature a debate on Climate Change: George Monbiot and Elizabeth May take on Bjørn Lomborg and Lord Nigel Lawson. Tickets to attend the event live are sold out, but you can sign up for seats at an overflow showing or sign up to watch the debate as a webcast.

Two More Gov 2.0 Links

2009-11-13T15:36:53+00:00

The Toronto Star’s Map of the Week shows where in Toronto the Canadian casualties of WWI lived. Humbling.
The Sunlight Foundation’s goal is greater transparency in government. Laudable, but I think the inverse of Gilmore’s Rule will hold as it always has: criminality will interpret openness as a threat and route around it. Still, a Red Queen’s Race is better than outright defeat.

The Definition of “Exotic”

2009-11-13T15:25:44+00:00

Someone other than Oscar Wilde once defined “exotic” as “anywhere that people poorer than us can’t afford to go”. The same cynicism fits techno trends as well. Take, for example, this recent post by Marc Drapeau about last Friday’s Audience Conference in New York. No laptops, no cellphones, no twittering—in short, none of the things that cool people were calling cool five years ago. Why? Because if everyone’s doing it, it can’t be cool. Next up: a Wired article about the rebirth of attention, followed by a new book from Chris Anderson…

WWTD?

2009-11-13T15:18:51+00:00

The saddest part of Terry Pratchett’s Alzheimer’s is, well, the Alzheimer’s. The second saddest is that no one else writing in English today would be able to use tree-eating crabs in a story quite as deftly as he could. Food falls down from the sky without anyone (or any crab) knowing where it comes from? Just think where he could go from there…

Me wanna Xtext

2009-11-13T04:56:01+00:00

(The funny part is that) I haven’t actually even touched Xtext yet, but I am utterly excited to participate to the Building DSLs with Xtext session at the coming Eclipse Modeling Day. I spent some time with Omar Badreddin at CASCON last week discussing, among other things, code generators and textual modeling and the like and Xtext came up at some point, sparking my interest for it.

But the thing that definitely changed my attitude from “hey this is interesting”, to the “ZOMFG I CAN’T WAIT!” electrifying nerdiness that I’m feeling right now was this fantastically interesting blog post by Sven Efftinge, on the usage of Xtext in the automotive industry (namely BMW). Do read.

(And in case you’re wandering, yes, that’s really Greek reggae and the band is called Locomondo)

Posted in blogging, eclipse, events, fun, self-reference Tagged: Eclipse Modeling Day, xtext

GPU Fail

2009-11-12T19:23:59+00:00

I’m getting screen corruption issues on my Macbook Pro; the one I’m using to run the study upon which I’m basing my thesis; the one I’m using to demo my project at next week’s RIA Showcase.

It’s a 3 year old machine with a battery that still lasts at least 4 hours. Unfortunately, it has the failure-prone nVidia 8600M GT soldered to the logic board.

I haven’t confirmed the GPU is the culprit. I’m currently backing up the drive (thank you Apple for Target Disk mode and Firewire 800).

I can’t take it to the Apple Store any time soon. I’m still anxiously waiting for my mom to be discharged from the hospital.

Aaaaaaaaaaargh >_

Damn you, Murphy’s Law!

tracSnap Evaluation Ideas

2009-11-11T18:06:00+00:00

I'm working on tracSnap as a CSC495 credit this year. Taking a half course spread out over the whole year is a strange experience... I can never find a large block of time to work on this! I will probably get most of this work done over Christmas break, so that I can focus. So far, I've been working with Jon and Steve to come up with a plan on how to evaluate the tool. Here's what we've come up with:

File relation stuff: Using the file relations graph, try to find some interesting implicit file dependencies that maybe Hadley isn't aware of. And then we tell them, and ask if its useful information. This will be difficult, because there are a lot of confusing file names I will have to make sense of in order to determine which relations are the less obvious ones.
Social/Expertise stuff: Jon has identified tickets that seem to be defects in the code. So, if we look at the data that tracSnap had access to at the time the ticket was reported, would tracSnap have predicted which experts fixed the defect? (i.e.) Are the experts/social contacts suggested by tracSnap the same people that commented on the ticket, and helped to fix the defect?

I've been working on (2.) so far. I've had to rewrite a fair amount of the code in order to incorporate the necessary temporal component. (Before, tracSnap just used the entire repository, from revision #1 to the most recent revision, to make its suggestions. If we want to see if tracSnap "predicts" the right people to talk to, we can only look at the data that happened before the ticket was created.)

So I think I've got the code re-written as required, and so at the moment I'm waiting as tracSnap calculates all the file relations and experts on the Hadley data (this takes a while! 15000+ revisions!)

Size and Activity

2009-11-11T14:35:38+00:00

Next term, I’m going to be teaching CSC302 (the second of our two-course sequence in software engineering). The mandate for the course is to introduce students to the tools and methods they need to deal with large applications; as part of it, I’m thinking of having each group of students go spelunking in a different pre-existing code base. I’d therefore like to find 15-20 applications that are:

Relatively well written in C, Java, or Python (the three languages I can be sure the students know).
Open source (for obvious reasons).
About 50,000 lines long (yes, I know that lines of code is a weak measure of complexity, but it’s easy to calculate).
Build and run on Windows, Linux, and Mac OS X.
Under active development (so that students have someone to turn to when they have questions).

I’d prefer complete applications to libraries, toolkits, or frameworks. Vim is a good example of what I’m after; I’d welcome pointers to others.

Following Up on the Toronto Innovation Showcase

2009-11-11T14:18:01+00:00

My Government 2.0 class didn’t meet this Monday, partly because I was on jury duty, and partly to give students a chance to catch up after last week’s two-day showcase at City Hall. Two things I’ve been watching to fill the void are:

City of Toronto Meeting Monitor — scrolling lists of proposals, amendments, referrals, and what not. I’d love to see something like this for the University of Toronto…
datato.org (which got a mention at O’Reilly from Nat Torkington). Neat to see that so many people want the same things my students did; kind of sad that some of that data (like household energy consumption) isn’t going to be available for a long time.

Four Links

2009-11-11T14:09:31+00:00

Email is down—strange how hard I find it to get started in the morning without my regular fix. While I’m waiting for someone to turn the firehose back on, here are four recent links:

The Best Hand Painting Art Ever. Gosh, we’re a creative species, aren’t we?
Ripley: Automatically Securing Web 2.0 Applications Through Replicated Execution. A new project at Microsoft Research that tries to prevent evildoers from subverting rich internet applications by duplicating execution server-side. Neat idea.
The CIX Top 20. Lots of interesting companies here; lots of good ideas.
The Go Programming Language. I’m underwhelmed: it’s as if the last 20 years of programming language research hadn’t happened. Now, Google backing Haskell… That would be interesting.

Achieving Climate Sustainability?

2009-11-11T01:51:18+00:00

I’m visiting Microsoft this week, and am fascinated to discover the scope and expertise in climate change at Microsoft Research (MSR), particularly through their Earth, Energy and Environment theme (also known as E3). Microsoft External Research (MER) is the part of MSR that builds collaborative research relationships with academic and other industrial partners. It is currently headed by [...]

Cross-Country Projects In January

2009-11-10T20:53:14+00:00

We’re going to run some cross-country undergrad capstone projects again starting in January—the newly-updated UCOSP FAQ has some details, and I’ll post a note on that blog when we finalize the project list. Please help us get word out to profs and students who might want to take part.

What Would It Take To Convince You?

2009-11-10T20:12:03+00:00

Jon Pipitone, a grad student here at the University of Toronto, has posted a question on his blog: what would it take to convince you (a software engineer) that a piece of complex climate modeling software was working correctly? It’s a hard question — he’d welcome your suggestions.

zuzelvp

2009-11-10T16:26:59+00:00

The paper “How do we read algorithms?: A case study” by Martha E. Crosby and Jan Stelovsky got my attention from the ones listed on the annotated bibliography about Code Reading and Program Comprehension. The authors monitored eye movement of 19 volunteers (randomly selected from the University of Hawaii’s computer science program) while reading an [...]

Hyper-V Performance

2009-11-10T03:17:51+00:00

I was taking a look around for some Hyper-V performance guidelines when virtualizing production workloads such as Exchange 2007/2010 on Windows Server 2008/2008 R2.

Checklist: Optimizing Performance on Hyper-V
http://msdn.microsoft.com/en-us/library/dd722835(BTS.10).aspx

Looking for that last ounce of Hyper-V performance? Then try affinitizing your VM to a NUMA node
http://blogs.msdn.com/tvoellm/archive/2008/09/28/Looking-for-that-last-once-of-performance_3F00_-Then-try-affinitizing-your-VM-to-a-NUMA-node-.aspx

Hyper-V Performance Counters - Part five of many - "Hyper-VM VM Vid Numa Node"
http://blogs.msdn.com/tvoellm/archive/2008/09/29/hyper-v-performance-counters-part-five-of-many-hyper-vm-vm-vid-numa-node.aspx

Performance and capacity requirements for Hyper-V
http://technet.microsoft.com/en-us/library/dd277865.aspx

Microsoft Support Policies and Recommendations for Exchange Servers in Hardware Virtualization Environments
http://technet.microsoft.com/en-us/library/cc794548.aspx

Exchange Server 2007 and Hyper-V
http://blogs.technet.com/scottschnoll/archive/2008/06/15/exchange-server-2007-and-hyper-v.aspx

Should You Virtualize Your Exchange 2007 SP1 Environment?
http://hypervoria.com/hyper-v/should-you-virtualize-your-exchange-2007-sp1-environment.aspx

Hyper-V How To
http://blogs.virtualizationadmin.com/davis/tag/hyper-v-how-to/

Simplifying the UML - the Request for Proposals is out

2009-11-10T01:32:47+00:00

The OMG has just released a draft UML Simplification RFP . Unfortunately, right now this document is only available to OMG members. However, this proves that the OMG is really serious about the need of simplifying the complexity of the current version of the UML language and this is good news for all of us!!

Scientific software quality: what would it take to convince software engineers?

2009-11-09T22:21:15+00:00

As I mentioned, last week at the CSER conference I presented a poster summarising my study. I got some great feedback about the coherence and integrity of my study, as well a great question: what would it take to convince the software engineering community of anything about the quality of scientific software?

I had about eight people stop by and actually engage in conversation with me. For most people (mostly other students), this was their first time thinking about scientific software as a type of software worth studying on its own. I had to walk slowly through the first slide to explain how computational scientists use software models to do science. I think this motivated the research questions well, since after the first slide most people understood that climate models are being built and used in an entirely different context from, say, accounting software: there isn't the same certainty about what correct output should look like.

I then explained my approach and findings. Almost everyone had one, or both, of the following reactions:

I'm not convinced.

Yup, just the one. I knew this was their reaction because either people would say so, or they would stare at the poster uneasily until I asked them questions about what they felt about the study. Some felt that the whilst the different parts of the study were reasonable in their own right the study didn't "gel" in a way that either built up (or eroded) their confidence in the quality of climate modeling software. They felt just as unsure about it as before I had explained my study and results.

Here's one example. With one fellow, a professor of software engineering (let's call him Mr. B), I pointed to the defect density chart. This one here:

And I asked whether it gave him any insight or feel for the software quality of the models. He said that yes, this chart led him to believe that the Model B and C were of good quality, and model A was not of great quality. When I asked whether it was the absolute defect rates or the comparison between the different models that led him to believe this, Mr. B said it was the relative rates. But then he also agreed that all of the models had defect densities that were "good". Hmmm... wait, what?

After going through the rest of the study, I asked him what he thought of the software quality of climate models and he said that they were "not very good". He maintained this even when I pointed to the defect density chart earlier on. I asked why he felt that, and he said that because there are so many ways for the climate scientists to interpret and represent their results that they may, even unknowingly, present their models as being more accurate than they really are. He felt that the models could be full of bugs and the scientists may never notice.

To be clear, I'm not convinced either way about the software quality of climate models. I'm trying to leave my opinions for later, once I've collected more interview data at least (and I've expressed before how poor I think measures like defect density are at gauging quality). But, one thing I've come to understand about the climate modelers is that they care very deeply about the correctness of their code -- they most certainly do not take it for granted. I pointed this out, to Mr. B, and I explained some of the ways I've learnt that the climate modelers verify and validate their models.

Mr. B was still unconvinced. And so, as I say, neither were many others.

I understand this. I'm not sure that what I've done convinces me that climate modeling software is "good" or "bad" either. But maybe good/bad and high/low isn't a meaningful way to thinking about quality. Still, I have this urge to distill what I've learnt about the scientific software quality into a quantity. I dearly want some objective measuring stick or benchmark to be able to judge/compare/assess software quality. I think this is what the folks at the poster session wanted too, and didn't get from my poster.

Near the end of the conversation, I asked everyone this final question: what would convince you, as a software engineer, that a climate model is of good software quality or not? I asked this question at the CASCON workshop as well. No one had an answer. In fact, most people just dismissed the question with a laugh. Is it that silly of a question? I think it's a great one (but sure, maybe a touch rhetorical).

I've asked a few climate scientists the same question in earnest: what convinces you that climate model software is of good quality or not? The answers have been quite varied. Knowing the history of the model, or the development team, the state of the documentation, whether they've seen the model code or not, and generally how open the development is, are some of the things that factor into their assessment. Defect densities do not.

What I appreciate about the climate scientists' answers is that they focus more on the internals of the development process than they do on metrics. My interpretation of this is that knowing that the modeling group is following the right processes for building the model, and being able to verify that yourself, is a better indicator of the software quality than any defect metric. (I'm of the mind that this is true in general of software quality assessment... but this is quite a new thought for me and I haven't spent much time thinking through the implications.)

But anyhow, what this leads me to is the idea that maybe a more satisfying way of assessing software quality is some sort of "maturity" assessment, like what is done as part of the CMMI process. "Best practices" for developing and verifying climate models could be established by the community to which each modeling centre could be assessed against. From my interviews so far I can already suggest some of the factors that would go into such standard (check the last two slides).

I'd guess that this sort of assessment scheme would be acceptable to climate scientists, but would it convince the software engineering community? If not, what would need to be included in the assessment? Or, is it just a matter of educating the software engineering community as to the nature of scientific software?

Addendum: As Jorge suggested, maybe I received such suspicion at the poster session simply because of the venue. Poster sessions rank at the bottom of the academic credibility scale, so maybe everyone at the poster session is bound to be suspicious.

Climate Change Informatics Research Areas

2009-11-09T18:56:09+00:00

I’ve finally managed to post the results of our workshop on Software Research and Climate Change, held at Onward/Oopsla last month. We did lots of brainstorming, and attempted to cluster the ideas, as you can see in the photos of our sticky notes. After the workshop, I attempted to boil down the ideas even further, and [...]

zuzelvp

2009-11-09T18:02:22+00:00

I started reading papers about code reading, navigation and searching with a small paper: “Fluid Source Code Views for Just In-Time Comprehension”. The paper didn’t give me information about how developers actually read, navigate and search source code, but through it I noticed another paper of Dr. Chris Exton that maybe will contain more information. [...]

Registration Problems with Google Groups

2009-11-09T17:19:18+00:00

I set up a Google Group to communicate with the contributors to the book on evidence-based software engineering that I’m pulling together for O’Reilly, but have run into some registration snags. I invited everyone into the group using their regular email addresses, but Google isn’t letting some of them confirm (which means they don’t get email sent to the group). They can confirm when I invite them via a GMail address, but they don’t read that regularly, and anyway, this stuff is just supposed to work, right? Has anyone run into this before? If so, what’s the cure?

How Many Ways Can OneZone Fail?

2009-11-09T13:42:46+00:00

Stop me if you’ve heard this one… I’m at the court house waiting to find out if I’ll be selected for jury duty. The only WiFi network available is OneZone, but:

The signup pages are all HTTP, not HTTPS (which means my credit card information and home address just flew threw the ether unencrypted), and
Once I sign up (yeah, I did it anyway, I’m that desperate for bandwidth) the system echoes back my user ID and password on another unsecure page in plaintext.

Who are these people? And why are they still employed? *sigh*

Later: it may be insecure, but at least it’s slow…

Vegetables

2009-11-09T12:42:36+00:00

Yep, this post is not about research. It's about vegetables. Specifically, vegetables I just received from two of my friends, Tarrah and Nathan. Just two years ago they bought a farm up near Neustadt, Ontario. They are making a go of starting up an organic, mixed vegetable and livestock farm: Green Being Farm (website under construction).

If you know me, you know that I often affectionately talk about working there last summer. And if you know me, you may have also had the opportunity to eat some of the produce grown at their farm last year (those of you in the lab may remember the potato fairy delivering piles of potatoes to your desk last summer), or you may have been one of the lucky folks to get in on their amazing pastured pork, chicken or turkey.

To eat good, local organic food is a real treat. To eat good, local, organic food that friends of mine grew is amazing. I'm so proud of them and what they are doing, and I'm so proud to be eating their produce. I regularly have doubts about the benefit of my research to the world, but I never doubt the benefit of what people like Tarrah and Nathan are up to.

Code-Generation conference 2009 - Slides available

2009-11-08T22:30:05+00:00

Slides for many of the talks in the Code Generation 2009 conference are now online . Some of the featured talks are:

MDD: The Good, The Bad and The Ugly / MDD: The Best, The Worst and The Ugliest - Steven Kelly (MetaCase) & Markus Völter
Pragmatic Model-Driven Software Development with CodeFluent - Omid Bayani & Daniel Cohen-Zardi
Dynamic modelling and code generation patterns in CodeFluent - Omid Bayani & Daniel Cohen-Zardi
Template Specialization - Kathleen Dollard

Speaking at Queen’s

2009-11-08T19:35:16+00:00

I will be giving a repeat of my DevDays talk on evidence-based software engineering at Queen’s University on Thursday, November 19. Details are in the flyer; if any regular readers are in the audience, please say hello.

Toronto DemoCamp #24: Dec 3

2009-11-08T17:53:19+00:00

David Crow’s announcement has the details. It is becoming more focused on social media than hard-core geekery, but it should still be fun.

Green Web Hosting

2009-11-08T17:44:38+00:00

Useful page comparing the green options and claims of various web hosting services. My host, Site5, isn’t even listed; renewal is coming up, and I may switch.

Modeling the solutions to climate change, part I complete.

2009-11-07T23:35:08+00:00

A couple of weeks ago I mentioned a side project a few of us in the software engineering group are undertaking. In short, the purpose is to explore the idea of modeling (graphically, not in code) the various solutions to climate change proposed in a couple of recently published books. The objective is not only to model the solutions, but to model them in such a ways to make it easy to explore the differences and similarities between the different solutions. We have restricted ourselves to just looking at wind power, as a way to make this experiment do-able in a shortish period of time.

Our plan has been to look at each book in turn and completing several different types of model for each book: an ER model, a goal model, and maybe a systems dynamics model. Once we've completed those then we'll explore how to relate them to one another.

We began by looking at a David MacKay's book. Last time I posted the ER diagrams and the start to the goal models. Well, we're done the goal models now, so here they are.

This first goal model covers the first chapter of the book. This has very little to do with wind power, but gives us the the actors and some context from which to build out the wind power model.

And here's the wind power model:

Even if you don't understand i* syntax you should be able to follow these diagrams with some success. Again, I should note that these models represent the view of MacKay himself, as interpreted by our motley group.

Having completed these models marks the end of the first stage of this experiment. The next stage is to do make the same types of models for another book in the series.

I have a few observations to make on the modeling process and this project. This is the first time I've ever been involved in formally modeling anything. The process was, on the whole, frustrating and unsatisfying. I would leave each session feeling like whilst we had put up more boxes and arrows on a canvas we were somehow missing the essence of the text. When we made the entity-relationship diagrams we often seemed to be forcing syntax into the labels on the relationships. The goal models seemed equally inadequate albeit in a different way: there were concepts we all wanted to represent (e.g. facts like "typical wind speed is 6 m/s") but there was no way to say this is in i*. Jen, our modeling Guru who ran most of our sessions, would often tell us that what we were trying to say wasn't easily expressed in a goal model or ER diagram and that this frustration was something she was familiar with from her own work. It felt like putting on a one-armed sweater and fishing around with one hand for the other, non-existent, arm hole. This concern of mine is really just me saying I don't think that the modeling languages we are using are rich enough to really capture what is important. Or also that they don't seem to capture it in "the right way"; that they don't "carve nature at its joints" (plato).

I'm open to this one possibility though: maybe this dissatisfaction points to what is actually a helpful and normal aspect of using several different modeling approaches. Each approach on its own may not be able to represent the full meaning of the text, but taken together they may. And also, by constraining the modeling to specific concerns (entities in the ER diagrams, and goals and actors in the i* models) we are actually providing a more clear picture of the text, and in the future, a more clear picture of the differences between different texts. For instance, it may be easier (at the very least in terms of visual clutter) to show the differences between the basic concepts used in the texts by somehow showing differences in the ER diagrams, without having to, at the same time, navigate actors and goals and process concepts.

This is all just speculation, of course. I'm just trying to say, that my uneasiness may just be because I'm unfamiliar with thinking about something in parts like this.

There's another thing I'm unsatisfied with. If our goal is to show the differences between several types of texts, I fear we may be going about doing this in an unhelpful way. I have nothing to base this on other than just a sinking feeling I get. To me, the real work in this challenge is about how to represent and navigate differences is large and complex structures. In some sense, doing the modeling is easy, or at least known. Trying to explain how one model is different from another in a way that is actually helpful and useful... I have no idea. What we don't have is what we want this model "diff" to look like.

One thought I had: we start by writing up our own summary of the differences between two of the texts. That is, we read two of the books on the list and then collectively write up an essay that compares and contrasts the two authors' solutions in as much detail as we think is relevant. Then we model that document. Doing this will give us the goal posts and probably teach us heaps about what and how to make a visual comparison useful.

What do you want?

2009-11-06T23:15:29+00:00

I was asked today what I wanted in a job.

It was surprisingly difficult to answer.

I like achievable challenges. That’s obvious. But—getting some CSS to display just right on Internet Explorer 6 is an achievable challenge, but it’s not work I’ll enjoy. Crushing a hard bug is fun, but debugging code that should never have made it through review is not. Writing efficient code is fun. Dealing with low-level memory management, not-so-much. Type errors in loosely typed languages feel preventable. Not fun. Designing a good API: Fun. Dealing with legacy constraints: Not fun.

These days it’s popular to filter software engineers: Front-end or back-end? Applications or systems?

But this isn’t a key distinction for me. It doesn’t capture the essence of what I care about. I could be happy working on front-end or back-end code. I could be unhappy working on front-end or back-end code.

Now, moments after my interview ended, I’ve managed to figure out the answer…

Accidental complexity: Not fun. Intrinsic complexity: Fun.

Misa Sakamoto on DB2 Technology Explorer

2009-11-06T19:31:00+00:00

One of our PEY students, Misa Sakamoto, has an article up on DeveloperWorks about the stuff she’s doing at IBM — yay!

Planet DCS@UofT

Serendipitous and Unexpected

Adaptation just as important as Mitigation

Modeling tools news - Nov. 20 2009

Will Peak Oil Save us from Climate Change?

Academic Genealogy

Methods & Tools - a free software development magazine

Animated Database Courseware

Does Microsoft still believe in model-driven development? - the Oslo move

Engineering the Software for Understanding Climate Change

One Trillion Tonnes of Carbon

First evaluation result

Special Issue

zuzelvp

What He Said

Top 5 most popular posts in October 2009

What we actually know about software development by Greg Wilson

Don't miss Johan den Haan series of posts on basic concepts of MDD

First dollar with the Amazon Affiliate program

I’m Just Not Very Creative

Evolution in Action

Embodied Social Proxies

Evaluation of me as a TA

Why Am I Not Surprised…

When I Said “The Last Twenty Years…”

Speaking at CUSEC 2010

Bend It ‘Til It Breaks

Pitfalls of informal modeling languages (cartoon)

Three posters at the AGU meeting

Bad surveys

My sample Google/Microsoft interview question

What do we want from climate informatics tools?

The Munk Debates: Climate Change

Two More Gov 2.0 Links

The Definition of “Exotic”

WWTD?

Me wanna Xtext

GPU Fail

tracSnap Evaluation Ideas

Size and Activity

Following Up on the Toronto Innovation Showcase

Four Links

Achieving Climate Sustainability?

Cross-Country Projects In January

What Would It Take To Convince You?

zuzelvp

Hyper-V Performance

Simplifying the UML - the Request for Proposals is out

Scientific software quality: what would it take to convince software engineers?

Climate Change Informatics Research Areas

zuzelvp

Registration Problems with Google Groups

How Many Ways Can OneZone Fail?

Vegetables

Code-Generation conference 2009 - Slides available

Speaking at Queen’s

Toronto DemoCamp #24: Dec 3

Green Web Hosting

Modeling the solutions to climate change, part I complete.

What do you want?

Misa Sakamoto on DB2 Technology Explorer