Scale Changes Everything: Understanding the Requirements for Systems of Systems

Steve Easterbrook, University of Toronto

Keynote talk originally given at the 6th IEEE International Conference on COTS-based Software Systems, Banff, Alberta, Canada, February 26 - March 2, 2007 (ICCBSS 2007)


Software technology now penetrates almost every aspect of our lives in complex ways. The reality of 21st century software development is that software itself is but one part of a complex system-of-systems that includes a broad technological infrastructure along with a wide set of human activities. The technological systems and the human activity systems have a symbiotic relationship - each shapes the other in complex ways, such that neither can be understood in isolation. A recent report from the SEI on Ultra-Large Scale (ULS) Systems accurately characterized the nature of these systems-of- systems: they have no centralized control; experience normal failures and continual evolution of heterogeneous elements; and their requirements are inherently conflicting, diverse and often unknowable. For design purposes, the boundary between people and software disappears - design is as much about shaping the human activities as it is about constructing the software.

In this talk, I will argue that these challenges are now true of most software development. The engineering approaches we use today for software development only work when we take a very narrow view of the requirements, as well-defined sets of features and interfaces, which can be fully specified. This approach helps us to build components that conform (in a narrow sense) to their specifications. But we cannot tell in advance whether they will be any use in any of the many different systems-of-systems in which they may be deployed. Our engineering techniques rapidly break down when we attempt to scale up our design ambitions. The result is a growing gap between expectations and practice in the software industry. We can build very reliable software at the small scale, for tightly constrained problems. But we cannot build reliable software for complex socio- technical domains.

To make progress on these challenges we need to abandon the idea that we can write complete, consistent specifications. Instead, we need to capture the multiple, conflicting requirements for each software component that arise from its different contexts of use. We need to be able to express our partial understandings of the broader systems- of-systems in which our components will be deployed. And we need to be able to reason about the properties, and end-to-end behaviours of these systems, without resolving all the unknowns and inconsistencies in our models. I will end the talk with a survey of recent research in requirements engineering that tackles these challenges. In particular I will discuss techniques for managing large, evolving collections of fragmentary requirements models, and show how it is possible to tolerate inconsistency when we reason about the properties of these models.

Further Reading

Gerald Weinberg: An Introduction to General Systems Thinking

The ideas that originally inspired this talk come from my reading of works on systems theory and systems thinking. The best introduction to this is Weinberg's book. Incidentally, I believe that Weinberg was the first person to identify the "Principle of Complementarity", as I haven't found any one else describe this idea. To me, it seems fundamental to all the conceptual modeling we do in software development, but sadly, few modelers seem to be aware of it.

Linda Northrup et. al.: Ultra-Large Scale (ULS) Systems

This is the report I quote from at length in the talk. The chapter on the characteristics of ULS, and how they challenge software engineering assumptions is great. However, I'm not so sure about the research agenda they set out in response to these challenges.

Charles Perrow: Normal Accidents

Perrow covers the idea that in large scale systems, things go wrong all the time. He coined the term 'normal accidents' to describe this phenomenon. Should be required reading for anyone interested in complex systems.

Michael Jackson: Software Requirements And Specifications

The distinction between requirements and specifications that I use in the talk is due to Jackson. This book is perhaps the most readable introduction to those ideas. The analysis of the aircraft landing on a wet runway is also from Jackson.

Pamela Zave: An experiment in feature engineering

I don't think Pamela has ever published her limerick, but she has written extensively about the challenges of analyzing feature interactions in evolving systems. This paper is a nice introduction to her work.

George Monbiot: Heat: How to Stop the Planet From Burning

A must read on global warming. Monbiot does some interesting analysis on why some strategies really don't work well. In particular, he has some nice analysis on why energy conservation (alone) might not deliver the reductions that it is supposed to, because of systems interaction effects.

Graham Priest: In Contradiction: A Study of the Transconsistent

If you're ready for a logician's account of inconsistency, this is it. Priest argues for dialetheism, the view that there really are true contradictions in the world, and hence we need to capture these in our logics if we want to use logic as the foundation for human reasoning. And he explains how paraconsistent logics work.

...and of course, some of our own papers cover the research described at the end of the talk:

N. Niu, and S. M. Easterbrook, So, You Think You Know Others' Goals? A Repertory Grid Study. IEEE Software, Vol 24, No 2, pp53-61, March/April 2007.
Describes our work with matching stakeholder terminology using ideas from pychology.
Nejati, M. Sabetzadeh, M. Chechik, S. M. Easterbrook, and P. Zave. Matching and Merging of Statecharts Specifications. Proceedings of the 29th International Conference on Software Engineering (ICSE'07), Minneapolis, USA, 21-25 May 2007.
Describes an approach to matching and merging of software models, in this case, behavioural models expressed as state machines.
G. Brunet, M. Chechik, S. M. Easterbrook, S. Nejati, N. Niu, M. Sabetzadeh, A Manifesto for Model Merging. Workshop on Global Integrated Model Management (GaMMa'06) at the 28th International Conference on Software Engineering, Shanghai, China, 20-28 May 2006.
Describes our overall approach to model management, and identifies a set of operators for manipulating partial, inconsistent models.
M.Chechik, B. Devereux, S. M. Easterbrook and A. Gurfinkel Multi-Valued Symbolic Model-Checking. ACM Transactions on Software Engineering and Methodology (TOSEM), Vol 12, No 4, p371-408. Oct 2003.
Shows how we can reason in paraconsistent logics to draw conclusions about models that contain inconsistencies.
M. Sabetzadeh, S. Nejati, S. Liaskos, S. M. Easterbrook and M. Chechik Consistency Checking of Conceptual Models via Model Merging. Proceedings, 15th IEEE International Requirements Engineering Conference (RE'07), Delhi, India, October 15-19, 2007.
Describes our latest work on consistency checking for distributed collaborative modeling.

Photo Sources

Most of the photos used in this talk are from

Photo Credits:
1) Steve & Jamie © Sarah Easterbrook (not available)
2) TransCanada PipeLines Pavilion © Weeza1@flickr
3) Downtown Banff © Steve Easterbrook
4) Cascade Mountain © Steve Easterbrook
5) Mount Rundle © Steve Easterbrook
6) Banff from Sulphur Mountain © Steve Easterbrook
7) Canadian Rockies © Google Earth
8) Athabasca Glacier © eckiblue@flickr
9) Athabasca Falls © junnn@flickr
10) Moraine Lake © champy1013@flickr
12) ICCBSS panel session © Steve Easterbrook (not available)
13) Highway 401 in B&W © Lone Primate@flickr
14) Westjet aircraft © tnano@flickr
15) Highway #1 near Banff © JoLoLog@flickr
16) How I missed my flight © Dru!@flickr
17) A chronicle of obsessions © romanlily@flickr
18) Bulb © Andrew Coulter Enright@flickr
19) On the way to Banff © Craig James White@flickr
20) Limestone Quarry © rogueleadZERO@flickr
21) New Learning Centre © The Banff Centre
22) Death's Day © judÄ™@flickr
23) Greenhouse Effect © Me vs Gutenberg@flickr
24) Vapor trails © CaptPiper@flickr
25) Earth from space © NASA
26) mc-50 poster with icons © GustavoG@flickr
27) lost legos 1/3 © massdistraction@flickr
28) Time Warp at 14h19m35s © Claudia1967@flickr
29) Emett's Dream Machine view 1 © Steve Easterbrook
30) Emett's Dream Machine view 2 © Steve Easterbrook
31) Fire Bug © homo_sapiens@flickr
32) drive-thru © Trisha GG@flickr
33) The Wow starts now © hkdigit@flickr
34) Pickard as a Locutus © Paramount
35) Stress and Multitasking is the Future © Ye Santenko@flickr
36) Things to do this year - 2006 © bumpoowilly@flickr
37) die computer die 2.22.07 © kmevans@flickr
38) Threadless few © //d@flickr
39) emotional baggage claim part 1 © xxxtoff@flickr
40) NASA mission control & shuttle launch © NASA
41a) Buying Food © Clonny@flickr
41b) money animal market © kdriese@flickr
42) Sulphur Mountain Viewpoint © Steve Easterbrook
43) Stairway © amyallcock@flickr
44) In Fact and KMZ on Chateaudun © bluehour@flickr
45) Whiteboard © Michel Vuijlsteke@flickr
46) Writing on Slate with Charcoal ©
47) Bulbs © VancouverDuane@flickr
48) A is for Auntie's House © pumpkinoodle@flickr
49) Zihuantanejo © ektarama@flickr
50) Hey don't turn off the lights © nishiology@flickr
52) Landing gears on a wet runway © hsivonen@flickr
53) flickr screenshot © flickr
54) LOGO2.0 © Stabilo Boss@flickr
55) The bezel under the screen © Marco Wessel@flickr
56) Rainbow Colors © NY_doll@flickr
57) DIY Server Rack © phatmonkey@flickr
58) Day 20 © j.reed@flickr
59) wall of spam © chotda@flickr
60) slash with mackerel © ScottS101@flickr
61) Helpful terminology © sono salvo@flickr
62) Jennifer Aniston and Brad Pitt. Source: all over the internet
63) Brad Pitt and Angelina Jolie. Source: all over the internet