The challenges of software engineering
On november 30th the Club of Amsterdam organized its 'Future of Software
Architecture' conference. The meeting was hosted by Syntens, a government
sponsored 'Innovationnetwork' for entrepreneurs, who have their office
in the center of Amsterdam in a modern highrise overlooking the water.
Cargo ships lazily coasted by, as the winter sun gave way to the long
winter evening. The meeting proved to be what is becoming a trademark
of the Club of Amsterdam's events. New insights arose out of a series
of constructive confrontations between radically opposing viewpoints.
The big surprise was that a seemingly straightforward technical subject,
in appearance almost dull, generated such different, almost diametrically
opposed, perspectives. The contrast inspired a frantic discussion about
complexity, history, values, and the role of technology in today's and
Maarten Boasson, professor at the University of Amsterdam and chair of
the Industrial Complex Computer Systems Department, started of the evening
in minor mode. The title of his presentation "Do we want to keep software
Architecture alive" says it all; software architecture is a terminal patient.
Boasson emphasised that in his view software architecture concerns itself
with the technical design of systems. This is a relevant remark, since
the term software architecture has acquired such diverse meanings over
the years. From a purely technical term it has evolved into an all inclusive
concept, containing everything from generic properties of computer systems,
to users, to a wider context of stakeholders, 'software architecture'
can describe all possible relations and structures that exist between
these elements. In fact this conceptual inflation of the term 'Software
Architecture' will be reflected by the diverse opinions that will be exchanged
during the evening's discussion.
The succes of the digital revolution, or perhaps we should say semi-conductor
revolution, has left no aspect of society untouched. The explosion of
computing power, expressed by Moore's law, has generated a myriad of complex
systems that pervade all corners of reality. The discipline of software
engineering seems to be split between the drive to design robust functioning
systems which exhibit predictable properities and the ever quickening
development of business needs, discovering new market opportunities which
require ever newer applications. In this dynamic we are constructing our
tower of Babel on an imperfect foundation of frantically, and shabbily
integrated legacy systems and components. In short: software architecture
is a mess. There is no comprehensive theory, and practical (i.e. business)
considerations drive most of software development, leaving little room
for reflection on design and the pursuit of quality. Education and research
are key to creating quality systems, but these considerations are completely
marginalized by forces driving the development of technical systems. Software
is unstable and unreliable, and we're not just talking about PC operating
systems. Onboard computers in airliners also do fail for no other reason
than bugs in the programming; which is a daunting realization. (For more
on this: http://catless.ncl.ac.uk/Risks
Unfortunatele we cannot step back and start all over again from scratch.
Boasson's presentation amounts to a passionate plea to start focusing
on quality and appreciate the crucial impact of great minds. Really good
software, if it exists at all, is without exception the product of a few
brilliant programmers, whose skills cannot be matched by a mulitude of
mediocre developers. The road ahead is unclear, but if we want improvement,
we should learn from our mistakes and develop general principles for understanding
how architecture is related to specific applications and not merely adopt
the fashion of the moment. Whether this will actually happen, or even
has a chance of happening, is uncertain. The final conclusion is depressing,
looking back on three decades of systems engineering, software architecture,
as a way to mindfully design the best possible systems, is more or less
A greater contrast with the next presentation can hardly be imagined.
Maarten Visser, a self-styled Performance, Productivity and Integration
evangelist takes us into the promises of Web 2.0 and the emerging reality
of an ever more complex array of communication and information processing
applications. His focus is on communities which organize themselves around
online applications, and he explains how these community needs will become
important drivers for the development of next generation information and
communication tools. The massive amount of information available is so
overwhelmingly large and complex that we will need new layers of interpretation
between the information and our needs. Such 'semantic' web technologies
are to help us find what we need in the exploding informational complexity
that is the internet. Visser bounces with energy as he performs his fast-paced
sermon with an exuberance and optimism that indeed resembles divine inspiration.
A numbing list of emergent technologies are presented: blogs, XML and
RSS (Really simple syndication) feeds are already superseded by SSE (simple
sharing extensions), RDF (Resource Description Framework) and an new ambitiously
named standard called WOL (Web ontology language). Things don't appear
to become less complex folks! Everything is to become connected to everybody
everywhere, with machines and applications taking care of the semantics
of it all…
Niek Jetten's subsequent presentation brings home the point already bemoaned
by Boasson: software engineering in general, and software architecture
specifically, are driven by business interests, and as we all know: business
has a dynamic all of its own. In effect we learn why the interests of
business (low-cost, high-pressure, short time-to-market, scalability)
are so diametrically opposed to the interest of software architects attempting
to build robust quality system with predictable properties, requiring
research, reflection, and ample time.
Given the contrasting views in the presentations the discussion promised
to be lively, and so it was. In spite of Boasson's critical assessment
of the quality of most software, the general focus seems to be on new
developments, and since these developments have an impact beyond the technology
itself the discussion quickly expanded into non-technical areas. Software
architects value robust systems, web communities seek intangible values
in their interaction with each other, business is interested in financial
value and healthy bottom lines, these represent very different value domains.
Even though the high expectations of Web 2.0 may be exagerated (it wouldn't
be the first time the internet will have inspired fantasies) the internet
will increase its influence, as certain as its internal chaos will increase
as well. As the internet is sucking in information and communication functions
that are traditionally provided by other media like radio, television
and telephones, it will become the dominant knowledge and and communication
system of the global village, and its architecture will be a crucial asset
(or liability) in our brave new world. It seems clear that software engineering
will increasingly be driven by what users, and communities of users, want
and need. Isn't that a good thing? Perhaps, but the internet already is
such an intense cocktail of interrelated technologies, components, systems
and protocols. How are we to guarantee its stability and sustainability?
How are we to rely on a system containing so many possible glitches?
A real solution to the problems outlined by Boasson does not present itself.
So what if we just reinterprete the problem? Could software engineering
become somewhat akin to a behavioural science; empirically studying autonomous
organic systems? So that we recognize the inherent instability of systems
and admit that the level of complexity is beyond the reach of our understanding?
Properties of a system then become emergent rather than predicted by design.
Boasson is visibly appalled by this suggesting while Visser is eagerly
nodding, having found a metaphor that strikes a chord with his visions.
Did we discuss the future of software architecture? Not everyone agreed.
To some the discussion lacked technical content, to others the entire
session had too limited a focus on technology and insufficiently addressed
the humanity of it all. Well, we did end up discussing quite a range of
topics. What do value systems, human needs, communities and social developments
have to do with software architecture? Well, as with many things, context
determines everything, whether we like it or not, and systems are an integral
part of their supersystem. As such the "Future of Software Architecture"
exemplified the Club of Amsterdam's for diversity, creativity.
Mathijs van Zutphen, Philosopher, educator, artist and creator of VISH,
participates as a philosopher in the Knowldge Stream Life Sciences of
Summit for the Future on Risk.