Reading through "Software Engineering Essentialized", Part I
(There are four parts to the book. What follows is only about part one. I am in the process of reading the rest. Hereafter, "the book" refers only to part one.)
There are no end of methodologies when it comes to software engineering. I’m not exactly a researcher in the field, but I’ve experienced lot of different approaches to development in my work life, and during the course of my thesis I read a fair bit about it. I’ve yet to be convinced that following any known method will bring project success.
With that in mind, I started reading Software Engineering Essentialized, a book in the works from some of the folks behind SEMAT. It aims to describe the essential elements of software engineering. To quote:
[…] this book is fundamentally different. The things being described are key aspects that are selected because they are prevalent in every software engineering initiative. They are the essential things to work with, the essential things to do, and the essential competencies needed when you develop software. They are not just examples of things or typical things. They are selected because they are the things that underpin all recognized ways of developing software.
The basis of this book is the Essence standard, which was written by the SEMAT community and adopted by OMG.1 Essence is "a common ground, a kernel or a foundation of software engineering". That is, it captures the common themes and elements that permeate all the known software methods. Basically, if you follow some method, Essence should be able to describe it.
As a teaching technique, the book introduces a character: Smith. Smith’s story, such as it is, is that of a technology enthused teen who becomes a keen college student, culminating into a career programmer for a fictional company (known as “TravelEssence”). We observe Smith’s knowledge acquisition through experience, and empathize (somewhat) with the feeling he can’t quite express about the things he sees in his work and other’s work that seems to be universal.
Don’t get attached to Smith though. He goes long stretches without making an appearance. His story is a MacGuffin.
The story is really about Essence. And cards.
An explanation is in order.
Software engineering history is littered with the remains of failed methodologies. Many of them aspired to bring order to the insanity that is software development by attempting to proceduralize its various aspects. I think it’s fair to say that none of them have been a rousing success. The closest is arguably the “Agile” movement, but it’s too fractured, fad-ish, and ill-defined to be considered a methodology. (Although maybe that lack of definition is what makes it kinda, sorta work.) And by “methodology” we mean a set of prescriptive practices and guidelines that are proven to produce quality software with a reasonable cost.2
Essence is an attempt to grasp the commonalities in all these methods and lay the groundwork for a theory upon which to build a discipline of software engineering. It wants to use those remains as fertilizer and try to grow something out of it.
It’s an admirable goal, and a welcome one. I read about SEMAT’s goals a few years ago and while I was skeptical, I was open to the idea. As the authors lay out in the book, so much of the industry is immature — it’s more akin to the fashion industry with what’s hot at any given time — that it needs something, anything, to get a start on really understanding how and why some software projects succeed while most fail.
As the story of Essence unfolds in the text, it seems reasonable. The concepts and examples are relatable. I appreciated the breakdown of elements into what to work with, what to do, and competencies. Maybe its even accurate. The notion of “patterns” comes up and isn’t well explained, but it’s still familiar. There’s a lot to see of yourself as a software developer in the details (despite the absence of that poor chap Smith): customer interaction, understanding the stakeholders, appreciating business value, gathering requirements, working with a team, planning for maintenance. The book reminds you that as much as don’t want it to be, software is about much more than coding.
After finding yourself in the story, things start to get weird.
First, you notice that there is a lack of talk about how much of a creative effort software engineering really is. And usually a multi-person creative effort, at that. In fact, the creative aspect of it may explain why it is has been so resistant to methodologies thus far. There is some mention of it, but only in passing. That seems to be an error of omission. More on this shortly.
When the prose dips into the explanation ocean, we soon find ourselves swimming with the consultancy sharks. We’re introduced to a diagram language to describe pair programming. At least, I think it’s pair programming, because like Smith, it’s left hanging there. It is only a vehicle to talk in detail about some of the key elements of Essence, the first being alphas. Alphas are the most important things to deal with when developing software. They have states — and are carefully described using a prop.
To help remember the states of the Requirements alpha, Essence provides a poker-sized card description as shown in Figure 13. As we will discuss later, such poker-sized cards will also serve as teaching tools, and even make software development more fun through games.
Seeing as Essence is part of an effort to “re-found software engineering based on solid theory”, it’s very odd that this requires cards. This is no hyperbole. The cards are featured heavily in all the explanations and are described, in gory detail, in the Essence specification (see §9.7).
Act three is where the story really falls apart. After getting through all the elements (alphas, activities, competencies, patterns), including the cards for alphas and their states, the outlandish claims show up. There are various conclusions presented that are assumed to follow from the fact that the authors are experienced and have spent the time to write this down. Take, for instance, the states of alphas. Where do they come from? It turns out that they were “selected by a team of experts” because they saw them as important. I have no quibble with the states themselves, but that’s a foundation of sand for any discipline.
The statements as to the effectiveness of Essence are definitive despite having nothing to back it up. The entire end of this hot mess is an appeal to authority. Essence is “practical” and will help you understand software engineering for your career. This raises more embarrassingly simple questions, like “how”. Up to this point, Essence has been presented as a way to describe practices and methodologies. It has not been shown, in any way, how to use it and even if you did, how it would help. Nor has it been shown to help anyone in practice — but apparently more and more companies have “accepted” it.
This is consultant fodder. The reason we see no serious effort to address the creativity aspect of software engineering here is that it is unimportant when you’re trying to describe things instead of make predictions, like a decent theory would. There does seem to be something that could be the start of useful kernel of software engineering, but there is literally nothing, aside from anecdotes and self-citation, to sway an evidence-based thinker.
It’s unfortunate the book takes this route because there is some good discussion in it. The section on the history of software engineering is enjoyable and accurate. It sets the stage nicely for the reader to appreciate the shortsightedness of the industry: we really do forget our past. The unstated implication here is that software engineering methods are not academic, but monetary. They’re the domain of consultants. Despite what it purports, with this book (so far), you’re getting more of the same.