Main

Blog (RSS)

Book review: From intuition to methodology in debugging

This is the second in a series of reviews of debugging books.

If you’ve been debugging for a while, you may start to notice there are patterns to successfully solving problems. Explaining those patterns can be difficult, which is why debugging is a great skill to learn through mentoring. But how do you know know what tool to use and when to use it? What is a good strategy for dealing with that latest bug report? How do you even tackle a problem in the first place?

Agans’ Debugging[1] describes nine rules that address these questions. And frankly, if you elect to ignore them, “you’re gonna have a bad time.”

Let me get the quibbles out of the way first. Agans comes across as the lovable, geeky uncle whose jokes don’t land because he tries to be both formal and folksy. And two words: Comic Sans. There’s no better way to make a diagram undermine a good point.

The advice he provides, however, is fantastic. It’s unlikely there’s better book on debugging in my reading queue. What’s wonderful about Agans’ book is its applicability, terseness, and relatability. If you’ve worked through some tough problems, the advice will be familiar. If you haven’t, then you’ll surely learn something. The use of “war stories” grounds the advice and gets the point across. It’s the great strength of the book, alternating between the abstract and the concrete. Even better, it’s not too specific. The text does not drag by trying to illustrate tool usage. The book’s war stories may discuss dated technologies—that were dated when the book published—but it doesn’t matter because they are not the central characters: it’s the journey from problem to solution. Debugging without these rules would be like trying to make a movie without a script.

Notably, the rules do not prescribe any particular process. Instead, they aim for principles of thought. The rules, in an odd way, undermine the book in that, once you read them, you may not feel the need to read the book at all.[2] They just feel intuitively correct.

The authors behind The Developer’s Guide to Debugging expand on Agans’ rules and try to put some formalism behind it. Grötker et al. give a general sounding title but aim squarely at C/C++ developers. For their target audience, they succeed admirably. The advice is solid. And they don’t pollute the prose with full code listings, saving those for appendices. The discussion doesn’t even require that you read the code listings to understand what is going on; I’m fairly certain the code was included out of a sense of completeness.

Of paramount importance to the authors is a structured approach to debugging. They attack this by looking at the data flow of C/C++ development. This gives you the locations (in time and space) to consider. After that, they take an expanded version of Agans’ rules—The 13 Golden Rules of Debugging—and say how to find issues at these locations. An ordering on the rules gives it more formality, although I didn’t find the ordering to be strictly necessary; it’s really a partial order. The extra rules are to emphasize elements of software development and while practical, are not a necessity in the act of debugging.

A gratifying aspect of The Developer’s Guide to Debugging is its treatment of tools. Instead of using them to illustrate the point, they emphasize that to actually act on the golden rules you must have a good set of tools, and know how to use them. Furthermore, those tools must have certain capabilities. The majority of the book is spent looking at different classes of problems and what tools for those classes should be able to accomplish. Inevitably there are examples of specific tools but those are kept mercifully short. Having worked with a fair bit of C/C++ over the years, I found the techniques and pitfalls they describe worth reading.

Grötker et al. do a good job of taking the rules of Agans and tailoring them for a particular domain. It almost feels like a procedure, but a loosely structured one. If you’re interested in splitting semantic hairs, I’d call it more of a method than a procedure. I’m not convinced Grötker et al. hit the mark on providing the structured process they talk about, but they come pretty close.

What both books do, spearheaded by Agans’ rules, is capture the essence of problem solving. After all, what is debugging if not problem solving? You are given a situation in which something does not do what it’s supposed to do and your job is to make it do the right thing. This is the intuition that Debugging and the “Guide” are working from. The rules are there to give you perspective and make you ask questions, to make you think carefully about what it is you are trying to do. Tools are necessary to get at the details, but the most important tool is that of your intellect, experience, and a decent set of heuristics.

Debugging by Thinking goes for that angle, with considerably more ambition. Metzger’s approach is to provide methods of debugging from six different personas: a detective, a mathematician, a safety expert, a psychologist, a computer scientist, and an engineer. It is a daunting tome of over 550 pages, and significant departure from the concision of the other two books.

Each persona gets a detailed description of the methods used to find bugs. These methods are based on the worldview of their corresponding fields. The detective and mathematician personas get considerably more attention than the other four, with nearly 70% of the text devoted to them. There are two (large) chapters with case studies that demonstrate how to apply some of the methods and are largely unnecessary, unless you find reading 20+ page C++ programs and terminal output enjoyable.

Describing Debugging by Thinking is difficult because it touches on so many topics. Metzger is not satisfied with the lack of rigour set forth in other approaches and aims to provide an explicit methodology for debugging. He characterizes his methodologies as cognitive (thought-based) instead of constructive (example-based). I didn’t find this compelling. The way he describes “debugging by thinking” makes it sound as if other debugging approaches don’t advocate thought. I have not found constructive approaches present themselves that way, even unintentionally. What they tend to lack is a good set of questions you should be asking, or a generalization of what actions are presented. That doesn’t imply they are mechanistic or without reflection. Typically they assume the reader has already done some thinking and needs help with the details. That is, the developer has already asked some useful questions but doesn’t know how to deal with a particular situation or setup.

Thankfully most of the book doesn’t dwell on this premise and mostly sticks to the task at hand: coming up with methods for debugging. The amount of material covered is extensive, with particular attention to literature review. There is a lot of space devoted to describing the background of the persona’s discipline, usually with an overview of pertinent studies to provide both perspective and evidence. The references and recommendations are incredibly thorough and worth exploring. The discussion on the psychology of errors is of particular interest.

The problem is that it never coalesces into something coherent. There are a few reasons for this, which is unfortunate because there are many good points raised. One reason it doesn’t gel is that the book emphasizes structure and factual presentation over summary. In order for a methodology to be effective, it must be presented clearly. The irony of this text is that the methodologies are left for the reader to discern. In fact, “methodology” is never really defined. The best way I found to get an overview of the methods was to read the (extensive!) table of contents.

When looking at the contents to get the gist of the methods, you can see the rules of Agans and Grötker et al. peeking through. A summary of that nature is not presented in the book. Instead, it’s a long collection of subsections. If you use the table of contents to look at the three chapters devoted to detectives[3], for example, the methods make sense, but then, after reading the text, you have to wonder why it took so long to make the point. Is it really important to provide so much background (and citations!) on fictional characters in a technical text? Most people are familiar with the concept of a detective. Going on for forty pages makes you miss the forest for the trees.

This leads to the significant shortcoming of the book: the writing. Never have I read such stilted prose. I don’t like calling out such things, but it can’t be overlooked. It’s the kind of writing that only a natural language processor could love. It’s so mechanical it’s disorienting. There are long stretches where you can practically see the template at work. The repetition is stultifying. And if the repetition doesn’t get you, the pedantry will. Lists enumerating all possibilities are written out in mind-numbing detail where a simple sentence would do. Or worse, it’s not even a list and the options are written out in full sentences, with each sentence having the exact same structure. The blandness and verbosity of the writing overshadows the point of the book.

Debugging by Thinking partially succeeds in providing a deeper understanding of bugs and how to fix them, but there’s a lot of work for the reader.

February 4, 2018

Footnotes:

[1]

The title of this book is a travesty, in my opinion. It should really just be called “Debugging.” That subtitle is sensationalist. I blame the publisher.

[2]

The rules are available as a poster on the book’s website, debuggingrules.com. The design of the poster, however, is best described as “kitsch”.

[3]

The detectives profiled are Sherlock Holmes, Lord Peter Wimsey, and Professor Solomon. You’re probably only familiar with one of them. I had no idea who Professor Solomon was and strangely enough, the book doesn’t tell you.

Send comments to wozniak.ca with the local part being comment

Last modified 2018-03-25