A Practical Taxonomy of Bugs and How to Squash Them, by Kylie Stradley
#bugs #instinct #debugging

Abstract

Catching software bugs is a mysterious magic, unknowable by science and untouchable by process.

False! Programming bugs, like real bugs, can be organized into a taxonomy. Come with me and I'll show you how classification can help you build “programmer’s instinct” into a logical debugging process.

Description

Talk Format Preview

Biologists use taxonomy to categorize living things, including bugs, which are not fundamentally different from programming bugs. Programming bugs and living bugs both have attributes and behaviors. Phenetics is a branch of classification that groups organisms by their observable attributes. In this talk I’ll describe some common types of programming bugs: Heisenbugs, Mandelbugs, Bohrbugs, Schroedinbugs, Hindenbugs, etc. For example:

Let’s look at the Bohrbug. The Bohrbug is named after the Bohr atom model because it consistently manifests under consistent and well-defined conditions.

The Bohrbug itself will be illustrated as a Bohr model of a stable atom with six little legs. Then I’ll classify each of those bugs by their observable attributes and behaviors. For example:

Are you observing this bug in production? (yes) Can you reproduce the bug easily in your local environment? (yes).

If so, you might have a Bohrbug on your hands. Bohrbugs are deterministic (meaning they always provide the same output for a given input) and are characterized by their repeatability and reliable manifestation. Bohrbugs are often found as GitHub issues with many +1s or reported to your developer twitter account for projects of sufficient popularity.

Once I’ve built a taxonomy of common bugs, I’ll step through a workflow built from my own experiences and those I’ve collected for resolving each bug. For example:

The Bohrbug is the friendliest of bugs and the easiest to catch. We’ll catch it first because it will provide a model for each of the other bugs:

Replicate the Bohrbug manually on your machine.
Write a test that mimics the replication process you just did manually. Assert that the correct thing (and not the Bohrbug) is taking place in the environment that the bug lives in.
Bag the bug by writing the simplest possible implementation.
Rewrite your implementation so that it is highly readable. Readable code prevents bugs from sneaking back into your code base when someone else had to make changes.

The resolution steps for the Bohrbug might seem contrived but they lay the groundwork for the more complex resolution steps for other types of bugs. Next, let’s look at a more complex example that is a variation on the Bohrbug

…
After identifying, classifying and showing resolution tools and process for each type of bug I’ll review the idea of “narrowing the universe” that a bug lives in.

Details

Notes for Reviewers

Debugging software is the swiss army knife in a developer’s toolbelt. Adept debuggers seem to be able troubleshoot and resolve every issue from databases that refuse connection all the way up to “undefined is not a function.” Always knowing the next question to ask and the next path to take.

Their deft actions make it easy to talk about debugging as if it comes from intuition. Having worked on consulting projects at a previous job and joining a long running product company recently, I’ve become accustomed to hearing this phrase:

“As you get used to working on this app, you’ll build some instincts for debugging these kinds of problems”

In my experience though, that intuition often ends up turning out to be memorized rulesets, ie:

“Whenever I see something like this happening the first thing I do is scan the logs to see if this process is completing or is sending a weird message.”

This statement reveals a lot about the tools in this developer’s toolbelt:

When something is going wrong, they look to the logs. The logs contain information about the status of a process The logs contains information about the messages being sent and if those affect the status of a process. This developer knows what a “normal” completion or message looks like for this process.

There are many things that can go wrong in deployed code. Sometimes it seems like those problems are based on so many complex requirements and environmental conditions that surely they are unique to a specific codebase. More often than not, the more bugs you see, the more patterns materialize.

Once you have identified a bug by its attributes and behaviors, it’s possible to narrow the scope of causes. In this talk, I’ll classify several common types of software bugs by their attributes, narrow the scope of possible causes and discuss debugging tools (both software and process tools) that can be used to resolve each bug.

Debugging may be instinctual but those instincts come from seeing, identifying and diagnosing patterns. In this talk, we’ll use phenetic taxonomy to build a debugging decision network based on attributes and behaviors of common bugs. Newer developers will learn how to apply systematic pattern matching to debugging while seasoned developers will learn how to use their instincts to teach debugging.

Only for Keep Ruby Weird:

How is your talk “keeping it weird”? Any other info you’d like to provide (personal website, talk videos, love letters to the organizers, &c.)?

At the core of this talk, I break down one of the key components of software development, debugging, and get people thinking about turning their "debugging instincts" into to repeatable knowledge. Translating our instincts into process is valuable because while instincts are fun and hand-wavy, knowledge processes are teachable. Teaching is what moves software development forward. I'm sure that doesn't sound "weird" but, because I think it's important, I've taken the time to make it weird. For each of the bugs I'll describe, I have a little hand-drawing and for each of those little hand drawings I will have a disturbing illustration of the bug being squashed. I think this will give the audience comfort as I ask them to give up their beloved instincts in favor of shareable, teachable knowledge processes.

A presentation I have given in the past year that I am quite proud of is this one: https://www.youtube.com/watch?v=bSbla50tqZE. When I wrote that talk, I wrote it all in one evening because I knew that I absolutely had to share this idea. I wrote most of this talk also all in one evening because I felt the same excitement. This is how I know that this talk is worthwhile and that the way I have chosen to present this idea is sufficiently weird. Several of the compliments I received when asking friends to review my proposal: "huh", "what an upsetting visual", "please don't make it gross", "what if you made it less about insects?".

Edit proposal