Blog

Many Scientists, from Newton to Einstein to Sagan, have talked about a spirituality they find in science. But aside from some vague descriptions of “mathematical elegance”, they never really explain. I think there is a connection between spirituality and information theory which can be explained in simple terms, but in order to do that, I have to tell a story.

High school calculus, my classmate raises her hand. “I can apply the rules to get the correct answer, and I know it’s correct because I can un-apply them to get back to the question. But I don’t know why. I can do the math, but I don’t understand it.” The teacher, a petty, bitter man of limited imagination, dismissed her question as irrelevant nonsense. In retrospect, I suspect he didn’t understand what he was teaching either, because that was one of the most insightful and profound questions I’ve ever heard.

In the lobby of the Museum of Science in Cambridge MA, they used to have a machine built from Tinker Toys that had been designed to play tic-tac-toe (the featured image on this piece). A grid of levers represented Xs, Os, and unplayed squares; a crank on the side, when turned, would result a lever flipping to indicate the next move.

The design of the machine implements what is called a “model” of tic-tac-toe. There’s nothing special about Tinker Toys; the model could be implemented in many different mediums. For example, another medium which implements logic are the flows of systems of pressurized air or water; hoses connect to switches which turn on or off when pressure is supplied. Although these mediums appear very different, the model is always the same.

Computers are a natural choice for many models’ implementation, because they can support so many types of complexity. When I teach JavaScript, I use tic-tac-toe as an example model to implement. But models don’t have to be implemented directly; in more complex implementations they may arise by themselves.

Imagine if we modelled the Tinker Toy machine. Anyone who’s seen a modern first-person video game can picture this. First person video games use what’s called a ‘physics engine’, a program which models how reality behaves. First, we’d give the physics engine information about the Tinker Toy pieces, qualities like flexibility and weight. Then we’d specify how the pieces were connected: this rod connects to this hole in this hub. The physics engine would model that when pressure was applied to the rod, the hub would turn, and so forth. Now we’d be able to “play” the Tinker Toy tic-tac-toe inside of the computer.

Often when we talk about computers, we unconciously give them understanding. We say the computer is “thinking hard”, or that it doesn’t “want” to do some task. It’s obvious that the Tinker Toy machine doesn’t “understand” the game, that the understanding comes from the knowledge of the person who designed it. Does the computer “understand” that a model of tic-tac-toe is embedded within the model of the Tinker Toy machine? That’s harder to say. The physics engine could be “asked” to “predict” what the next move will be, and based on the physics alone, it could predict the answer correctly.

If the both the computable physics and our human “understanding” both give an accurate prediction, it’s hard to say that the human is “thinking” and the computer isn’t. So let’s just put aside the sticky question of whether the machine is “thinking” or “feeling”, and focus on results.

If two processes reliably generate the same results, then they have to be equivalent in some way. When a person executes the rules of tic-tac-toe and a machine executes them, they have something in common, something interchangable. This is the feeling my classmate complained about; she knew in her concious mind that the experience she was having while doing her homework was not that of a mathematician, but a calculator. She knew how to flip the levers and turn the crank, but not how to design and build the machine, embedding the model within it.

The human brain is often referred to as the most complex object in the universe. That’s true on at least two levels. One is the physical arrangement; microscopically, the neurons in the brain connect to and communicate with each other in fantastically complex ways. Another is the number of levels of models the brain can hold; to model the first-person version of the Tinker Toy machine, the programmer has to simulataneously hold models of physics, and the physics engine, and the implementation of the tic-tac-toe model, and also a model of how the computer itself works… in order to produce a fourth model, which would be the first-person program itself. Maybe another model, which is that of why the program is useful to the user and worth writing. The number of levels is not infinite, but it can go hella deep.

However, there’s another way in which that statement is not true at all. The cost to having such a complex brain causes human children to be dependant on their parents for an extraordinarily long time compared to other animals, who may be mostly self sufficient weeks down or even minutes after birth. During the years we are being raised, we unavoidably absorb language and culture and knowledge about the world from other humans around us. One human brain, with all its complexity, has never existed in isolation; it has to come from a network, a community of other brains.

There is therefore a result which can not be produced by one brain, but is only possible to get from many brains interacting. I’m still avoiding the question of whether many-brains experience a “feeling” in the same way there is a “feeling” of being one brain, or a machine; because it doesn’t matter. The result, the complexity is measurably greater.

We can’t model the complexity of many-brains within our one brain. But potentially we could someday model it within a computer. Current neuroscience is capable of modelling a mouse brain; that is, producing the same result inside the medium of the computer. It seems unlikely there is a limit which will prevent increases in computing power and technique from some day being able to simulate many-brains.

There is a limit to simulation though. Many-brains require bodies, which require breathable atmosphere, abundant fresh water, nutrients, gravity. You can simulate all of that, the entire planet, solar system, galaxy. But you can’t totally simulate the universe with a simulation inside it: if the simulation includes itself then the universe does too, which must then be simulated. It becomes a recursive problem, reflecting itself without end.

This result — a level of complexity we are prevented from ever directly interacting with — is indistinguishable from what is commonly called “a higher power”. They say the Lord moves in mysterious ways, and so do the inner workings of the cosmos.

I am humbled and staggered by the ultimate complexity of the universe, I fall on my knees and raise my arms to it. I beg for greater understanding, even while I accept my limitations. I look for the reflection of eternity in the eyes of my fellow humans.

My spirituality is scientific. I worship a force greater than myself by observing reality, which I am both inside and a part of. Which I can never truly understand, but has led me here for a reason.

All things are part of a greater whole. None of us are separate.

The reason is love.

Donald Knuth, one of the legends of programming, has since 1962 written a multi-volume encyclopedia called “The Art of Computer Programming”. In a perfect irony, he hasn’t finished it. But I want to talk about the title.

Knuth, who I feel can reliably be labeled an authority on the subject, chose the term “Art” to describe programming. Most people think of computers as a science. What does that mean, the Art of programming?

Science concerns itself with the search for truth, so scientists only interest themselves in exploring new ideas. As programmers, we tend to tread the same ground over and over again. But so does art.

Art is another word for communication. The better an artist you are, the more you communicate. You work in some medium, and on one level the artifact produced in that medium is the communication. Art has to work at the literal level. Does it work as intended? A program that doesn’t do what it’s supposed to fails to communicate.

If you’re a working programmer, you need to produce working programs, just like a chef needs to produce edible food. But a chef is judged on much more than whether their food is safe to eat. Presentation matters. Historical and cultural references enhance the experience. Details matter, feelings matter.

The better an artist you are, the more levels on which you communicate. Great programmers express more than programs that simply work: they compose experiences that are a pleasure to use. So like all artists, they must become the person capable of that communication.

The best programs are the ones you enjoy the most. Programs have many quantifiable properties, speed and capacity and the like, but user satisfaction is the most important.

Write programs for others to enjoy, and you’ll be a great artist.

You may have heard this statement, “If you’re not outraged, you’re not paying attention.” The first search results I found this attributed this to Heather Heyer, who was killed protesting white supremacists in Charlottesville, NC.

I respect Heather’s sacrifice. I’ve frequently felt called, as Mick Jagger put it, to go down to the demonstration, to get my fair share of abuse. You’ve got to put your money where your mouth is and your body on the line.

But I reject the demand that I have to be outraged. I challenge that statement on several terms.

The simplest: it’s not a call to action, and therefore has no possibility of causing any change. Okay, I’m paying attention, now I’m outraged, check, check. What next?

What next?

Being outraged can’t tell you how to help fix anything, what to do. And I think trying to figure out what to do is the central problem we all deal with most of the time.

Next challenge: that statement constitutes an order; a directive about how I should feel.

I say no one has jurisdiction over how I feel except me. I decide if I want to be outraged or not. What if you and I accomplish all the same goals, but I’m just not mad about it? Is outrage a requirement to be involved in making the world a better place?

I reject negativity, which sounds like a contradiction in terms until you flip it. Not anti-negativity, I am pro-peace. Instead of talking about outrageousness, DO something positive in the world.

Instead of posting about how evil your enemies are, you could sign a petition. Instead of worrying about global warming, you could figure out how to eat less meat. Instead of proclaiming your views on subjects like gun control and abortion, you could volunteer in your community.

The fourth result for the search linked to a store where you could buy a t-shirt with that statement printed on it. As if you don’t have enough conflict in your life already, so you wear a sign inviting more.

And you pay someone for the privilege. Be suspicious of those who encourage you to be angry. It’s frequently the glitter in your eyes, the magic trick distracting you from the grift.

Beware anyone who wants to make you feel anything except love.

Lots of people say we’re in a crisis right now, but for different reasons. Some claim that ‘alternative facts’ are leading us to a ‘post-truth’ world, while others warn about ‘fake news’ turning humanity into ‘sheeple’.

I think they’re all wrong.

It seems to me that the loudest voices on both sides have something to gain from fear and uncertainty. They want to focus on the depth of our differences instead of the breadth of our commonality. They make truth into a competition.

The answer, like much in life, is simple — but not easy: truth is not a binary, either/or condition; it’s a percentage, a probability. When we say something is true, that’s an oversimplification. What we really mean is it’s highly likely to be true, unlikely to be proven false.

Ultimately the decision about what is true is made by a person, and people have feelings. The significance your emotional state has on your decision making process resists overstatement — as anyone who has felt the low blood-sugar state referred to as “hangry” knows, your physical and emotional state can greatly affect what seems true.

But there are things that are permanently, unchangably true …aren’t there?

I started trying to think of the truest truths. What can all people, across all times and places agree on? One of the best I came up with: falling.

Everyone understands falling. Nobody can fly. Objects which are not supported by a force equal to their weight fall towards the ground, until they’re supported by a force again. The further they fall, the faster.

Anyone can measure the speed of falling. Since Galileo, it’s been measured millions, maybe billions of times, by students, engineers doing construction, scientists conducting experiments. A baby investigates falling when it pushes things off its high chair.

You’ll notice I mentioned falling, and not gravity. Falling is as close as you can get to an absolute fact. Gravity is not a fact, not a measureable quantity, but an explanation for why and how things fall. It’s a tremendously useful explanation, because it makes accurate and useful predictions about other measurements. But while the measurements never change, explanations can, and do.

Newton invented the modern concept of gravity largely to solve errors in astronomy. You can sum up his invention simply: the same rules that describe falling here on earth can also describe relationships in the heavens.

And for over 300 years, that explanation has proved useful over and over again. The rules Newton used to describe gravity — calculus — can be used to predict other super useful things, like where a cannonball will go when shot from a cannon. The same rules describe launching something into space! If you fire a projectile so far that it falls over the horizon, it keeps “falling” around the Earth, and enters orbit.

So Newton’s truth is enough to get man to the Moon. But if you want to build GPS, Newton won’t cut it. At the speed of light, communication between earth and orbit exposes a deeper truth about gravity: a clock in orbit runs slightly slower than one on the ground.

Unlike falling, this truth is super weird, and you can’t really measure it yourself. But it’s also uncontroversial. Relativity has produced the most accurate predictions in human history. If you accept that GPS works, you accept relativity as an explanation.

So is Newtonian gravity false? No. If you’re shooting a cannonball, Relativity is unnecessarily complicated. Simplicity counts. You should always prefer the simplest possible explanation that yields acceptable results.

Einstein didn’t prove Newton wrong, he extended and clarified him. By the same token, it’s unrealistic to assume that nothing about Einstein will be extended or clarified.

That means some thing experts assert as a matter of fact at this moment will someday be found wrong. Galileo thought the tides were caused by water sloshing around as the world rotated. Newton’s other interests included alchemy. Einstein went back and forth about the cosmological constant.

There’s potentially something wrong with everything we think we know, we just don’t know what yet. We usually only find out the wrong parts one by one, over a long time.

Recap: things true in the past have been proven false later. So when we say true, we express our confidence in the likelihood the current explanation will not change. But we have to closely examine our emotional state to minimize the chance of influence and distortion.

Or tl;dr: consider the possibility you might be buying into your own con job.

Nothing has brought me more success, personally and professionally, than the habit of questioning my every action. I can hear some people object they don’t want to feel uncertain all the time, but I find this attitude makes me more confident in what I know. I just have to be more flexible about explanations.

When people become emotionally attached to an explanation, they get upset when asked to consider the possibility it isn’t true. It’s hard to question assumptions which make you feel good about yourself. But everything good in life is hard.

Simple ain’t easy. Try harder.

If we could look to the skies with ideal eyes and see as far as eyes could see
We could observe the difference in the boundaries between my universe and thee
Now wouldn’t that be keen… if once we knew what we mean

Now time is short, and time is long, and time can fly and crawl
So I suspect the description that fits every situation cause it really fits none at all
Not sayin’ it’s null… it’s just a wall
Can’t win ’em all… so take it easy when you fall

It’s always shorter on the way back, the way I long knew
It’s always shorter on the way back, without the headwind that blew
It’s always shorter on the way back, in the direction I grew
It’s always shorter on the way back, the way back to you

Now I know time is real because I feel it when I’m far away
Away from you my love and from our home and then I have to stay
Day after day… they stretch and fray
To heal we pray… hey hey hey

It’s always shorter on the way back, the way I long knew
It’s always shorter on the way back, without the headwind that blew
It’s always shorter on the way back, in the direction I grew
It’s always shorter on the way back, the way back to you

Lately it seems like the sadness is relentless, crushing. So many loved ones passing, so many illnesses, disasters, catastrophes, so much coming to an end. The other day I read a post from a suicidal, abused teenaged girl. She wrote her truth so brave, so fierce, my heart broke for her. Seems like my heart breaks every day these days.

Part of it has to come from the news we just bathe in now. Back in the day, it was possible to miss out on hearing about a tragedy, but the algorithm sure as hell won’t let that happen any more. There’s a reason they call it a feed — we gorge on misery 24/7.

But part of it has to be perspective. I was at a memorial service with my Dad — I think it was for Uncle Paul, but it might have been Mom’s, there’s just been so many. Anyway, Dad was looking at a picture, of him and his mother and my brother, and with melodramatic flair he announced, “I guess I’m the only one left from this picture.” The self pity irritated me, so I snapped, “Well if you live long enough, that’ll be true of all the pictures you’re in.”

I always regret when I allow myself to get irritated with Dad, but sometimes truth comes out of it. I started thinking, the only way to avoid the feeling of being the last one left in a picture is to not be in any pictures. Having family, friends, willingly engaging with the world, it makes you vulnerable to loss.

As a culture, we shun death. We fear it, avoid it, try to obliterate it with health obsessions and hospitals. That’s cheating. You don’t get to have the love and the friends and the good times without the hard times too. They are one and the same.

Earlier today, I wrote I was sad to learn of Sara Romweber’s death. I take that back. I’m not sad about it. I’m thankful that I get to live a life where I can learn people like Sara exist, where I have to appreciate them right now, because they might not be here later. I’m thankful I got to see Sara lay down a beat just one time, one beautiful, ecstatic night of rock n roll. I’m thankful I get to participate in this mad moment we call the twenty first century, where all the information is available to us, and the challenge is to manage that embarrassment of riches.

All these moments shine in my mind like a chest full of jewels, but they only shine because the edges define them. A good story needs a beginning, middle, and end. Don’t short change the ending — it’s the best part of the story.

Death and darkness are half of life. When we turn away from them, out of fear, we fail to fully live. I am slowing learning to love death and sadness, just like you love family. I accept them for what they are, I sit with them in silence, I draw strength from their constancy.

Some day, I will only be in the picture, instead of looking at it. When you look at my picture, I want you to know: I tried really hard to live well.

Love is always the right answer.

I’ve spent half my life now practicing the art of software development. Like any other art, some development principles are generally applicable to any human endeavor. I’d like to share these with you.

Programming consists primarily of two related but separate tasks. The first is translating definitions from one language to another; the second is managing complexity as those definitions grow.

When you think about translation, you probably think first of translation between natural languages: English to French, Japanese to Turkish and so forth. Programming has a remarkable similarity, except the translation goes from any natural language to an artificial one, which is essentially arithmetic.

At their core, computers aren’t much more than glorified calculators. Programs whose output appears rich, organic and analog generate those results through massive amounts of calculation – basic arithmetic, just performed millions of times per second. Since every program’s output consists of this arithmetic, another way to think about a program is as a giant mathematical equation. Facebook to Fortnite, Instagram to Indesign, from the calculator’s point of view it’s just an equation to be solved, with different variables provided at the time of execution.

Between natural languages, concepts can get “lost in translation”. Some phrase in one language doesn’t have an exact match in the other, and the translator has to construct an equivalent. Programmers hone in like bomb-sniffing dogs on words like “active” and “valid” and “soon”, because they are loaded with implication and inference, which the calculator cannot tolerate. Software development has taught me that natural language is inherently imprecise. Someone says something and you nod, thinking you agree, only to find a couple sentences later that you and they are not talking about the same thing at all. Worse, I’ve learned if you question people for further clarity, you often find that they don’t know exactly what they want.

Natural language intentionally trades away precision for speed and brevity. Someone says, “I saw a car”, you can guess what they mean based on probability: four wheels, enclosed body, less than 40 or 50 years old. Programmers think a lot about what we call “edge cases”: is a big pickup truck with dual back wheels still a “car”? what about those three wheel, two seat, open body vehicles? If that weird thing is a car, is a motorcycle?

Very smart people have come up with terribly clever strategies to help manage this complexity, but ultimately these techniques still fall on the developer to implement. On a large project, developers can’t handle the collective weight. In the software, this results in bugs; for the developers personally, frustration and burnout.

In the late 90s, a group of developers invented a non-technical strategy called “Agile”. Agile has gotten a bad rep from some folks over the years, but people experience a bad instance and then blame the concept.

Agile acknowledges the difficulty of imprecision in natural language, instead of pretending it doesn’t exist. In a situation where definition is vague or incomplete, and complexity building, Agile provides a method to keep making forward progress. People make Agile way more complicated than it needs to be, because it can be summed up very simply, and it goes like this.

Pick the most important thing. Work on that first until it’s done. Then move on to the next small piece, while checking to make sure the current work hasn’t affected the previous work. If it did, it means the two were somehow related, and you took on too much at once. But now you should have enough information now to go back and change the first piece as needed. Once you have two independent and stable pieces, move to the third, and repeat indefinitely.

Novice software developers make this mistake more often than any other: they try to do too much at once. They change several parts of the code at the same time, and when something breaks, they can’t tell which part caused it.

Every January, many people make the same mistake in their personal lives. They resolve to get more exercise, eat right, quit drinking and write a novel. Then they get surprised and disappointed when none of those things work out.

Agile philosophy suggests you pick only one of those things, the one most important to you, and break it down into the smallest achievable step. Don’t get a gym membership and hit the free weights; don’t even buy a FitBit. Instead, just walk around the block at lunch. But do it every day, every week, every year. The smallest piece is always hard enough. Small changes you can consistently keep up are a million times better than lofty goals you can’t. Maintain your gains.

Which brings us back to the practice of programming. Like music or yoga or any other practice, success requires the courage and patience to start working on one small thing at a time, and the humility to accept you can never achieve total perfection. It’s not really about the arithmetic, or the coding languages; the thing we’re really doing when we program is interpreting. We express our interpretations in different ways, and that expression is what we practice.

It’s hella hard, but worth it. I wish you the best in your practice.

This might come across at first as a humblebrag, but really it’s an apology, a confession, and a promise.

People have told me from time to time that they’re jealous of part of my life. It’s certainly flattering. I understand they intend it as a compliment, but I can’t accept it. I know that if they got the thing they desired, they’d be surprised at how much unpleasantness they’d have to take along with it. From the outside, my life may look appealing, but it takes its shape from suffering.

I’ve never wanted to talk about that suffering. Saying something without value is complaining, and complaining feeds the ego. And I haven’t felt I have anything of value to share. Until now.

On Friday I lost another friend to suicide. This one really hurt because we were close, and I failed to be there the way I should. If any of many friends I have lost to this disease held any misconception about what a mixed blessing this life is, I need to set the record straight publicly.

Depression runs in my family. My parents have both suffered from it. Someone asked me if I inherited it, and of course I did. Regardless of genetics, growing up in negative, chaotic circumstances shaped me. But rather than suffer from it, I struggle against it.

Like a lot of teenagers, I developed resentment towards my parents for their flaws. I swore I’d never end up like my Dad, but soon I realized lashing out at the world was having the effect of making me more like him.

I decided that to rebel against negative, chaotic energy, I had to engage in consistent positive actions. I started making better decisions — eating well, exercise, quitting smoking, career moves — and one good decision made the next one easier.

But they all drew their fuel from negativity. I was doing the right things, but motivated by comparison, self judgement, self denial.

Negativity ran out of fuel in 2013. Nothing was good enough for me anymore. I stopped playing music, moved way out of town, checked out. Mad at everyone for their shortcomings, I imagined peace and quiet alone.

Bad decision. Quiet and alone provided the perfect pool for negativity to settle. After getting deeply miserable, I realized the reason I do so many things is for the interaction with people. I never needed negativity. I needed human acceptance, for myself and others.

Which makes losing someone so hard now. Did I do everything I could have for my friend? Of course not. I’m not perfect. I know exactly where I failed.

But I choose not to look back at my failures in a negative light. I choose to look towards the chances I’ll have, armed with this hard earned knowledge, to do better in the future.

Improving is hard. Commitment is hard. Positivity is hard. You know what else is hard? Negativity, depression, anger and fear. One way or the other, you’re going to suffer. But you get to choose what you get out of it in between.

Choose people, choose life.

Love is always the right answer.

I have taught programming to the entire range of developer experience, from complete novices to experienced engineers. Both groups have a harder time learning javascript-stack development if they do not develop fluency with the command line interface (abbreviated CLI, also called the terminal).

I use the word fluency intentionally, just like language. If you can’t “speak” CLI, you’ll struggle with many of the tasks js-stack development requires. Once you establish CLI fluency, you can switch to one or more graphical user interface (GUI) based tools, but you need to establish fluency first.

Even the switch to GUI tools illustrates why CLI fluency is so important. You may need three or four different programs to cover the common tasks that the CLI handles. Picture those three programs, with an average of, say, five menus, and fifteen commands per menu. That’s 225 different places to look to find the action you need. Worse, once you learn where to find the action, the tool gets rearranged in the next version and you have to re-learn it. And I didn’t even mention the time cost of installing and supporting those tools.

More fundamentally, GUI tools are a layer on top of the CLI. Whenever you add a layer, you also add the potential for gaps between them. Every experienced programmer has had a situation where some layer fails to accurately capture its underneath.

It’s not the fault of any specific tool, fixable by switching to another, but instead a fixed cost that comes from introducing extra complexity. GUI tools simplify visually, but have to connect to the same API the CLI does. The complexity comes from that connection.

To resolve a discrepancy between any GUI tool and the CLI, you have no choice but to look at the CLI. This holds true for nothing more strongly than version control application Git. If you are not comfortable with the Git CLI, then you will struggle with any GUI tools.

That’s worth restating more strongly. If you do not understand the Git CLI, then you don’t understand Git.

I see users of Git GUIs performing cargo cult rituals, like fetching before pulling, in the hopes (wishes, prayers) that will solve some entirely unrelated problem. Common and easily solved issues like merge conflicts or uncommitted work become major, painful obstacles. If you don’t understand Git at the CLI level, your actions in the GUI are just guesses.

So you have to understand the CLI before using a GUI, and some tasks will require you to leave the GUI and go back to the CLI. Switching back and forth between the two approaches has a cost: you spend mental energy. Costs associated with everyday tasks like these add up fast.

Finally, front-end development is web development, and the web has a strong bias towards Linux. The structure of URLs reflects this: they use the forward slash character because it corresponds to the Linux directory separator (as opposed to Windows’ backslash). Even if you develop completely in a stack like .NET or IBM Java, you’ll have to deal with that Linux bias once you start working with the frameworks and libraries that front-end now requires. Having CLI fluency within Windows through a tool like Git BASH or PowerShell makes a huge difference.

Hopefully I’ve convinced you of the importance of CLI fluency. Like a spoken language, nothing except practice produces fluency. In the next post, I’ll show you some tips for how to do just that.

Software engineers practice debugging over the course of their entire career. However, it is a prime example of tacit knowledge: an engineer who can expertly solve a bug in record time may have a difficult time explaining the process they used.

The results I found when googling for debugging described how to use a particular tool, but assume the user already has a debugging process. This guide is intended for novice programmers who need help getting started, regardless of the technology involved. I am primarily a web developer, so I’ll give mostly web-based examples, but the principles apply to any complex system. The system doesn’t even have to be a computer — keep reading to find an automotive example.
A guide, because no map exists

Software engineering seems to the beginner like a totally “hard” scientific discipline, and much of it is. In daily practice though, extenuating requirements like time and resource constraints make success a function of craftsmanship, and therefore an art.

No guide can give you explicit step by step instructions to solve your particular problem, so that’s not what you’ll find here. Instead, these principles illustrate good habits which will make you a better debugger faster. Read them, put them into practice, and then revisit later.
Anatomy of a bug

A good general definition of a software bug is a condition where an operation results in undesired behavior. Note the critical term in that definition is undesired behavior. You can describe every bug as a discrepancy between the desired behavior and the system’s actual behavior.

That point is so important it bears restating.

Every bug stems from some difference between your conception of how the program works and how the program actually works.

The computer behaves with absolute determinism. That’s a ten dollar word that simply means non-random: the same input always produces the same output. If a computer’s behavior appears illogical or chaotic, it’s only because we don’t have enough information about all of the input to identify the pieces responsible for the behavior. Remember the term “input” isn’t limited to user actions, or even system data like program instructions and variable values; it extends to any and all things that affect the system, like hardware and temperature.

I’ve heard so many programmers exclaim “That’s impossible!” when confronted with some behavior, even though the stark fact of the behavior’s existence proves not only its possibility, but also its inevitability. In reality, the behavior is only impossible within their mental model.

This explains exactly why finding the cause of a bug can be so difficult and frustrating: your mental model has to change.
Mnemonic

A mnemonic helps to keep our objectives in mind at all times. I like three (pseudo) E’s: empirical, unequivocal, and efficient.

Empirical means basing everything on observable evidence — no speculation, hunches, or theories.

Unequivocal means there can be no underlying or alternative explanation — rule out all other possible causes.

Efficient seems obvious, but it’s actually the most difficult element. Remember the definition of a bug hinges on something you don’t know. Finding what you don’t know in the shortest possible period of time is quite a challenge.
Empirical means ‘prove it’

How you find out what you don’t know? Just like in school, you test. However in school, you tested on a single subject. When doing social studies, you weren’t expected to catch examples of incorrect math.

In a complex system like modern software apps, different layers interact in subtle ways. Since your bug could hide at different (and possibly multiple) levels within the software stack, you have to test everything.
Prove it

The first rule of debugging is absolute skepticism. Adopt the attitude that you will accept nothing as a fact until you have proven it empirically.

Think your jQuery selector is returning the correct element? Prove it. Think the server is returning a certain piece of data in an ajax response? Prove it. Positive that some subroutine is getting called? Prove it.

It follows that the best weapon in your debugging arsenal is the log statement. Log everything. Prove that all values are what you think they are. When I help novice programmers debug, the most common errors they make involve assumptions:

• treating a variable as if it contained a different value; for example, an object nested inside an array

• mistaken identifiers: similar names, right name but wrong context, id instead of class

• operation not actually getting called, typically because it’s inside a conditional which never gets satisfied

• making changes in the wrong file, failing to restart the server after changes, etc.

To prove you’re looking in the right place, try intentionally breaking code. The program should now fail at that point. If not, investigate that first.
Prove it again

If everything looks correct at first glance, go back and look closer. A single character can make all the difference, but novices have a tendency to see what they want to see. The best technique to support double checking is character-by-character, line-by-line comparison.

Go ahead and put your finger on the screen. Don’t be embarrassed — after almost twenty years, I still do it — it’s necessary to help your eyes identify minute differences. For each character, ask “why is this here?”. Pay close attention to the display of each character. Common offenders include so-called smart quotes, and whitespace generated by non-printing ASCII characters.

Read the first error message returned, google it, and learn what it means. Don’t skip to the second or third error message just because it contains a word you recognize and the first one doesn’t.
Unequivocal means ‘prove it in isolation’

Back in high school, my ’78 Malibu had trouble starting. The battery would die if the car sat for as little as one day. I measured the current with the engine running and found it on the low side. Obviously, the battery wasn’t getting charged, so I replaced the alternator. A few days and eighty bucks later, the battery died again. So frustrating!

The running current now tested fine, so I went on to test the current with the car shut off. Lo and behold, it showed a current drain even with the ignition and accessories off. Come to find out the switch controlling the interior passenger door light had shorted, so the light stayed on constantly… even when the door was shut, where I couldn’t see the light.

Bugs in complex systems can have complex causes. Don’t allow your excitement at discovering an important fact to mislead you into assuming it is the root cause of the problem. Instead, devise a test that will prove it (sound familiar?) before investing time and effort.
Efficiency means prioritize

Given infinite time, you’ll always find the missing piece of knowledge that solves your mysterious bug. We want pretty much the opposite of infinite time though. Efficiency requires you to prioritize activities which have a higher likelihood of producing results.
Follow Best Practices

It sounds facetious, but the most efficient way to solve bugs is to not create them in the first place. Following software engineering best practices results in more understandable systems, which are therefore easier to debug.

Develop a strong, ethical commitment to good code. When you’ve identified the cause of a bug, always replace the problem with a solution that improves the quality of your code. Implementing a low quality solution is like cleaning using dirty equipment: you’re just moving the problem around.
Reproducibility

A bug is reproducible if you can re-produce it, or make it happen on demand. If you have a bug you can’t reproduce, don’t spend any time in code, but instead focus on collecting more information on how to reproduce it. You might need to go so far as to watch a screen share — users are notoriously bad at telling you what subtle thing they’re doing differently than you expect.
Simplest first

Once reproducible, always always always start by checking the simplest things. Think about how sad you’ll be if you spend hours pursuing some far-out theory, only to find a variable name was misspelled, or the server needed restarting, or some other quickly solved issue. Remember KISS (regardless of how awesome they were, I mean the acronym “Keep It Simple Stupid”, not the band).
Intermittents

If a bug happens some, but not all of the time, it’s called an intermittent. A computer is a deterministic device, meaning every bug must have a determinable cause, but finding the conditions which lead to an intermittent can consume breathtaking amounts of time. You’ll need to practice triage to decide how much time to spend.

Quickly assess how frequently the intermittent occurs: attempt to reproduce the bug, counting the ratio between positive and negative results. Repeat the test as few times as needed to gain a high statistical confidence in the frequency. If you find the bug occurs 4 out of 5 times, you can have reasonably high confidence the bug will continue to occur upwards of 50% of the time over a larger number of tests. But if you get a positive result 2 out of 5 times, you really need to test 5 more times, as the true rate could range from 20% (2 out of 10) to as much as 70% (7 out of 10).

One you have reasonable confidence in the frequency, perform a risk assessment: multiply how often the bug occurs by its severity, and weigh that factor against the time cost of investigating. For example, something that happens 15% of the time but only results in a cosmetic error doesn’t deserve spending as much time as something that happens at a 0.5% rate but results in a security breach or data corruption.
Check your self

Bugs are tiny little mysteries, and a lot of people feel they can not rest while a mystery remains unsolved. However, a lack of rest makes one tired, and performing any kind of task while tired increases the risk of making a mistake.

Any activity will tire you to some degree, but debugging also has the potential to increase frustration in a way that other activities can’t. A bug’s defiant refusal to obey (your version of ) rules and logic can take on an almost insulting quality.

Frustration can sneak up and overwhelm you while all your attention is devoted to debugging. Take your mental temperature frequently. Ask yourself how you’re feeling. It seems paradoxical, but in order to remain efficient on longer problems you have to step away when needed.

Take a break and do something unrelated and physical. Go for a walk, wash the dishes, whatever. The physical element is critical: if you lay on the couch and watch TV your mind will stay stuck in its rut. You might still think about the bug while washing dishes but there is something special about physical movement that enables your mind to free itself. You’ll be astounded how often you come back to the problem after a break and see the cause immediately.
Write everything down and (maybe) ask for help

Sometimes the most efficient way to solve a bug is to get someone else to solve it. Sometimes. The more knowledge another programmer has, the more likely they are to be busy using that knowledge to their own benefit. Even if you post on a super awesome site like Stack Overflow, the answers that come back quickly have a much higher chance of being incorrect. And an incorrect answer can be disastrous to efficiency, let alone effectiveness.

If you’re just utterly stumped, write down everything you know about the bug. Everything: environment and conditions that cause it, when it started, a detailed description of the bug itself, everything you can possibly think of. Don’t edit or format, just let it pour out in a stream of consciousness.

There’s a marvelous phenomenon called Rubber Ducky Debugging. First printed in the book The Pragmatic Programmer, the story goes that a programmer kept a rubber duck on his desk, and when confronted with a problem, he would ‘explain’ it to the toy, realizing the cause during the process. While most certainly apocryphal, it does describe an experience that every programmer has at some point (my theory about the origin: just like most people don’t know that Bert is evil, Ernie is actually a l33t hax0r).

Same thing happens when you write everything down: as you pour out all the details, one of them sizzles with crackling energy in your mind. Something about the mental process of describing the bug unlocks a connection you just couldn’t make consciously.

There’s a second benefit to writing it down: even if you don’t get the eureka moment, you have written great documentation! Take some time now to edit and clean it up before using it to ask for help (and if you’re not familiar with the ettiquette of asking for help, read up on that first too).
Thank you sir, may I have another?

Like any skill, debugging requires devotion to practice. You solved the big gnarly bug? Congratulations, now get started on the next one! Learn to see the never ending supply of bugs as A Good Thing. The opportunity to expand and refine your understanding never ends.

In that spirit, I will update this guide as needed. If I’ve missed something important, please let me know.