Blog

  • Giving Away Secrets: Surely Disaster

    I'm excited to announce I have a new project. I'm giving away all my secrets.

    The secret that will probably be most useful to everyone is how to practice the art of programming. I've created a five day course to introduce this art to anyone who's interested, regardless of experience. Right now the course is in beta testing - still secret! - but I'll announce when it's open to everyone.

    Is it a recipe for disaster to give away your secrets? Don't you lose all your power when everyone knows what you know?

    I've never believed that, and my life has proved it. I've made it a principle to give my knowledge to others as I was given knowledge when I started. Years ago, I made my living teaching guitar, and it made me a better musician. Having more musicians around means better bandmates. A rising tide lifts all boats.

    Likewise with teaching programming. Whether as a consultant for Fortune 500 companies like American Express and VMWare, or working with novice developers coming from disadvantaged backgrounds, giving away secrets has been some of the most rewarding stuff I've ever done.

    There's a deeper component to secrets. The concept of "authenticity" got discussed to death a few years ago, but I still believe the core message: the closer your internal life resembles your external life, the stronger you become.

    There's two ways to make those lives match. You can make the internal external, by releasing it into the world. Or, you can make the external internal, by accepting the world as it is, but changing how you choose to view it.

    Here's another secret: I'm excited about my new project, but excitement can sometimes be indistinguishable from fear. Developing this course has required me to get familiar with a lot of skills I didn't have, like email marketing and business plans. Learning new things can be scary, but I choose to view fear as excitement.

    So, I'm looking forward to giving away all my secrets. My greatest hope is that people will find them useful. Thanks as always for your attention - I promise I'll treat it with the utmost respect.

  • Command Line: Diff

    Programming involves working with data. Computers process data natively, but humans process information, which is data arranged in an understandable format.

    The more similarities two pieces of data have, the harder it can be for humans to see the differences. diff is a tool which shows the differences between two files, on a line by line basis.

    There’s three ways a line can differ between two files. It can be present in the first file and not the second; the opposite, present in the second but not the first; or present in both but slightly different.
    Example

    The squishy word in that description is “present”. How does it get decided whether or not two different lines are matching? Imagine two files:

    a b c d e r t x f rf g h i j a b c d ek l m n o k l o m np q r s t p q r s t

    • “a b c d e” is a full match in content, but not in position.

    • “k l o m n” is a partial match; same line position, slightly different content.

    • “p q r s t” is a full match, both in content and line position.

    • “r t x f r” and “f g h i j” both have no match.

    We need our tool to identify all these cases.
    The Process

    Enter the least danceable rhythm, the algorithm. Like algebra, the word algorithm comes from the name of an ancient Persian mathematician, al-Khwarizmi. Unlike algebra, everyone understands what an algorithm is: it’s a recipe or process for producing data. Long division is an algorithm — find the largest multiple of the divisor, subtract it, then repeat until the remainder is less than the divisor. Balancing a checking account is an algorithm (I said everyone understands what it is, not that it’s easy!).

    In this case, the best suited algorithm is called the “longest common subsequence”. You can look up the details if you like, but the important part is that the algorithm needs to look for that subsequence anywhere in the second file, potentially even on an earlier line number. In our example, it needs to recognize “a b c d e” is a match even though it appears on different line numbers.

    diff takes two file names as arguments. It compares the entire contents of the first file for the longest common subsequences for each line, and then repeats the process for the second file. Finally, it displays the results in a format which indicates where the matches were found.

    Note that diff also operates on directories, but we’re going to focus only on files for now.
    The Format

    diff needs to present its results in a text-only format so that other command line tools can accept it, including STDOUT for display. The format consists of three basic elements. Let’s look at the output for our example:

    $ diff this.txt that.txt0a1> r t x f r2,3c3< f g h i j< k l m n o— -> k l o m n

    Any lines which match are not displayed. If all lines match, diff returns nothing.

    For convenience, diff displays the content of the lines which were different. The angle bracket indicates which file the difference came from, corresponding to which argument the file name was passed as.

    The first output element indicates the line numbers of the differences, and in what way they were different: ‘a’, ‘c’, or ‘d’, corresponding to Add, Change, and Delete. In the first group, diff recognizes the first line in ‘that.txt’ has no match in ‘this.txt’.

    Numbers separated by a comma represent groups of lines. If diff finds a match which is spread out or mixed up across multiple lines, it reports the entire area as a difference. In this case, it recognizes the subsequence k+l+m+n in both files, it treats the line starting with ‘f’ as part of the same change.

    Why did diff not report line ‘f’ as a deletion? Remember, diff is looking for the “longest possible subsequence”. diff can tell that the k+l+m+n line is similar, so it looks at its neighbors to see if they are also similar. It reports the line starting with ‘f’ as a change based on the comparison of the longest subsequence (using capitals to highlight):

    “c d e F G H I J k l M/o N/m O/n p q r”
    Using the format

    diff is an indispensable tool when working with data files, including code. Getting comfortable with reading the format allows you to instantly see tiny discrepancies.

    There is another use for the output of diff which doesn’t get used as much but helps to understand why it is the way it is. diff has a corresponding program, called patch. patch takes the output of a diff and applies the changes to one of the files. In other words, it reverses the diff; after a patch both files are identical.

    patch used to be a very common task before automatic software updates, when users had to literally “patch” their own systems. It’s still useful today if you want to review and edit many changes before applying them all in one action.
    Exercises

    1. Perform the diff in the section “Example”
    • create both text files using the text editor of your choice

    • execute the diff command

    • see something unexpected? Some programs like Word and Pages add their own formatting to the file. Try using Notepad or TextEdit.

    1. Diff two HTTP sessions
    • combine two previous lessons by piping the output of the curl command to a file

    • Careful! Piping output can overwrite target files.

    • curl -v google.com &> first.txt

    • curl -v google.com &> second.txt

    • diff first.txt second.txt

    • what’s different between them?

    • try again, but point curl at ‘www.google.com' instead

  • Music And Programming: the Loop

    On the surface, music and programming couldn’t be more different. Music seems subjective and emotional, programming inhuman and cold. Beneath the surface, they have many similarities. The most fundamental is the idea of the loop.

    In the parlance of our times, the word ‘loop’ literally refers to the electronic process of sampling, taking a small piece of one recording and playing it repeatedly, nested within another.

    The verse/chorus/bridge pattern of any song is a form of branching loop. Musical notation has looping elements built in; indicators for playing a section again, or starting over from the beginning.

    At the lowest level, sound is a loop. All sound is vibration, which is a form of oscillation: a change that departs from a point and comes back to it. At the highest level, an entire piece of music is itself a loop. You listen to it once, then go back to the beginning and listen again.

    Programming can be described in the same terms of manipulating loops at different levels. The input and output of computer hardware is a loop: check for input, process data, send output, repeat. User interfaces are a loop between the human and machine — one acts, the other responds, they take turns. Entire programs are a loop in the same way as pieces of music.

    Since all these similarities are based on the concept of loops, they are similar at a more fundamental level: they’re all about time. In the ancient classical view of knowledge, music was considered a form of science: the expression of number through time, analogous to geometry’s number through space. We don’t discuss programs’ functionality in terms of managing timing nearly enough, which is our loss. Even tasks like algorithms and data structures are largely about timing — presenting the proper data at the right moment.

    Science has difficulty defining time. It’s a very squishy, human concept — imagine two people coming out of a concert, describing the same event as flying by and crawling slowly. Acknowledging the importance of timing in programming helps to bring the focus back to humans; and using the rigorous tools of programming in music helps create solid foundations built on numbers.

  • Command Line: Redirect Streams To Files

    The art of programming requires “standing on the shoulders of giants.” We take for granted so much work that needed to come before to make the modern programming environment possible. Sometimes things that seems ridiculous or complex make much more sense once we understand the history of how they came to be.
    Background

    So far, with all the command line exercises, we’ve typed commands and the results output on the screen. In the beginning of programming, this was not a given. All programs had to explicitly make their own connection to the operating system in order to receive input and transmit output. You can imagine this was incredibly tiresome, error prone, and also prevented programs from being moved between operating systems.

    Unix invented a concept called “standard streams”. In the last lesson, we played around with piping streams from one program to another. There were two streams we were using without being aware of it: ‘standard in’ and ‘standard out’, often referred to as STDIN/STDOUT (there’s a third one we’ll cover later).

    When a process starts, it inherits standard streams from its parent process. If they haven’t been changed from the defaults, STDIN will come from your keyboard, and STDOUT will display data on the terminal screen. Not having to set up these common connections makes it super easy for us to get started with whatever job we want to do.
    Streams and Files > Abstractions

    Sometimes we want to do other things with data. The most common case is to save the results of an operation; we want to write STDOUT to a file. With complicated or repetitive input, we might want to store the input in a file and send it to the program instead of using the keyboard.

    From the point of view of the operating system, standard streams are always files. In order to make the reading and writing process standard, the keyboard and the terminal are treated as if they were files (the specific terms are ‘file handle’ or ‘file descriptor’, depending on the context).

    This is an example of abstraction. To abstract something is to separate what’s similar from what’s different. Abstraction has a cost — it increases conceptual difficulty. In this example, we now have to keep in mind the concept of files. But the cost is greatly offset by the savings in setup (above).

    Pro tip: When used correctly, abstraction should always simplify more than it complicates. Unfortunately, some programmers really enjoy abstraction, and apply it everywhere regardless of cost/benefit. Always view abstraction skeptically, but embrace it when it genuinely provides value.
    A Character Of Their Own

    Because we’re dealing with file handles, standard streams get their own characters for piping. The angle brackets, ‘<’ and ‘>’, indicate which stream is being redirected (in or out). So reading a file as input into the “word count” program looks like so: wc < file.txt. And saving the result of a command to a file: curl google.com > capture.txt. Both together: wc < input > output.

    Pro tip: the single right bracket overwrites the target file. To keep the existing file, and append the new content, use double brackets: curl google.com >> multi-capture.txt.
    Murphy’s Law

    In the ’70s, Unix started getting used for commercial purposes, including big industrial printing operations. A couple of those early print jobs got ruined: the gigantic machine started up, ran a bunch of paper through, but instead of news headlines or book pages, there was some garbled computerese. Expensive mistake — what happened?

    The printing program failed to start, but having only one output, was forced to write its error message to the printer stream. Later versions of Unix introduced the third standard stream: STDERR, standard error. STDERR allows us to send error messages to a different place than ordinary output. This supports lots of neat uses, like writing log files, and keeping STDOUT clean.

    To redirect STDERR, use this symbol: ‘2>’. Why 2? It’s the file descriptor. The file descriptor for STDOUT, 1, gets inserted by default when you use ‘>’ by itself. The append rules still apply for ‘>>’. All these settings can be mixed: command 1> overwrite-output-file 2>> append-to-error-file. To send STDERR to STDOUT, as if it was a file, use an ampersand followed by ‘1’ to indicate the file descriptor: command > any-output-file 2> &1.
    Exercises

    Start by opening a terminal. On Mac or Linux you have one built in; on Windows I highly recommend (Git Bash) [https://gitforwindows.org/].

    Caution: sending output to a file changes data on your drive. To minimize risk of modifying an existing file, first create a new directory to work in: mkdir exercises && cd !$ (we’ll explain that command in a future article!).

    • Send output to a file: ls > directory-contents.txt
    • Read input from a file: cat < directory-contents.txt
    • Redirect output and error: cat directory-contents.txt nonexistentfilename > output 2> errors

    curl www.google.com | wc -m will produce an error “wc: stdin: Illegal byte sequence”. Try assigning output and errors to separate files.

  • Command Line: Pipes And Streams

    When your internet goes out, you quickly realize your computer isn’t very interesting when it isn’t moving data from one place to another. Same with the command line. The way to move data around on the command line is to “pipe” a “stream” from a source to a destination.

    The pipe character is the plain, upright bar, usually on the upper right hand side of the keyboard, between backspace and enter. It looks like an ‘l’ or an ‘I’ without decorations: ‘|’. In *nixes — any of the flavors in the Unix family — the pipe represents a connection between two streams. Streams of what? Data!
    Pipes and Streams

    We’ve already seen an instance of a stream. When we executed the ‘curl’ command, data was delivered to the screen via a pipe. Anywhere data goes in a *nix system — output to the screen, saved to disk — it goes via a pipe.

    Piping data between programs gets super useful. We can take the output of ‘curl’ and send it to another program. A program named ‘wc’ handles tasks related to “word count”. We can combine the two like so: curl www.google.com | wc. The output of ‘curl’ is “piped” to the input of ‘wc’, which outputs the number of lines, bytes, and words by default.

    Although it’s called “word count”, what ‘wc’ really does is count various types of data in a stream: words, characters, bytes. So if we want to count characters, we can pass an option to it, like so: curl www.google.com | wc -m.

    As we learned before, you can use the ‘man’ command to find out more about the options ‘wc’ supports, by running man wc (to exit ‘man’, type ‘q’ to quit).
    Exercise

    Start by opening a terminal. On Mac or Linux you have one built in; on Windows I highly recommend (Git Bash) [https://gitforwindows.org/].

    Pipe some stuff around:

    ls | wc
    ls -la | wc
    curl www.google.com | wc
    curl www.google.com | wc -m
    man wc | wc

  • Command Line Tools: Curl

    #Text is King

    One of the great things about web development — almost everything is text based. You don’t need a special program to view or edit information. Open source software development, *nix, is built around a core of many small text tools. These tools share conventions; that is, they expect to be used in similar ways. Learning the conventions gives you a lot of power to use many tools.

    Previously, we looked at how the whole Internet Protocol suite delivers content, but stopped before looking at the content. Today we’ll take that step using a program named curl. On Mac or Linux you have curl built in; on Windows I highly recommend Git Bash.
    How to “See URLs”, and instructions, as text

    HTTP is a text-based protocol. When resources get sent from the server to the client, they’re wrapped up in HTTP’s text format.

    Normally, the browser calls the server to request a resource, and when the server responds, the browser handles it. We’re going to have to make the call ourselves if we want to look at the raw response.

    curl (commonly pronounced like the word meaning ‘not straight’, but named as a pun: “see url”) will make the HTTP call for us. The command looks like this:

    curl -v example.com

    The first element is the program name, curl. Everything that comes after the program name are called arguments. All arguments change how the program executes, but there are some conventions around how they’re used.
    Arguments, Options, Parameters

    The elements that come directly after the command name are usually of a type called “options”. In our example, ‘-v’ is an option. By convention, options come in a pair of formats, short and long. The short version is a single dash followed by a single letter; the long is two dashes followed by one or more full words. The long version is almost always lowercase; the short version is case sensitive, meaning ‘-v’ is a completely different option than ‘-V’.

    In curl, ‘-v’ shortens the long form ‘ — verbose’. Using this option means curl will output as much info as it can. As the name implies, options are always optional. When not specified, an option will default to some standard behavior. In the case of the ‘-v’, you could think of as the opposite of verbose, like “succinct” or “brief”.

    Arguments which are not options are called parameters. Parameters (often referred to casually as ‘params’) are pieces of data the command will use. Parameters are different for every command, because each command needs different data.

    Some options are common across commands, like ‘-a’ often means “show all”. There’s no explicit standard though, so never assume you know what an option does.

    Instead, there’s a different text tool we can use to learn options: man, the command that displays manuals. Execute the command man curl to see the options for curl. Note that in this example, “curl” is now an argument, the name of the command we want to see the manual for. For any *nix command, you can run man — try man ls, or even man man (to see the instructions for the ‘man’ command itself). To exit ‘man’, type ‘q’ for quit.
    Real World Example & Exercises

    Remember separation of concerns? Somewhere inside the browser, there’s code that does the same thing as curl: check DNS to get the IP address for the host, call the IP address and ask for the resource. However, the browser goes a step farther, and attempts to interpret and render what it gets back. curl stops, and outputs whatever it gets back as text.

    Exercises:
    — curl with and without the verbose option, long and short form
    — curl a few of these sites:
    http://google.com
    http://www.google.com
    https://www.google.com
    https://duckduckgo.com/
    — use the man command
    man curl
    man ls
    — keep using man as we look at other tools

  • Can We Use Coronavirus To Improve

    Woke up this morning thinking about my friends in the service and entertainment industries who have been laid off, or are facing the threat of it. Between coronavirus and the stock market tanking, there is surely more hardship to come.

    I want to help support them; but a lot of the suggestions I’ve heard (get takeout, buy gift certs) benefit the business, not the staff. The suggestions to support staff are piecemeal and short-term, like sending them tip money via Venmo.

    I have a different idea.

    The crisis provides an unprecedented opportunity to help in a long term way. It’s not every day we get to press the “reset” button on an entire industry.

    Here in Maine, Portland has been declared a restaurant destination by media like Bon Appetit magazine. But the Portland Press Herald and Mainer News have reported on several popular restaurants which closed last year, in large part because they couldn’t find staff.

    Becoming a foodie destination has caused rents in Portland to skyrocket, like they have in other, much bigger cities. The workers who make the restaurant scene function can’t afford to live close enough to work there. This situation would have become unworkable everywhere eventually, the current crises have just accelerated it.

    What if we, customers and staff together, pledge that when businesses reopen, we will give significant preference to those with a living wage/no tipping policy?

    This model can work — several restaurants I love in Austin and San Francisco already operate this way. In Europe it’s always been the culture. Usually we say “well, we can’t change our culture”, but now we have been given exactly that opportunity.

    Searching for “no tipping” comes back with a bunch of results about how the industry already wants to move in this direction, but hasn’t been able to because of the ‘tragedy of the commons’ — no one wants to go first, alone, so no one goes at all. Now’s the chance to jumpstart that dynamic.

    This crisis will have damaging effects on our favorite establishments, there’s no way to avoid that. Let’s start thinking about how we’re going to rebuild now.

    What am I missing? What’s the best way to get started? What can we do at the same time to help support staff right now? I’m volunteering to put in the leg work to get this started, but I need your feedback, your experience, and your support. Please join the conversation on Twitter.

    Let’s rebound from this challenge, eating right, together!

  • Internet Protocols Demystified

    Protocols sound scary and complicated. But while the details of protocols themselves are complicated, they exist to make other things simpler. Let’s investigate protocols with some everyday examples, and a command line exercise.

    What is a protocol?

    What’s the first thing you think when you hear the word “protocol”? Maybe “diplomatic”? Diplomats specialize in making agreement and transfers across borders. To do that, you need a protocol: a set of behaviors to agree on.

    In a web page, separate parts of the code are responsible for separate jobs. HTML, CSS and JavaScript all have specific jobs; but before they can do their jobs, they have to get to your computer. Getting there is the job of the Hypertext and Internet Protocol Suites.

    Real World Example

    These protocol suites are a group of behaviors that clients and servers agree on to make transfers. Your browser handles several of these for you. Let’s look at the most important four:

    URL > DNS > IP > HTTP

    That Senator said the internet is a series of tubes, but the internet is like the old-timey telephone system.

    The URL is like the caller stating who they’d like to talk to. The browser reacts like a telephone operator, and starts the process of connection by looking up the name of the callee.

    DNS is the Domain Name Service, which functions like a telephone book, translating the alphanumeric domain name into an IP address.

    The IP in ‘IP Address’ stands for Internet Protocol. IP is responsible for the intermediate communication between the parties. In the phone analogy, IP is equivalent to normal conventions of speaking on the phone — for example, if one party doesn’t hear the other for a particularly long time, or if there’s noise on the line, they check if the other is still receiving.

    Hyper Text Transfer Protocol covers the conventions of a conversation. Both parties start by saying hello, and agreeing on a language. The request may specify how the answer will be formatted, the response may demand the user identify themselves or limit what they can ask.

    Exercise

    Start by opening a terminal. On Mac or Linux you have one built in; on Windows I highly recommend (Git Bash) [https://gitforwindows.org/].

    The nslookup utility allows us to interact with the DNS protocol. Give nslookup a domain name, and it will return the associated IP address:

    $ nslookup google.com
    Server: 209.18.47.61
    Address: 209.18.47.61#53Non-authoritative answer:
    Name: google.com
    Address: 216.58.192.174

    The first “server” and “address” represent where nslookup got its information from (in our telephone analogy, which telephone book got used). The ‘address’ in the answer is the IP we can expect to find the domain at (“non-authoritative” means your local DNS server is not in charge of this domain, and got this information from some other DNS server).

    Now take that IP address, and paste it into the address bar of the browser. You should get the Google home page, because the browser knows how to look up IPs as well as domain names (this doesn’t always work — sometimes the IP is an intermediate machine, like a load balancer).

    Speaking of intermediate machines, let’s look at how IP directs our traffic by using traceroute:

    $ traceroute 216.58.192.174
    traceroute to 216.58.192.174 (216.58.192.174), 64 hops max, 52 byte packets
    1 192.168.0.1 (192.168.0.1) 2.308 ms 1.556 ms 1.368 ms
    2 142.254.209.133 (142.254.209.133) 12.264 ms 12.465 ms 13.588 ms
    3 agg63.scbome0802h.northeast.rr.com (24.58.224.229) 33.023 ms 23.830 ms 23.815 ms

    Each “hop” is the “packet” of information being passed from one intermediate machine to another. Notice the final hop ends at the IP which DNS gave us. If you ever find a situation where someone is getting a totally different response than you, traceroute helps to check if it’s because their network is routing the request differently.

    What happens next?
    Once the URL > DNS > IP chain reaches its final destination, then HTTP can start its job. In an upcoming lesson, we’ll explore HTTP in depth.

  • Agile Demystified

    There’s an old joke that goes, “If there’s such a thing as a good toupee, I’ve never seen one.” An open secret in Hollywood: leading-man actors such as Ted Danson and Burt Reynolds wear toupees. But not raccoon rugs — extremely expensive, finely crafted ones that no one can identify as different from natural hair.

    Agile shares this quality: when it’s good, you don’t notice it. In fact the original Agile manifesto asserts this in its principles: “Prefer people over process.” But too often, organizations bend Agile process to avoid changing difficult internal dynamics. This leads to what Agile manifesto signatory Kent Beck calls ‘Scrum But’, as in, “We practice Scrum, but…” (Scrum — discussed later — is so closely related to Agile that they’re often used interchangably).

    Software has no physical constraints, and consequently, any software project can rapidly get as complicated as its participants’ wildest imaginations. Sometimes this leads to wonderful new products, but often, it leads to bloated, broken experiences, behind schedule and over budget, burnt out team members. Agile provides an alternative method to triage the scarce resource of team member time. It can be summed up super simply:

    Deliver the most important features as soon as possible.

    Simple, right? Of course, simple ain’t easy! People frequently overcomplicate Agile because, like a good toupee, they’ve never seen it done right.

    The most comprehensive way for me to show you the right way to do simple Agile would be to have you join a project at the beginning, and to “ride along” as we go through the steps. Since that’s not possible in an article, I’ll have to describe the process instead. In the spirit of Agile, I’ll do it as simply as possible, but since simple ain’t easy, it won’t be short. Let’s break it up into three parts:

    1. Basic principle
    2. Daily practice
    3. Story points: estimating software development
      Basic principle

    In watchmaking, anything except for the hour and minute hand — say, a chronometer, or the date — is called a complication. It makes the mechanism more difficult to implement. But this doesn’t make a watch with only the hour and minute simple! That operation is complex enough, without any added complications.

    Simple and complex are not opposites. Simple is the opposite of complicated; complicated means to be more complex than necessary.

    Agile strives to deliver features as soon as possible by keeping process to an absolute minimum. But there are minimums, below which lie failure. One of those minimum components is testing.

    What is the single most important feature of your project, anyway? There’s a saying some sports folks use: “Second place is just ‘first loser’.” Kind of a rough sentiment, but Agile requires ruthless prioritization. If your most important feature doesn’t work, then does anything else in your project have value?

    You might think of the ‘special sauce’ that makes your app a killer. But that’s useless if your users can’t get to it. So features like “log in”, and before that, “create an account”, are actually more important.

    The most important feature in all software projects is something along the lines of “it needs to run.” If the website or app or library doesn’t compile and execute, none of the other features matter.

    Agile projects require automated testing, full stop. If you’re not testing, you’re not Agile. As we add more and more features down the line, the features delivered previously — by definition more important than later ones — have to remain functional. If a less important/later feature breaks a more important/earlier one, we’ve violated our priorities.

    Manual testing can work at first, but as the project grows it becomes inefficient. Also, decades of best practices have shown that code written to be easily testable has immense advantages: drastically fewer bugs, greater support for additional features, and much more. Writing testable code starts out very difficult, but eventually gets easier. Not doing it from the beginning is not an option though, unless you want “development hell” at the end of the project, when changing any one part of the code causes problems somewhere else.

    Start with a so-called “smoke test”. The name comes from electronics hardware: if you put power on a circuit and the solder joints start smoking, something basic is terribly wrong. A smoke test just asserts that the page loads, or an object exists, or some other super basic criteria. It also asserts that the test setup works! As the project evolves to support more complex features, the smoke test still has a benefit: when many feature tests fail at the same time, the state of the smoke test helps to quickly determine whether or not the problem affects the entire system.

    Pro tip: tests which appear trivially simple can still have a great benefit. Don’t judge a test by its coverage 🙂

    Another place where it appears tempting to cut process actually subtly connects the development and business teams: the handoff from design to development. Newcomers to Agile process often suggest that instead of having business stakeholders write acceptance criteria, developers should work off of mockup images that come from the design team. This seems like an easy way to save time, but it introduces a lot of risk.

    “A picture is worth a thousand words” is true; but the value of most of those thousand words is about how the design should look. When you give a developer a picture and ask them to invent how the software should behave, you’re asking them to make business decisions. Developers should know how to program — asking them to perform so far outside their area is inefficient.

    Some developers are really good at understanding the business, interpreting business requirements into UI design, and so forth. If you have someone like that, count yourself lucky, but you may still be vulnerable to the greater threat.

    “Premature Optimization” is software jargon that means “solving a problem you don’t yet have.” Developers notoriously think about all the requirements the software might ever have. Sometimes they focus on performance, calculating load times based on an imaginary projected number of future users; sometimes it’s about features, making every element of the app infintely configurable “just in case”.

    Developers can waste spectacular amounts of time on premature optimization, and the complications added to the codebase can quickly grind progress on valid features to a halt. But they fixate on these points because they’ve gotten burned in the past. Acceptance criteria serve as a guarantee and a guideline for developers, a plan they can stick to with confidence.

    The basic principle comes down to making process as simple as possible, but no simpler. Next time we’ll talk about how to do this on a daily basis.

  • First Lesson Of Programming: Simple Ain’t Easy

    I would like to teach you everything I know about programming. This is the first lesson, and while it is simple, it is not easy.

    Have you ever had a computer problem? It’s a ridiculous question. We’ve all had some problem at some point. Some task you want to accomplish, and you can’t do it.

    The computer is a tool that offers the promise of infinite possibility. We can see it, shining like a mirage in the distance. Sometimes not far in the distance — the closer you are to your goal, the more painful the inability to get there.

    The primary skill of a programmer is not intelligence. It’s not how smart you are, how much code you can write, how much magic seems to come from your fingers on the keyboard.

    The primary skill of every programmer is managing frustration.

    The computer never, ever gets frustrated. No matter how mad you get, how many swear words you try, the computer gives you the same answer over and over again. Sometimes the answer seems different; but constant difference can also be a frustrating kind of sameness.

    The only way to succeed is not to quit. And the only way not to quit completely is to quit constantly.

    If you’re having a problem, and you sense yourself getting frustrated, stop. Once your mind is frustrated you are useless as a programmer. Go get a glass of water, go outside and walk around the block. Make dinner, wash the dishes, watch a movie, go to bed. Do whatever you have to do in order to let the frustration go.

    You will find, as you confront frustration again and again, that it gets easier every time. You get better at managing frustration. Eventually, problems that would have wasted a whole day, you can fix in a heartbeat and keep going.

    What do you keep going on to though? Bigger problems. The Haitians have a saying, “Mountains beyond mountains.” You solve one problem, and the next problem sits there waiting patiently for you.

    At first this seems like hell. Eternal problems! But that’s simply an incorrect way of looking at the situation.

    World class artists face this. Led Zeppelin turned down Woodstock, and played in Atlantic City that weekend instead. The comedian Dave Chappelle walked away from his career at its peak, because it wasn’t right for him. How hard must it be to operate at that level, where your decisions are so monumental!

    But this is what we all aspire towards: having bigger problems.

    When you start programming, you will never be bored again in your life. There’s always something more you can learn. Another approach you can try. Different languages, different techniques.

    But perhaps not all right now. You can’t try every approach tonight. I have found so many times that as soon as I let go, and walk away, the solution just jumps in my face.

    The opposite of frustration is patience. Give up constantly, and come back constantly.

    It’s simple. But simple ain’t easy.