Development Is Production

Quick, think of a “dev server.” What comes to mind? Probably a barely running mess full of old temp files, tarballs named after people, scripts that aren’t in version control and nobody’s quite sure if they’re still used, SSH keys from developers who left years ago...in short, a big ramshackle mess.

Okay, now think about your QA environment. Does it fully work? Does it really? Or are there a bunch of integrations stubbed out? Maybe there are jobs that run in production that can’t run in QA. Probably the database isn’t very realistic, because the production data has PII that can’t be copied around. Do you have high confidence that passing tests in QA means the software will work in production?

Maybe you’re in the minority. If your image of a dev server is a fresh virtual machine with a known configuration, that’s great! Maybe your image of QA is a whole environment stamped out by the same automation tools that deploy to production, with an anonymized sample of production data from within the last week. If so, you’re doing quite well.

Most organizations treat their development environments like a shantytown. Stuff only works there because the developers run their own power by daisy-chaining extension cords from a nearby settlement. QA doesn’t match production in topology or scale, and multiple dev teams are trying to get into QA but can’t because there’s only one environment. (Hint: There’s no “right number” of QA environments. Virtualize them so every team can create its own on-demand QA environment.) In short, development environments are treated with utter disregard.

This is kind of odd when you think about it, because developers are creating content all the time. They build software that has to go into version control (a service), get constructed in CI (another service), tested in QA (a service), and stored in a repository (yet another service). When these services are down, developers can’t do their jobs. Let’s look at an analogy. Suppose your company’s content management system went down so copywriters couldn’t do their jobs. That would be at least a Severity 2 outage, right?

The tools, services, and environments that developers need to do their jobs should be treated with production-level SLAs. The development platform is the production environment for the job of creating software.