I’ve been frustrated with the unsolved problem of configuration management (CM) my entire career. I’ve used several systems. They all did the basics, and none of them were disastrous, but they all sucked in the same fundamental ways.

In Part 2 I’ll share an idea for a path forward. But first, Part 1. Let’s talk about what’s wrong.

It’s Documents All the Way Down

Most CM systems are document-centric.¹ They are designed to control MS Word, PDF, PowerPoint, Visio, and other file-based content. Writing and drawing are wonderful thought technologies, but documents — narrowly defined here as monolithic products with print-first layouts that are stored as separate files in a folder structure — are relics. Word, in particular, is stuck in a skeuomorphic, paper-based paradigm that brings with it the worst baggage of the pre-internet era.

A document stored on a fileserver has no inherent way to canonically identify itself (filenames aren’t unique, file paths are computer and OS dependent, and both can be changed by anyone at will). This means there’s no means to permalink to content. These popular file formats also have no native mechanism to show how content evolves or capture who has read, reviewed, or approved it.

But these are critical configuration management functions, so CM systems are created to apply a series of hacks and conventions to fill the gaps. Document numbers identify and support cross-referencing, revision letters and logs track versions, and ancillary documentation like Change Requests (CR), review routings, and signature pages capture CM meta data.

The resulting process is a Kafkaesque love child of Brazil and Metropolis, characterized by tedious busy work that groans under the weight of its own bureaucracy.

If you’re nodding in agreement, just skip to Part 2; otherwise, let me use a story to describe a typical CM workflow and its issues.

A Month in the Life

Our protagonist, Hiro, is writing the first version of a system-level document that needs to be released by the next design review. He’s written lots of these, but this project is pretty novel, so he can’t just modify some prior art.

He starts with the company’s Word template. It’s a hodgepodge of styles created through careless copy-and-paste, bloated with boilerplate,² and a case study in poor taste in fonts. Word (and the other common formats — I’m looking at you PowerPoint) is flexible, but also fussy. It’s easy to spend lots of time making things marginally better — or more often, worse. It doesn’t help that most people’s only training in the visual display of information are the Office defaults, which are bad.

But Hiro is experienced, doesn’t get bogged down in minutiae, actively communicates with the larger team on the content, and quickly gets to a decent draft. Only God and Hiro know where this file is stored. Maybe there are 30 drafts thoughtfully saved, with commit messages, to a Git repository that’s cloned to multiple locations. But it’s probably on his desktop with a bunch of other files distinguished only by their last modified dates and the version numbers appended to their filenames.

Revision control of early drafts aside, the document is ready for peer review, but it’s not time for the CM system yet. The CM system is heavy weight, and our boy, like everyone, hates having his documents rejected. So he emails out a copy to the key stakeholders and asks for comments.

There’s a non-trivial activation energy required for someone to open a long document with the intent of thoroughly reading it and providing thoughtful feedback.³ Hiro’s prose is parsimonious, but it’s a complicated system and a long document, and he has to nag everyone once before they review it.

Two weeks later, everyone responds:

• Ethan replies to all with some big-picture thoughts.
• Sam’s input is more tactical, captured section-by-section as tracked changes, and emailed to everyone.
• Janet adds her thoughts, mostly copy edits, to Sam’s file.
• James sends out a third file, with only his changes, and only to Hiro.

Hiro incorporates the feedback. He tries to use Word to merge the inputs, but Ethan’s feedback involved structural changes that confused the algorithms; ultimately, he does most of it manually, change by change. But it gets done and a clean file is almost ready for release in the CM system, pending a few fiddly details:

• He updates the figure and table caption numbers and cross-references. A few things moved around, which broke the links, so he tediously recreates them.
• He updates the table of contents, tables, and figures.
• He creates a table of referenced documents at the front and acronyms at the back, manually.
• He updates the header and cover page with a release date, document number, and revision. Pulling the document number was this project’s first interaction with the CM system, so this involves a little yak-shaving.

And Signatures All the Way Up

The next step is initiating the formal signature process in the web-based CM tool. Click, click, click to upload the file and then select and notify reviewers. Another round of nagging folks over the activation-energy hump uncovers the first point of friction: the reviewers gave conflicting feedback, but they didn’t know it because some of Ethan’s and all of James’s comments were private.

The disagreement had to do with the content, not the CM tools and processes, but it is the tools’ fault (Word and email in this case) that Hiro was the only person who saw all the comments. He thought his resolution would make everyone happy, but it didn’t. Janet and Sam approve, but James rejects the route, which sends it back to Hiro for an update. Hiro helps resolve the fundamental conflict and makes a quick update; this clears away Janet and Sam’s approval, so the latest version goes back out to all four.

Even if there was no fundamental conflict, this process would still suck because the new route features a “clean” document, and the reviewers can’t tell what changes Hiro made or if their comments were incorporated. James has been burned before, so he finds his emailed comments and meticulously double-checks everything. He doesn’t really have time for this, so he’s a little resentful at Hiro, the process, and the tools. This sucks, because James is a hell of an engineer whom we want to keep happy.

Janet is more trusting; sadly, one of her key insights was overlooked and won’t get fixed until Rev C, when the latent mistake becomes a problem.

Finally, all four reviewers approve. We have initial release!

But things change fast around here and soon Hiro is making changes to a draft revision that no one can see; meanwhile, the team works to an out-of-date (but approved!) copy. The activation energy tax is most pernicious now. The next revision won’t get approved until the value of an update outweighs its cost (in people’s time). But we just saw this cost is high, which means revisions don’t get made until things are really disconnected.

I’ve seen a few hacks around this: working to redlines, using lighter-weight documentation to aggregate and approve small changes until the next major revision, delaying initial release, removing signatories, and doing a less thorough peer reviews.

There has to be a better way…

To be continued in Part 2 — A Path Foward.