Software development meets the McNamara Fallacy

The McNamara Fallacy is the idea that it is an error to make decisions purely on measurements or quantitative data.

Robert McNamara was the US Secretary of Defense during most of the 1960s, during the first half of the Vietnam War. He came to the position after a successful career in industry—most notably as the first president of the Ford Motor Company who was not from the Ford family—after his innovative use of statistics in business. 

McNamara took the view that anything that can’t be measured should not be taken into account in the decision-making process. He considered all measurable data to be useful, and made metrics the sole means of determining the correct course of action.

He brought this statistical viewpoint to his job as Secretary of Defense and applied it to his management of the war. Problematic results, such as using enemy body counts as a measure of military success, ultimately led to the coining of the phrase “The McNamara Fallacy.”

I think the software industry is in danger of falling victim to the McNamara Fallacy. 

While metrics are useful, relying on metrics alone and ignoring qualitative factors that certainly come into play can easily lead you to focus on things that don’t matter. Ultimately it can lead to failure.

Numbers are easy

First, measuring the software development process is becoming easier and easier. With Git becoming the de facto source control system and tools like LinearB, JellyFish, and Plandek providing deep insight into what is happening within software repositories, it is very simple to get metrics that tell you a lot about what your team is up to. 

It is comical that the industry once seriously took something as pathetically simple as “Lines of code per day” as a serious metric. Today’s tools allow managers to see things that were not previously observable. Simple team metrics like deployment frequency, pull request review time, and cycle time are readily available to help pinpoint bottlenecks and aid in decision-making.

We’ve gotten to the point where metrics are easy, and “soft” measurements are hard. Don’t get me wrong—the granular metrics we now have are useful and valuable. But the temptation is great to rely on easy numbers instead of doing the hard work of figuring out the impossible-to-measure stuff. Robert McNamara fell into that trap, and it is easy to see the software industry doing the same. 

If we lose sight of the non-measurable things, we lose sight of what makes software development successful. If we focus on “computer science” and “software engineering,” we can lose sight of the human factors in writing code that make or break a successful project.

Software development is a team sport. Although individuals can and do shine as part of a team, the team results are what really matter. Sports fans thrive on statistics, but they know that ultimately, it’s not the statistics that win championships. It’s the intangibles that make the difference between first and second place.

Intangibles are hard

Despite our best efforts, we don’t have a means of measuring “writes good code.” It takes years of experience to recognize “good code” from “bad code,” and we can’t (yet?) measure it objectively. Maybe AI will figure it out someday. One could argue that AI can write good code today, but the ability to recognize good code is still uniquely human. 

Similarly, how does one measure “works well with others”? How about “team morale”? There is not, and probably won’t ever be, a way to measure these kinds of things, but they are important, and knowing it when you see it is a key to success. Recognizing and encouraging these intangibles is a critical skill for a software development manager to have. 

Finally, over-indexing on metrics can be detrimental to morale. No one wants to feel like a number on a spreadsheet. 

I encourage you to use the cool measurement tools out there today. But as you review what things like the DORA (DevOps Research and Assessment) metrics are telling you, remember to consider the things not revealed in the numbers. Sure, metrics are important, but understanding what your gut is telling you and listening to your intuition can be just as valuable, or even more valuable. 

Measure what you can, but always be sure to listen to the still, quiet voice telling you things that no statistical package ever can.