To Measure or Not to Measure

Written by Yegor Bugayenko / Original link on Jun. 23, 2020

The question was asked on StackExchange nine years ago (just around the time the site was launched): “If not lines of code, then what is a good metric by which to measure the effectiveness of remote programmers.” The answers, not surprisingly, were all along this line: programmers are not supposed to be measured! I bet those who answered were programmers themselves. Indeed, why would a programmer be interested in being measured and being reduced to a mere number?

better-call-saul.jpgBetter Call Saul, Season 5 (2019) by Vince Gilligan et al.

First, let’s see why putting a metric on a programmer may be considered bad practice (my opinion: these are merely excuses from overly-paid programmers/managers who are just trying to keep their jobs, doing nothing whatever they want, and wasting employers’ money):

How do you like that? “Keep paying me, but don’t expect me to give you anything back! And don’t you dare check my performance!”—this is what I hear and I’m not surprised. What’s happening is called the performance management revolution and the gist of it is this: modern management is so weak that we desperately need an official name for this chaos, to avoid confusion. Agile is the name.

Managers are not in charge anymore, programmers are. And it’s sad.

However, not everybody believes in this anarchy. Some experts think that metrics are actually helpful. Bradley Kirkman et al. claim that “recognizing a single team member seems to have a positive and contagious effect on all the other members in the team,” and the famous Vitality Curve suggested by Jack Welch in 2001 is still with us.

I believe the all the negativity aimed at metrics is caused by their incorrect usage. Indeed, if the only performance metric of software engineers, for example, is the amount of hours they stay in the office, that will suggest only bad things about all metrics. Such metrics do hurt, no doubt about that. But the absence of good metrics hurts even more. How to chose them—is the real question. Let me suggest a few, instead of the famous and incorrect LoC. Ideally you would pick a combination from this list, or even use them all together:

I can’t stress this enough: each metric must have a quality control mechanism. Just measuring bugs reported without checking the quality of them would lead to cheating: programmers will report whatever they like, just to bump the numbers up. Each bug must be verified by an architect: duplicates or low-quality bug reports are to be rejected. The same is true for every metric: trust without control leads to cheating.

Trust without control leads to cheating.

It is also worth mentioning that features, bugs, pull requests, and documentation pages may have different complexity, urgency, and severity, which also should be taken into account, increasing or decreasing the numbers in each metric.

Most of these metrics can be collected automatically, without any human interaction, for example via the GitHub API.

In an ideal world of ideal management, the project compensates the work of its programmers according to the metrics collected. Instead of salaries, programmers get money for features, bugs, documentation pages, and so on. How far your project is from this utopia—is the indicator of your professionalism as a project manager. Lousy managers don’t measure anything and make everybody “happy” by keeping wages high and control low … until the project runs out of money. On the other hand, exceptionally good managers let metrics control everybody, making the best happy and the worst … quickly find the way out.

Which one are you?

yegor256 calevans

« Improve Performance - Unit test naming conventions »