Developer productivity

There is widespread agreement in the software-engineering literature that it is common that there is a huge difference in productivity between two developers with the same formal training and the same experience. Clearly, this observation is not backed up by any solid scientific proof, because we do not even know how to measure the productivity of software developers. For example, we know by now that it would be a disaster to attempt to measure productivity as the amount of code written per time unit. One of the earliest papers suggesting such a big difference in productivity is "Exploratory experimental studies comparing online and offline programming performance." by Sackman, Erikson, and Grant from 1968. In that paper, differences of up to a factor 28 are cited. Though a more recent article "The 28:1 Grant/Sackman legend is misleading, or: How large is interpersonal variation really?" by Lutz Prechelt (from 1999) suggests that typical differences are much lower (a factor 2 or so), even though he suggests that factors of up to 10 can be observed.

Even though there is a definite lack of hard scientific data suggesting huge differences in productivity, there is some evidence, and many practitioners would agree that they have observed the same thing. Certainly, it is consistent with our own experience that there is surprising variation in productivity.

Now, let us say that it is possible that there might exist differences in productivity of a factor 10. To get an appreciation for how much a factor 10 is, let is do some back-of-the-envelope calculations. First, let us imagine the consequences on developer salaries. Let us say that current typical salaries represent a developer with a productivity of the geometric average between the one with the worst productivity and the one with the best productivity, and let us assume that this typical salary is around 4000€ per month. If salaries were proportional to productivity, we would then observe low salaries around 1300€ per month and high salaries of around 13000€ per moth for developers with the same formal training and the same experience.

Clearly, since we do not observe such differences in salary, either this perceived difference in productivity is purely imaginary, or (more likely) employers not only have absolutely no clue when it comes to estimating developer productivity, but they are not even aware of the possibility for the difference in productivity to be this great. If they had the slightest hint that the difference could be that great, they would invest heavily in research towards methods for evaluating productivity, given the huge potential savings they could make by only hiring highly productive developers.

Another way of getting an appreciation for how considerable a factor 10 might be, we can imagine a team of (say) 100 programmers with an average productivity of again the geometric average between the worst and the best. By using only highly productive programmers, a factor 3 could be gained, so that the same production could be obtained with 35 or so programmers instead of 100. Anyone with some experience in software development furthermore knows that production is not proportional to the size of the group, because as the number of members of a team increases, communication takes a larger and larger proportion of the available time. It is therefore likely that the same work could be done with 30 or even 25 highly productive developers.

So what should an employer do who wants to improve the situation? Well, there are many actions that should be considered simultaneously.

First, employers need to become more serious when it comes to screening candidates. In Europe (more so than in the USA) there is a tendency to avoid asking technical questions during hiring interviews, and if technical questions are asked, they do not attempt to determine productivity. Why is this? Again, very likely because employers can not fathom the possibility that the difference in productivity can be as great as a factor 10.

Second, employers need to create an internal program with the purpose of lifting the productivity of its developers. At the center of this program would be a number of continuing-education courses, the contents of which would be adapted to the needs of the employer. However, we firmly believe that it is important to identify highly productive developers and have them serve as mentors for less productive colleagues. The reason is that high productivity is very likely the result of a particular attitude toward continuous (daily) small improvements in productivity, and that this attitude needs to be transmitted to all developers.

Third, employers need to realize that the tools that are used by developers play a significant role in productivity. Furthermore, the tool that plays the absolute most important role in this respect is the programming language that is used. There is a tendency to believe that all programming languages are roughly the same, but there is some scientific evidence that the programming language can play a major role in terms of productivity. We will discuss this idea in a separate article in the future.

But what if you do not believe that there could be a factor 28, or even a factor 10 difference in productivity. What if there is "only" a factor 2? Well, in our opinion, even a factor 2 is huge. At least employers seem to think that even a factor 1.01 is significant because that is the order of magnitude of a typical pay raise. In fact, a factor 2 is so large that I think it is fair to say that an employer who ignores such a factor is not living up to the expectations of the shareholders and should therefore be relieved of any responsibilities.