Readers of this book should have an affinity to the reflective practitioner view [Jarvis 1999], [Argyris and Schön 1996], where practitioners reflect on what they do in their daily work and develop better ways of performing work tasks. To get hold of the usually implicit understandings of practitioners should be a main focus of disciplines such as ours. I must mention that our HR manager from earlier also stated that her company didn’t really want their programmers to spend time reflecting too much upon what they did. Rather, they should just implement what they were told to implement. I’m sure most of us, together with Joel Spolsky, would not find this view particularly appealing, but it is all too easy to revert to such strategies in pressed lead-time situations, i.e., getting trapped in the urgent/important quadrant at the expense of the important/not urgent planning and development quadrant [Covey et al. 1999].

With our insights, however, we can lay a logical trap for our already time-trapped HR manager: her previous remark on IQ entailed that (1) GMA predicts programming performance, but (2) since GMA predicts learning ability first and foremost, this must mean that she wants programmers who are fast at learning, and (3) learning is done through reflective practice, but (4) once she has her programmers, she doesn’t want them to reflect, hence she doesn’t want them to learn. I’ll call this the unreflective time-trapped HR manager’s paradox.

In fact, our HR manager isn’t totally unreflective. She knows that intelligence matters. Maybe she has also read some of the literature on it cited in this chapter. But perhaps she didn’t turn to the page where it says more specifically that intelligence is good for acquiring skill rather than good for predicting skill already acquired. Intelligence is also more than IQ (which is just a measure of one aspect of intelligence). It’s easy to lose the details when reading scientific literature, and this wouldn’t be the first time. An urban legend in software engineering tells us that this was the case for Royce’s article [Royce 1970] where he argued against the waterfall model, but the industry failed to turn to the next page, where it said that the model on the previous page was not to be recommended. There are sure to be many more incidents of getting only half the picture; among practitioners and researchers alike. Remember to include the fuller picture when basing your decisions on evidence!

So how did we fare with our three questions posed at the start of this chapter? The first question concerned whether one can define what a good software developer is. The answer is that for certain tasks of software development (e.g., programming), it seems we might soon be able to. For other tasks (e.g., software effort estimation), it is much harder to define what a good performer is. Before you object that a good performer on the task of estimation is simply a person who estimates accurately and reliably with the correct level of confidence, remember that our discourse in this chapter filled this first question with a great deal more meaning than that. To define a good performer means to define the task and the expertise to go with it, to the degree that you can say what it takes for a novice to become an expert, and an expert to become an even better expert.

The answer to the second question follows suit. For certain tasks it seems that we’ll be able to measure expertise and task difficulty, but for other tasks the road there seems much longer. Note that we’re asking to determine a degree of expertise by efficient means: that is, without having to observe performance over a long period of time. This, therefore, includes predicting future performance.

The two protagonist tasks in this chapter—programming and software effort estimation—are quite different in nature; one relates to planning, and the other relates to doing. And our answers to our first two questions might seem to broaden this divide. But just because we have a beginning grasp on one task but not the other doesn’t mean that the two tasks are unrelated. On the contrary, it is natural to ask whether performance on one coincides with performance on the other.

Tentatively, it seems that programming skill and estimation accuracy and reliability on one’s own tasks are positively related, which goes against the common belief that even good programmers can’t estimate their own effort. If the positive relationship turns out to be true, you could ask your best programmers to estimate and then calculate the correction to this estimate according to the skill levels of the programmers who will actually do the job. This would be a vast improvement on the skill-insensitive compensation factors that are currently in circulation. Using one concept vicariously for another concept in this manner is an interesting prospect. For example, can you improve estimation skill indirectly by improving programming skill?

The third question asked whether one should focus on tools if one can’t reliably determine expertise. If one does understand the task and the expertise needed for it, then one should definitely focus more on expertise, since large bodies of research show the increased business value of just increasing the organization’s expertise a few percent. But clearly, Glass’s facts aren’t useful unless you know how to recognize expertise in software developers. When you do not have a grasp on expertise and how to develop it, the alternative is to rely on the environment and to develop facilitating tools and techniques. The relevant skill then becomes once removed to that of mastering the tool or technique. Maybe this is the way to transform ill-defined tasks such as software effort estimation into at least an inconsistent task, if not a consistent task.