Software engineering has several characteristics that that distinguish it from other disciplines. Software is developed in the creative, intellectual sense, rather than being produced in the manufacturing sense. Software processes are development processes, not production processes. In other words, they are not replicated over and over again. This unique aspect of the discipline is probably the most important one, and greatly affects how we learn. We always need to be on the lookout for the effect of context variables. Because software engineering is a human-based discipline, there will always be variation in study results and we will never be able to control or even identify all the context variables. The discipline creates a need for continual experimentation, as we explore how to modify and tailor processes for people.

The variation in context springs from more than just the people who create the software; each piece of software itself is different, as are software development environments. One consequence of this is that process is a variable, goals are variables, etc. That is, we need to select the right processes for the right goals for the environment we are analyzing. So, before we decide how to study a technique, we need to know something about the environment and the characteristics of what we are about to build.

A second distinguishing characteristic of the software engineering discipline is software’s intangibility, or one might say the invisibility of its structure, components, and forms of development. This is compounded by a third characteristic, the field’s immaturity, in the sense that we haven’t developed sufficient models that allow us to reason about processes, products, and their relationships. These difficulties intensify the need to learn from the application of ideas in different situations and the requirement to abstract from what we see.

A final problem is that developing models of our experiences for future use (that is, reuse) requires additional resources in the form of money, organizational support, processes, people, etc. Building models, taking measurements, experimenting to find the most effective technologies, and feeding back information for corporate learning cost both time and money. These activities are not a byproduct of software development. If these activities are not explicitly supported, independent of the product development, they will not occur and we will not make quality improvements in the development process.

All this makes good experimentation difficult and expensive. Experiments can be confirmatory only in the small scale and are subject to problems in understanding scale-up, the integration of one process with another, the understanding of the effect of context variables, etc.