This chapter covers the following topics:

INTRODUCTION

Agile is a lightweight software development approach that evolved in the mid-1990s and resulted in an Agile Manifesto and an accompanying set of ‘12 principles of agile software’ published in February 2002. Since then, it has grown in popularity and is now an extremely common and fashionable umbrella term for a number of methods and processes. It is used by thousands of development teams across the world, and is the subject of numerous books, papers and training courses. In that time, it has evolved and changed, and is not only applied to software products, but also to other types of development.

There are many synergies between agile and business analysis. Agile favours highly collaborative working between the customer and product development team; a focus on the early and frequent delivery of tangible solutions; and an iterative approach that is highly responsive to changing requirements. While the major focus of the agile approach is on the development of software, the concepts can apply to business analysts working on change projects, where the project focus is on business rather than technology changes. However, to do this successfully, business analysts need to understand the rationale and philosophy that underpin the agile approach.

While the rest of this book will focus on how it may be applied to business analysis, this chapter explains how agile has developed and provides an introduction to the agile philosophy, principles and methods.

THE ORIGINS OF AGILE

Before the term, ‘agile’ was coined, software had been successfully created for decades. Until the early 1990s, linear planning techniques were commonly used, which isn’t surprising, given that software development originated from computer systems engineering. These linear systems are commonly referred to as ‘waterfall’, due to the way that they were drawn in Winston W. Royce’s influential article from 1970, ‘Managing the development of large software systems’. Interestingly, Royce did not actually use the term ‘waterfall’ in his paper; however, he is renowned for developing this approach and its resulting use ever since.

Royce represented a common approach to developing complex software systems, as shown in Figure 2.1.

 

Figure 2.1 Waterfall development approach

Unfortunately, Royce’s paper was widely misunderstood. He presented the above model as ‘risky and invites failure’, and was proposing modifications to make it much more iterative and incremental. However, that element of his work is largely forgotten, and his waterfall picture remains in common use, although the names of the stages have changed a little over time.

As discussed in the BCS publication Developing information systems, the waterfall method and its variations are effective for some types of development. However, as Royce himself stated, it is less suitable for many types of project, as capturing and baselining all the requirements early in the development life cycle, and expecting them not to change, is a significant drawback for many, if not most, projects.

The evolution of iterative methods

New methods began to emerge in the 1980s and 1990s as a response to some of the challenges that developers faced when using a waterfall approach. One of the first to become popular was Rapid Application Development (RAD), which involved creating prototypes and using them to elicit requirements, validate designs and evolve toward usable solutions. RAD began at the New York Telephone Company and was refined by IBM in the 1980s. A book, Rapid application development, by James Martin, with contributions from several others, was published in 1991.

Although RAD is a method in its own right, it also spawned a number of alternative approaches that are based on some of the same principles. One popular example is the Spiral model, developed by Barry Boehm (1988), one of James Martin’s collaborators. This model introduced the concept of incremental planning, definition, design and development of software so that each time around the spiral the risks are reduced and the end result is more predictable.

A different iterative development method emerged in the late 1990s that combined some of the rigour and whole life cycle elements of the waterfall life cycle, with incremental delivery and a particular focus on modelling. It was based on the research of three men who originally collaborated to create their most famous work, a modelling notation known as the UML. James Rumbaugh, Grady Brooch and Ivar Jacobson (known as ‘The Three Amigos’), developed ‘The unified software development process’ in 1999 with Philippe Kruchten, who provided the overall architecture for the iterative approach.

At the time, they were working for the Rational Software Corporation and their process therefore became available as the Rational Unified Process (RUP). RUP is risk driven, iterative and takes a use case- and architecture-centric approach.

The founding agile methods

Through the 1990s, a number of alternative ways to organise and structure the development of software products began to emerge and shape the thinking that eventually evolved into ‘agile’ in 2001. These approaches were referred to collectively as ‘Lightweight Methods’. The key developments are described in Table 2.1 below.

The Agile Alliance

It was Kent Beck who first sowed the seeds of the agile movement, with the publication of his book in 1999 and his invitation to various leaders in XP to join him at a retreat in Oregon. Significantly, he didn’t just invite XP thought leaders, but also several other people who had similar interests. These included Alistair Cockburn (Crystal), Jim Highsmith (ASD) and Dave Thomas (co-author of The Pragmatic Programmer) (2000). At this initial meeting, they discussed the similarity of XP with other methods and decided to meet again, with a broader range of people, to explore common ground.

 

Table 2.1 Agile development methods

On 11 February 2001, a group of 17 people met at ‘The Lodge’ at Snowbird Ski Resort in Utah to talk, ski, relax and try to find common ground between their thoughts on software development. They represented the leading thinkers from XP, Scrum, DSDM, ASD, Crystal, FDD and Pragmatic Programming, along with others who wanted a viable alternative to heavyweight, documentation-driven development processes.

Two days later, they had agreed on a name to replace the term ‘lightweight’ (which had been used up to that point but was not well liked), a manifesto that embodied the values they all believed in and a set of guiding principles. The Agile Alliance was born.

The agile mindset

One of the remarkable things about those two days at the ski lodge in Utah is that 17 people with very different backgrounds and perspectives managed to reach agreement on what agile means. They were applying different approaches, and had different ways of achieving similar outcomes. Yet, they were still able to agree a set of principles and values that they could all sign up to.

This is why agile should be regarded as a mindset – a way of thinking, and not a method to be learned. This mindset should be supported by methods and techniques that help projects to adhere to the principles and values, as shown in Figure 2.2.

If agile is thought of in this way, it is possible to see why there are different agile approaches and methods that can all support the same principles, even though they differ in execution. Unfortunately, this also means that it is not possible to instruct someone in how to be agile; they have to adopt the mindset. While you can teach a process, you cannot teach a way of thinking, and just applying an agile process or technique will not mean that you are applying the agile principles and values.

Agile is a philosophy to be felt and comprehended, such that it guides the way in which work is approached. It is not a method to follow.

 

Figure 2.2 The main elements of agile

Although this is not a universally held view, it is held strongly by the authors of this book. While it can be possible to treat agile as a recipe to be followed for straightforward projects, as complexity increases this becomes less and less likely to succeed. In practice, projects are not simple, and in order to succeed with an agile approach, the team must have or gain an agile mindset.

THE AGILE MANIFESTO

The most important and most visible result of that meeting in Utah, shown in Figure 2.3, was the publication of the manifesto for agile software development.

 

Figure 2.3 The manifesto for agile software development

The manifesto for agile software development (often shortened to the Agile Manifesto) is short but powerful; in a few words it manages to encapsulate a wide range of concepts, ideas, prejudices, values and experiences of software development. Although the key focus is on software development, once unpacked, there is very little that is actually software specific. That’s why various commentators over the years have proposed amendments and changes. Indeed, in Chapter 3, we shall propose our own version, reflecting a business perspective and more applicable to business analysts. However, the original words remain unchanged on the website, and, for the most part, still hold true for many situations.

Although simple, the Agile Manifesto is often misunderstood. In particular, the final clause is often forgotten or ignored: ‘That is, while there is value in the things on the right, we value the items on the left more.’ It is really important to remember that agile projects still require processes, tools, documentation and plans; it is just that these are not the most important artefacts or deliverables. It is the outcome delivered by the project that is the actual value.

Let’s look at the individual statements of the Manifesto in greater detail.

Individuals and interactions over processes and tools

Creating solutions is a team sport. Team members need to communicate with one another. They need to ask questions, discuss ideas, debate ways forward and support one another. Particularly with more complex problems, the ancient adage that ‘two heads are better than one’ still holds true. Of course, tools and processes are also important, sometimes critical. They can help teams be consistent, encourage the right things to be done in the right order and optimise repetitive tasks. In fact, without tools, some common agile practices like continuous integration or version control would be much harder. But, on balance, agile teams should get the people parts right first.

Working software over comprehensive documentation

The perspective of a software development team is that it exists to produce working software, that has been tested and delivers an expected outcome to the users to a desired level of quality. Good software should also be properly documented, of course, and poor documentation can diminish or nullify the value offered to the customer. For example, if the customer isn’t able to set up the software because they don’t understand the controls, they will not realise any value from it. Similarly, ongoing maintenance and further development of the software will require the relevant documentation.

Having said this, no matter how good the documentation, at the end of the day it is the software, or a working solution, that the customer needs. An important word in the Manifesto value is ‘comprehensive’. There can be a tendency in some teams to generate huge volumes of documentation in great detail. It is often as if there is a feeling of comfort in producing documentation – if we analyse and document everything, we won’t be caught out! Similarly, where a method permits tailoring (as many do), inexperienced or cautious project managers have a tendency to include more artefacts than required. Does an architectural prototype intended to help the customer understand how the interface might work really require a test plan and a support guide? Probably not. When it comes to documentation, as with most other elements of agile development, teams should aim for Just Enough, Just in Time.

This is just as true for non-software solutions. Overly focusing on the word ‘software’ can be dangerous, particularly for larger, more integrated solutions, or when trying to solve business problems with a holistic solution. Business analysts often work at higher levels of abstraction, where IT or technology is only one aspect of the solution, and may use models like POPIT™ (discussed in Chapter 3) to explore how people, the organisation or processes also need to be changed.

Business analysts may find that this value challenges some of their work within project teams, particularly with regard to requirements documentation. However, experienced business analysts are skilled in deciding the requirements approach, and this includes determining the artefacts that will best deliver the project objectives. To achieve this, the Lean principle of Just Enough, Just in Time should be borne in mind. The requirements should provide sufficient detail to allow working solutions to be created; there should be a much lighter touch for requirements that are not to be implemented within the current release.

Customer collaboration over contract negotiation

This value is based upon the premise that the people who know best what a system should do are the customers who will be using it. They will know what business outcomes they are trying to achieve, and, as your system takes shape, they will be best able to judge whether it looks like it will help them. Since most projects are complicated and have to encompass numerous requirements of different types and of varying complexity, it is hard to define exactly what those needs are in advance. This is the fundamental problem encountered when applying the Waterfall development life cycle, where requirements must be agreed and signed off before design and implementation begin.

Inevitably, at some point, there will be changes made to the requirements or an earlier misunderstanding about what is needed will emerge and require clarification. Under the Waterfall model, this would require a formal change request process to be followed, with all the consequential effort this would demand. Since this situation is common, agile teams like to pre-empt it with contracts that are less prescriptive and with high levels of engagement and collaboration with customers.

This presents many problems. Customers are often very busy or they may be located at different, or dispersed locations. There may also be different types of customer (explored in Chapter 7), all of whom may have a different perspective on the system under development. This is where business analysis can be invaluable. Business analysts are able to analyse and manage stakeholders and as they will typically hold knowledge of the business domain, can help the customer to better understand the potential the software will offer and the real business needs to be met. The level of collaboration is expected to be much greater on an agile project and, as a result, business analysts can expect to have a different kind of relationship with customers.

Responding to change over following a plan

This is probably the single most important element of agile software development; the expectation that things will change, and the adoption of processes, practices and principles that not only expect change but embrace it.

This is also the element of agile that causes the most angst, and challenges orthodox development methods the most. It is inherently logical to want to know what you can expect, when and for what cost. Most project management and systems engineering approaches aim towards delivering a certain set of things, on a certain date, with a certain amount of budget, in the hope that this will reduce risk. Senior stakeholders want to be able to make promises to others about what they can expect, and it seems perfectly sensible to want to be able to give them an answer.

In reality the only certainty is that things change: technologies, business strategies, competitors, personnel. Solution development methods that expect, anticipate and deal with change will be more likely to navigate those changes with the least impact and the highest chance of succeeding; that’s why agile teams value responding to change.

This does not mean that plans are bad. On the contrary, plans are essential, but they must be able to change when the environment that affects them changes.

THE 12 AGILE PRINCIPLES

The Agile Manifesto is supported by a set of 12 principles, drafted during the original meeting in February 2002 and finalised over the following few months. Table 2.2 shows the 12 principles, and briefly describes why they are important.

 

Table 2.2 The 12 agile principles explained

Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.		The customer is the most important stakeholder, and what is most important to them is knowing that you will solve their problem for them. It is even better if they can receive something of value early.
Welcome changing requirements, even late in development. Agile processes harness change for the customer’s competitive advantage.		Requirements change for all sorts of reasons. Agile teams expect this and anticipate it.
Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for the shorter timescale.		The best way to know if something is right is to see it in action. This helps to refine requirements for future releases, raises customer confidence in the software development team and offers the potential to realise value early.
Business people and developers must work together daily throughout the project.		Most projects are too complicated to assume that written down requirements will capture every detail. Being able to ask questions and clarify understanding throughout the project is essential – the best way to do that is face to face.
Build projects around motivated individuals. Give them the environment and support they need, and trust them to get the job done.		People build solutions, and people do better work when they are motivated, empowered and have the right tools for the job. The impact on quality and productivity caused by not doing this should not be underestimated.
The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.		While other forms of communication are important, for many things, face to face is by far the best.
Working software is the primary measure of progress.		It is better to measure progress in terms of the actual thing you are delivering, rather than other factors (like effort spent) since that’s what the customer really cares about.
Agile processes promote sustainable development. The sponsors, developers and users should be able to maintain a constant pace indefinitely.		People build solutions, and people don’t do good work when they are overworked, stressed or neglecting other parts of their life. Good agile teams don’t rely on a hero culture.
Continuous attention to technical excellence and good design enhances agility.		Delivering quickly is not an excuse for poor engineering. In fact, good design can make it easier to add new capability quickly.
Simplicity – the art of maximising the amount of work not done – is essential.		It is easy to make things hard, big and complex. Often, it is harder, but far more valuable, to make things simple.
The best architectures, requirements and designs, emerge from self-organising teams.		A self-organising team that is fully focused on the goal will offer more relevant answers than those imposed upon them.
At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behaviour accordingly.		No team is ever perfect and the environment it operates in is never static. The best teams identify regularly the adjustments they should make in order to improve.

The agile principles are largely self-explanatory, and although seemingly heavily weighted toward software development, in the main, may be applied to other types of project or solution. They embody the values in the Manifesto and provide concrete examples about how the values should be demonstrated.

The principles describe not only how an agile team works, but how its members should think, behave and feel. Compromising these principles, perhaps when changing a process to suit a particular project, can cause inconsistency and a lack of coherent focus; this will often lead to problems at a later stage.

AGILE APPROACHES

As discussed earlier, several ‘lightweight’ development processes existed at the time the Agile Manifesto was created; additional such methods have emerged since. They are now described collectively as ‘agile’ methods. Chapter 5 will describe some of the more popular methods in detail. However, in overview, the key methods are:

There are many other agile methods in use today. This includes hybrid methods such as ScrumBan and numerous in-house customisations that individual companies have developed.

AGILE PRACTICES

Given the popularity and widespread adoption of agile, and the profusion of different approaches, it is not surprising that there are numerous agile practices and techniques. The Agile Alliance maintains a guide to these agile practices on their website. Some of the most commonly used practices are listed in Table 2.3 below and are explored throughout this book.

 

Table 2.3 Agile work practices

Requirements practices		• Backlog: a prioritised list of requirements or work items that is frequently updated • Definition of done/definition of ready: setting acceptance criteria for a requirement • Personas: a way of identifying and describing users of the system • User stories: a way of capturing requirements • Story mapping • Story splitting: breaking down stories that are too big • 3Cs: a way of structuring user stories: Card, Conversation, Confirmations
Estimation practices		• Planning Poker • Point estimates • Relative estimation • Velocity: a way of predicting how much work the team can do
Team leadership and organisation practices		• Iterations/timeboxing • Daily meeting (Scrum) • Burndown chart: a measure of progress in this iteration • Task board/Kanban board • Retrospective: a review meeting to identify things to change in the next iteration • Scrum of scrums: a way to manage multiple teams working on related projects
Software development practices		• Pair programming • Test driven development (TDD) • Behaviour driven development (BDD) • Refactoring
Testing and release practices		• Automated build • Continuous integration • Version control

CONCLUSION

This chapter has discussed the history and development of agile, and has described the values and principles at its core. Having originated from a software development base, the Agile Manifesto and principles have a strong software focus. However, agile may be applied to a much wider context and can be beneficial if used on other types of change project such as process improvement or skill development.

REFERENCES

Anderson, D.J. (2010) Kanban: successful evolutionary change for your technology business. Washington, DC: Blue Hole Press.

Beck, K. (1999) Extreme programming explained: embrace change. Canada: Addison Wesley.

Boehm, B.W. (1988) A Spiral Model of software development and enhancement. ACM SIGSOFT Software Engineering Notes, August. Available from: http://csse.usc.edu/TECHRPTS/1988/usccse88-500/usccse88-500.pdf [6 December 2016].

Cadle, J. (ed.) (2014) Developing information systems: practical guidance for IT professionals. Swindon: BCS.

Highsmith, J.A. (2000) Adaptive software development: a collaborative approach to managing complex systems. New York: Dorset House Publishing Co. Inc.

Jacobson, I., Booch, G. and Rumbaugh, J. (1999) The unified software development process. Reading, MA: Addison Wesley.

Martin, J. (1991) Rapid application development. USA: Macmillan.

Poppendieck, M. and Poppendieck, T. (2003) Lean software development: an agile toolkit. Boston, MA: Addison Wesley.

Royce, W.W. (1970) Managing the development of large software systems, Proceedings of IEEE Wescon. Available from: www-scf.usc.edu/˜csci201/lectures/Lecture11/royce1970.pdf [6 December 2016].

FURTHER READING

Agile Alliance. Available from: www.agilealliance.org [6 December 2016].

Ambler, S. (2009) The Agile Scaling Model (ASM): adapting agile methods for complex environments, IBM Rational. Available from: www.webfinancialsolutions.com/wp-content/uploads/2011/10/Adapting-Agile-Methods-for-Complex-Environments.pdf [6 December 2016].

Ambler, S. and Lines, M. (2012) Disciplined Agile delivery. New Jersey: IBM Press, Pearson.

Hunt, A. and Thomas, D. (1999) The pragmatic programmer. Reading, MA: Addison Wesley Professional.

Interview with Jeff de Luca (2007) ‘Jeff de Luca on Feature Driven Development – interview April 2007’. Available from: www.it-agile.de/fileadmin/docs/FDD-Interview_en_final.pdf [6 December 2016].

Paul, D., Cadle, J. and Yeates, D. (2014) Business analysis: 3rd edition. Swindon: BCS.

The Agile Manifesto. Available from: http://agilemanifesto.org [9 January 2017].