Chapter 3. Understanding the Different Types of Agile

This chapter describes some of the more widely used Agile tools that are available, describes the similarities and differences between Agile methodologies, and illustrates how Cayman Design has implemented these techniques. Table 3.1 on the following page provides an overview.

How do you know when a software development organization has become Agile? Is there such a thing as a completely Agile organization? An Agile organization is quite simply one that is adaptive to change. This could be the software development arm of a company or any other area, such as marketing or finance. These areas have put policies and tools in place that enable them to be highly reactive to issues and opportunities as they arise.

An Agile software development organization is one that uses Agile tools, techniques, and cultural norms to develop their products. Becoming an Agile organization is more of a journey than a destination. Most organizations use a combination of Agile tools and methodologies that best support their goals and objectives. To date, there is no certification process that an organization can undergo to become an official Agile-certified software development organization. There are, however, Agile certifications that individual team members can pursue to distinguish them as Agile experts, such as the Scrum Master certification and the Agile Certified Professional from the Project Management Institute. More details on such certifications are covered at the end of this chapter.

Techniques and principles from Extreme Programming are now some of the most popular in Agile software development and focus on the following key themes:

• Frequent releases, short development cycles

• Pair programming

• Regular builds and integration tests

• Quality and avoiding code breakage

• Simplicity of code, coding only what is needed

• Rapid and regular feedback

The process also relies heavily on an automated unit-testing strategy known as test-driven development (TDD). When practicing TDD, developers write automated unit tests before writing the corresponding production code; code is not considered complete until all automated tests pass. Test-driven development is discussed in more detail in Chapter 8, “Tracking and Reporting.”

There are some basic guidelines that programmers should follow when they are pairing. The first is that the development space should accommodate two developers working side by side. The developers share a keyboard and either use dual screens or share a large screen. Both developers should be active participants in the creation of the code, and the pairing should end when one or both developers feel tired and unable to concentrate.

Despite Scrum’s overwhelming popularity with companies that decide to adopt Agile methodologies, many are still uncertain about what it entails. Many consider XP to have a technical or developer focus, but Scrum tends to address more of the project management and teaming aspects of software development.

Scrum emphasizes the importance of organizing a project into specific durations, known as time boxes, that help the team know what they need to focus on each day and encourage a sense of urgency. Work requirements are grouped into chunks of work that can be completed in one- to six-week time frames called sprints. All design, development, test, and customer validation work is contained within the sprint; the goal is that by the end of the sprint, the new functionality is ready to deliver to customers.

The product owner selects work requirements for each sprint. The product owner maintains a prioritized list of requirements for the product called a product backlog. The requirements are written in the form of user stories or stories about the problem that needs to be solved by the requirement. User stories are covered in more detail in Chapter 5, “The New Way to Collect and Document Requirements.”

Tracking is also an important part of the Scrum approach. Scrum teams meet daily in what is commonly referred to as a daily stand-up or daily Scrum meeting (see Figure 3.3). A Scrum master is assigned to lead these meetings to ensure that they remain brief and focused and that all team members have a chance to contribute. These meetings are designed to be short (usually around 15 minutes), and each team member answers the following questions:

• What have I done since yesterday?

• What am I doing today?

• Any roadblocks?

The team also uses burn down or burn up charts to track progress through the sprint. More information on tracking is covered in Chapter 7, “Testing, Quality, and Integration.”

The diagram in Figure 3.4 depicts the general flow of a project using the Scrum methodology.

The future chapters in this book emphasize the Scrum process, because it is the most widely adopted Agile methodology.

The success of this initial use of FDD encouraged De Luca to document the best practices of his approach and share them with other software development professionals. In 1999, he began by teaming up with Peter Coad and Eric Lefebrve on their book, Java Modeling in Color with UML, and dedicated a chapter to describing the FDD methodology. He has also published extensive information on his web site (http://www.nebulon.com/fdd/index.html). A few years later, in 2002, Stephen Palmer wrote a more comprehensive book on FDD with Mac Felsing entitled A Practical Guide to Feature-Driven Development.

FDD has been adopted by the Agile Alliance as a recognized Agile methodology. Product features are developed in two-week iterations and focus on the following activities:

1. Create the model—Perform an initial customer walk-through that determines scope and context and creates the model.

2. Develop the feature list—The model determines the features. Features are often described as a sentence in a format that looks something like this:

<verb> the <something> for/of/to a(n) <something else>.

<verb> the “<result>” by|for|of|to a(n) <object>

For example, for a movie review site, the features might say:

a. Calculate the average star rating for a movie

b. Calculate the “go” or “don’t go” recommendations for a movie

c. List the local movie theaters to show the current movie locations <calculate> the <average star rating> for a <movie>

Features are then ordered and assigned to classes.

1. The plan is determined by features—Unlike other Agile methodologies, FDD advocates that teams create a development plan that is based on ordering features and assigning them to classes. Classes are then assigned to an owning developer.

2. The design is based on the features—The owning developer assigns a small set of features that can be developed in two weeks. Detailed sequence diagrams are created based on features, and the overall model is updated to reflect the changes. Classes are written, and a design review is held.

3. The build is based on the features—Code is developed based on feedback from the design review. Once the developer completes the code and unit testing, then a code review is held to analyze whether final changes are needed. Final updates to the code are made and the code promoted to the current build.

DSDM is part of the Agile Alliance and focuses on the project management aspects of Agile software development and non-IT projects; it is often considered a complementary approach to Extreme Programming. DSDM relies on nine core principles, all of which must be followed to ensure the best chances of a successful project:

1. Active user involvement is critical for any project.

2. Teams must be empowered to make decisions.

3. Teams need to focus on frequent product delivery.

4. Criteria must be in place for an approved deliverable.

5. All development must be done iteratively and delivered incrementally.

6. Any change introduced during development must be reversible.

7. Product requirements are baselined as high-level themes rather than low-level changes or improvements.

8. Product testing is performed throughout the project life cycle.

9. Team interactions are collaborative and cooperative.

In addition to the nine principles, four major phases are used to iterate through during the project, including

1a. Feasibility Study—The project team identifies the functionality that will be included, creates and tests a functional prototype, and develops a timeline for the project.

1b. Business Study—The business processes and associated requirements are analyzed by the project stakeholders using the MoSCoW requirement prioritization method. MoSCoW prioritizes requirements based on what the product “must have,” “should have,” “could have,” and “won’t have.” Schedules are developed using the time boxing method to create and schedule and ensure quality can be maintained throughout the project.

2. Functional Model Iteration—Requirements from the business study are translated into functional models that are tested and reviewed by user groups. The project team iterates on the functional models with users until there is agreement that the models are ready to proceed to a design prototype.

3. Design and Build Iteration—All of the functional models are brought together to create a design prototype; this prototype incorporates all product functions and again is tested and reviewed by user groups. The result is a working prototype that includes user documentation.

4. Implementation—The final phase includes user acceptance of the product and implementation in their environment. Users are trained, and the project team reviews the business objectives to ensure the user goals have been achieved.

DSDM relies heavily on time boxing, prototyping, and the MoSCoW rules during the project phases. These tools are discussed in more detail in later chapters. Visit http://www.dsdm.org/ for the latest DSDM framework and best practices.

In 2003, Mary Poppendieck and Tom Poppendieck penned the first book on this topic, entitled Lean Software Development: An Agile Toolkit. Their book describes the seven principles that guide Lean software development; these principles are close derivatives of the Lean manufacturing principles. They offer an additional 22 tools that software developers can use, and illustrate how the Lean principles are applicable to Agile development environments.

Following are the seven principles of Lean software development:

• Eliminate waste—Remove anything that does not add customer value

• Amplify learning—Regularly test code, learn by doing rather than documenting or planning, gain customer feedback through prototypes

• Decide as late as possible—Wait to get facts rather than making assumptions

• Deliver as fast as possible—Focus on speed to market

• Empower the team—Managers allow team members to make key decisions

• Build integrity in—Pieces work together as a whole

• See the whole—Focus on the entire product experience rather than the pieces

Lean principles and tools are covered in more detail in Chapter 5.

The visual signboard and work-in-progress pull method, also known as the Kanban system, is the most popular tool that organizations adopt when they use the Kanban method. The Kanban system is covered in more detail in Chapter 8.

The Kanban method is based on four key principles:

• Foster leadership at all levels of the organization—Everyone in the organization is encouraged to act as leaders, from the entry-level employee to the executive team.

• Start with what you know—Kanban is not prescriptive in nature and assumes that organizations are not all the same. Instead, it is important to understand where you are today and use change management approaches to evolve from that point.

• Focus on incremental and evolutionary change—The goal is not to overhaul the process, culture, and product overnight. Instead, it is important to make small but impactful changes often.

• Respect current methodologies and roles—The current process was put there for a reason; something about it has worked, and that is why people have continued to use it. It is important to preserve what has worked and to change what is no longer helping the team achieve their goals.

Six core practices are central to the Kanban method:

• Visualize workflow—Teams need to visually understand their work so that they can optimize their workflow. The Kanban method often uses a board with columns and cards to show the workflow (see Figure 3.5).

• Limit work in progress, and pull in new work only when time becomes available—The team cannot be efficient and focused if too much work is on their plate; it is important to focus on one task at a time and not to start a new task until the current task is completed.

• Manage the flow through the system—Just as with a relay race, the team needs to understand how the work progresses toward the goal. The team needs to ensure that the appropriate level of work is flowing through the system and that bottlenecks are being kept to a minimum.

• Provide explicit policies—The team needs to understand how to work within their organization in order to eliminate ambiguity. For example, a team may enact the policy that they do not pull a new feature until the current feature has been through unit test.

• Improve collaboratively and evolve experimentally—A team can best improve if members first have a shared vision and work together to generate ideas for improvement. Evolution cannot happen through guesswork or intuition; the team needs to try new approaches and objectively evaluate their merit.

• Ensure feedback is part of the process—Both positive and negative feedback are essential in understanding how the organization needs to change and evolve.

One important difference of Kanban from other key approaches such as XP or Scrum is that it does not rely on iterations. Kanban allows the team to focus by limiting the work that is in progress and advocates a continuous flow of work. Kanban does not prohibit the use of iterations or time boxes, and it is common for teams to use these tools with Kanban, but specific time frames are not considered a necessary element of this approach.

Cockburn used the term “crystal” for the methodology because no two crystals are exactly the same, based on their unique colors and hardness; the analogy is that no two projects are the same. Also, a color (x) continuum is used to select the number of people needed for the project team and the project “heaviness” that is required: The darker the color, which ranges from clear (quartz) as the lightest to the deepest (sapphire) as the heaviest, the more team members and heavier process would be needed for the project. For example, you would expect a ruby project to have significantly more team members than a quartz project, and it would also require a heavier process (more communication, documentation, coordination). The y continuum signifies the hardness of the project: The higher the number, the more hardness or robustness of the process required. In Crystal, the hardness represents the severity of damage if the system fails to operate properly. Figure 3.6 shows how a project can be evaluated for color (number of team members) and hardness (criticality).

For example, if an organization were working on software that shows a patient’s heart rate, then there would be dire consequences if the program failed to show a spiking heart rate and did not warn doctors of an impending heart attack; a project with such serious consequences would rate high on the hardness (criticality) scale. This project would have a low tolerance for error because it is life critical (L), but may not require a large team to develop the product, rating it lower on the color (people involved) scale, in the 20-person range. Given these criteria points, it would fall in the L20 (orange crystal) methodology range. An orange crystal project would likely involve a lot of face-to-face communication, but it would not be as formal because it would not require the formalities of a 500-person team. The fact that it rates high on criticality would introduce the need for extensive testing, more documentation, and close collaboration with the target users.

Rather than give a specific set of guidelines for each color, Cockburn uses case studies to illustrate common approaches that are used for a specific color. There are, however, specific values and rules that are applicable to the entire Crystal Family:

• Values

• High tolerance

• Focus on people and communication

• Rules

• Development must be incremental and last less than four months.

• The team must conduct reflection sessions before and after the project. Holding a mid-project reflection meeting is also encouraged.

Examples of Crystal projects can be found in Cockburn’s 2004 book, Crystal Clear, and also in Agile Software Development: A Cooperative Game, which was published in 2001 and revised in 2006.

The team returned from the course excited to try some new techniques and recommended that they start with three of the more common tools from Extreme Programming, Scrum, and Kanban. The Cayman Design team brought in an Agile consultant to teach the team the basics, and they got started by reorganizing their work environment to implement these techniques. The team installed Kanban white boards on the walls and arranged pairing stations for team members to use during the workday, and they assigned team members to Scrum teams and developed a code repository that could create daily code builds.

• Extreme Programming is an Agile approach that encourages rapid development, high-quality code, and close interaction with users. Paired programming is a widely used technique where team members code together to offer real-time code reviews and cross-training.

• The Scrum methodology focuses on teaming, organizing, and prioritizing the work.

• Feature-Driven Development focuses on the specific features that address customer requirements during the planning and development phase. This method emphasizes timely delivery of working software and strong internal and external communication about the project.

• Dynamic Systems Development Method is a project delivery framework that uses a structured approach to rapid development. A collection of nine principles and four major project phases guides this approach that is often considered complementary to Extreme Programming.

• Lean development derives from the idea of Lean manufacturing. This method encourages teams to eliminate wasteful activities, create a learning environment, and take a holistic view of the product.

• Kanban method uses a pull approach to keep a continual flow of work through the system. The Kanban board is a popular tool that helps teams organize their work into a flowing and visible system.

• Crystal Family advocates that when a team is using an iterative process, the amount of structure should be tailored to the dynamics of the team. It favors choosing the best method for the situation rather than a “one size fits all” approach and uses a color and hardness scale to determine the appropriate amount of structure.

• A variety of certifications are available for a general practitioner (e.g., PMI-ACP); for specific methods, such as a Certified Kanban Practitioner; and for specific roles on Agile teams, such as a Certified Scrum Master (CSM).

Kristin and Sondra: Has anything surprised you about the adoption of Agile methodologies in general? Have you experienced any unexpected responses specific to the Crystal Family?

Alistair: Probably what surprised me is not what you might expect. People generally think that I (we) would be surprised by how badly people adopt the Agile methodologies, how they misinterpret the message. This certainly didn’t surprise me, and I don’t think it really surprised the other authors very much. My view is that you will either be misquoted or ignored, that there isn’t a sweet spot in the middle where people understand your message and follow it.

That having been said, one thing did surprise all of us who wrote the manifesto, and that is how quickly it was picked up by the industry—literally within months. We wrote it in February of 2001, and already by summer there were panels on it and a conference already in 2002. It appeared as contractual terms in a contract between two large companies already in late 2003. This is very fast adoption, even though the PMI and IEEE waited almost ten years to pick it up.

The other surprise, to me, personally, was the adoption of reflection workshops and methodology adaptation as core to the Agile approach by the industry. At the time of the 2001 meeting, the idea of holding monthly reflect-and-improve meetings was, to my mind, a long stretch for most teams. The methodologies presented at the 2001 workshop were largely fixed in form, not as malleable as Crystal—they were “agile” but not particularly “adaptive” (for an example to see the difference, RUP was “adaptive” but not particularly “agile”). The surprise to me is that by five years later, most practitioners had associated Agile with making changes to their methodology, as a core concept and practice. This surprise was a welcome and happy surprise to me, since adaptive methodologies have been in my blood since the very beginning, back in 1991.

Kristin and Sondra: What is the most important idea you would like people to take away from the Crystal Family methodology?

Alistair: Start lighter than you think you need, focus on the quality of conversations between people, change something every month to experiment with ways to get better.

That may look like three things, but there is no part of that you can remove and get Crystal.

Kristin and Sondra: What recommendations do you have for practitioners who are new to using Agile approaches?

Alistair: Deliver to real users early and often, focus on the quality of conversations between people, change something every month to experiment with ways to get better.

Kristin and Sondra: What is your vision for Agile methodologies as we look toward the future of software development?

Alistair: I don’t know if it is a vision or an aspiration. In the 1990s, we suffered under the burden of the NATO 1968 and 1969 conferences on software engineering and all the resulting discussion, which gave us Waterfall development and heavy, impersonal, predictive methodologies. With the Agile Manifesto, we broke free from that burden. My, and other, research has shed a lot of light on what improves team results in design tasks, so we now know why Agile works and what needs adjustment.

The shortfall in Agile adoption is shoddy thinking practices and sloppy behavior and pretending that Agile makes all that okay. It doesn’t. My aspiration is that people will keep the people aspects and learn to sharpen their thinking to get even better results (I write this up these days as “Disciplined Learning”; see my web site). Then we can get the best of both worlds.

Will it happen? I don’t speculate on things like that. We’ll see.

Kristin and Sondra: Do you think there will be a point where all software development will be done using Agile methodologies, or do you think traditional methodologies such as Waterfall will always have an important role in software development?

Alistair: Waterfall is a scheduling and staging strategy in which there is only one integration and delivery at the end. There are still projects (rarely in software) where there can be only one integration and delivery at the end, and so the need to keep knowledge of how to behave in such situations will remain. There will also remain, in all likelihood, people who simply refuse to switch over, and so the topic will plague us for decades to come, as far as I can see.

Astels, David, Miller, Granville, and Novak, Miroslav. (2002). A practical guide to Extreme Programming. Upper Saddle River, NJ: Prentice Hall.

Beck, Kent. (2000). Extreme Programming explained. Boston: Addison-Wesley.

Beck, Kent, and Fowler, Martin. (2001). Planning Extreme Programming. Boston: Addison-Wesley.

Cockburn, Alistair. (2004). Crystal clear: A human-powered methodology for small teams. Boston: Addison-Wesley.

Cockburn, Alistair. (2006). Agile software development: The cooperative game (2nd ed.). Boston: Addison-Wesley.

Highsmith, James A. (1999). Adaptive software development: A collaborative approach to managing complex systems. New York: Dorset House.

Palmer, Stephen, and Felsing, Mac. (2002). A practical guide to Feature-Driven Development. Upper Saddle River, NJ: Prentice Hall.

Poppendieck, Mary, and Poppendieck, Tom. (2003). Lean software development: An Agile toolkit. Boston: Addison-Wesley.

Schwaber, Ken, and Beedle, Mike. (2002). Agile software development with Scrum. Upper Saddle River, NJ: Prentice Hall.

2. What three questions are typically covered in a daily stand-up meeting?

3. What five activities need to be performed in an FDD project?

4. What four requirement categories are used for DSDM?

5. What are the seven principles of Lean software development?

7. What are the four key principles of Kanban?

8. Why was “crystal” used to name the Crystal Family?

9. What two factors are considered when assessing a color level in Crystal Family?

10. What Agile certification is available through the Project Management Institute?