5

IMPLEMENTING AGILE: DELIVERING IN AN AGILE ENVIRONMENT

5.1 CHARTER THE PROJECT AND THE TEAM

Every project needs a project charter so the project team knows why this project matters, where the team is headed and what the project objective is. However, the project charter itself may not be enough for the team. Agile teams require team norms and an understanding of how to work together. In that case, the team might need a team charter.

The chartering process helps the team learn how to work together and coalesce around the project.

At a minimum, for an agile project, the team needs the project vision or purpose and a clear set of working agreements. An agile project charter answers these questions:

• Why are we doing this project? This is the project vision.
• Who benefits and how? This may be part of the project vision and/or project purpose.
• What does done mean for the project? These are the project's release criteria.
• How are we going to work together? This explains the intended flow of work.

A servant leader may facilitate the chartering process. A team can coalesce by working together, and the project charter is a great way to start working. In addition, team members may want to collaborate to understand how they will work together.

Teams do not need a formal process for chartering as long as the teams understand how to work together. Some teams benefit from a team chartering process. Here are some chartering ideas for team members to use as a basis for their social contract:

• Team values, such as sustainable pace and core hours;
• Working agreements, such as what “ready” means so the team can take in work; what “done” means so the team can judge completeness consistently; respecting the timebox; or the use of work-in-process limits;
• Ground rules, such as one person talking in a meeting; and
• Group norms, such as how the team treats meeting times.

The servant leader together with the team may decide to address other behaviors.

Remember that the team's social contract—its team charter—is how the team members interact with each other. The goal of the team charter is to create an agile environment in which team members can work to the best of their ability as a team.

5.2 COMMON AGILE PRACTICES

Sections 5.2.1 through 5.2.8 describe a few of the most common agile project practices.

5.2.1 RETROSPECTIVES

The single most important practice is the retrospective because it allows the team to learn about, improve, and adapt its process.

Retrospectives help the team learn from its previous work on the product and its process. One of the principles behind the Agile Manifesto is: “At regular intervals, the team reflects on how to become more effective, then tunes and adjusts its behavior accordingly.”

Many teams use iterations—especially 2-week iterations—because the iteration prompts a demonstration and a retrospective at the end. However, the team does not need iterations in order to retrospect. Team members may decide to retrospect at these key times:

• When the team completes a release or ships something. It does not have to be a monumental increment. It can be any release, no matter how small.
• When more than a few weeks have passed since the previous retrospective.
• When the team appears to be stuck and completed work is not flowing through the team.
• When the team reaches any other milestone.

Teams benefit from allocating enough time to learn, either from an interim retrospective or an end-of-the-project retrospective. Teams need to learn about their work product and work process. For example, some teams have trouble finishing work. When teams plan enough time, they can structure their retrospective to gather data, process that data, and decide what to try later as an experiment.

First and foremost, a retrospective is not about blame; the retrospective is a time for the team to learn from previous work and make small improvements.

The retrospective is about looking at the qualitative (people's feelings) and quantitative (measurements) data, then using that data to find root causes, designing countermeasures, and developing action plans. The project team may end up with many action items to remove impediments.

Consider limiting the number of action items to the team's capacity to address improvement in the upcoming iteration or work period. Trying to improve too many things at once and not finishing any of them is much worse than planning to complete fewer items and successfully completing all of them. Then, when time allows, the team can work on the next improvement opportunity on the list. When the team selects the improvements, decide how to measure the outcomes. Then, in the next time period, measure the outcomes to validate success or failure of each improvement.

A facilitator from the team leads them through an activity to rank the importance of each improvement item. Once the improvement items are ranked by the team, the team chooses the appropriate number to work on for the next iteration (or adds work to the flow if flow-based).

5.2.2 BACKLOG PREPARATION

The backlog is the ordered list of all the work, presented in story form, for a team. There is no need to create all of the stories for the entire project before the work starts—only enough to understand the first release in broad brushstrokes and then sufficient items for the next iteration.

Product owners (or a product owner value team that includes the product manager and all relevant product owners for that area of the product,) might produce a product roadmap to show the anticipated sequence of deliverables over time. The product owner replans the roadmap based on what the team produces. (See Appendix X3 on Agile Suitability Filter Tools for examples of roadmaps.)

5.2.3 BACKLOG REFINEMENT

In iteration-based agile, the product owner often works with the team to prepare some stories for the upcoming iteration during one or more sessions in the middle of the iteration. The purpose of these meetings is to refine enough stories so the team understands what the stories are and how large the stories are in relation to each other.

There is no consensus on how long the refinement should be. There is a continuum of:

• Just-in-time refinement for flow-based agile. The team takes the next card off the to-do column and discusses it.
• Many iteration-based agile teams use a timeboxed 1-hour discussion midway through a 2-week iteration. (The team selects an iteration duration that provides them frequent-enough feedback.)
• Multiple refinement discussions for iteration-based agile teams. Teams can use this when they are new to the product, the product area, or the problem domain.

TIP

Consider using impact mapping to see how the product fits together. Under normal circumstances, the product owner leads this work. A servant leader can facilitate any necessary meetings as a way of serving the project.

Refinement meetings allow the product owner to present story ideas to the team and for the team to learn about the potential challenges or problems in the stories. If the product owner is unsure of the dependencies, the product owner can request the team to spike the feature in order to understand the risks.

There are many ways for the product owner to conduct backlog preparation and refinement meetings, including for example:

• Encourage the team to work as triads of developer, tester, business analyst/product owner to discuss and write the story.
• Present the overall story concept to the team. The team discusses and refines it into as many stories as required.
• Work with the team to find various ways to explore and write the stories together, making sure all of the stories are small enough so the team can produce a steady flow of completed work. Consider becoming able to complete a story at least once a day.

Teams often have a goal of spending not more than 1 hour per week refining stories for the next batch of work. Teams want to maximize the time spent doing work as opposed to planning work. If the team needs to spend more than 1 hour per week refining stories, the product owner could be overpreparing, or the team may be lacking some critical skills needed to evaluate and refine the work.

5.2.4 DAILY STANDUPS

Teams use standups to microcommit to each other, uncover problems, and ensure the work flows smoothly through the team.

Timebox the standup to no longer than 15 minutes. The team “walks” the Kanban or task board in some way, and anyone from the team can facilitate the standup.

In iteration-based agile, everyone answers the following questions in a round-robin fashion:

• What did I complete since the last standup?
• What am I planning to complete between now and the next standup?
• What are my impediments (or risks or problems)?

Questions like these generate answers that allow the team to self-organize and hold each other accountable for completing the work they committed to the day before and throughout the iteration.

Flow-based agile has a different approach to standups, focusing on the team's throughput. The team assesses the board from right to left. The questions are:

• What do we need to do to advance this piece of work?
• Is anyone working on anything that is not on the board?
• What do we need to finish as a team?
• Are there any bottlenecks or blockers to the flow of work?

One of the antipatterns typically seen in standups is they become status meetings. Teams who have traditionally worked in a predictive environment may tend to fall into this antipattern since they are used to providing a status.

Another antipattern typically seen in standups is that the team begins to solve problems as they become apparent. Standups are for realizing there are problems—not for solving them. Add the issues to a parking lot, and then create another meeting, which might be right after the standup, and solve problems there.

Teams run their own standups. When run well, standups can be very useful, provided the nature of the team's work requires intense collaboration. Make a conscious decision about when the team needs, or can effectively use, standups.

TIP

Encourage any team member to facilitate the standup instead of a project manager or leader to ensure it does not turn into a status meeting, but instead is used as a time for the team to self-organize and make commitments to each other.

5.2.5 DEMONSTRATIONS/REVIEWS

As the team completes the features usually in the form of user stories, the team periodically demonstrates the working product. The product owner sees the demonstration and accepts or declines stories.

In iteration-based agile, the team demonstrates all completed work items at the end of the iteration. In flow-based agile, the team demonstrates completed work when it is time to do so, usually when enough features have accumulated into a set that is coherent. Teams, including the product owner, need feedback to decide how early to ask for product feedback.

As a general guideline, demonstrate whatever the team has as a working product at least once every 2 weeks. That frequency is enough for most teams, so team members can get feedback that prevents them from heading in a wrong direction. That is also frequent enough so that the teams can keep the product development clean enough to build a complete product as often as they want or need to.

A fundamental part of what makes a project agile is the frequent delivery of a working product. A team that does not demonstrate or release cannot learn fast enough and is likely not adopting agile techniques. The team may require additional coaching to enable frequent delivery.

5.2.6 PLANNING FOR ITERATION-BASED AGILE

Each team's capacity is different. Each product owner's typical story size is different. Teams consider their story size so they do not try to commit to more stories than there is team capacity to complete within one iteration.

When people are unavailable (e.g., holidays, vacations, or anything that prevents people from participating in the next set of work), the product owner understands that the team has reduced capacity. The team will not be able to finish the same amount of work as it finished in the previous time period. When teams have a reduced capacity, they will only plan for work that meets that capacity.

Teams estimate what they can complete, which is a measure of capacity (see Section 4.10 for examples). Teams cannot predict with 100% certainty what they can deliver, as they cannot know the unexpected. When product owners make the stories smaller and teams see progress in the form of a finished product, teams learn what they are able to do for the future.

Agile teams do not plan just once in one single chunk. Instead, agile teams plan a little, deliver, learn, and then replan a little more in an ongoing cycle.

TIP

Draw the team's attention to the antipattern and help the team to discover how to improve its standups.

5.2.7 EXECUTION PRACTICES THAT HELP TEAMS DELIVER VALUE

If the team does not pay attention to quality, it will soon become impossible to release anything rapidly.

The following technical practices, many of which come from eXtreme Programming, may help the team to deliver at their maximum speed:

• Continuous integration. Perform frequent incorporation of work into the whole, no matter the product, and then retest to determine that the entire product still works as intended.
• Test at all levels. Employ system-level testing for end-to-end information and unit testing for the building blocks. In between, understand if there is a need for integration testing and where. Teams find smoke testing helpful as a first look at whether the work product is any good. Teams have found that deciding when to run regression tests and which ones helps them maintain product quality with good build performance. Agile teams have a strong preference for automated tests so they can build and maintain a momentum of delivery.
• Acceptance Test-Driven Development (ATDD). In ATDD, the entire team gets together and discusses the acceptance criteria for a work product. Then the team creates the tests, which allows the team to write just enough code and automated tests to meet the criteria. For non-software projects, consider how to test the work as the team completes chunks of value.
• Test-Driven Development (TDD) and Behavior-Driven Development (BDD). Writing automated tests before writing/creating the product actually helps people design and mistake-proof the product. For non-software projects, consider how to “test-drive” the team's designs. Hardware and mechanical projects often use simulations for interim tests of their designs.
• Spikes (timeboxed research or experiments). Spikes are useful for learning and may be used in circumstances such as: estimation, acceptance criteria definition, and understanding the flow of a user's action through the product. Spikes are helpful when the team needs to learn some critical technical or functional element.

5.2.8 HOW ITERATIONS AND INCREMENTS HELP DELIVER WORKING PRODUCT

Iterations help a team create a cadence for delivery and many kinds of feedback. Teams produce increments of value for delivery and feedback. The first part of this delivery is a demonstration. Teams receive feedback about how the product looks and operates through a demo. Team members retrospect to see how they can inspect and adapt their process to succeed.

Demonstrations or reviews are a necessary part of the agile project flow. Schedule the demonstration as appropriate for the team's delivery cadence.

5.3 TROUBLESHOOTING AGILE PROJECT CHALLENGES

Agile approaches were born out of the need to solve issues associated with high rates of change, uncertainty, and complexity on projects. Due to these origins, they contain a variety of tools and techniques for dealing with issues that present problems in predictive approaches. Refer to Table 5-1.

TIP

Teams should demo often for feedback and to show progress. Encourage the PMO and other interested parties to watch demonstrations so the people deciding on the project portfolio can see the actual progress.

Table 5-1. Agile Pain Points and Troubleshooting Possibilities

Pain Point	Troubleshooting Possibilities
Unclear purpose or mission for the team	Agile chartering for purpose–vision, mission, and mission tests
Unclear working agreements for the team	Agile chartering for alignment–values, principles, and working agreements
Unclear team context	Agile chartering for context–boundaries, committed assets, and prospective analysis
Unclear requirements	Help sponsors and stakeholders craft a product vision. Consider building a product roadmap using specification by example, user story mapping, and impact mapping. Bring the team and product owner together to clarify the expectations and value of a requirement. Progressively decompose roadmap into backlog of smaller, concrete requirements.
Poor user experience	User experience design practices included in the development team involve users early and often.
Inaccurate estimation	Reduce story size by splitting stories. Use relative estimation with the entire team to estimate. Consider agile modeling or spiking to understand what the story is.
Unclear work assignments or work progress	Help the team learn that they self-manage their work. Consider kanban boards to see the flow of work. Consider a daily standup to walk the board and see what work is where.
Team struggles with obstacles	A servant leader can help clear these obstacles. If the team doesn't know the options they have, consider a coach. Sometimes, the team needs to escalate stories the team or servant leader has not been able to remove.
Work delays/overruns due to Insufficiently refined product backlog items	Product owner and team workshop stories together. Create a definition of ready for the stories. Consider splitting stories to use smaller stories.
Defects	Consider the technical practices that work for the environment. Some possibilities are: pair work, collective product ownership, pervasive testing (test-driven and automated testing approaches) and a robust definition of done.
Work is not complete	Team defines definition of done for stories including acceptance criteria. Also add release criteria for projects.
Technical debt (degraded code quality)	Refactoring, agile modeling, pervasive testing, automated code quality analysis, definition of done
Too much product complexity	For software and non-software encourage the team always to be thinking “What is the simplest thing that would work?” and apply the agile principle of “Simplicity--the art of maximizing the amount of work not done”. These help reduce complexity.
Slow or no improvement in the teamwork process	Capture no more than three items to improve at each retrospective. Ask the servant leader to help the team learn how to integrate those items.
Too much upfront work leading to rework	Instead of much upfront work, consider team spikes to learn. In addition, measure the WIP during the beginning of the project and see what the team's options are to deliver value instead of designs. Shorten iterations and create a robust definition of done.
False starts, wasted efforts	Ask the product owner to become an integral part of the team.
Inefficiently ordered product backlog items	Rank with value including cost of delay divided by duration (CD3) and other value models
Rush/wait uneven flow of work	Plan to the team's capacity and not more. Ask people to stop multitasking and be dedicated to one team. Ask the team to work as pairs, a swarm, or mob to even out the capabilities across the entire team.
Impossible stakeholder demands	Servant leadership to work with this stakeholder (and possibly product owner).
Unexpected or unforeseen delays	Ask the team to check in more often, use kanban boards to see the flow of work and work in progress limits to understand the impact of the demands on the team or product. Also track impediments and impediment removal on an impediment board.
Siloed teams, instead of cross-functional teams	Ask the people who are part of projects to self-organize as cross-functional teams. Use servant leadership skills to help the managers understand why agile needs cross-functional teams.

5.4 MEASUREMENTS IN AGILE PROJECTS

Transitioning to agile means using different measurements. Using agile means looking at new metrics that matter to the team and to management. These metrics matter because they focus on customer value.

One problem with status reporting is the team's ability to predict completion or to use traffic light status to describe the project. For instance, project leaders describe the project as “90% done.” At that point the team tries to integrate the pieces into a product. The team discovers missing requirements or surprises, or finds that the product doesn't integrate the way they thought it would.

The project is only partway done, and the traffic light status reporting does not reflect the real state. Too often, the project team realizes it will need just as much time to complete the remainder of the project. For too many projects, the team realizes they are—at most—10% done because of issues the team discovered.

The problem with predictive measurements is that they often do not reflect reality. It often happens that a project status light is green up until 1 month before the release date; this is sometimes referred to as a watermelon project (green on the outside, red on the inside). Oftentimes project status lights turn red with seemingly no warnings, because there is no empirical data about the project until 1 month before the release date.

Metrics for agile projects contain meaningful information that provide a historical track record, because agile projects deliver value (finished work) on a regular basis. Project teams can use such data for improved forecasts and decision making.

Surrogate measurements such as percent done are less useful than empirical measurements such as finished features. See Section 4.10 for more information on value management. Agile helps teams see problems and issues so the team can diagnose and address them.

In addition to quantitative measures, the team can consider collecting qualitative measures. Some of these qualitative measures focus on practices the team has chosen and assess how well the team uses those practices, for example, the business satisfaction with delivered features, the morale of the team; and anything else the team wants to track as a qualitative measure.

5.4.1 AGILE TEAMS MEASURE RESULTS

Agile favors empirical and value-based measurements instead of predictive measurements. Agile measures what the team delivers, not what the team predicts it will deliver.

A team that is accustomed to having project baselines and estimates of earned value and ROI might be puzzled about working on a project and not managing to a baseline. Agile is based on working products of demonstrable value to customers.

Baselines are often an artifact of attempted prediction. In agile, the team limits its estimation to the next few weeks at most. In agile, if there is low variability in the team's work and if the team members are not multitasking, the team's capacity can become stable. This allows better prediction for the next couple of weeks.

After the team completes work in iteration or flow, the team can replan. Agile does not create the ability to do more work. However, there is evidence that the smaller the chunk of work, the more likely people are to deliver it.

Software product development, like other knowledge work, is about learning—learning while delivering value. Hardware development and mechanical development are similar in the design parts of the project. Learning takes place by experimenting, delivering small increments of value, and getting feedback on what has been accomplished thus far. Many other product development projects incorporate learning also.

Because learning is such a large part of the project, the team needs to balance uncertainty and provide value to the customers. The team plans the next small part of the project. The team reports empirical data and replans further small increments to manage the project uncertainty.

Some iteration-based projects use burndown charts to see where the project is going over time. Figure 5-1 shows an example of a burndown chart where the team planned to deliver 37 story points. Story points rate the relative work, risk, and complexity of a requirement or story. Many agile teams use story points to estimate effort. The dotted burndown line is the plan. In Figure 5-1, the team can see by Day 3 that they are at risk for that delivery.

Some project teams prefer burnup charts. The same data used in Figure 5-1 is shown in Figure 5-2 in a burnup chart.

Burnup charts show the work completed. The two charts in Figures 5-1 and 5-2 are based on the same data, but displayed in two different ways. Teams may prefer how to see their data.

When a team sees what it has not yet completed as it works through an iteration, the team may become dispirited and possibly rush to complete the work without meeting the acceptance criteria. However, the team could have any number of good reasons for not completing work as it expected. Burndowns show the effect of team members multitasking, stories that are too large, or team members out of the office.

Especially with teams new to agile, the burnup will show changes in scope during the iteration. Burnups allow teams to see what they have accomplished, which helps the team proceed to the next piece of work.

Whether teams use burndown or burnup charts, they see what they have completed as the iteration progresses. At the end of the iteration, they might base their next measure of capacity (how many stories or story points) on what they completed in this iteration. That allows the product owner along with the team to replan what the team is more likely to succeed in delivering in the next iteration.

Velocity, the sum of the story point sizes for the features actually completed in this iteration, allows the team to plan its next capacity more accurately by looking at its historical performance.

Flow-based agile teams use different measurements: lead time (the total time it takes to deliver an item, measured from the time it is added to the board to the moment it is completed), cycle time (the time required to process an item), and response time (the time that an item waits until work starts). Teams measure cycle time to see bottlenecks and delays, not necessarily inside the team.

TIP

Teams might discover it can take four to eight iterations to achieve a stable velocity. The teams need the feedback from each iteration to learn about how they work and how to improve.

Lead time is useful to understand cycle time from the first look at a particular feature to the length of time it took to release it to the customer. The work in progress (WIP) limits at the top of each column, shown in boxes here, allows the team to see how to pull work across the board. When the team has met its WIP limits, the team cannot pull work from the left into the next column. Instead, the team works from the right-most full column and asks, “What do we do as a team to move this work into the next column?”

Each feature is unique, so its cycle time is unique. However, a product owner might notice that smaller features have smaller cycle times. The product owner wants to see throughput, so the product owner creates smaller features or works with the team to do so.

Burnups, burndowns (capacity measures) and lead time, and cycle time (predictability measures) are useful for in-the-moment measurements. They help a team understand how much more work they have and whether the team might finish on time.

Measuring story points is not the same as measuring completed stories or features. Some teams attempt to measure story points without completing the actual feature or story. When teams measure only story points, they measure capacity, not finished work, which violates the principle of “the primary measure of progress is working software” (or, other product if not software).

Each team has its own capacity. When a team uses story points, be aware that the number of story points a team can complete in a given time is unique to that team.

TIP

When teams depend on external people or groups, measure cycle time to see how long it takes for the team to complete the work. Measure lead time to see the external dependencies after the team completes its work. Teams can also measure the reaction time, the time from ready to the first column, to see how long it takes them—on average—to respond to new requests.

When teams provide their own units of measure, teams are better able to assess and estimate and deliver their work. The downside of relative estimation is that there is no way to compare teams or add velocity across teams.

The team can measure completed work in a feature burnup/burndown chart and in a product backlog burnup chart. These charts provide trends of completion over time, as shown in Figure 5-4.

Feature burnup/burndown charts may show that requirements grew during the project. The features complete line shows that the team completes features at a regular pace. The total features line shows how the project's total features changed over time. The features remaining burndown line shows that the rate of feature completion varies. Every time features are added to the project, the burndown line changes.

Earned value in agile is based on finished features, as shown in Figure 5-5. The product backlog burnup chart shows completed work compared to total expected work at interval milestones or iterations.

A team can only finish one story at a time. To complete a large feature that contains several stories, the team will have remaining stories to complete and may not complete that entire feature until several more time periods have passed. The team can show its completed value with a product backlog burnup chart as shown in Figure 5-5.

If a team needs to measure earned value, it can consider using this burnup chart in Figure 5-6 as an example: Note that the left Y axis represents story points as scope, and the right Y axis represents the project spend.

Traditional EVM metrics like schedule performance index (SPI) and cost performance index (CPI) can be easily translated into agile terms. For example, if the team planned to complete 30 story points in an iteration, but only completed 25 then the SPI is 25/30 or 0.83 (the team is working at only 83% of the rate planned). Likewise, CPI is the earned value (completed features value) to date divided by the actual costs to date or, as shown in Figure 5-6, $2.2M / $2.8M = 0.79. This means a result of only 79 cents on the dollar compared to plan (but of course this assumes that the prediction is still correct.)

A cumulative flow diagram, illustrated in Figure 5-7, shows the work in progress across a board. If a team has many stories waiting for test, the testing band will swell. Work accumulation can be seen at a glance.

Teams have trouble with accumulating work: the team has work in progress instead of work completed. When teams have a lot of work in progress, they delay their overall feature delivery. The longer it takes for a team to deliver, the more pressure a team will have for yet more features in the same period of time.

Adapt this cumulative flow to the project task board.