A Team Lead, Architect, and Project Manager Walk into a Bar...

Dan is a lead developer and architect at a company that makes coin-op games and kiosks. He’s worked on projects ranging from arcade pinball machines to ATMs. For the last few years, he’s been working with a team lead, Bruce. They bonded on a release of the company’s biggest product, a Vegas slot machine called the “Slot-o-matic Weekend Warrior.”

Joanna was hired a few months ago as a project manager to head up a project building software for a new line of streaming audio jukeboxes, which the company wants to introduce and sell to bars and restaurants. She was a real prize—they poached her from a competitor that already has a successful jukebox on the market. She’s been getting along really well with Dan and Bruce, and she’s excited about getting started on a new project with them.

Dan and Bruce are much less excited about the new project than Joanna. They all went out for drinks after work one day, and Bruce and Dan started explaining why the team came up with their own name for the slot machine project: the “Slog-o-matic Weekend Killer.”

She wasn’t happy to learn that in this company, failing projects are the rule, not the exception. The last three projects were declared a success by the company’s managers, but they only got out the door because Dan and Bruce worked incredibly long hours. Worse, they held their noses and took shortcuts in the code that are causing support nightmares today—like when they hastily patched up a prototype for one feature and pushed it out into production, and it later turned out to have serious performance problems because pieces of it were never built to scale up.

As they talked, Joanna recognized the pattern, and knew what was causing the problem: the company follows a particularly ineffective waterfall process. A team following a waterfall process tries to write down as early as possible a complete description of the software that will be built. Once all of the users, managers, and executives agree on exactly what the software must do (the requirements), they can hand a document that contains those requirements (the specification) to a development team, and that team will go off and build exactly what’s written. Then a testing team comes in afterward, and verifies that the software matches the document. Many agile practitioners refer to this as “big requirements up front” (sometimes abbreviated BRUF).

But Joanna also knew from her own projects over the years that theory often differed from practice, and that while some teams have a very effective waterfall process, many of them struggle with it. She was starting to think that this team might be one of those that struggled.

Figure 2-1. The waterfall model.

As they talked, Bruce and Dan confirmed a few things that reinforced her opinion. Just as Joanna suspected, there were a lot of specifications sitting in large binders and gathering dust on shelves all across the company. Somehow, everyone expected a group of users, managers, and executives to create a perfect requirements specification. In real life, the spec had a nasty habit of changing so that it would be inaccurate by the time the team got their hands on it, and would progress to being disastrously wrong by the time the team finished building the software. Bruce, Dan, and a lot of other people at the company knew that it was unreasonable to expect the perfect spec, but they still ran their projects as if it were possible.

As the evening went on, Bruce got more comfortable (and tipsy), and he brought up another problem that had been nagging him: that many teams he’d been on at their company had a lot of trouble actually building their software. Even if the users got the requirements right (which rarely happened), and even when the team understood those written requirements perfectly after reading them (which had yet to happen to this day), they often used inferior tools and had a lot of trouble with software design and architecture. This led to Bruce’s teams repeatedly building software that had a lot of bugs, and was often an unmaintainable mess.

Both of these problems led to many failed projects, especially because they considered any project where they had to work many 90-hour weeks and deliver buggy code to be a failure. Joanna explained that the biggest cause of those failures was the inability of the waterfall process the company followed to handle change. In a perfect world, the waterfall process would work just fine, because at the start of the project everyone would know exactly what they’d need at the end. That way, they could write it all down in a neat spec and hand it off to the team to build. But real-life projects never seemed to work out that way.

Dan and Bruce were now officially drunk, and deep into a marathon gripe session with Joanna. Dan told her that on almost every project that they’d worked on, the customers decided partway through that they needed the team to build something different than what they’d originally planned on. Then everyone had to go back to the beginning of the waterfall. According to the strict waterfall process the team was following, they were supposed to write entirely new specifications, come up with a different design, and build a whole new plan. But in reality, this almost never happened, and it was rare that they had time to throw out all of the code that had been written so far. Instead, they usually ended up hacking together a solution out of the existing code. This rework led to bugs, because taking software that was designed for one purpose and hastily modifying it to do something else often results in messy, tangled code—especially when the team is under pressure. Adapting it to the new use would have taken up precious project time, so they ended up with poor workarounds and brittle code.

What Dan, Bruce, and Joanna were starting to realize by the end of the night was that their project’s problems were caused by overly rigid documentation, poor communication, and bugs, which led to projects that could not keep up with normal project changes.

At the end of the evening, the bartender called taxis for all three of them. Just before they left, Dan said he was relieved to get a lot of that off of his chest. Joanna was happy to have a better picture of the project that she was joining...but a lot less optimistic. Will she be able to find a way, she wondered, to fix some of these problems?

No Silver Bullet

Today we know that there’s no single “best” way to build software. But while that’s not a controversial statement now, for much of the last century many people in the industry would have loudly disagreed. There was a general feeling among many practitioners that they could discover a single, highly prescriptive “silver bullet” method that would solve project problems for everyone, everywhere. Many people felt that developers could build software just by following a set of instructions, like following a recipe or assembling a product on an assembly line. 

(Ironically, one of the most quoted papers in software engineering is Fred Brooks’s 1986 essay, “No Silver Bullet,” in which he shows conclusively why this is an impossible goal. That has yet to stop people from trying to find one!)

There were many proposed silver bullet solutions to these kinds of problems. They typically came in one of two forms: a methodology that claimed to give teams a foolproof way of building software, or a technology that programmers could use to prevent or eliminate bugs. The idea was that if a company decided on a methodology and a technology, then all the team had to do was to follow the company orthodoxy, and great software would start pouring out.

Dan and Bruce know firsthand that this doesn’t work, because they lived through years of managers in their company throwing methodologies and technologies at their projects without any real, lasting improvement. The company’s attempts at finding a silver bullet software process were usually enormously disappointing for everyone involved—and especially for Bruce and Dan, because they were asked to follow an ever-changing series of processes that they hadn’t asked for.

Joanna was also no stranger to this from her own career. At her last job, she was routinely handed a rigid set of requirements, and ordered to come up with a plan to build them out into software. Teams were then given her plan, and ordered to follow it to the letter. Teams that “planned the work, and worked the plan” were doomed to build software that was often outdated and not useful to their users even on the day that it was deployed.

One thing that gave Joanna pause was that some teams she worked on actually did manage to get great software out the door by following a waterfall (or waterfall-like) process that was heavy on up-front documentation. She’d managed some of her best projects at a company that built software for medical devices using waterfall practices.

Waterfall really can work. In fact, if you actually know what you need up front, then writing it down first is a pretty efficient way to build software. Joanna’s medical device software projects were rare examples where the requirements were actually right from the beginning, and needed very few changes during the project.

But it takes more than just stable requirements to run a successful waterfall project, which is why they run into so many problems.  Teams that build great software using a waterfall process typically have a few common characteristics:

• Good communication, because the teams that were successful in a company that mandated waterfall were the ones that consistently talked to their users, managers, and executives throughout the project.

• Good practices, especially ones like code reviews and automated testing, which are aimed at finding bugs as early as possible in the process. They usually called this “defect prevention,” and it required teams to actively think about how those bugs got into the code in the first place.

• Drawers full of documentation that have rarely been opened, because the people on the team understand that the act of writing down the plan—and the questions that get asked during that planning—is more important than mindlessly sticking to it afterward.

There’s one other piece of the puzzle. Dan started his career after the 1990s revolution in software development tools and technology, so he’s only ever worked on teams that used object-oriented development to build better software designs. Version control systems, automated testing, better integrated development environments (IDEs) that include features to automate coding practices like refactoring and class design, and other revolutionary tools all help Dan keep control of the code. Bruce has been working on projects longer than Dan, and he watched developers on his teams increasingly adopt software tools over the years to automate routine and repetitive tasks. Bruce and Dan know from their own projects that the most successful ones made effective use of these practices, tools, and ideas. That left more time for them to talk to their users and teammates, and to think about the problems they had to solve instead of fighting with the code.

And as it turns out, when waterfall projects are run effectively, it’s because their teams take to heart many of the same values, principles, and practices that agile projects follow. Projects that are run using some agile techniques and practices—but that don’t really follow the agile values and principles—often end up running into the same kinds of problems that plague waterfall projects.

Unfortunately for Bruce, Dan, and Joanna, they’re about to learn this the hard way.

Agile to the Rescue! (Right?)

You probably know what a waterfall process feels like, even if you’re just learning the term “waterfall” for the first time.³ Joanna, Bruce, and Dan do, too. Before jumping into planning the jukebox project, they talked about how waterfall processes had caused problems for their teams in the past.

On their last project, Bruce and Dan worked with Tom, a customer account manager at the company, who spent a lot of time on the road helping customers at arcades, casinos, bars, and restaurants install and use their products. The three of them spent the first few weeks of the project building up a specification for the new slot machine. Tom was only in the office half the time, which gave Bruce and Dan time to start designing the software and planning the architecture. Once all three of them had agreed on the requirements, they called a big meeting with the CEO and senior managers of the company to review the requirements, make necessary changes, and get approval to start building.

At that point, Tom went back out on the road, leaving the work up to Bruce and Dan. They broke the project down into tasks, dividing them up among the team, and had everyone start building the software. When the team was almost done building the software, Tom gathered a group of business users, project managers, and executives into a big conference room to do a demo of the nearly complete Slot-o-matic Weekend Warrior software.

It didn’t go nearly as smoothly as they’d expected.

At the demo, they had an awkward conversation where the CEO asked, “The software looks great, but wasn’t it supposed have a video poker mode?” That was an unfortunate discovery. Apparently, the CEO was under the impression that they were working on software that could be deployed to either the slot machine hardware or the video poker hardware. There had been a lot of discussion of this between the senior managers, the board, and the owners of their two biggest customers. Too bad nobody had bothered to tell the team.

The worst part about it was that if Dan had known about this earlier in the project, it wouldn’t have been difficult to change direction. But now, they had to rip out enormous amounts of code they’d already written, and replace it with code retrofitted from the video poker project. They spent weeks troubleshooting weird bugs that were caused by integration problems. Dan spent many late nights complaining to Bruce that this was 100% predictable, and that this almost always happens when code built for one job is hastily repurposed to do something else. Now he’s going to be stuck with a tangled mess of spaghetti code. It’s going to be difficult to maintain this codebase, and the team is frustrated because it clearly didn’t have to be this way.

And this wasn’t just a problem for Dan and Bruce. The project manager for that project was so unhappy that he left the company. He had trusted the team’s estimates and status, which were completely destroyed by the video poker change. The team had no idea that they would have to deal with the unexpected hardware change, but that didn’t make the project manager’s life any easier. Even though the facts on the ground had changed, the deadline was set in stone. By the time the project ended, the plan was out of date and basically useless, but the project manager was being held accountable for it anyway. After being raked over the coals by the senior managers in the company, all he could say in his defense is that his team didn’t give good numbers to start with. Pretty soon, he left the company for another job—and Joanna was hired shortly after that.

Tom may have been the most frustrated of the group, because he was the one who had to face the customers when they ran into problems. Their biggest customer for this product was the Little Rock, a casino in Las Vegas that wanted to customize all of their slot machines with themes to match their reproductions of cities in Arkansas. The customer had requested this new feature so they could change games between shifts without having to move kiosks around. Their engineers kept running into bugs left in the software by Bruce’s team, which meant that Tom and Dan had to spend weeks on the phone with the engineers coming up with patches and workarounds. The Little Rock didn’t cancel any contracts, but their next big order was with a competitor, and it wasn’t a secret that the CEO and managers blamed the Slot-o-matic Weekend Warrior project for the loss of business.

In the end, everyone knew that the project went wrong. And each person had a pretty good idea of how it was somebody else’s fault. But nobody seemed to have any idea how to fix these types of recurring problems. And the software that they’d delivered was still a mess.

A Fractured Perspective

Teams have been running into problems for as long as they’ve been building software. In fact, back in the 1960s, people openly talked about the idea that software development was fundamentally broken. They used the term software crisis, which was coined at the same NATO Software Engineering Conference in 1968 as the term software engineering was.⁴ The “software crisis” referred to the state of software development in a typical company in the 1970s and 1980s, because serious (and now-familiar) problems were very common and led to failed projects. Over time, our industry started to understand a major source of the software crisis. An important milestone was when Winston Royce, an engineer at Lockheed, published a paper in 1970 that described a model of software development that was very popular, yet highly ineffective. By the early 1980s, this method was widely known as the waterfall process; it took another 10 to 20 years before many teams made any real headway past blindly adopting it. Like Bruce, Dan, and Joanna, many teams have discovered that agile practices can help them with the problems of a typical waterfall process—but many have also found that it’s not as straightforward as they’d expected.

Developers use software tools every day to build their code, and a developer who’s proficient with many tools is always more marketable than his neighbor who doesn’t know as many. So when many developers first encounter agile, they immediately see it as a set of tools, techniques, and practices. Almost any developer who has been working with agile for a few months has updated his résumé to include the practices that he’s been working with. This first impression of agile is a good thing, because it helps make the agile practices attractive to developers who might otherwise be uninterested in them.

But seeing only the tools, techniques, and practices is just the first step in “going agile,” and it has a problematic side effect. Think about things from Dan’s perspective. As a developer and architect, he’s going to concentrate mainly on things that directly affect development: techniques that help him improve the code by removing or preventing bugs, tools that make builds faster and easier to run, and practices that help improve the way the code is designed, reviewed, and built. A project manager like Joanna, on the other hand, cares deeply about how much work it will take to build the software and the quality of the results. So she’ll concentrate on the tools that help her understand and communicate schedules, estimates, and effort. A business user like Tom is generally interested in the value that the software delivers to the business, and will be attracted to any practice that helps the team understand exactly what the users need, so that the software they build is valuable. And team leads like Bruce want to make sure that everyone on the team is moving in the same direction, communicating well, and learning from their experiences. They look for the tools that help with those things.

Take the agile practice of writing user stories, for example. A user story is a way to express one very specific need that a user has. It’s usually written out as a few sentences, often on an index card, sometimes in a strict structure but other times in a flexible format. One user story from the jukebox project, for example, went like this: “As a bar patron, I want to be able to play the newest hit that was just released today.” 

Each of the people on the team sees user stories a little differently:

• Joanna, the project manager who’s trying to become a Scrum Master, sees a user story as work to be done, neatly packaged up and ready to build. She wrote each of the user stories on index cards and taped them all to a whiteboard to help keep everyone on track.

• Dan, the lead developer and architect, sees a user story as a small piece of functionality, written out in a simple, easy-to-understand way. He can break it down into tasks, create an index card for each of them, and write his name on a task card when he starts working on it. When he’s done building it, he moves it to a section of the whiteboard reserved for completed tasks.

• Tom, the product owner, sees a user story as value that will be delivered to the company, because it lets him see a clear connection between what the team is building and what the users will do with the software once it’s built. The user stories help him talk to the customers at the accounts he manages so he can figure out what they’re looking for in the jukebox software, and he makes sure that each story represents something a user actually needs.

• Bruce, the team lead, sees a user story as a specific goal that the team can organize around. He helps them figure out which stories should be done next, and uses the progress to keep them motivated.

Adding user stories to a team like this will help improve the way they build software, because each of the people in those four roles will see a way that user stories helps him or her personally.

But this can work against the team, too. On Dan’s past projects, he had a detailed spec that left little room for flexibility, and now he had the freedom to make broader decisions about what he built. That can be a good thing, but it led to some problems on the project. When Dan wrote the code for the “newest hit” user story, he thought it meant building a feature to let bar patrons play any hit as soon as it was uploaded to their server. But when he demoed that feature to Tom at the end of the iteration, it led to a big argument. Tom had to explain that the newer songs played on the jukebox meant that bar owners had to pay higher royalty fees. He’d worked out details with them to let the patrons play the latest hits often enough to be happy to hear them, but not so often that it ran up excessive costs. Dan was pretty upset about this, because it meant that he had to go back and completely rewrite a large part of the feature, and Tom was angry because it meant that the first release of the software wouldn’t include the feature.

Had Dan understood that the user story was valuable to Tom for validating what the users needed, he might have had a discussion with Tom about what the software had to do before he started coding. Conversely, if Tom had taken the time to understand a little more about how Dan was going to build the software based on his limited knowledge of the user story, he might have made sure that conversation happened at the beginning of the iteration. But they didn’t have that conversation, and the project ran into the same kind of problems their past waterfall projects ran into: developers making incorrect assumptions, jumping into programming, and having to make changes to the code that could have been avoided, and that leave it more fragile.

When each person thinks only about his role and that one particular way the user story helps him, and doesn’t look past it to see how the entire team uses the user story—or any other agile tool, technique, or practice—it can cause exactly the kind of problem that Dan and Tom ran into. We call this a fractured perspective, because everyone has a different view of the agile practice.

Let’s leave our streaming audio jukebox team to their project—we won’t be seeing any more of them in this book. Will they overcome their problems and deliver their software? As you read the rest of this chapter, see if you can spot ideas that might help them.

Why Does a Fractured Perspective Lead to Just Better-Than-Not-Doing-It Results?

What people deliver will often vary based on what they’re focused on. The more people focus on their own goals, and not on the goals of the team, the less likely it’s going to be that what they deliver has real value for the company.

This presents a paradox for a team trying to go agile. Teams that concentrate only on the individual practices do end up seeing improvement, but only in the areas that they’re already pretty good at. The reason is that team members can only concentrate on what they know already, because for any one of them to expand past what they already know, they would need to have a good handle on what they don’t know yet. Asking a team to improve what they don’t know seems like a tall order!

Figure 2-2. Individual team members tend to add agile practices in areas of the project that they’re already good at, so the team only improves in those areas—and that’s why a fractured perspective leads to better-than-not-doing-it results.

That’s why a team that adopts agile practices one at a time will often only get better-than-not-doing-it results. They’ve applied better practices to the things that they already do, so they’ll do those things better. But they haven’t touched the areas of the project that they don’t already pay attention to, because the practices that affect those areas don’t appeal to anyone on the project; therefore, any problems affecting those areas won’t be improved. And those areas may be exactly what’s keeping the teams from being hyper-productive and delivering those astonishing results.

How does a team get past this problem?

The Agile Manifesto Helps Teams See the Purpose Behind Each Practice

The Manifesto for Agile Software development, more commonly known as the Agile Manifesto, was created in 2001 by a group of 17 like-minded people who came together at the Snowbird Retreat in the mountains outside of Salt Lake City, Utah, to come up with a solution to the software development problems they had all seen throughout their careers. After days of discussion, they agreed on a core set of ideas and principles (and also on the name “agile”). They bundled them into a single document, which started a shift in thinking across the software development world.

The Agile Manifesto contains four simple values. Here’s the entire Manifesto:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan
• That is, while there is value in the items on the right, we value the items on the left more.

Understanding and effectively working with agile starts with understanding these values.

Responding to Change Over Following a Plan

There’s an old project management saying: “plan the work, work the plan.” Unfortunately, if you work the wrong plan, you’ll build the wrong product. That’s why teams need to constantly look for changes, and to make sure that they respond appropriately when there’s a change in what the users need, or in how the software needs to be built. If the circumstances change, the project needs a new plan.

It’s not uncommon for the person or people who built a plan to resist change, because it takes work to change a plan. For example, there may have been a lot of effort put into breaking the work down into packages, and estimating each one of them. A change could require a project manager to have to redo all of that work, and if he values following a plan over responding to the change, he might dig his heels in. This makes for a smoother project, but, if the change is really needed, it will be much harder to make it later on, after the code is more complete.

Figure 2-3. Agile teams often use task boards to show tasks and track progress. They’ll write tasks or user stories on index cards, and move them across the board as they progress. Many teams also draw charts on their task boards to help them track progress.

A task board is a good example of a practice that helps a team make the right decisions about responding to changes. Each element of the work (typically a user story) is written on an index card and attached to a board—like the one that Joanna used for the jukebox project—usually in columns that show the status of each one. A task board can also be managed with a computer program, but many teams find it more effective to physically put it on a wall because standing in front of the board, talking, gesturing, and moving stories is a much richer form of communication than simply talking. The board is set up in a way that anyone can rearrange the order of tasks, and is even encouraged to do so. If changes happen, everyone is encouraged to add index cards to the task board, rather than having to clear each change through a single project manager hub. This helps keep everyone in the loop on changes, so the plan doesn’t get stale.  

Understanding the Elephant

Lyssa Adkins,  in her book Coaching Agile Teams, explains how a metaphor can be a valuable tool to help understand a concept:

Metaphor is a powerful thing. Professional coaches have known this for a long time. In fact, “metaphor” is a core skill taught in professional coaching courses. ... Coaches ask questions that help clients create their own metaphor, one that is visceral and resonant. Clients use the metaphor to guide them through the events unfolding in their lives.⁸

There’s a useful metaphor that can help us get a better handle on what it means to have a fractured perspective, and why that leads teams down a less effective path. It’s the story of the blind men and the elephant.

Six blind men were asked to determine what an elephant looked like by feeling different parts of the elephant’s body. The blind man who feels a leg says the elephant is like a pillar; the one who feels the tail says the elephant is like a rope; the one who feels the trunk says the elephant is like a tree branch; the one who feels the ear says the elephant is like a hand fan; the one who feels the belly says the elephant is like a wall; and the one who feels the tusk says the elephant is like a solid pipe.

A king explains to them: “All of you are right. The reason every one of you is telling it differently is because each one of you touched the different part of the elephant. So, actually the elephant has all the features you mentioned.”⁹

Teams that get better-than-not-doing-it results from agile are often teams who are already able to get software out the door reasonably well before starting with agile, and were hoping that agile adoption would help them make a good project even better. The problem is that even before the team started adopting agile practices, they were already experiencing problems—not the serious, software crisis problems that caused their projects to fail outright, but problems that caused friction and discomfort on the team. This is what “fractured perspective” really means: the developers think about developer stuff, project managers think about project manager stuff, and they throw the code over the wall to a business user who thinks about business stuff. Everyone is really busy thinking about his or her own project work—so much so that they actually use phrases like “throw it over the wall,” which explicitly divide the team and kill collaboration. When each person only thinks about his or her own practices, there isn’t a lot of communication between people, and they’re really functioning as individuals working separately toward compatible goals, not as a team. 

That’s where the “Blind Men and the Elephant” story comes in. In a fractured agile adoption, each person uses only the practices that impact his or her work, the same way each of the blind men feels only one part of the elephant. Developers concentrate on, say, test-driven development, refactoring, and automated builds. Project managers like task boards, project velocity, and burndown charts. Business users use release planning and user stories to get a better grasp on what the team is doing. Team leads use daily standups and retrospectives to manage and improve the team. Everybody wants something different from the project, and they each see a few practices that do something specific to help them. (Again, we’ll cover each of these practices later in the book, so don’t worry if you aren’t familiar with them yet.)

Now, adopting those practices individually will definitely improve things, because agile practices are really good. The problem is that because everyone—developers, project managers, business users, and team leads—sees the project from a different perspective, they’ll concentrate on only those practices that immediately appeal to them. There’s a paradoxical effect (“See! I was right all along”), where each person now sees only the part of agile that affects his specific project work, and draws the conclusion that agile is all about getting everyone else to come around to his point of view.

So while the agile “elephant” is made up of many great practices, the whole thing is even greater than the sum of the parts. And if you only see the individual practices—especially if you’re only looking at the practices that directly affect your project work—then you’ll only see one small piece of agile. The “elephant” of agile is made up of the day-to-day practices, but it’s much bigger than just those practices. 

Figure 2-4. The agile “elephant” is greater than the sum of its practices.

A team whose members only see the practices and don’t think about the principles will miss out on the important interactions between people. Their perspective stays fractured; the team members stay separate, and don’t really function as an effective unit. They’ll still get their work done, but they miss out on the great team interactions and collaboration that make agile really powerful.

This is built into agile. Take another look at the very first value in the Agile Manifesto:

• Individuals and interactions over processes and tools

Processes, methodologies, and tools are still important (which is why the Manifesto ends with, “...while there is value in the items on the right, we value the items on the left more”). But even more important than specific practices are the individuals and interactions. It’s these values (along with the 12 principles, which you’ll learn about in Chapter 3) that show us how the practices work together, and serve as a guide for how teams adopt those practices.

Methodologies Help You Get It All in Place at Once

There’s a big gap between understanding the values in the Agile Manifesto (and the principles behind it) and actually changing the way your team builds software. Luckily, there’s another important aspect to agile development that specifically helps with this. There are agile methodologies that are actually intended to help teams adopt agile and improve their projects. 

Agile methodologies are valuable to you because they help you see the practices in context. They are especially useful for teams that are not familiar with all of the agile practices. Each methodology was developed and improved over the years by agile experts focused on all of the different parts of the elephant. Adopting a methodology in full means you’ll be following a tried-and-true path that takes you from the beginning to the end of a software project, without the trial and error that can lead to a fractured perspective.

An agile methodology is a collection of practices combined with ideas, advice, and often a body of knowledge and wealth of experience among agile practitioners. An agile methodology will outline different roles and responsibilities for everyone on the project, and will recommend certain practices for each of them at various stages of the project.

The VersionOne State of Agile Development 2013 survey results contain a list of the most popular methodologies, and topping the list are Scrum, followed by a hybrid of Scrum and XP. Respondents also reported using Lean and Kanban, which are not agile methodologies (as you’ll learn about in Chapters 8 and 9), but are still core to agile.

Alistair Cockburn has this description of Scrum in Agile Software Development: The Cooperative Game, 2nd Edition:

Scrum can be summarized (but not executed) very simply:

• The team and the project sponsors create a prioritized list of all the things the team needs to do. This can be a list of tasks or a list of features. This is known as the product backlog.

• Each month, the team pulls off the top section of the list, which they estimate to be one month’s worth of work. They expand it to a detailed task list, called the sprint backlog. The team promises to demo or deliver the results to the sponsors at the end of the month.

• Each day, the team meets face-to-face for five to ten minutes to update each other on their status and on any roadblocks that are slowing them down. This is called the daily standup meeting.

• One particular person is designated the Scrum Master. It is this person’s assignment to remove, or get someone to remove, whatever roadblocks the team mentions in the standup meeting.¹⁰

For many teams beginning their move to agile, this translates to specific practices (which we’ve highlighted in boldface here, and which we’ll explain in detail in Chapter 4):

• The product owner creates and maintains a product backlog, a list of requirements for the software.

• The team runs timeboxed month-long sprints, where they pull a month’s worth of requirements from the product backlog to build, test, and demo. The requirements for the current sprint are called the sprint backlog. (Some Scrum teams use sprints that are two or four weeks long.) 

• The team meets for a daily standup meeting in which everyone talks through the work they did the day before, the work they plan to do today, and any obstacles in their way.  

• The Scrum Master acts as leader, coach, and shepherd to guide the team through the project. 

But adopting Scrum requires more than just adopting those great practices. Every one of those practices could potentially be used in a way that doesn’t reflect the values and principles of agile. A daily standup, for example, works very well if the team uses it to collaborate and work together to move the project forward. But it could also be a meeting where a project manager informs the team of their individual assignments and gets status updates from each of them, one at a time. And each developer uses the meeting to tell the project manager, “Here are obstacles blocking my way—you go deal with them.” If each person clings to his role—“That’s your responsibility, not mine!”—he starts to treat each new obstacle as someone else’s problem. The meeting turns into a contract-like negotiation, instead of an opportunity for collaboration. A team that falls into this trap may have adopted Scrum-like practices, but they aren’t using Scrum.

(You’ll learn more about how Scrum works and its practices later on in the book.)

A second methodology is eXtreme Programming (or XP). The Art of Agile Development, by James Shore and Shane Warden, summarizes XP like this: “Using simultaneous phases, an XP team produces deployable software every week. In each iteration, the team analyzes, designs, codes, tests, and deploys a subset of features.” (Many XP teams use iterations that last one week, but others use two-week or month-long iterations. Scrum can also be adapted to use different iteration lengths. You’ll learn more about adapting agile methodologies later in the book.) XP prescribes specific development practices aimed at improving collaborating with users, planning, developing, and testing. But XP goes beyond that, using those practices to help the team build simple, flexible software designs that are easy for the team to maintain and extend.

Scrum and XP have many things in common, including the fact that they are both iterative. This means that the project is divided up into iterations, in which the team performs all of the activities of a complete project to produce working, deployable software at the end of each iteration. Many XP teams use iterations that last one week, while many Scrum teams use one-month iterations. Putting a limit on how long an iteration lasts is called timeboxing, and it helps users to know when they can expect new features to be delivered.

Many teams find adopting methodologies—especially Scrum and XP—to be more effective than simply adopting individual practices. While adopting individual practices allows each team member to select those practices specific to his or her job, adopting an entire methodology encourages the entire team to get together and figure out as a team how to adopt all of the practices for that methodology. To do this, they need to change the way they think about doing their jobs. Methodologies are constructed around the agile values and principles, so this change in attitude is generally toward collaboration and interactions, working software, and responding to change. This transition is often made easier by the books and knowledge already collected by other agile practitioners, and encouraged by existing communities formed around those methodologies.

Lean is not a methodology.  Rather, it’s a name for a mindset—it has its own set of values, and thinking tools to help you adopt it. Lean is just as important in the agile world as XP and Scrum, and you can learn a lot about what it means to be agile by understanding what’s common among all three of them. Kanban is an agile method for improving the way that a team builds software. It’s built on the values of Lean, and it includes its own practices to help a team improve and evolve.

The practices in XP, and in many ways much of the focus of XP, are different than the practices and the focus of Scrum. Lean and Kanban take yet another approach, with different practices and focus. How is it possible that these different approaches to agile can have entirely different focus and practices, yet all be considered agile? It’s because all agile methodologies are based on the same principles, and they all rely on everyone on the team to work together and collectively own every aspect of the project. The values and principles of the Agile Manifesto are what tie all methodologies and methods together.

Figure 2-5. Scrum, XP, and Lean all have the agile values at their core, and share some values, ideas, and practices with each other.

Where to Start with a New Methodology

By coming together and working toward the single goal of adopting a methodology, everyone on the team begins talking about each other’s practices, ideas, and perspectives. This is the opposite of a fractured perspective. By looking at the methodology as a whole, teams start to understand how the individual practices interact with each other. And that’s where Bruce, Dan, Joanna, and Tom want to be—but they don’t know quite how to get there.

When the team initially tried out the new practices and ideas, they didn’t yet understand how those new practices would relate to the ones they were already familiar with. That understanding will come as the team gains experience with the methodology. This works because an agile methodology is a complete system that includes a set of practices that are known to interact well together, and which teams have used successfully to become more productive. Adopting a complete set of practices will help give the team a foundation for learning about those interactions.

But adopting an entire methodology is more difficult than picking and choosing practices that fit in with the team’s current way of working. If a team can adopt the methodology all at once, they stand a much better chance of getting the most out of their agile effort. In part, this is because in addition to the practices that are similar to what each person already does, they also adopt practices and ideas that they may not think they need at first.

As we learned, the jukebox team ran into problems because Bruce, Dan, Joanna, and Tom all approached agile practices independently. To get the greatest benefit of agile in the future, instead of just diving into the practices, they should first sit down together and have a real discussion about what each of those practices will do for them and their project. But this is a challenge, because they don’t know how to start that discussion. Like many teams, they face a dilemma: if they already knew what the agile practices would do for their project and how to work together to put them in place, they wouldn’t need to have that discussion. But because they don’t know that already, the discussion is difficult to have.

There’s a solution: the 12 principles that go hand-in-hand with the values of the Agile Manifesto. You’ll learn about them in Chapter 3.