Starting with a squad

In Spotify, the smallest functioning unit is a squad, which has the following characteristics:

• Autonomous: Each team defines what to build, how to build it, and how to work together (and, if necessary, with other squads) to build it.
• Close proximity: Team members sit in close proximity to one another to facilitate communication and collaboration.
• End-to-end responsibility: Each team is responsible for one or more features from beginning to end — design, commit, test, deploy, maintain, and anything else that may be required.
• Long-term mission: Every team has a long-term mission, such as to make Spotify the best place to get music or to build infrastructure for A/B testing — comparing two versions of a feature or product to determine which is better at producing the desired outcome.

One of the key benefits of having small, autonomous squads is speed. Squads make decisions locally and are given the freedom to fail, so they’re not wasting time seeking permission or approval. It also minimizes handoffs (passing a task to another team to complete the work), waiting, and time spent coordinating efforts.

Spotify doesn’t endorse any one type of agile team, nor does it prescribe a methodology. Squads are similar to feature teams in LeSS, in that they focus on a feature in a product. (See Chapter 5 for more about LeSS.) The differences are that Spotify teams are generally smaller, and they’re not required to use Scrum. Squads can use Scrum, Kanban, Lean Software Development, Extreme programming, or any other approach, or they can mix and match agile team models or create their own model. Whatever works.

The only limits on teams is that they align their activities with the squad’s mission, the company’s product strategy, and the squad’s short-term goals, which are reviewed monthly.

Not all squads work on product delivery. Squads may work in operations, on infrastructure, or in other functional areas of the company. Whatever the squad’s mission, it is expected to communicate and collaborate with other squads as needed. Spotify encourages a “we’re all in this together” culture. Although squads are generally self-sufficient, they’re not silos.

Maintaining autonomy and alignment

Although squads are autonomous, you don’t want every squad going off in its own direction. Spotify prevents the fall into chaos by requiring that teams be aligned with product strategy, company priorities, and other squads. As Spotify puts it, each squad is expected to “be a good citizen in the Spotify ecosystem.” The company’s mission is more important than that of any of the squads. Ideally, squads are autonomous but aligned.

Spotify sees autonomy and alignment as two different dimensions (see Figure 7-2). It charts the two with autonomy along the x-axis and alignment along the y-axis to create the following four quadrants:

• Low-alignment, low autonomy: A high-control culture in which teams do what they’re told.
• High alignment, low autonomy: Leadership defines the problem and prescribes the solution. Teams execute the solution.
• Low alignment, high autonomy: Teams do whatever they want, wandering off in different directions, which leads to chaos.
• High alignment, high autonomy: Leadership defines the problem and leaves it up to the teams to figure out how to solve it. Spotify tries to stay in this upper-right quadrant.

At the squad level, alignment is primarily the responsibility of the system owner who keeps the squad aligned with its mission and creates and maintains a prioritized list of work items to help the team prioritize its work. She also communicates with other system owners in different squads to help coordinate the work and even manage any dependencies. (For more about the system owner role, see the later section “Having a system owner.”)

Forming tribes of squads

For large products, such as Spotify, squads are grouped into tribes (see Figure 7-3), with each tribe focusing on a specific area in product development. For example, Spotify has three tribes:

• Client app tribe: Each squad in this tribe facilitates the release of features on one specific platform, such as iOS or Android.
• Feature tribe: Each squad in this tribe focuses on a specific feature area, such as search or playlist management. The squad builds, ships, and maintains features in that area for all platforms.
• Infrastructure tribe: The squads in this group focus on making other squads more effective by providing tools and routines for operations, monitoring, testing, and so forth.

The tribes are built on a self-service model. Tribes don’t push what they have on other tribes; they simply make it available to the tribes to use the services when needed. The client app tribe enables and supports the feature tribe, and the infrastructure tribe enables and supports both the client app and feature tribes.

Setting up chapters

Within each tribe are chapters that cut across the squads (see Figure 7-4). Each chapter has a chapter lead and is composed of people with a shared competency, such as software development, web development, data management, or testing. The chapter lead acts as line manager, but without some of the traditional management responsibilities. The chapter lead is more of a service position, coaching and mentoring the chapter members. She doesn’t assign work or supervise employees, but she may set salaries, schedule vacations, or help squad members obtain additional training.

One of the key benefits to having chapters is that people can move from one squad to another without losing their chapter lead. In that way, a chapter is more like a traditional department, although Spotify would never think of calling it that.

Sharing interests and knowledge in guilds

To encourage knowledge sharing, promote professional development, and support the wider community (beyond squads and tribes), the Spotify approach uses guilds — informal groups that form around a shared interest. A guild is basically a community of practice (CoP) (see Figure 7-5), which is common in other enterprise agile frameworks, such as SAFe and LeSS.

Each guild has its own coordinator, who sets the time and location of meetings and may create a loose agenda based on the guild members’ needs. These meetings, often referred to as “unconferences,” are a time for different parts of the organization to get together and share ideas in an informal setting.

Guilds offer numerous benefits beyond knowledge sharing among a group of people with shared interests, including the following:

• Guilds can be a great tool for improving understanding across squads and across chapters. For example, a frontend web developer may join a guild on backend development practices to gain a better understanding of the challenges involved in backend development.
• Guilds can be great for cross-training. A tester may be interested in attending a guild on software development to develop knowledge and skills in another area that may help his squad fulfill its mission.
• Guilds tend to drive innovation. Having expertise in more than one area creates a synergy that often sparks fresh ideas.

Guilds serve two primary purposes: They build and spread knowledge, and they bring together people from across the organization to create a greater sense of a common purpose.

Use your guilds as a barometer of your culture of continuous learning and professional development. When guilds go dormant, that’s usually a good sign that everyone’s spending too much time on product delivery and not enough on professional development. When your people aren’t learning and growing, your organization is at greater risk of stagnation.

To keep your guilds thriving, make them a priority by doing the following:

• Set aside time during the normal workday for the guilds to meet. If you require that they meet after work to avoid workplace interruptions, attendance will suffer.
• Add some perks. Spring for beverages and snacks or pizza.
• Build an online community that enables your guilds to communicate, share knowledge, and make themselves known to the rest of your organization.
• Keep the meetings short, an hour at most.
• Make attendance optional, but provide an agenda in advance of the meetings that’s compelling enough to draw a crowd.
• Provide the guild the resources it needs — books, magazines, subscriptions, training, and so on.

The vitality of a guild depends a great deal on its coordinator. Generally, the person or people who form the guild are the most passionate about their shared interest, but someone else in the organization may be better suited for that role. You want someone with passion, knowledge, and charisma. Encourage your guilds to choose the best person for the role and look past ego and politics. The best coordinator is the best for everyone in the guild.

Discourage guild coordinators from trying to set standards. It’s easy for a guild to devolve into a sort of political action group that tries to set standards or rules for the entire organization.

Driving agility and innovation through culture and values

Agile frameworks tend to communicate culture through manifestos and principles more than anything else. Although Spotify doesn’t exactly list its principles, the creators of its approach highlight what the organization values in terms of inequalities. As you read through this list of Spotify’s “values,” note how nearly every one of them supports squad autonomy and innovation over methodologies and practices:

• Agile > Scrum: Being agile is more important than adhering to any agile framework, such as Scrum, Kanban, or Lean Software Development.
• Chaos > Bureaucracy: Although Spotify doesn’t exactly yearn for chaos, it strives to have the least amount of structure and process (bureaucracy) to avoid total chaos. It relies on its foundation of community and culture to keep from tipping into chaos.
• Community > Structure: A thriving community drives innovation, whereas a highly structured organization tends to stifle it. As Spotify puts it, “If you always need to know exactly who is making decisions, you’re in the wrong place.”
• Cross pollination > Standardization: Allowing practices to spread from squad to squad leads to a healthier balance of consistency and flexibility than does requiring that all squads adopt a specific set of practices.
• Enable > Serve: Facilitating other people’s work is more effective than doing it for them.
• Failure recovery > Failure avoidance: The Spotify approach doesn’t punish failure. Learning from failure and using the lessons learned to improve the product and process is what matters most.
• Impact > Velocity: A feature isn’t done by a certain deadline or at the end of a given sprint but when the product achieves the desired impact — when it produces the desired results.

• Innovation > Predictability: Spotify sees innovation and predictability as opposite ends of a spectrum. It wants some predictability but prefers to be more innovative than predictable. The focus is on delivering value, not on meeting deadlines.

  When higher predictability is needed (for example, to coordinate a release with a planned marketing activity), Spotify may slide closer to the predictability end of the spectrum and use standard agile planning techniques, such as epics and user stories.

• People > *: An organization’s people are its most valuable asset. Mutual respect is essential.
• Principles > Practices: Guiding principles promote agility, whereas established practices may just get in the way.
• Servant > Master: Servant leaders are more empowering than process masters. In line with this value, Spotify changed the name Scrum Master to Agile Coach.
• Trust > Control: Trust gives people the freedom and confidence to try new things and take initiative. Control hinders agility and stifles innovation by creating a culture of politics and fear.

Reducing the negative consequences of failure

To prevent experimentation and failure from negatively impacting customers (or to minimize that impact), Spotify introduced the concept of having a “limited blast radius.” That is, if failure occurs, it negatively impacts only one feature or feature set and very few customers. Spotify limits its blast radius in two ways:

• Decoupled architecture: The decoupled architecture (see the earlier section “Decoupling systems and squads”) enables each squad or tribe to work on an isolated system within the product, so any mistakes affect only that system.
• Gradual rollout: When the squad or tribe deems a feature good enough to roll out, it makes the feature available to very few users. It then monitors how users respond to the change to determine whether it had the desired impact; for example, whether users are logging on more often or sharing more music. If the feature functions well and has the desired results, it’s gradually rolled out to more and more users.

Having a limited blast radius reduces the squad’s fear of making mistakes. As a result, it’s a great way to encourage your squads to take chances.

Encouraging innovation

Throughout the year, Spotify encourages squads to spend ten percent of their time to invent and build whatever they want with whomever they want. The squads set aside one of every ten days as a hack day — innovative play time. Twice a year, the entire company joins in a hack week at the end of which the company has a party and reveals the inventions developed over the course of the week.

The hack day is nothing new. Several companies, including Sun Microsystems, allocate play time to give programmers the opportunity to engage in exploratory programming and explore ways to improve the product. They could rework the code or even research new ideas. At Spotify, squads are encouraged to create anything they like — the idea is to get the creative juices flowing.

Developing an aversion to waste

One way Spotify promotes creativity while improving efficiency is by eliminating waste — anything that doesn’t add value to the product, such as time reports, handoffs, separate test teams, task estimates, useless meetings, and corporate nonsense.

To develop an aversion to waste, simply give your squads license to stop doing whatever doesn’t help them or whatever doesn’t contribute value to the product. People naturally stop using tools that don’t make their jobs easier; they just need permission to do so.

Engaging in continuous improvement

Like all other enterprise agile frameworks, Spotify’s approach stresses the need for continuous improvement. The company drives continuous improvement in the following ways:

• Conducting retrospectives: Squads and tribes meet regularly to discuss what’s working, what’s not working, what’s getting in their way, and what they need to do about it.
• Experimenting and capturing the learning: Squads and tribes experiment and then analyze the results to figure out ways to improve both the product and the process for creating it. By measuring and collecting data, squads are able to make data-driven decisions instead of having their decisions driven by ego, authority, or opinion.
• Using improvement boards: Improvement boards, such as the kata board, shown in Figure 7-6, help the squads move from problem to solution. The improvement board Spotify uses has four quadrants:
  • Top left: A description of the current situation — the problem.
  • Bottom left: The definition of awesome — a description of what the situation will look like when the problem is gone.
  • Top right: A realistic target that brings the situation closer to awesome.
  • Bottom right: Three concrete steps the squad can take to achieve the realistic target. (After taking these steps, the squad reassesses the situation and sets more steps to reach the realistic target or, if it has already hit that target, it sets another realistic target that brings them even closer to awesome.)

Continuous improvement must be driven from below and supported from above.

Strengthening community and culture overall

The very foundation of the Spotify approach is community and culture. According to the creators of this approach, “Healthy culture heals broken process.” Spotify nurtures community and culture with the following:

• People operations dedicated to supporting the needs of community members
• About 30 Agile Coaches that spread across all squads
• Boot camps for new hires, during which new hires form a temporary squad and work together to solve a problem while learning the Spotify approach to enterprise agility
• Storytelling, which spreads the culture throughout the organization via blogs, post-mortems, retrospectives, demonstrations, or even conversations over lunch

Changing the game plan for larger products

While Spotify’s usual approach works for small products, such as a feature, it’s less successful for large, complex products. Spotify tries its best to steer clear of larger products by breaking the product down into smaller products, so it can use the “Think it, build it, ship it, tweak it” approach described in the previous section. However, if that’s not an option, it takes a more traditional approach to product development.

For big products, Spotify adds a small, tightly knit leadership group, which typically includes a tech lead, product lead, and (sometimes) a design lead. The leadership group isn’t exactly management. They don’t assign and supervise work. Their function is to communicate the vision and maintain alignment among the squads and tribes.

The squads track progress visually using a Kanban board. They conduct daily syncs to facilitate coordination and collaboration and ensure alignment. And they produce a weekly demo to check how the product is coming along. The idea is increase collaboration and reduce risks by keeping the feedback loops as short as possible.

Spotify sees big products as a major source of risk, the biggest risk being building the wrong thing.

Having a system owner

For large enterprises and large products, Spotify recommends adding a role called a “system owner.” Even though “system owner” sounds like one person, it’s actually the role of two people — a developer and an operations expert, each of whom works in her own squad. Every so often (on “system owner day”), they get together to make system-level decisions.

The system owner role is based heavily on the DevOps (development and operations) role, which is an attempt to combine two roles that may seem incompatible at first glance:

• Development: Development is more in the sphere of agile, relying on innovation and experimentation through frequent iterations to achieve continuous incremental improvement.
• Operations: Operations relies on planning ahead to make sure all components or systems of a product work seamlessly together.

Having a systems owner is an attempt to strike a healthy balance between predictability and innovation on large products. The system owner should always be pushing to achieve a healthy balance between frequent changes and the stability of the whole.