Deming on Business Transformation

While researching for the second edition of this book, we came across key principles for managing business transformation published in 1982 by renowned engineering and business professor W. Edwards Deming. Many of the key points of FinOps echo Deming’s principles, so it’s worth looking at a couple of Deming’s key principles.

Deming focused on constancy of purpose. His view was that the best way to drive a company to long-term success was to foster long-term thinking and be constantly implementing continuous improvement.

He also stressed the importance that all teams involved in a transformation need to work as a single unit, breaking down the barriers between departments to enable the whole organization to identify problems early on and collaboratively work toward solutions. Likewise, a successful cloud and FinOps transformation requires each team involved in the management of cloud to understand that the transformation is everybody’s job.

Leadership and education feature heavily in Deming’s principles. He did not recommend that leadership manage primarily via targets and numbers, but to manage the system in a way that allows staff to contribute with a pride in their work. In Chapter 24, we’ll discuss the related concept that leadership support for FinOps is not optional and that mandates to reach arbitrary targets will ultimately fail. Rather, leadership needs to support and drive a cultural shift that empowers teams to make decisions that balance the cost versus the value of cloud.

Deming also added that massive training is required to instill the courage to break with tradition, and that every activity and every job is a part of the process.

Who Does FinOps?

From finance to operations to developers to architects to executives, everyone in an organization has a part to play. Take a look at the wide variety of job titles quoted throughout this book, and you’ll see what we mean.

It’s everyone’s job to help the company go faster. In an agile world, we are all servant leaders to help the engineers deliver what they are doing. Everyone else should be there to unblock delivery.

David Andrews, Senior Technology Manager, Just Eat

Whether it’s a small business with only a few staff members deploying cloud resources or a large enterprise with thousands, FinOps practices can and should be implemented throughout the organization, albeit at different levels of required tooling and processes and at different cadences depending on the value delivered in each place.

So what skills and roles does a FinOps team need to have? Later in the book, as we work through the optimize phase, we’ll go into ways a FinOps team generates recommendations that need to be considered by the wider organization, such as changing resource configurations or making commitments to cloud service providers. While these will undoubtedly save the organization money, trust is required between product, finance, and engineering to carry them out.

To create this trust as quickly as possible, the FinOps team must have cloud expertise. Without it, recommendations might be wildly incorrect, or engineering teams may waste valuable development time trying to explain their way out of making necessary performance modifications. A FinOps team with cloud domain expertise builds trust and credibility.

FinOps must have executive support, be capable of sharing best practices, and use a common language that ties disciplines together, as we’ll discuss in the next chapter. Not only does this help to spread the message that FinOps is important to the company, but it also ensures that staff will have time allocated to perform required tasks. When done well, FinOps doesn’t increase conflict between technical and business teams. Instead, that conflict is removed. This is accomplished when teams use a common language that helps create alignment around business goals, thus making sure that everyone understands and supports those goals.

A central FinOps enablement team is the driver of a FinOps mindset across the organization, evangelizing best practices for incorporating cost considerations into development plans. FinOps practitioners perform some centralized tasks like managing and reporting on commitment-based discounts to the cloud providers or centralizing reporting to make sure everyone is looking at the same numbers. This prevents other teams from reinventing the wheel while taking advantage of economies of scale in relation to commitments.

However, the actual implementation of many FinOps best practices must be decentralized to the engineers inside various product and application teams across the company. A good example of this is rightsizing and idle usage reduction. While a central FinOps team may help with data and goal setting, engineers in various lines of business must make data-driven decisions about when to make changes to the infrastructure based on business value and level of effort.

When you can give finance an alert about a spend spike in 24 hours as opposed to a quarter later, it buys a lot of credibility. If it takes you multiple months to figure out what happened to cause an overrun, it won’t give confidence to finance that tech knows what they are doing.

Joe Daly, formerly Director of Cloud Optimization, Nationwide

Another vital role for the centralized FinOps team is to facilitate the conversations among teams and to foster trust. Finance teams must partner with engineers, using shared data and reporting, to enable everyone to quickly find and solve the situations that need addressing.

Armed with the reporting and billing knowledge from the FinOps team, engineers will be able to show where, when, and why a plan or cost exceeds budget. This shared knowledge builds confidence, trust, and efficiency.

Instead of a focus on centralizing spend approvals, FinOps builds visibility of the spend to the appropriate areas to create accountability. The cloud-aware expertise of the FinOps team allows the other teams to understand how each specific billable item can be distributed into chargeback and showback. These are terms used for how you display and handle costs. We’ll dig into this subject more in Chapter 11.

Why a Centralized Team?

We have so far referred a lot to the FinOps team and to the centralized functions a FinOps team performs. It’s worth pointing out here that we’ll continue to refer throughout the book to a centralized FinOps team, but there are a number of different FinOps Team models that work. How your FinOps team is structured will largely depend on the type of organization you have and the way it will support a cross-discipline, coordination machine that the FinOps team represents.

While we will still refer to the FinOps team, interpret that as the coordinating body that drives FinOps best practices, but understand that whatever organizational design that takes, there are things that a FinOps team should do organization-wide to make the transformation as successful as possible.

The unbiased central team, with ties to business leaders, engineering teams, and finance teams, shares objective best practices and recommendations. The members of this team are never seen as pushing a specific agenda to benefit themselves, which builds more trust in the advice they give. Critically, they must also work to incorporate business objectives for each cloud workload.

When this central group drives the rules of cost allocation and pushes out clear communication about how teams are doing, everyone is reassured that they’re being managed and measured against the same criteria. If one of the budget-holding teams drives the process, which happens when a group is responsible for the lion’s share of cloud spend, it can raise questions about the integrity of information. If cost allocation is done independently by each team, there will inevitably be overlap or gaps in spending data. In either case, doubt grows. And when teams lose trust in data, they can’t be held accountable to it.

When individual teams try to build out their own cloud reporting processes, disagreements arise about whose cost was whose. An organization-wide FinOps team solves this issue, fostering agreement that the spend data is the correct data and then objectively attributing each cost to the appropriate engineering group.

Finally, the central team defines what the organization uses as a business metric. In fact, this is a key early task of the FinOps team. A dollar is not just a dollar when it comes to cloud spending. One can choose to look at cost amortized or not amortized, and with custom rates applied or excluded. Shared costs of an application from another service may be included or invisible. When the business metric used across teams is clearly defined, everyone speaks the same language. In the next chapter, we’ll dive deeper into the language of FinOps.

The FinOps Team Doesn’t Do FinOps

It’s easy to assume that the daily work of FinOps can be assigned to a central team and siloed from the work of the rest of the business. But FinOps isn’t only happening inside your FinOps team; they are primarily an enablement team that supports the rest of the organization with subject matter expertise in the very real complexities of cloud billing data and pricing models, centralized data and reporting, commitment management, and related functions while helping the distributed engineering teams with the data they need to make daily decisions about their infrastructure. It should not be expected to be a central team for managing infrastructure or performing the bulk of optimization activities. That responsibility is pushed to the groups that understand their own infrastructure and their business objectives the best. The FinOps team aims to propagate repeatable patterns throughout the organization to accelerate FinOps practices at the edges.

The other teams in our organization ask for me to tell them about FinOps and explain how we are going to protect them from overspending. To which I say, I’m not, I’m gonna help you protect yourself.

Alison McIntyre, Cloud FinOps at Lloyds Banking Group on The FinOpsPod, 2022

A FinOps team will do more of the work early on in an organization’s cloud maturity, as everyone becomes familiar with their new responsibilities. But as time goes on, teams will become more self-sufficient, and the FinOps team will be able to tackle fewer of the routine tasks and focus on more complex projects related to cloud efficiency.

Why is subject matter expertise in cloud billing needed? As an example, AWS’s S3 storage service has at least 6 different types of “actions” you can get charged for—storage pricing, request and data retrieval pricing, data transfer and transfer acceleration pricing, data management and analytics pricing, replication pricing, and the price to process your data with S3 Object Lambda. There are also at least 8 different storage types from Standard to Infrequent Access to Glacier to Deep Archive. That is 48 combinations of pricing options for a single service. Amazon currently has hundreds of different services all with different pricing options. Subject matter expertise in a FinOps team can help navigate this complexity.

Ashley Hromatko, formerly Director of Cloud FinOps at Pearson

FinOps teams also might, as they mature, pick up more responsibility for automating the really routine tasks that aren’t a result of engineering teams being inefficient. There is a certain amount of technical debt introduced when moving to cloud, such as putting storage in a more expensive tier than it needs to be or purchasing higher tiers of service than required, that the FinOps team might be empowered to fix on its own, once the FinOps team has established the credibility to do so and the trust with engineering teams.

This enablement pattern is not a new concept; consider security in the cloud. It’s a shared responsibility model. Just as the security team in an organization is not solely responsible for security, the FinOps team is not there to optimize all the things. It’s there to provide the resources for the engineers closest to the cloud resources to decide on the right optimizations at the right time, or to not optimize them using informed ignoring, covered in Chapter 2.

The work that your central FinOps team does enables and accelerates your organization to use cloud more effectively. It doesn’t replace the daily duties that other engineering teams and business leaders have to continuously optimize their own infrastructure.

The Role of Each Team in FinOps

Figure 3-1 demonstrates how organizations operate in the FinOps model. A cross-functional team, such as the Cloud Center of Excellence (CCoE), manages the cloud strategy, governance, and best practices and then works with the rest of the business to transform how the cloud is used.

Figure 3-1. Team interaction around FinOps

Individuals at every level and in every area of an organization have different roles to play in the FinOps practice. These include the following.

A New Way of Working Together

Each of the previous functions needs to integrate like never before. Some say engineers have to think a bit more like finance people and finance people have to start thinking like engineers. That’s a great start, but the entire organization needs to shift from a centralized cost control model to one of shared accountabilities. Only then can the central FinOps team empower the organization to move and innovate faster.

This cultural shift also enables those in leadership positions to have input into decision making in a way they currently don’t. Based on leadership input, teams make informed choices about whether they are focused solely on innovation, speed of delivery, or cost of service. Some teams go all-in on one area with a growth-at-all-costs mindset. Eventually the cloud bill gets too big and they have to start thinking about growth and cost together. For example, “Move fast, but keep our cost per customer transaction below $0.45.”

Where Does Your FinOps Team Report?

More often than not, FinOps teams report to the technology leader, but they can also report to finance or other functions. The State of FinOps survey asks where FinOps teams report, and as you can see, the results in Figure 3-2 show that FinOps teams report to many locations within different organizations. Most commonly, the FinOps team is aligned within or alongside technology teams.

Figure 3-2. Most common executive FinOps reports to, from the 2022 State of FinOps report

This means it is typically part of the CTO or Head of Technology function. Some organizations also place it within a CIO function, though this is declining over time. Having the technology leadership watch the technology spending may be akin to the fox watching the hen house since the group spending the money is also the one monitoring it. However, there are numerous advantages to having FinOps sit in the technology organization. Initially, more weight and credibility are given to its recommendations by engineering and tech ops teams, who might be more dubious about recommendations coming from a finance team. Most importantly, the technology teams are the ones who best understand the complexities of cloud and are able to most quickly set up the practice of managing highly variable cloud spend. Finance-driven FinOps often initially attempts to apply existing financial reporting and forecasting systems not designed for a cloud bill, which may have millions of individual charges each month that change daily at variable usage and rates. Increasingly, we’re seeing a hybrid function emerge where there is a COO or CFO for Technology under the CTO. A story from Equifax demonstrates this structure in Chapter 6.

Regardless of where they sit, the central FinOps team must partner closely with the finance and engineering organizations. Small companies might not have a dedicated place in the organization for a FinOps team. A common pattern in that case is to assign a single person or split part of an individual’s time to practice and promote FinOps within the business. A large, complex spender may have 10 or more people in the central FinOps team.

Understanding Motivations

Trust usually comes when there is empathy for other people’s problem domain.

Sascha Curth, Head of Cloud Economics and Governance at OLX

Whenever a disparate group of smart people come together from different parts of an organization, there will invariably be some friction, largely because they have different goals and targets. Finance focuses on cost efficiency and ensuring budget targets aren’t exceeded, while operations teams think about offering high-quality service with minimal service issues and outages.

Let’s take a deeper look at the motivators of various personas involved with FinOps.

FinOps Throughout Your Organization

It’s no longer acceptable for teams to consider only their own priorities. If the technical operations team doesn’t take into account the impacts of their cloud spending, finance will have an impossible challenge of forecasting and budgeting an organization’s cloud spend. Alternatively, if finance takes full control of cloud spend and requires approvals for every resource, the organization will struggle to take advantage of the speed and agility of having variable spend resources and on-demand infrastructure. Remember, the advantage of the cloud is the speed of innovation.

Hiring for FinOps

It’s important to note that you can’t facilitate a cultural shift to FinOps just by hiring practitioners or bringing in a contractor. With FinOps, everyone in the organization has a part to play. A FinOps requirement should be added to all hiring, from executives to finance to operations, such as listing FinOps—or more specifically the management of the value derived from cloud—as part of each job description or providing FinOps training as part of onboarding new hires into the organization. Ask engineers how cost metrics play a part in good service design, and ask finance how the variable spend of cloud changes normal finance practices. Without hiring new talent with a FinOps understanding or ongoing training for the new members joining your teams, the culture you’ve built will slowly be eroded by new staff lacking the correct mindset.

Roles will evolve to meet the needs of FinOps. More poly-skilled employees who have a business head, fiscal thinking, and technology acumen will emerge. Finance people will learn cloud, just as IT people will learn finance.

Think back to when the idea of full-stack engineers was new; now it’s time to start thinking about full-business people.

Here are some of the common skill sets that you’ll need to fill when building out your FinOps practice:

Policy governance

Creates policy documents, standards, key performance indicators (KPIs) and objectives and key results (OKRs), and governance frameworks to guide the organization’s cost allocation and cloud use.

Technical writing

Creates documentation about FinOps processes, helps socialize standards (tagging), and sends budget alerts.

Analysis

Digs into cost abnormalities, learns about cloud cost models and explains them to engineers and to finance, and creates and delivers reports to executives.

Engineering

Considers the costs of architectural decisions and assists in automation of billing data, optimizations, reporting of budgets and forecasts, and governance.

Automation engineering

Focuses on automating cloud tooling, either automating the management of cloud billing mechanisms like budget alerts and commitments, or automating optimization recommendations.

Data engineering

Builds systems that collect, manage, and normalize raw data into usable information for data scientists and business analysts to analyze and report on. Data engineers implement methods to improve data reliability and quality.

Training

Gets your teams up to speed on what FinOps means for them; requires training materials to be made available and management of courses either self-paced or in person.

Evangelism/marketing

Builds the profile of FinOps within an organization; requires effort to build a promoter base, while winning over the detractors by helping them understand the value that FinOps provides.

The preceding skill sets may initially be covered by a single jack- or jill-of-all-trades but over time, as the practice and spend grows, they will likely be split across a larger number of specialists. Chapter 6 talks about common team sizes based on maturity and lays out some sample team structures.

FinOps Culture in Action

As an example of how a FinOps culture, along with the language of FinOps, can enable the business, let’s take a look at how the introduction of containerization impacts cost visibility. With the introduction of containerization, operations teams are able to pack more services onto the same compute resources. Services such as Kubernetes give control and automation to operations teams like never before. It’s easy to see that for operations teams, implementing containerization is a great opportunity. And it has played out across the industry in the large-scale adoption of container environments.

You might think that having more services on the same compute resources would mean more efficient costs, but those benefits can come with a loss of visibility. Cloud billing data has the underlying compute instance costs, but there may be no details to help finance work out what containers from which applications are operating on top of each instance.

Consider how this plays out without a FinOps culture.

The finance team will understandably want to know how to split out the costs of the underlying compute resources to the correct business units. So they ask the engineers for help. This request means that engineers must stop what they are doing and shift their focus to costs and financial data. Because that shift will ultimately result in a lack of productivity—the finance team might conclude—and then start championing the idea to executives, that containerization is bad for business operations.

However, when you apply FinOps and introduce FinOps practitioners to the conversation, you end up with a different outcome. The practitioner will have deep knowledge of what data is or isn’t available from examining the cloud service provider’s billing files. Finance will learn and understand the basics of containerization and why it benefits the business. Meanwhile, the operations team learns about the importance of chargeback and showback.

When finance asks for the cost breakdowns for the containers running on the cluster, a FinOps practitioner will understand the finance team’s perspective: they see only the overall cost of the cluster. On the other hand, while engineering teams know which container is scheduled on each cluster instance, they can’t easily associate this data with the cloud bill. But when they share that information, the FinOps practitioner can take on the burden of working out the costs.

The FinOps team then takes this allocation data and performs the needed analytics to combine it with the billing data. Now finance gets the reports they need, and already busy engineering teams can keep working. Because of this cross-functional approach, finance can now see containerization as an enabler of efficiency, not as a blocker.

Difficulty Motivating People Is Not New

For the last two years, in the State of FinOps survey the number-one challenge FinOps practitioners face is motivating engineers to take action, as shown in Figure 3-3.

Figure 3-3. Survey responses in the FinOps Foundation State of FinOps 2022 survey, showing motivating engineers as the top challenge

It is easy to fall into the trap of thinking this problem is unique to the cloud. As a FinOps practitioner, you identify opportunities for optimizations and report the opportunities to your engineering teams. Recommendations require your engineers to spend time reviewing them and then making the changes necessary to their cloud resources. These individual recommendations have specifics on how various cloud resources are charged and configured and alternate options available. You can see how cloud-specific this problem can appear at an individual level and how new this practice of highlighting opportunities to engineers to follow up is.

It’s important to step back from the cloud-specific concepts and begin to focus on the core issue, which is: how do you motivate others to make changes and incorporate new things into their workflow? Consider the task you are trying to achieve when you generalize the challenge to be as simple as “you need engineers to dedicate some time to perform a recommended action.”

As seen in Figure 3-4, the action scale visualizes the balance between how likely engineers will be to implement your recommendations. The scale has many factors that tip either in favor of action or against.

Figure 3-4. The action scale

Conclusion

Throughout this chapter, we’ve emphasized that every team must adopt a FinOps mindset. All teams inside your organization are able to work together to understand one another’s goals alongside a centralized FinOps team that is helping to build out reporting and practices to assist everyone in achieving them.

To summarize:

• All teams have a role to play in FinOps.

• Teams have different motivators that drive spend and savings.

• Teams need to work together with a balance of empathy for one another’s goals.

• FinOps practitioners help align teams to organizational goals.

• Motivating engineers to take action is not a new cloud problem but an age-old challenge humans have faced.

• The action scale helps visualize the items that contribute to getting engineers to take action and the items that detract from it.

• To tip the action scale in favor of action, you need to implement programs to help the contributing factors and reduce the detractors.

In the next chapter, we’ll look into how teams talk to each other. You’ll discover that good collaboration requires more than placing teams in the same room. The language of finance can be vastly different from that of engineering, so everyone must embrace a common language.