Table of Contents

What is FinOps?

The cloud has become the ideal operations model for modern, data-driven organizations. However, migrating processes and applications to the cloud can entail significant effort and resources, with even the most sophisticated IT teams experiencing unexpected or runaway cloud costs. 

To take advantage of all that the cloud promises and offers, it has become imperative for enterprises that rely on the cloud to optimize their consumption of cloud resources and services. But that, in itself, is a challenge. This challenge has ushered in a new discipline: FinOps or Financial Developer Operations (DevOps).

FinOps Meaning

FinOps—also known as Cloud FinOps—is a term born from the union of two core disciplines—Finance and DevOps, or technology teams. It is a collaborative organizational approach to managing and optimizing cloud spending decisions across an organization. 

According to The FinOps Foundation:

“FinOps is an operational framework and cultural practice which maximizes the business value of cloud, enables timely data-driven decision making, and creates financial accountability through collaboration between engineering, finance, and business teams…”
(Definition Updated: Feb. 2023 by the FinOps Foundation Technical Advisory Council)

In practice, the meaning of FinOps is for an organization to align teams to achieve maximum ROI in the cloud, identify and implement FinOps tools and best practices to optimize cloud resources for maximum efficiency, and ultimately increase cloud revenue.

Why Does Cloud FinOps Matter?

In the past, organizational teams often operated in silos, with leaders focusing separately on financial aspects and operational or development decisions, resulting in inefficiencies and unpredictable costs. What cloud FinOps aims to do is mitigate those silos and bridge the gap between finance and developer operations.

The first step is about cost control. By getting a firm handle on cloud costs between multiple stakeholders, engineers and technology departments can focus their brain power on innovation versus optimization. 

From O’Reilly Media’s Cloud FinOps, 2nd Edition: 

“FinOps is all about removing blockers; empowering engineering teams to deliver better features, apps, and migrations faster; and enabling a cross-functional conversation about where to invest and when.” (Chapter 1, Data-Driven Decision Making).
Another important mention from the book is that the three parts to a successful FinOps practice are Real-time reporting + just-in-time processes + teams working together. While FinOps teams can have organizational structures and processes in place, real-time reporting data—or real-time cost optimization for that matter—is critical for ensuring maximum ROI from your cloud FinOps approaches

By working collaboratively towards common goals, with each team taking ownership of its own cloud usage and working towards optimized price/performance, successful FinOps organizations grow nimble in product delivery, more effective in budgeting and planning in the deployment of their human resources, and more efficient in predicting and managing cloud spend.

CASE STUDY: How Extole Established Its FinOps Culture Through Real-Time Cost Optimization

“No one outside of the operations or finance teams [now] has to think about the cost management for reporting, and that… is immensely beneficial to the business.”

—Ben Smith, VP of Technical Operations, Extole

Cloud Spend: The FinOps Driver

Cloud Spend: Sometimes More Than Anticipated

The rise of cloud computing has disrupted the way modern businesses operate. Cloud computing has become imperative for today’s data-driven enterprises to grow, thrive, and succeed. However, embracing the cloud also means a new emphasis on cloud cost management.

Gartner projects that public cloud spending will grow to almost $679B in 2024. Yet while almost two-thirds of respondents expected to realize cost savings by moving to the cloud, only 46 percent are achieving that outcome. (Gartner’s Public Cloud Initiatives Study 2023)

In the FinOps Foundation’s latest report, The State of FinOps 2024, 50 percent of survey participants labeled reducing waste or unused resources as their highest priority. While organizations have migrated to the cloud with the promise of lower operational expenses, enhanced efficiencies, and more substantial output, many are finding difficulties in maintaining an optimal resource spend in line with performance.

Chart Current FinOps Practitioner Key Priorities Insights V2

Image taken from State of Finops by FinOps Foundation

FinOps More Crucial as Cloud Usage Grows

The ability to quickly scale and deploy resources for new applications is ever enticing, but the dynamic nature of applications makes continuous optimization of resources incredibly difficult.

Resource underprovisioning of application capacity can lead to failures in job execution, so engineers tend to overprovision resources to ensure that their applications run—which can easily lead to overspending.

FinOps can help enterprises achieve the balance between overprovisioning to ensure jobs run with adequate resources and underprovisioning to save costs. With stakeholders and collaboration across multiple departments, there is a greater focus on maximizing the effectiveness of an organization’s cloud spend.

The Foundation of a FinOps Strategy

It can be challenging to start a FinOps strategy from the ground up. FinOps is still an emerging methodology, and every company has its own unique culture and practices, so every strategy is different and iterative. There is no universal template for success. Here are a general set of principles, stakeholder roles, and values to establish for a practical FinOps foundation.

1. The FinOps Framework

The top priority amongst companies with a FinOps culture has become reducing waste or unused resources—showing that cloud cost optimization is the pivotal goal across both small and large organizations.

finops state report graphic

In order to solve for this goal, a strategy needs to be laid out to ensure the business is achieving an optimal ROI from its cloud investment. There are many in-house exercises that can help progress an organization’s FinOps culture, but a FinOps tool or solution is truly needed for reducing waste. 

2. The FinOps Team

FinOps is a team sport, where collaboration across the enterprise toward the enterprise’s goals—with a balance of empathy for one another’s goals—is essential to success. 

While there are no fixed requirements for FinOps team members or its composition, identifies the five major stakeholders that typically collaborate in a FinOps environment:

Executives. CIO, CTO, CFO

Product owners. Director of Product Development, Cloud Analyst, or Business Operations Manager

Engineering. Software Engineers, Systems Engineers, Cloud Architects, and Engineering Managers

Finance/Procurement.Procurement Specialists, Financial Planners, and Business Financial Advisors.

FinOps Practitioners. FinOps Team Lead, FinOps Analyst, FinOps Practitioner

finops framework

3. The FinOps Lifecycle

There are three phases to the FinOps lifecycle—inform, optimize, and operate.

Inform. The first stage of the FinOps lifecycle is about empowerment through gathering and analyzing all the critical information to drive strategic and correct business decision-making. The Inform phase ensures the enterprise has the accurate information and insights to fuel its allocation, benchmarking, budgeting, and forecasting of its cloud spend.

Optimize. Fueled by highly actionable insights, organizations and teams proceed to optimize their cloud footprint. This is done by taking advantage of cloud-provider pricing models and eliminating waste at both an organization’s infrastructure layer and application platform layer.

Operate. The Operate stage entails establishing cloud governance practices and accountability within the organization itself. This can be by improving communications gaps between stakeholder teams, building accountability, or even establishing training programs for best cloud FinOps or cost optimization practices. 

What Are Some FinOps Tools to Optimize Cloud Costs?

Stakeholder buy-in from executives and finance is necessary for a truly successful FinOps culture, but the most important piece lies within engineering—specifically Platform Engineers, DevOps teams, and developers themselves. 

These roles account for managing how many resources certain workloads need as well as when they need them. The resource needs for cloud workloads, such as Apache Spark, change dynamically based on a variety of factors, and keeping up with the requirements of those app resources is practically impossible at scale.

Application Overprovisioning Graph

Five Optimizations That Help But Only Partially Address Your FinOps Problems

While the cloud is an attractive environment for application development and resource scaling, those same factors typically drive cost overruns. 

There are several tools and platforms that may appear as cloud FinOps solutions, but they only optimize at the infrastructure layer, and don’t fix another core, underlying problem—application waste at the Apache Spark task and executor level. 

Here are explanations of what those optimizations do on the surface and why they don’t fully address the problem of application overprovisioning: 

  1. Observability and Monitoring
    Observing where your waste is and having a solution to recommend a next step helps developers identify and quantify the waste in their data stack. That said, finding the waste isn’t fixing the waste. Recommendations also just generate more tasks to complete, which becomes impossible to implement at scale. Observability still requires manual tuning just so you can tune for peak resource usage—which leaves application waste because non-peak times are still driving peak-level costs.
  2. Managed Autoscaling
    Autoscaling enables the automatic addition and removal of instances to match the volatile demands of Spark workloads. However, this solution also fails to solve the problem that Spark tasks tend to waste the resources requested by the autoscaler.
  3. Instance Rightsizing
    Choosing the right instance is one of the first steps in the optimization process, but it doesn’t prevent inefficient applications from creating waste. Application resource requirements change in real time, and selecting the optimal instance type doesn’t address the problem of resource overprovisioning.
  4. Manual Application Tuning
    Manual tuning is both time consuming and tedious. Developers have to adjust requested resources to as close to peak as possible, but manual optimization at scale is practically impossible because of the dynamic nature of modern data stacks.
  5. Apache Spark Dynamic Allocation
    Also at the application level, the Spark scheduler can be configured to add more tasks to executors and kill idle executors whenever possible. However, this does not address the problem that Spark tasks can be arbitrarily overprovisioned and therefore arbitrarily wasteful.

We encourage you to implement any or all of these solutions, but none of them addresses application waste through Real-Time Cost Optimization.

A True FinOps Tool: Pepperdata’s Augmented FinOps Solution for Amazon EMR and EKS

According to Gartner, the only metric that matters in a cloud project is the true price/performance ratio, and an Augmented FinOps tool is the most viable solution for keeping that metric stable. While building a cross-departmental FinOps foundation is essential to maximizing the cloud’s value, total optimization of resources is impossible without an autonomous and continuous optimization solution.

Pepperdata Capacity Optimizer is a real-time cost optimization solution for Amazon EMR and Amazon EKS that augments your FinOps practices without practitioners needing to: 

  • Manually tune configurations or workloads
  • Apply recommendations
  • Make application code changes

Reclaiming Cloud Spend at the Application Layer

Using patented algorithms and Continuous Intelligent Tuning, Pepperdata provides the system scheduler with real-time visibility into what’s actually available in your application clusters, second by second, node by node, instance by instance. 

There are several optimizations you can do at the infrastructure layer, and while those practices may very well improve your overall system performance, Pepperdata takes a completely different and complementary approach by optimizing the capacity with applications themselves. 

Pepperdata removes the blinders from the task scheduler and provides it with a real working picture of what’s going on, intelligently informing the scheduler which instances can take more workload, dynamically and in real time. It’s Augmented FinOps in real time.

With Pepperdata Capacity Optimizer, FinOps teams can achieve immediate cost savings without a major organizational transformation. There’s no need for a crawl-walk-run phase with Pepperdata Capacity Optimizer—simply turn on “The Easy Button,” and you can immediately start optimizing all your applications to maximize revenue and minimize costs. 

Discover how Pepperdata can support your FinOps strategy and help your organization maximize the cloud to its fullest potential without compromising control, speed, and innovation.

060624 thumbnail finserv ebook

Optimizing GenAI Spark Workloads for Performance and Cost: A Primer for FinServ

CO classic datasheet pg1

Capacity Optimizer Datasheet

thumbnail benchmark spark emr 052924

Pepperdata Reduces the Cost of Running Spark Workloads on Amazon EMR 7.0 by 51%

Explore More

Looking for a safe, proven method to reduce waste and cost by up to 47% and maximize value for your cloud environment? Sign up now for a free Cost Optimization Proof-of-Value to see how Pepperdata Capacity Optimizer can help you start saving immediately.