FinOps for AI - A practical blueprint for enterprises (Part-2)

In Part-1 of this series, we looked at why AI FinOps or FinOps for AI is similar but a genuinely different discipline from cloud FinOps, what the full cost stack looks like from silicon to SaaS, and why three distinct AI consumption patterns each need their own governance approach. This second part provides a practical blueprint for what carries over, what must change, who should own what, and where enterprises should begin

The good news is that you do not start from zero. A meaningful portion of your existing FinOps practice transfers directly. The caveat is that more needs to be rebuilt than most guidance admits, and the rebuild is not primarily a tooling project. It is an operating model redefinition.

What carries over from Cloud FinOps

Several FinOps disciplines carry over, although most require some adaptation for the speed and variability of AI workloads.

Cost allocation and showback still apply wherever you have infrastructure you control. If your AI workloads run on cloud-hosted GPU instances, your existing tagging taxonomy, account structures, and showback processes work. Apply them with the same rigour you would to any cloud workload.

Commitment-based discounts follow the same break-even logic as compute reservations. For sustained, predictable inference workloads on a self-hosted model, reservation arithmetic is familiar and the savings are comparable to what you see with EC2 reserved instances. The difference is that AI workloads evolve faster than traditional workloads, so shorter reservation horizons are generally more appropriate.

Anomaly detection and budget alerting transfer in mechanics but not in cadence or thresholds. AI costs can move within hours in ways that weekly review cycles will never catch. Agentic workloads can exhaust monthly budgets in a day if a loop has no stopping condition. The same alerting infrastructure applies, but it needs to be tuned for much shorter detection windows and lower absolute thresholds.

The governance culture, which is the hardest FinOps asset to build, is the most portable. If engineers take ownership of cost, product managers treat cost as a key product requirement, and finance participates in trade-off decisions rather than simply receiving invoices, the organisation already has a strong foundation for AI FinOps.

Three deployment tracks, three governance models

The consumption patterns introduced in Part-1 explain how value should be measured. Deployment tracks explain where costs appear, what telemetry is available, and which controls can be applied.

The most useful reframe that emerges from working across AI infrastructure is that governance should be designed per deployment track, not as a single unified framework. Three structurally different deployment models exist, and each has different cost drivers, different tooling, and different ownership.

For self-hosted AI infrastructure, where organisations run their own GPU clusters either on-premises or as cloud IaaS, the governance model looks most like traditional infrastructure management. Cost drivers are hardware utilisation, serving stack efficiency, and workload scheduling. The primary instrument is NVIDIA DCGM for GPU metrics, not provider billing dashboards. No LLM gateway is relevant here because costs sit below the API surface entirely. Unit economics must be computed manually by dividing infrastructure billing cost by tokens served. The FinOps practitioner in this track works through infrastructure and ML engineering teams to implement changes; they cannot optimise directly.

For API-consumed AI, where teams call model provider APIs such as OpenAI, Anthropic, AWS Bedrock, or Google Vertex AI, the LLM gateway is the foundational governance instrument. An LLM gateway deployed as a mandatory shared platform service, through which all API calls must route, provides the attribution data that makes cost governance possible. Without it, you are looking at provider invoices with no ability to connect spend to business context. Model routing, prompt caching, and output token constraints are the primary optimisation levers. The governance approach differs further depending on whether the workload is a simple chatbot, a RAG pipeline (where 40% of cost often sits in embedding calls and vector database queries, not the LLM call), or an agentic application requiring task-level budget limits.

For SaaS and productivity AI, much of the cloud FinOps toolkit has limited relevance. Microsoft controls the model, the API, and the billing for M365 Copilot. Governance here is ITAM work combined with an HR-led adoption program. The metric that matters is active users divided by licensed seats, and the target should be above 60%. Most enterprises sit at 20 to 30%, which means the cost problem is an adoption problem, not a tool problem. At 5,000 seats and $30 per user per month, getting from 25% active to 60% active does not just improve the ROI case; it also gives you the usage data you need to negotiate the renewal with Microsoft from a position of knowledge.

Hybrid environments, which is what most large enterprises actually operate, require separate governance tracks running in parallel. The FOCUS (FinOps Open Cost and Usage Specification) standard is the most practical normalisation layer available today for aggregating cost data across these three tracks into a unified executive view.

Unit economics as the primary framework

Cloud FinOps rewards frugality. The primary signal is cost reduction. AI FinOps rewards value density. The primary signal is value generated per dollar of spend, and there is a new term for this: Tokenomics

This means building unit economics into the governance framework from the beginning. Not at the level of tokens or GPU hours, but at the level of business outcomes. Cost per support ticket resolved. Cost per purchase order processed by an agentic workflow. Cost per AI-assisted document completed. Cost per fraud transaction screened.

This framing changes every governance question. A model upgrade that increases cost per token by 30% may be correct if it increases task completion rate by 50%. Quantising a model to cut inference cost by 40% is a poor trade if it reduces task accuracy by 15% on your specific workload. These are trade-offs that require collaboration between ML engineers, product managers, and FinOps practitioners, and they cannot be resolved by looking at a token spend dashboard.

In our experience managing AI infrastructure for enterprise clients, the organisations that measure AI cost in tokens or GPU hours plateau at cost visibility. The organisations that measure it in business outcomes per dollar are the ones that can make rational optimisation decisions and demonstrate AI value convincingly to leadership.

The tooling landscape, impartially assessed

The AI FinOps tooling market is more mature than it was two years ago, but it is not yet complete. Here is an unbiased picture of what exists. This is a point-in-time assessment of a rapidly changing market. Cloud billing platforms such as AWS Cost Explorer, Azure Cost Management, and most popular third-party Cloud FinOps tools work well for IaaS-hosted AI workloads. They see GPU instances, storage, and networking. They have no visibility into what those instances are doing, no understanding of model serving efficiency, and no concept of tokens or AI-specific cost levers. Some of them have started adding visibility into AI subscription usage.

LLM gateways, like Portkey, Helicone, LiteLLM, and TrueFoundry, are the most practically useful new tools for AI Inference FinOps, even though most practitioners do not frame them that way. Deployed as mandatory shared services, they provide the attribution data, routing policy enforcement, and budget limit capabilities that make API-consumed AI governable. They are well suited to simple inference, RAG pipelines, and embedded product AI. They are less effective for agentic workloads where task-level cost visibility requires framework-level instrumentation that gateways cannot provide alone.

Agentic observability platforms such as LangSmith, Arize, and Weights and Biases trace multi-step agent workflows and can surface per-task cost alongside quality and reliability metrics. These tools were built for ML engineers, not FinOps teams, but they contain the task-level cost data that agentic FinOps depends on.

For SaaS-delivered AI, the relevant tools are Microsoft Copilot Analytics and Viva Insights for M365, vendor dashboards for domain SaaS, and SaaS management platforms such as CloudNuro for seat governance.

Then we have AI Tokenomics (the unit economics equivalent of Cloud FinOps) which acts as a layer above the infrastructure. It focuses on the consumption and value of AI. Because AI inference bills scale based on tokens, tokenomics tracks exactly which features, agents, or end-users are driving up costs, connecting the AI spend directly to business value

The gap is in providing unified visibility across all three deployment tracks. There is no mature, purpose-built tool today that integrates from GPU utilisation to token consumption to business outcome on a single pane. The tooling market is converging toward this, but it isn't there yet. For enterprises managing AI at scale, this means a near-term architecture of: cloud billing tool + LLM gateway + observability platform + a business metrics layer that you build and maintain. It is not a clean off-the-shelf answer, but it is a buildable one.

Who needs to own what

The roles and responsibilities picture for AI FinOps is considerably broader than the current Cloud FinOps structures, and it is the piece most enterprises underestimate.

In self-hosted inference, the FinOps practitioner sets unit economics targets and reports on them, but the infrastructure team makes the hardware and serving stack decisions that determine cost, and the ML engineering team makes the model optimisation decisions that move cost per token. The FinOps practitioner cannot optimise directly. They govern through the infrastructure and ML teams.

In training workloads, the most cost-determinative decisions are made by the ML researcher who designs the experiment, before a single GPU is provisioned. Architecture choices, parameter counts, training duration, and experiment plan quality have more cost impact than any infrastructure decision. FinOps governance here is primarily a stage-gate model: pre-run estimate requirements, go or no-go gates at phase transitions, and post-run reconciliation. Monthly reviews are too slow and too coarse for training cost governance.

In API-consumed workloads, the engineering team owns prompt engineering, which at scale is a financial decision. A 30% reduction in average prompt length is a 30% cost reduction for that workload. The FinOps team deploys and enforces the LLM gateway as a shared platform service. Product managers own the cost-per-user review.

In productivity AI, the primary value lever is owned by corporate functions like HR and Learning and Development. The adoption program drives active usage, which is what determines whether the investment pays off. FinOps governance here is seat cost management and renewal preparation, which requires ITAM skills more than cloud FinOps skills.

The practical implication is that AI FinOps is even more a cross-functional capability, not a centralised function. The FinOps team provides the governance framework and financial accountability model. The infrastructure, ML engineering, product, and HR teams provide the levers.

A practical starting sequence

Given the scope, the risk is trying to govern everything before you have the fundamentals in place. A staged approach works better.

Audit spend: Start by understanding what you actually have. Audit AI spend across all three deployment tracks. Pull cloud billing for GPU workloads, review SaaS procurement records, and survey teams for API subscriptions. Most organisations find their actual total AI spend is 20 to 40% higher than what finance is tracking, largely because of shadow AI and untracked developer tool spend.

Instrument the largest spend category: Then get visibility into your largest spend category before optimising anything. If most of your AI spend is in SaaS licences, pull adoption data from vendor dashboards. If it is in API consumption, deploy an LLM gateway before doing anything else. If it is in self-hosted infrastructure, instrument GPU utilisation via DCGM. You cannot govern what you cannot see, and it is worth investing in instrumentation before moving to optimisation.

Define unit economics: Then define unit economics for your top three AI use cases by spend. Not token costs, but business outcome costs. This is the metric that connects AI spend to business value and that makes executive conversations about AI investment tractable.

After that, the optimisation activities, the tooling decisions, and the R&R conversations all have a foundation to build on. The accompanying AI FinOps Decision Map helps teams identify the relevant governance path based on deployment track, consumption pattern, cost visibility, and business-value maturity.

Where this is heading

AI FinOps will mature quickly. The FinOps Foundation's expansion of its mission from cloud value to technology value signals where the discipline is going, and AI is the primary catalyst. The tooling gaps that exist today will close. The frameworks will converge.

The organisations that will govern AI economics well are not waiting for the market to solve it. They are building the cross-functional capability now: the ability to connect a GPU procurement decision to a serving stack configuration to a token consumption pattern to a business outcome, and to make architecture trade-offs on that basis.

That capability is what a serious AI FinOps practice looks like and that capability doesn't come from a tool. The tooling will change quickly.

The durable advantage will be the operating model: infrastructure, engineering, finance, and product teams making AI investment decisions from a shared view of cost, performance, and business value.

AI FinOps: Decision Map Tool (https://www.microland.com/finops/ai-finops-map.html)* various industry sources.