Every enterprise AI roadmap reaches the same meeting. The architecture is sound, the use cases are ranked, the pilot worked. Then someone from finance asks where the money comes from, and the roadmap moves to next year’s planning cycle.
The assumption underneath that meeting is that AI needs new budget. For most organisations it doesn’t. It needs budget that is already being spent, badly, on infrastructure sized for a workload that no longer exists.
What is inference economics?
Inference economics is the study of what it costs to run AI models in production, as distinct from what it costs to train them. For most enterprises, training is now someone else’s problem and inference is the entire cost story, a shift analysts have started calling the inference flip.
What makes this urgent is agentic architecture. A single chatbot call is one inference. An agent that plans, retrieves context, invokes tools, evaluates its own output and retries is many inferences per completed task. Gartner’s March 2026 analysis found that agentic workflows consume between 5 and 30 times more tokens per task than a standard chatbot query. EY puts the same shift in currency: a customer-service interaction that cost around $0.04 as a simple retrieval-and-response flow in 2023 costs roughly $1.20 in 2026 as an orchestrated agentic system.
That multiplier is the reason pilot economics so rarely survive production. Nothing broke. The workload simply changed shape, and the budget was built on the old one.
Why does an agentic AI roadmap need a FinOps discipline before it needs GPUs?
Because every inefficient habit that costs you a manageable amount on general-purpose compute costs many times more once accelerated compute and per-token billing are involved. The discipline has to exist before the expensive workload arrives, not after.
AI FinOps, sometimes called inference FinOps, is the practice of governing AI compute the way mature organisations govern any other consumption-based spend: per-workload attribution, demand-matched capacity, routing decisions made on cost as well as quality, and an owner attached to every line. The metrics that matter are cost per inference and, for agentic systems, cost per completed action, since one business outcome may involve a dozen model calls, several tool invocations and a retry.
Almost nobody stands this up in advance. It gets built after the first bill that nobody can explain.
Where is the money hiding in a legacy estate?
In capacity that was sized once, at migration, against the busiest day anyone could imagine, and never revisited. It resolves into a handful of specific, measurable leaks. Most enterprise estates have several running at once.
Provisioned for peak, billed at peak, used at trough. Fleets sized against a quarter-close still carry that capacity in February at two in the morning. Average utilisation across a typical lift-and-shift estate sits well below what anyone would defend if they saw it on a dashboard. Nobody sees it, because nobody owns it.
One workload per instance. The virtual machine was the unit of deployment, so it became the unit of billing. A service needing 1.5 vCPU gets a 4 vCPU instance because that was the next size up. Multiply by a few hundred services.
Instance generations frozen at the date of migration. Workloads that moved in 2019 are frequently still on 2019-era families. Current generations and ARM-based compute deliver materially better price-performance for identical work, and for containerised services the change is often a rebuild rather than a rewrite. The saving is real. The reason it hasn’t been taken is that nobody has a mandate to touch a system that currently works.
Everything on-demand, forever. Steady-state baseline at on-demand rates. Interruptible batch at on-demand rates. Dev and test environments at on-demand rates across a weekend when nobody is logged in.
Managed-service floor prices on capability you barely use. A managed vector search tier with a monthly minimum around $350 is a rounding error once. It is not a rounding error across thirty environments.
Storage and telemetry that only ever grow. Older volume types never migrated, snapshots without lifecycle rules, logs retained at full fidelity for years because deciding what to delete requires a meeting.
Individually, none of these is large enough to escalate. Together they are frequently the largest uncommitted line in the infrastructure budget.
Why legacy compute is the right budget to reach for
This is the part that turns a cost-cutting exercise into a strategic one.
The capabilities that release money from a legacy estate are precisely the capabilities that make an agentic estate affordable. Bin-packing. Demand-driven autoscaling. Scale-to-zero. Per-workload cost telemetry. Environment parity. There is no second, separate discipline for GPU.
So the sequencing works in your favour. Build these once on general-purpose compute, where a mistake costs a manageable amount, and you inherit them on accelerated compute, where the same mistake is multiplied by unit cost and by the token multiplier above it. Teams that skip the step tend to discover it later, in production, at GPU prices. Gartner’s projection that a large share of agentic AI projects will be cancelled before 2027 is not primarily a story about model quality.
What actually moves the cost-to-performance ratio?
In rough order of effort against return:
- Instrument before you optimise. Continuous per-workload, per-team, per-agent cost attribution that names an owner. Not a monthly export reconciled by hand. Waste without an owner does not get fixed, and you cannot defend a reallocation you cannot evidence.
- Rightsize against observed demand, not the sizing document written during migration.
- Consolidate the fleet. Moving from one-workload-per-instance to a scheduled, bin-packed cluster is where the step change sits, because it attacks overprovisioning structurally rather than instance by instance. It is also the step deferred longest, because it looks like a migration programme. With IaC golden paths it is closer to a guided process.
- Take the architecture arbitrage. Current instance generations, ARM where the workload allows, spot for anything genuinely interruptible.
- Match commitment structure to demand shape. Committed discounts against the stable baseline, elastic capacity above it, nothing on-demand by default.
- Retire premiums you aren’t using. Self-hosted open-source components where the managed floor price exceeds the value of the management.
- Enforce retention at the platform layer, so the policy survives the next reorganisation.
None of this is exotic. The reason it doesn’t happen is almost never technical. No team has both the mandate and the capacity, and the work competes with a product roadmap it will always lose to.
The governance step everyone skips
If you take one thing from this post, take this one.
Savings that are not ringfenced in advance get absorbed. You run the programme, the infrastructure line comes down, finance recognises a permanent efficiency, next year’s baseline resets lower, and the AI roadmap is exactly where it started with a tidier cloud bill.
Agree the reallocation before the harvest, in writing. The proposal is not “we would like to spend less on infrastructure.” It is this: a defined quantum of recovered spend moves to the AI line, at a defined cadence, with the telemetry that proves it was recovered. That reframes AI from a new cost centre needing a new business case into a redirection of committed budget. In a flat-budget year, only one of those gets signed.
One trap on the way
There is a version of this argument that ends somewhere bad.
Optimise on unit cost alone and the cheapest-looking option for AI workloads is always the shared inference layer: nothing to provision, a per-token price that looks trivial at pilot volume. Two problems. It scales linearly with usage and never bends down, so it degrades precisely as the initiative succeeds, which is the whole content of the 5 to 30 times token multiplier above. And it is the option that moves your data outside your own boundary to get an answer.
For workloads touching client records, patient data, matter files or proprietary process IP, that second point isn’t a cost decision at all. It’s a decision about whether the workload can exist. Recovered budget is what makes the sovereign option viable: self-hosted inference on GPU you control has a real fixed cost, and past a certain volume it is the cheaper one as well as the compliant one. We’ve written up the regulatory side of that decision for European buyers here.
Cost and control are usually presented as a trade. Run the numbers past the crossover point on any high-volume workload and they stop being one.
Where XePlatform fits
XePlatform operates this layer inside your own cloud account: Kubernetes provisioning and lifecycle, GPU scheduling with MIG partitioning and autoscale-to-zero, demand-driven autoscaling across compute and inference, and per-workload cost telemetry unified across private GPU and managed endpoints. Customers typically see GPU spend reduced by 30 to 40 percent against self-managed clusters, without hiring a platform engineering team to get there.
The same platform runs your modernised core workloads and your agentic estate, because they are the same problem at different unit costs.
To size the crossover for a specific workload, use the AI Infrastructure Calculator. To walk through your own estate with someone, book a session.
FAQ
What is inference economics? Inference economics is the cost structure of running AI models in production, as opposed to training them. For most enterprises, inference now dominates AI spend, because agentic workloads invoke models many times per completed task rather than once per user query.
What is AI FinOps? AI FinOps is the practice of governing AI compute spend through per-workload attribution, demand-matched capacity, cost-aware model routing and clear ownership. Its core metrics are cost per inference and cost per completed action.
How much more expensive are agentic AI workloads than chatbots? Gartner’s March 2026 analysis found agentic workflows consume 5 to 30 times more tokens per task than a standard chatbot query. EY estimates a customer-service interaction has moved from roughly $0.04 in 2023 to around $1.20 in 2026 under an orchestrated agentic design.
Can we reduce GPU costs without moving off our current cloud provider? Yes. Most GPU savings come from scheduling rather than procurement: autoscaling to zero when idle, MIG partitioning so each GPU is shared across workloads, and cost telemetry that identifies which model or team drives spend. XePlatform runs inside your existing AWS, Azure or GCP account.
Is self-hosting models cheaper than calling an API? Past a volume threshold, yes. API pricing scales linearly with usage forever, while self-hosted inference has a fixed infrastructure cost that amortises as volume grows. Where that crossover sits depends on the workload, which is what the AI Infrastructure Calculator is for.
Do we need a platform engineering team to do this? No. XePlatform operates the Kubernetes, GPU scheduling, release engineering and observability layers inside your cloud account, which is typically what would otherwise require a dedicated platform hire.
