Six months ago, your AI pilot cost a few hundred dollars a month. Today, it’s a line item your CFO asks about by name. Nobody signed off on that increase, it just happened, one API call at a time.

This isn’t a hypothetical. A February 2026 survey of 500 finance leaders at large US and UK enterprises, conducted by Sapio Research and commissioned by DoiT, found that the large majority of enterprises had experienced AI cost overruns in the prior 12 months. Separate research from cost-governance firm Mavvrik, covering 372 enterprises, found that only about 15% of companies were able to forecast their AI costs within 10% of actual spend — and roughly a quarter missed by more than half.

The pattern shows up at the extreme end too. Uber’s CTO told reporters the company had burned through its entire annual AI coding budget in about four months, prompting a hard per-employee spending cap. Axios separately reported an unnamed large enterprise that spent $500 million in a single month after rolling out AI access without usage limits. Neither company was reckless by industry standards, they were early, and they ran into the same structural problem every organization on token-based pricing eventually hits.

This is the quiet crisis sitting inside almost every enterprise AI rollout right now: token-based pricing was built for experimentation, not for production at scale. And the gap between those two things is where budgets go to die.

Cost is only one symptom of a bigger shift, though. As enterprises move AI from pilots to production, pricing predictability turns out to be one layer of a much larger infrastructure problem alongside governance, routing, and security. This post focuses on the cost layer specifically; see our full breakdown of what enterprise AI infrastructure requires end to end for the bigger picture.

“The problem isn’t that AI is expensive. It’s that enterprise AI has outgrown the pricing model it started with.”

The Pricing Model That Made Sense at 1% Scale

Token-based pricing wasn’t designed for enterprise production workloads , it was designed for developers experimenting against an API. Cheap to start, no upfront commitment, pay only for what you use. That’s the right model for a proof of concept handling a few hundred requests a day, a single developer testing an idea, or a low-stakes internal tool nobody depends on yet.

It was never designed for multi-agent systems, always-on copilots, or production workflows an entire company routes through daily and that mismatch, not the pricing model itself, is the actual problem. The model isn’t broken. It’s being run far past the scale it was built for.

The problem is that nobody re-evaluates the pricing model when the project graduates from pilot to production. The same per-token meter that cost $200/month in the pilot phase scales linearly, sometimes worse than linearly as usage grows. And usage always grows faster than anyone plans for, because:

• Adoption compounds. Once one team sees value, three more teams want in. Nobody budgeted for four teams.
• Context windows keep growing. Every prompt now carries more history, more retrieved documents, more system instructions.
• Agentic workflows multiply calls. A traditional chatbot generates one inference per user message. A modern AI agent handling the same request can generate a planning step, memory retrieval, tool execution, verification, and reflection : each its own model call. Illustratively, a single user request can trigger somewhere in the range of 20 to 50 inference calls behind the scenes. The user still sees one interaction; Finance sees dozens of billable events for it.
• Retry and error-handling logic burns tokens silently. Failed calls, retries, and fallback chains all meter the same as successful ones.

None of this shows up as a single alarming spike. It shows up as a bill that’s 15% higher than last month, then 15% higher again, until a quarter-end review reveals the number has tripled or quadrupled with no corresponding decision point where anyone approved

How AI deployments scale from pilot through production to a governed enterprise platform.

Why This Catches Finance Teams Off Guard

Traditional infrastructure costs are forecastable. You provision a server, you know its capacity, you know its monthly cost. Token billing breaks that model in three specific ways:

1. Usage isn’t controlled by the people who see the bill. Engineering and product teams make decisions that directly move the cost needle; Finance sees the invoice weeks later with no visibility into which decision caused which increase.

2. There’s no natural ceiling. A server has fixed capacity and scaling is a visible, budgeted decision. Token metering has no such gate.

3. Vendor pricing itself is a moving target. Providers periodically change per-token rates or deprecate cheaper models, a cost model budgeted in Q1 may not hold by Q3.

The data backs up how widespread this is: separate 2025 research from Mavvrik and BenchmarkIT found that 80–85% of enterprises miss their AI infrastructure cost forecasts by more than 25%. And counterintuitively, the same DoiT/Sapio research found that organizations with the most mature FinOps practices reported the highest overrun rates, not because mature teams are careless, but because mature measurement is what actually surfaces problems that less rigorous organizations never detect.

Put together, this is what a “ticking time bomb” actually looks like: not a single catastrophic event, but a cost structure with no brakes, no visibility, and no fixed point where someone has to say yes before the number goes up again.

The Questions Worth Asking Before It’s Your Budget

• Do we know, right now, what would happen to our bill if usage doubled?
• Is there a single owner who sees both the engineering usage patterns and the resulting invoice?
• Are we metering cost per feature or per team, or is it one undifferentiated number?
• If a model vendor changed pricing tomorrow, would we know how exposed we are?

Most teams can’t answer these cleanly. That’s the actual risk , not that AI is expensive, but that the cost is unpredictable and largely invisible until it’s already a problem.

A Worked Example: What This Looks Like in Practice

The following is an illustrative example, not a real customer’s invoice. Every assumption is stated explicitly so the numbers can be substituted with your own and a couple of assumptions in an earlier draft of this example were too aggressive. Both are corrected here: the price-per-token used was closer to flagship-tier pricing than a typical production deployment, and it left out output-token cost, which every real invoice includes.

Assumptions

• 1,000,000 queries per month (a realistic volume for a company-wide internal assistant or customer support tool)
• 8,000 input tokens per query (a RAG-style prompt: retrieved document chunks + system instructions + user question), realistic for a system that isn’t actively managing context size
• 500 output tokens per query (a moderate-length response)
• Pricing based on Claude Sonnet 4.6-class rates: $3 per million input tokens, $15 per million output tokens — real, published mid-2026 rates for a commonly used production-tier model, not the cheapest or the most expensive option available

Pure token pricing, production-tier model

Input:  1,000,000 × 8,000 tokens = 8,000M tokens × $3/M = $24,000

Output: 1,000,000 × 500 tokens = 500M tokens × $15/M = $7,500

Total:  $24,000 + $7,500 = $31,500/month

If the workload runs on a flagship-tier model instead (roughly $10/$50 per million , a reasonable choice when output quality matters more than cost):

Input:  8,000M × $10/M = $80,000

Output: 500M × $50/M = $25,000

Total:  $105,000/month

So the honest range for “pure token pricing, unoptimized” on this workload is $31,500 to $105,000 a month — not a single number, since it depends heavily on which model tier is the default choice.

After optimization

Truncating documents, semantic reranking so fewer irrelevant chunks are sent, caching common queries, compressing prompts, and capping output length. Assume these bring average input context down from 8,000 to 4,000 tokens , a defensible ~50% reduction, achievable without heroics — and output down to 300 tokens via a tighter response cap.

Production-tier: 4,000M × $3/M + 300M × $15/M = $12,000 + $4,500 = $16,500/month

Flagship-tier:   4,000M × $10/M + 300M × $50/M = $40,000 + $15,000 = $55,000/month

Owned infrastructure

Self-hosted open-weight model on Kubernetes with dedicated GPUs, derived bottom-up from GPU-hour pricing rather than an assumed token rate:

Reserved GPU-class instances at ~$1.20/GPU-hour (realistic committed-use cloud rate)

2 GPUs running continuously: 2 × 24 × 30 = 1,440 GPU-hours/month

1,440 × $1.20 = $1,728/month raw compute

+ ~45% overhead (storage, networking, monitoring, redundancy) ≈ $2,500/month

Realistic range: $2,000–$3,500/month — largely independent of which API model tier you’d otherwise compare against, since you’re paying for GPU capacity rather than a per-token margin.

Monthly cost for the same 1M-query workload across three realistic scenarios.

That’s roughly a 6x to 7x cost difference between optimized production-tier API pricing ($16,500/month) and owned infrastructure (~$2,500/month) for the same workload , widening to roughly 30x to 40x against unoptimized, flagship-tier API pricing.

The multimedia add-on

Rather than assuming a single video’s content gets loaded into every one of 1 million monthly queries, an unrealistic usage pattern, since no retrieval system works that way — a fairer comparison holds retrieval frequency constant. Say a support video is relevant to and retrieved in 15% of monthly queries (150,000 queries), a heavily used but not universal reference:

Text-equivalent FAQ article (~600 tokens): 150,000 × 600 = 90M tokens × $3/M = $270/month

Video transcript + visual metadata (~1,500 tokens): 150,000 × 1,500 = 225M tokens × $3/M = $675/month

Video content costs roughly 2.5x more per retrieval than an equivalent text document carrying similar information — a real, durable effect worth planning for, just not a “$15,000 for one video” framing that assumes a usage pattern no real system follows.

None of this means token pricing is a bad choice for every workload , for low-volume, exploratory, or bursty use cases it’s still the right tool. But at real production scale, even the more conservative 6-7x gap is large enough to change the conversation from “how do we shave 20% off this bill” to “should this workload be on this pricing model at all” , and the answer gets more urgent, not less, the more a team defaults to flagship-tier models without a routing strategy.

Reframe the Number: Cost Per Business Outcome, Not Cost Per Month

A monthly total like “$31,500” is easy to argue about and hard to act on , it invites a debate about whether AI is “worth it” in the abstract. The more useful question is what that spend produces, broken down to the unit a business actually tracks.

The same total bill looks different depending on how you divide it:

• If the workload handles customer support, divide by tickets resolved: cost per support ticket.
• If it’s processing insurance submissions, divide by claims processed: cost per claim.
• If it’s reviewing contracts, divide by documents reviewed: cost per legal document.
• If it’s an engineering copilot, divide by pull requests reviewed or shipped: cost per PR.

This isn’t about producing a specific benchmark number — there isn’t a universal “right” cost per ticket or per claim, and anyone quoting one without your actual data is guessing. The point is the framing itself: “$31,500 a month” invites a debate about whether AI is expensive. “Cost per resolved ticket, compared to what a human agent costs per ticket” invites a decision. Track your own workload’s real unit cost before and after any optimization, and the conversation moves from abstract budget anxiety to a number finance can actually approve or reject.

The Alternative: Infrastructure You Can Actually Forecast

The fix isn’t “use AI less.” It’s decoupling your cost structure from a per-call meter you don’t control. Two approaches matter here, and they’re not mutually exclusive:

Fixed-capacity infrastructure. Running models on infrastructure you provision converts an open-ended variable cost into a forecastable fixed one. You know the ceiling because you own the ceiling.

Hybrid workload routing. Routing sensitive or high-volume workloads to owned infrastructure, while reserving metered API calls for genuinely bursty or low-volume use cases, gives you predictability where it matters most without giving up flexibility entirely.

This is the same logic that pushed most companies off pure pay-per-use cloud compute a decade ago and toward a mix of reserved capacity and on-demand scaling. AI infrastructure is simply catching up to a lesson the rest of the stack already learned.

The Real Takeaway

Token-based pricing isn’t inherently bad — it’s the right model for exploration and low-volume use cases. The danger is treating it as a permanent foundation for production workloads, where usage, context size, and call volume all move in one direction: up, and mostly out of Finance’s line of sight.

The organizations that get ahead of this aren’t the ones spending less on AI. They’re the ones who moved cost predictability up the priority list before a surprise invoice forced the conversation.

Cost, though, is just one piece of what it takes to run AI reliably at enterprise scale. Governance, routing across models, security, and observability all matter just as much once AI moves from a pilot to a production system many teams depend on. See the full playbook on building enterprise AI infrastructure for how these pieces fit together.

If your team is running AI workloads in production and can’t confidently answer what your infrastructure will cost at 2x current usage, that’s worth a conversation before it becomes a budget crisis. contact us here

Sources

• DoiT / Sapio Research, State of AI Spend Survey (500 US/UK finance leaders, February 2026)
• Mavvrik & BenchmarkIT, State of AI Cost Management Research (372 enterprises, 2025), as reported by Forbes
• Uber annual AI coding budget and per-employee spend cap, as reported by Forbes (citing The Information)
• Axios, reporting on an unnamed enterprise’s $500 million single-month AI spend : cited via Forbes and Larridin coverage
• Anthropic Claude Platform pricing docs : production and flagship-tier per-token rates