You Want Your Token Costs To Explode (But That's Also Why You Need A Governance Layer)

Early in our company history, one of our customer's top 10 most-used agents was one an employee had built to help him write a fantasy novel. More power to him—but not something that should happen on the company dime. The customer course corrected and the employee moved that work off the company workspace.

The point here is that management of AI is critical, and it's only possible if management and visibility is easy. "What are they actually using it for?" is one important question. Once that's solved, "Are they doing it efficiently?" becomes important. Once you've solved that, "What more could we be doing?" takes prominence. Companies that are creating the most value for customers are in the third paradigm.

Why costs are going up

Three things are happening simultaneously across every serious AI platform.

First, AI tools are getting better at fetching the right context before they respond. Rather than relying on the user to supply all relevant information in the prompt, platforms are pulling in role context, company knowledge, related data sources, and prior work automatically.

Caveat, this can be hard to measure. More context going in means larger individual requests, but it also means dramatically better output on the first try, which reduces the back-and-forth needed to get to a useful result. One larger request that lands correctly is cheaper per task than ten smaller exchanges that gradually get there. Per-request averages go up, per-task averages may go down, and the relationship between the two is wiggly enough that drawing conclusions from either number alone will mislead you.

Second, scheduling means AI is running far more frequently than it used to. Agents trigger on a cadence, process inputs, and surface results across the organization continuously. Usage compounds in ways that are easy to underestimate when you are only thinking about individual sessions.

Third, it is now more possible to string together complex workflows involving many tools. A single goal can kick off a chain of integrations, data fetches, drafts, and updates across multiple systems. Each step in that chain has a cost, and the chains are getting longer as reliability improves and ambitions grow.

All of that combines into a forecast: token usage will go up, probably significantly, and at a pace that tracks with how well your AI rollout is actually going.

Rising costs are the right outcome

Here is the thing most organizations get backwards. Controlling AI spend by limiting usage is a capability suppression strategy, not a cost management strategy.

The whole value proposition of enterprise AI is that your employees accomplish more meaningful work, faster, with less friction. If they are doing that, they are using more AI. If they are using more AI, they are spending more tokens. The line items move together.

A workforce that is aggressively using AI to produce higher quality output at greater scale is exactly what you paid for. Token costs going up is evidence that you are getting it.

The question is whether costs are going up for the right reasons

This is where the asterisk matters.

Increased token usage is a good sign if your employees are pursuing ambitious, high-value work. It is a bad sign if they are querying the wrong model for a simple task that a cheaper one handles just as well. It is a wasted sign if someone is running a personal project through the company's AI subscription. It is an invisible sign if half your team is using AI through a dozen disconnected SaaS tool integrations that no one has visibility into.

These are real concerns, the natural result of AI use spreading organically across an organization without infrastructure behind it.

You can improve the efficiency of token use

Are employees using the right model for each task? Are the agents using context effectively? Is there a scheduled task that runs every day, but no one uses the output? Is usage concentrated in a handful of power users while the rest of the organization sits idle? Are there workflows running redundantly across multiple tools? Is anyone actually doing anything they should not be? 

Without a governance layer, you have no way to answer any of those questions.

Why a fragmented stack makes this unsolvable

Many organizations have landed in a position where AI access is technically available everywhere and actually governed nowhere.

There is the Microsoft Copilot license that covers Office. There are the AI features built into the CRM. There is the model subscription some employees signed up for individually. There is the automation tool the ops team started using. There are the AI capabilities baked into the project management software.

Each of those has its own usage, its own costs, its own audit trail or lack of one. None of them talk to each other. None of them give anyone a coherent picture of what the organization is actually doing with AI. Maybe you have a unique API key for every tool, or every team. That still does not tell you much except for the size of the bill.

Signing employees into a dozen separate products with AI features does not give you enterprise AI. It gives you enterprise shadow IT with better PR.

The same problem applies to locking into a single model provider or a single software suite. The moment a better model for your use case exists outside that ecosystem, you either miss it or you add another disconnected tool to the pile.

What governance actually requires

A governance layer for AI is only possible if AI use is centralized, and centralization only works if the platform can do everything employees actually need.

That means any model, so employees always have access to the best option for their task and you can track which is being used for what. It means any integration, so workflows run inside the governed environment rather than outside it because the tool they need is not available. It means any employee, from technical to non-technical, so the population using the system represents the whole organization and usage data is actually meaningful.

When everything runs through one platform, cost visibility and workflow visibility become the same thing. You can see where tokens are going, what they produced, whether it was worth it, and where you would want to shift behavior.

The goal is high usage you can understand

The organizations that will get the most out of AI are the ones that actively encourage usage while building the infrastructure to see what that usage produces.

Rising token costs, in that context, become an indicator of organizational health rather than a budget line item to minimize. You want them going up. You just want to know exactly why.

‍