Pasting your OpenAPI spec into Cursor costs 150k tokens and is stale by lunch

An engineer is working with an agent, Cursor or Claude Code, on a change that calls an internal API. The agent needs to know the endpoints, so the engineer does the obvious thing and pastes the OpenAPI spec into the context. The spec for a mature API is large, on the order of 150,000 tokens. It lands in the context window, and three things happen at once. It crowds out the actual code the agent was reasoning about. It costs real money, metered, on that turn and every turn after it. And it is a photograph: the instant it is pasted, it stops tracking the API, so when a new version ships that afternoon, the agent is now writing code against a contract that no longer exists.

The engineer wanted the agent to know one operation, POST /charges. They paid for the entire catalogue to deliver it, and the catalogue was out of date by the time the agent finished reading it.

Paste is the only tool most setups have

This persists because the spec is the right thing to hand an agent and paste is the only mechanism most setups have to hand it over.

The instinct is correct. The spec is the contract; if you want the agent to work against the API, the spec is what it should consult. The problem is entirely in the delivery. Without a retrieval layer between the agent and the spec, “give the agent the API” collapses into “paste the document”, and the document is the whole thing: every path, every operation, every shared schema, all of it, when the task in front of the agent needed one operation and the handful of schemas it references. The unit of transfer is the entire API because there is no smaller unit on offer.

So you get the whole spec whether or not you need it, which is almost always not, and you get it as a copy, which is the second half of the problem. A copy does not update. The spec in the agent’s context is frozen at paste time, and the real one keeps moving.

graph LR
    SPEC["Full spec (~150k tokens)"]
    CTX["Agent context window"]
    SPEC -->|pasted whole| CTX
    CTX -->|evicts the code you are working on| DEG["Worse answers"]
    CTX -->|frozen at paste time| STALE["Stale by the next publish"]
    CTX -->|metered every turn| COST["Paid again and again"]

The tax on the engineer at the keyboard

Day to day, the cost is that the context window is a scarce, shared resource and you just spent most of it on documentation.

A 150,000-token spec does not sit politely in a corner. It evicts the files the agent was actually reasoning about, and it dilutes what remains: models attend less reliably to the middle of a very long context, so the more you paste, the less the agent reliably uses. You pay twice, once in money and once in quality, for information the task mostly did not need. And you re-pay it, because the cost is per turn, not per session, so a long conversation carries that spec on its back the whole way.

Then there is the re-paste tax. The spec changes, so to stay correct you would have to notice the change and paste the new version. You will not, reliably, because noticing every upstream API change is exactly the thing humans are bad at, so the agent quietly slips a version behind and keeps answering with total confidence. The expensive copy and the stale copy are the same copy.

The tax on the people paying the bill and trusting the output

One level up, this is money and risk at a scale that does not show up in any one interaction.

The money is straightforward. Tokens are metered, a large spec is a large number of them, and you are multiplying it by every engineer, every agent session, every turn within a session, and every large API they touch. The saving from not doing this is not a rounding error; it is most of the bill.

The risk is quieter and worse. The organisation is betting on agents to make engineering faster, and feeding them oversized, stale context undercuts the bet from underneath. An agent working from yesterday’s contract produces code that looks right, passes the review a human gives code that looks right, and fails at integration, which is the expensive place to fail. You are not just overpaying. You are paying to be confidently wrong at scale, which is a strange thing to buy on purpose.

The size of the gap

Put the two numbers next to each other. A large spec is around 150,000 tokens. The single operation the agent needed, with the schemas it references, is on the order of 300. That is not a tuning improvement, a percentage shaved off. It is more than two orders of magnitude, on a cost you pay every turn, to deliver the same answer.

And the 300-token version has a property the 150,000-token version cannot have: it can be current. A small slice fetched at the moment the agent asks is pulled from wherever the spec actually lives, so it reflects the contract as it is now, not as it was when someone pasted it. The large copy is expensive because it is large and wrong because it is a copy. The small query is cheap because it is small and right because it is fetched on demand. The size and the staleness are the same problem wearing two coats.

sequenceDiagram
    participant A as Agent
    participant R as Spec registry (over MCP)
    A->>R: get the operation POST /charges
    R-->>A: one operation plus referenced schemas (~300 tokens, current)
    Note over A: context stays small — the slice is current at query time

Why the obvious fixes miss

The alternatives each fix one face of this and leave the others.

Pasting the whole spec is the baseline, and it is expensive, stale and oversized all at once.

Retrieval over the spec, embedding it and pulling back the relevant chunks, is genuinely the right shape, and it is where the good instinct leads. What it leaves you with is an index to build and host per spec, a freshness problem that is now yours to solve, and the fiddly business of chunking a structured OpenAPI document whose operations lean on shared schemas that do not sit next to them. It is the correct idea handed to you as a project rather than a property of the system.

Bigger context windows are the fashionable non-answer: context is huge now, so paste away. This fixes nothing that matters. You still pay for every token, so the cost stands. It is still a snapshot, so the staleness stands. And a longer context does not attend uniformly, so the quality cost stands too. A bigger bucket is not a substitute for asking for less water.

Curating the snippet by hand, pasting just the one endpoint, is accurate and cheap and entirely manual, and it requires you to already know which operation the agent needs, which is frequently the very thing you were hoping the agent would work out.

What would have to be true

Argue it from the properties.

The agent should fetch the operation it needs, not receive the document that contains it. Retrieval, not paste, so the unit of transfer is the answer rather than the encyclopedia.

The slice has to be current at the moment it is fetched, pulled from wherever the live contract lives, so that being small also means being right rather than being a smaller snapshot.

It has to be a shared layer, not an index each developer stands up and re-freshes on their own laptop, because a per-person copy reintroduces exactly the drift and duplication the retrieval was meant to remove.

It has to return the minimal useful unit: the operation and the schemas it actually references, a few hundred tokens, complete enough to build against and no larger.

And the agent should decide when to fetch it, as a tool call in the middle of its reasoning, rather than a human pre-loading a guess about what it will need before it has started thinking.

How we come at it

Full disclosure: this is the spec0 blog, and serving specs to agents this way is the thing we build, so read this as interested rather than neutral.

spec0 puts a retrieval layer between the agent and your specs, over MCP. The agent queries the single operation it needs, on the order of 300 tokens, instead of being handed the whole 150,000-token document, and because the query resolves against the registry, the slice it gets is current at the moment it asks rather than a copy that started ageing when someone pasted it. It is one shared layer over your organisation’s specs, not an index each engineer maintains alone.

To stay honest about the edges: this is about reading the current contract efficiently. Asking what changed between two versions, so an agent can reason about a diff rather than a snapshot, is a direction we are moving toward and not something to claim as finished. Retrieval of the current operation, cheap and current, is the part that exists.

The part worth arguing about

Retrieval quietly assumes the agent knows what to ask for. Fetching POST /charges is only cheap if the agent already knows it wants POST /charges, and discovery, which API, which operation, out of an organisation’s whole surface, is its own problem that retrieval does not by itself solve. An agent that does not know what to ask cannot save its way out of the paste with a query.

So the tension I keep sitting with is how much of a map the agent needs before it can ask a narrow question, and whether assembling that map creeps back toward loading the whole spec by another route. There is a real line between “retrieve the operation” and “retrieve enough context to know which operation”, and I am not sure anyone has drawn it cleanly yet. If you have built agent tooling that stayed cheap without going blind, I would like to know how much you let it see before it was allowed to ask.