Budget for the retry: estimating AI agent tokens before you prompt

13 October 2026 · 11 min read

Every team that hands stories to an AI coding agent has had this afternoon. The first attempt looks promising, the tests go red, the agent tries again, then again, and somewhere around the fourth pass somebody opens the usage dashboard and goes quiet. Nobody did anything wrong. The story just never had a budget, so nobody knew it had blown one. This article is about giving it one before the first prompt, with a short round of planning poker and a deck of cards that count tokens instead of points.

The first try is the cheap one

Most rough guesses at agent cost go like this: "the change is small, the agent reads a few files, writes a few hundred lines, call it fifty thousand tokens." That's a fair guess for one attempt. The trouble is that work rarely takes one attempt, and the attempts don't cost the same.

An agent working on a story carries its history with it. When the tests fail, the second attempt doesn't start fresh: it re-reads the original instructions and files, plus its own first diff, plus the failing test output, plus whatever it reasoned about the failure. The third attempt carries all of that and the second round too. Each retry is a little heavier than the one before it, like a snowball rolling downhill. (Tools differ: some summarise or trim history between steps, and cached input is often billed for less. The shape stays the same; only the slope changes.)

A worked example: the snowball in numbers

Take a modest story: "Add a CSV export to the invoices list." Say the agent needs about 40k tokens of context to start: the task, the project's agent instructions and the files it reads. Each attempt writes roughly 6k tokens of output (the diff and its reasoning), and a failed test run feeds back about 4k tokens of logs. So every pass adds 10k tokens to what the next one has to read.

Three things jump out of that little table:

• The total grows faster than the number of passes. Five passes cost about seven times one pass, not five. Written out, n passes cost n × (start + output) plus the growth × n × (n − 1) / 2 that piles up from the history.
• The last pass is the most expensive. Pass five alone costs almost twice what pass one did, and it's the pass least likely to be the one that finally works.
• The starting context multiplies. Every pass re-reads it. Trim it from 40k to 25k (point the agent at the two files that matter instead of letting it explore) and three passes drop from 168k to 123k. Cutting context saves more than any amount of prompt polish, and it saves the most exactly when things go wrong.

What counts as a pass

Before you can budget retries you have to agree on what a retry is, and "one prompt" is the wrong unit. A single prompt can set off dozens of tool calls, file reads and test runs. We find it easier to count passes: one attempt at the change that ends in a check, whether that's the test suite, a type check or a human looking at it. A pass succeeds or it sends the agent round again.

Passes fail for three different reasons, and they don't cost the same:

1. The check fails. A test goes red, the build breaks. The agent sees the error and usually fixes it in one more pass. This is the cheap kind of retry, and the one people picture.
2. Review rejects it. The code works but it's wrong: a new library where an existing helper would do, a component that duplicates one next door. Now a person has to explain what they want, and the agent has to undo as well as redo.
3. The spec had a gap. Halfway through, someone realises "export" was supposed to include PDF, or that the dates have to follow the user's locale. This is the expensive kind: the story changes under the agent, the history is now full of work that doesn't apply, and often the cheapest move is to throw the whole thread away and start again.

The third kind is the one a team can prevent before any tokens are spent, and it's exactly the kind of gap a planning poker round is good at finding. We wrote about that in planning poker when AI agents write the code: a spread of votes is a map of everything the agent would otherwise have to guess.

Estimate the budget together

You could work out a token budget for every story with a spreadsheet, but you'd be guessing the inputs alone, and the guesses are where the trouble is. How many passes will this take? That depends on how well the tests cover this corner, how messy the module is, whether the ticket is clear. Different people on the team know different halves of that, which is the whole reason planning poker works.

So estimate the budget the same way you'd estimate points. Storyvote has an AI tokens deck for it:

10k · 25k · 50k · 100k · 250k · 500k · 1M · 2.5M · 5M · 10M+ · ?

A few things about it are on purpose:

• The steps grow, like Fibonacci. Nobody can tell 180k from 210k in advance, but everyone can tell 100k from 1M. Each card is roughly double or more the one before, so the vote is about the order of magnitude, which is the only thing you can really know.
• You vote for the whole story, retries included. Not "what does one attempt cost" but "what will it take to get this merged". In the example above, a team expecting three passes votes 250k, not 50k.
• Round up to the next card. 102k is a 250k story, not a 100k one. The point of the card is a line you'd be surprised to cross, and you shouldn't be surprised by 110k.
• 10M+ isn't an estimate. It means "this is a project, not a story". Split it before anyone prompts anything.
• ? is a real answer. It means "I can't size this because I don't know what it is". That's a spec gap, the most expensive kind of retry, caught for free.

One round, start to finish

Here's how the invoices export went for one team of five. Everyone voted privately, then the cards turned over together:

Five people, four different answers, and each one is a piece of the spec. Mia is right about the code: it's small. Sam knows why it will take more passes than it looks. Jonas has found a gap that would have turned into the expensive kind of retry halfway through. In ten minutes the team decided:

• PDF is a separate story, written up later. This one is CSV only.
• The ticket names the locales to test and links the existing CSV helper, so the agent doesn't write a new one.
• Priya trims the failing-test output for that suite to the failures alone, which cuts what every retry has to carry.

They voted again and everyone showed 250k. Not because they averaged their way there (the deck deliberately has no average: "about 400k" would mean nothing) but because the reasons for 50k and 1M had both been dealt with. Storyvote saved 250k as the story's estimate, the most-picked card, and the story went into the sprint with a number everyone understood. Our guide to reading a reveal has more on what shapes like that first vote mean.

Tripwires, not spending caps

A budget is only useful if something happens when you reach it. Hard caps don't help much: an agent cut off mid-change leaves a half-finished branch that someone has to clean up by hand. What works better is a tripwire, a point agreed in advance where the person driving the agent stops and talks to the team instead of pressing "try again". A few that teams use:

• Spend reaches the card. The story was a 250k and it's crossed 250k without passing review. Stop. Something about the story is different from what the team thought, and that's worth five minutes of conversation, not another pass.
• Two passes fail the same way. If the agent hits the same red test twice, a third try rarely helps. It's missing information, and only a person can give it.
• The diff wanders. Files nobody mentioned in planning start changing. That's the agent filling a gap with a guess.
• A new question appears. Anything that would have made someone vote ? if they'd known it in planning sends the story back for a quick re-estimate, just like a story that turns out bigger than its points.

Write the tripwires into the ticket next to the estimate. They take one line, and they turn "the agent is still going" from a mood into a decision. It's the same idea as giving every story a timebox in sprint planning: the clock doesn't end the work, it starts the conversation.

Start over or keep going?

The snowball has a practical lesson for the moment a tripwire goes off. Continuing in the same thread means every new pass pays for every old one. Starting a fresh thread with a better ticket means the next pass costs about what the first did, and the agent no longer has three wrong ideas in its history pulling it sideways.

So when the trouble was a spec gap, start over: fix the ticket, write down what you learned, and begin a clean attempt. When the trouble was a small, understood bug, keep going: one more pass with the right hint is cheaper than re-reading everything. Agreeing on that rule in advance saves an argument at the worst moment.

What a token budget isn't

• It isn't the cost of the story. Tokens are usually the cheap part. The expensive part is the hour a senior engineer spends reviewing the result. Keep estimating that the way you always have, in story points for complexity and risk. A story can be 25k tokens and an 8 for review, and that's the one to plan time for.
• It isn't a price. Prices per token change, differ between models, and cached input costs less than fresh input. Budget in tokens and convert to money only when you need to, with the rates of the day.
• It isn't a scorecard. Comparing tokens between people or teams tells you who had the messier module, not who works better. Like points, a token budget only means something inside one team, about its own work.
• It isn't precise, and that's fine. If a story lands anywhere under its card it went to plan. The budget is there to catch the story that needs 2.5M when everyone thought 100k, not to argue about 140k against 160k.

Calibrate after the sprint

The first sprint of token estimates will be rough, and that's expected. At the retro, put each story's card next to what it actually used (your agent tool or API dashboard shows usage per session or per key). You're looking for patterns, not individual misses:

• Stories that blew through their card almost always had a ? in the first vote, or a ticket nobody wrote down. That's an argument for the discussion, not for bigger cards.
• If almost everything lands at a third of its card, the team is padding. Move the reference story down a card.
• Modules that cost three times more than anyone expects, every time, deserve a better test setup or shorter agent instructions more than they deserve bigger estimates.

Storyvote lets you download a room's stories and estimates as a CSV, so the cards from planning are easy to bring into the retro and line up next to the actual numbers.

A five-minute checklist

Before an agent touches a story, the team should be able to say:

1. What a pass means for this story, and what check ends one.
2. The token card the team agreed on, retries included.
3. What the agent should read first, so it doesn't explore its way to 100k of context.
4. What's out of scope, especially anything that came up as a ? in the vote.
5. The tripwires: when to stop and talk, and whether to restart or continue.

None of this needs a new meeting. It's three extra minutes on stories you were going to discuss anyway, and it's the cheapest place to find out that a 50k story was really a 1M one.

Try it in Storyvote

Create a room, pick AI tokens as the voting system, and run your next refinement with it. If you want to size the human side too, keep a points deck and turn on AI tokens as an extra question in Settings, so each vote asks for both. It's free, there are no accounts, and the whole team joins from one link.

← All articles