The best model for coding agents depends on the ticket

Teams evaluating coding agents usually ask us some version of "which model is best?" It's a reasonable question with an unsatisfying answer: the best model for coding agents is a per-ticket decision, not a leaderboard lookup. A dependency bump and a cross-cutting refactor are different jobs, and the model that's clearly worth its price on one is often overkill on the other. We run agents across different models and repos every day; what follows is a decision guide built from that experience, labeled as experience, with no numbers invented to make it look like science.

Why "which model" is a per-ticket question

A background agent doesn't make one model call; it makes a long chain of them. It reads code, plans, edits, runs tests, reads failures, and edits again. That changes the economics and the failure modes compared with autocomplete. A per-token price difference that looks trivial in a chat window compounds across an agent run, and a model that's slightly worse at following instructions tends to wander off and produce the wrong diff entirely, rather than a slightly worse version of the right one.

Tickets amplify this. "Bump the library and fix the deprecation warnings" mostly tests whether the model can execute a known recipe without breaking things. "Extract the billing logic that's tangled through these four modules" tests judgment across a long run. Paying frontier prices for the first is waste; running a budget model on the second is how you get a PR you reject and a task you pay for twice.

What actually differs between frontier models

The public conversation fixates on scores. On real tickets, four less-quantified properties matter more.

Instruction-following over long runs. Any frontier model handles a three-step task. The differences show up forty tool calls in: does the model still respect the constraint you set at the start ("don't touch the public API"), or has it drifted? Drift is expensive because you find it at review time, not at run time.

Tool-use reliability. An agent's model has to run tests, read the failures, apply patches cleanly, and know when to stop and ask. Models differ in how consistently they do the boring mechanics, and a model that occasionally mangles an edit costs more than its price sheet says, because each failure burns a whole iteration.

Cost per task, not per token. What you care about is what a merged PR costs. A cheaper model that needs two extra rounds of review comments can cost more than an expensive one that lands the first try, and the reverse is true for simple work, where any competent model lands it.

Speed. For background agents, wall-clock time matters less than in an editor, since you're not watching. It still matters at the margins: faster iteration inside the run means more test-fix cycles fit in the same budget, and shorter runs mean less drift.

Context handling belongs on the list too, especially in monorepos, where the difference between "found the existing helper" and "reinvented it" is often a matter of how much of the repo the model could hold in view.

What we see in practice

These are qualitative observations from running multiple models on real tickets. They are our experience, not measurements, and your repo may disagree.

Stronger models earn their cost on gnarly work. On refactors that cross module boundaries, on bugs where the cause isn't where the symptom is, and on tickets that require reading a lot of code before writing any, the top-tier models take fewer wrong turns and need fewer review round-trips. The premium buys judgment, and judgment is what those tickets consume.

Cheaper models are fine for mechanical changes. Dependency bumps, renames, adding a test that follows an existing pattern, small config changes: the diff is easy to verify, the ceiling barely matters, and the economics favor the smaller model. If a reviewer can check the diff in two minutes, you probably overpaid if a frontier model wrote it.

Long context earns its keep in monorepos. Models that hold more of the codebase in view spend less of the run rediscovering structure and are less likely to duplicate something that already exists. In small repos the advantage mostly disappears.

Failure modes differ more than success rates. Models that land at similar quality on average fail differently: one over-edits, another under-tests, another asks for clarification too rarely. Which failure mode your team tolerates best is a legitimate selection criterion, and no leaderboard encodes it.

Leaderboards go stale fast

Public rankings are worth a look and not worth a bookmark. The SWE-bench leaderboards are the closest thing the field has to a shared yardstick for agentic coding, and they reshuffle with every model release, which lately means every few months. Any specific number we quoted here would be wrong before this post finished publishing. Use leaderboards to shortlist plausible models, then let your own tickets be the eval: the distribution of work in your backlog is a benchmark no public suite replicates.

How to decide

Start with a strong default. Pick a current frontier model and run it on everything for a few weeks. You're buying signal: which tickets it clears cleanly, where reviewers push back, what tasks cost.

Downgrade where diffs stay clean. When a class of work (bumps, renames, test backfill) keeps producing PRs that sail through review, move that class or that repo to a cheaper model and watch whether review friction changes. If it doesn't, the savings were free.

Make it config, not policy. Model choice should live next to the code it affects. In Sinatra it's a .sinatrarc.json file per repo:

{
  "agent": { "model": "opus" }
}

A quiet internal tool and your revenue-critical monorepo shouldn't share a model because a dashboard default said so.

Revisit at releases, cheaply. When a new model ships, rerun a handful of representative recent tickets on it and compare the PRs. That's an afternoon, and it's more informative than any launch-day chart.

There is no permanent best model for coding agents

Which is exactly why the switching cost matters more than this quarter's ranking. If your agent vendor locks you to one provider (none of the hyperscaler agents takes your own key), every leapfrog is their roadmap item instead of your config change. Bring-your-own-key turns model choice into a setting: connect your Anthropic or OpenRouter key (or a ChatGPT subscription you already pay for), pay provider prices with no markup, and change models per repo the week a release makes it worth doing. That's how Sinatra prices it, because we think the model market stays competitive and you should be able to act on that.

Common questions

Which model should I start with?

The strongest current frontier model your budget tolerates, applied to everything for a few weeks. Starting strong tells you what "good" looks like on your tickets, and downgrading from a known baseline is easy. Starting cheap tells you much less, because you can't distinguish "the model wasn't capable" from "the ticket wasn't clear."

Do bigger models always win?

No. They win more often on ambiguous, cross-cutting work, and they're frequently indistinguishable on mechanical work, where a smaller model is cheaper and sometimes faster. On easily verified diffs the review cost dominates the model quality difference, so the premium buys little.

Does model choice matter more than the ticket?

Usually not. A vague ticket fails on every model; a crisp ticket with clear acceptance criteria succeeds on most of them. In our experience the gap between a well-specified and a poorly specified ticket is larger than the gap between any two frontier models. Sharpen the tickets first, then tune the model.

If you want to run this experiment on your own backlog, Sinatra turns Linear and GitHub issues into pull requests on whatever model you configure. Start for free, or see the docs for per-repo model setup.