Why an AI agent should fix its own failing CI

Most background coding agents advertise the same finish line: give it a ticket, get a pull request. It's the wrong finish line. A pull request with red checks is unfinished work with a handoff attached, and the person it's handed to is you. An AI agent that can't fix its own failing CI has automated the first draft and left the debugging to you.

The wrong finish line

Consider what a red PR actually costs you. You open it, read the diff, read the CI logs, reconstruct what the agent was trying to do, and then decide whether the failure means the approach is wrong or a detail is wrong. That's the same context switch the agent was supposed to save you, moved two hours later and made worse, because now the code you're debugging is code you didn't write. A lot of earned skepticism about coding agents comes from exactly this experience.

The finish line that matters is a PR whose checks are green, and whose failures along the way were handled by the thing that caused them.

What the loop looks like when it works

The rework loop has a shape, and the order matters.

The agent runs your configured test command inside its own sandbox, while its pull request is still a draft. Most failures should die here, where a broken run costs nobody a notification. An agent that lets CI be the first place its code meets a test suite is outsourcing its testing to your build minutes and your attention.

The draft PR itself opens early in the run, and that part is a feature: you can watch the work take shape. What the tests gate is the review request, not the PR's existence. Checks run, and when one goes red the agent reads the actual failure output (the stack trace, the assertion diff, the log line) rather than just the red status. It patches, pushes to the same branch, and waits for the checks again. Only when they're green does it mark the PR ready and ask a human for review.

Two details in that loop carry most of the weight. Fixes land on the same branch, so the PR history shows what failed and what changed in response, and you review one coherent unit of work instead of a scatter of follow-up PRs. And review is requested after green, so the notification you get means "this is ready", not "come look at this".

What it takes for an AI agent to fix failing CI

Reading the failure is most of the job, because different failures demand different responses.

A flaky test that fails on timing and passes on retry is not a code problem, and "fixing" it by rewriting your code is worse than doing nothing. The right move is to rerun it, and if it passes, to say the test looks flaky rather than quietly take credit for a fix.

A missing environment variable in CI is a configuration problem. The code may be fine; the pipeline just can't see a secret or a service that the sandbox had. Patching application code to route around missing CI config buries the real issue for whoever hits it next.

A genuine logic bug caught by a genuine test is the case everyone pictures, and it's the one where patch-and-push is exactly right.

Blindly rerunning until green treats every failure as the first kind. Blindly patching treats every failure as the last. An agent that can't tell them apart is rolling dice against your build, and sooner or later the dice make CI pass for the wrong reason.

How Sinatra handles a red build

Sinatra opens its PR as a draft while it works, whether the task came from a Linear assignment or the Sinatra label on a GitHub issue. It runs your configured test command in the sandbox, and a failing check triggers rework: it reads the failure output, patches, and pushes the fix to the same branch. It also reviews its own PR before asking for yours. The intent is that the review request you receive means the checks are green and the obvious problems are already gone. The wider loop, from ticket to merged PR, is written up in this walkthrough.

Where the loop honestly stops

Some failures the agent should not try to fix, and it's worth being explicit about which.

If your CI needs secrets or infrastructure the sandbox can't reach (a private registry, a staging database, a license server), the agent can't reproduce the failure. The honest behavior is to say so on the PR, along with what it observed, instead of guessing at a patch it can't verify. A guess that happens to turn CI green without reproducing the cause is a landmine with a passing badge.

And some red checks aren't the PR's fault at all. A broken main branch, a CI provider outage, a queue that timed out: these fail every PR equally, and an agent that churns commits against them adds noise without adding progress. Knowing when to stop is part of the loop, not a failure of it.

What to look for in any agent

The CI story is a four-question test for any coding agent you evaluate. Does it run the tests itself, in its own environment, instead of letting CI be the first run? When a check fails, does it read the failure output or just the status? Do fixes land on the same branch, as follow-up commits you can review? And does it hold the review request until the checks are green? (A draft PR in progress is fine. A review request with red checks is not.)

An agent that passes all four hands you PRs that are ready for a real review when they claim to be. The green PR is the actual product. "Opens a PR" was only ever the halfway mark.

If you want to watch the loop run against a real test suite, assign Sinatra a small, well-written ticket in a repo that has one. You can start for free, or read the docs to configure the test command it runs.