Knowing When to Stop: The Art of Making a Loop Converge

How can an AI model know when its work is done? Well, how does a human know when our work is done. A programmer waits for the tests to turn green or waits for PR review from their team. A designer adjusts a composition, steps away, returns, and decides the remaining imperfections no longer matter. A writer submits a draft because the deadline has arrived or because an editor accepts it, not because the prose has reached some objectively final state. “Done” is rarely a property of the work itself. It is a judgment produced by the system around the work. Humans do not possess a universal detector for “done”. We rely on a patchwork of signals like tests, specifications, precedent, approval, deadlines, risk, and finding that point of diminishing returns. In each case, completion comes from outside the work itself. The Model that Could Continue Forever An AI model can almost always produce another answer. It can revise the paragraph again. It can try another implementation. It can generate another image with more detail, different lighting, and a stronger composition. It does not become tired of the work. It does not notice, unless we give it some way to notice, that the last three revisions made the result different but not necessarily better. This is part of what makes the recent idea of loop engineering so compelling. Instead of a human prompting a model, inspecting the result, describing what went wrong, and prompting it again, we can ask the system to perform the whole cycle itself. The person no longer has to sit inside every turn. The agent discovers the work, gives it to the model, checks the result, and decides what should happen next. Peter Steinberger 🦞 @steipete Here’s your monthly reminder that you shouldn’t be prompting coding agents anymore. You should be designing loops that prompt your agents. 6:58 PM · Jun 7, 2026 8.5M 1,796 1,414 19,839 However, the nuance when writing a loop is that the loop is only as good as the verifier at each step.

Take the standard coding-agent loop as an example: keep working until the tests pass. It sounds almost perfectly verifiable. But the tests are only a proxy for the task. In SpecBench, frontier agents routinely passed the visible tests while failing held-out tests that exercised the same features together. One agent produced a 2,900-line “compiler” that simply memorized the test inputs. The loop converged, but just on the verifier, not the user’s intent. The verifier is not just the stop condition. It also defines what the loop treats as progress. If the signal is incomplete, the loop can get better at passing the check without getting better at the task. Loop engineering is not the practice of making an agent retry. It is the practice of making each cycle reduce the distance between the current state and a desired state. A loop is not yet a direction. The Loops that Converged The first loops that worked well were coding loops. This is not an accident. Code is both editable and executable. An agent can change one function, run the program, read the test failure, and try again. The environment returns a relatively clear signal about what broke. The loop has both a precise way to act and a verifier that can measure progress. I wrote about a similar pattern in visual code generation. An SVG is not just an image; it contains paths, shapes, text, gradients, and layout. A Blender scene is not just a render; it contains geometry, materials, cameras, joints, and constraints. These representations give an agent something it can inspect and edit locally. If one curve is wrong, change the path. If one object is misplaced, move that object. The artifact can improve across iterations instead of being regenerated from scratch. But editability is only half of the problem. In open-ended image generation, another iteration often means generating another sample and choosing the best one. The feedback is global, and it is hard to map “this looks worse” to one precise edit. SVG and Blender loops can converge when the target can be expressed as a reference, geometry, constraints, or functional behavior from an articulated object. They struggle when the target is simply “make it better, with better taste, but you cannot ask a human”. Visual loops are not impossible. They are often extremely hard to verify. The Conditions of Completion If the verifier gives the loop direction, what does the loop need to converge? Based on many conversations with engineers and researchers across several domains, I think there are four things. 1. A target state The system needs a representation of what “done” means. For code, this might be a test suite, a specification, or a set of performance constraints. For an SVG, it might be a reference image, dimensions, colors, and layout rules. “Make it better” is not a target state. It is another prompt. 2. An observable current state The system needs to inspect what exists now. And that could mean files, diffs, test results, traces, a DOM tree, an SVG structure, or a Blender scene graph. A rendered output alone is often not enough. The system also needs to see the underlying structure so it can identify where the error came from. 3. A precise way to make changes The agent needs to change the part responsible for the error without regenerating everything else. Changing one function is better than rewriting the repository. Editing one SVG path is better than generating a new image. Adjusting one object in a Blender scene is better than rebuilding the scene from scratch. The more local the edit, the more likely the loop is to preserve what already works. In practice, this is the part people struggle to get right. Nearly every researcher I talked to said the same thing: their loop started working when they found the right set of tool calls and intermediate prompts. How do you discover the tools that meaningfully advance the loop? Right now, no one knows in advance. It is mostly trial and error. Which points to an uncomfortable implication: a loop is tuned to its stack. The tool calls that made a loop converge on one codebase encode assumptions about that codebase, and those assumptions stop holding somewhere else. A loop that worked for someone else is a starting point, not a guarantee. Bespoke loops do not generalize for free. And this is why we see both sides of the discussion: some people found magical loops that worked for them, but others found when they use publicly published loops they do not work at all. 4. A stopping rule The system needs a condition that tells it to stop. The condition should come from outside the generator: tests passing, constraints being satisfied, a score crossing a threshold, or a reviewer approving the result. The stop condition also needs to account for cost – a loop that reaches the right answer after 500 attempts may converge technically but not economically. One useful way to think about this is across two axes: how editable the artifact is, and how verifiable the result is. Code often sits in the upper-right corner. It is easy to edit, and it has relatively strong verifiers. Open-ended image generation often sits in the bottom-left. The system can generate another image, but it cannot easily repair one specific decision or verify that the result is closer to the user’s intent. One important property of the chart above is that the position of a task can potentially move by reframing a problem. The axes describe the representation, not the task itself. An open-ended image is hard to edit, but the same image, represented as SVG paths or a Blender scene, becomes editable — the task moves up. Give it a reference image or a set of constraints to check against, and progress becomes verifiable, which moves the task to the right.