A conversation about stopping, which is the part nobody plans and everybody needs.
How many retries should I give each item?
Three, unless you have a reason.
One is too few. A lot of first failures are the AI misreading something, and it fixes them immediately once it sees the actual error.
Ten is theatre. If it failed three times with the error message in front of it, attempt seven isn’t going to be the one. You’re paying for the same wrong answer repeatedly.
Go lower than three when each attempt is slow or costs real money. Go higher only when failures are usually flaky rather than real, like a network call that times out sometimes.
What happens to the ones that fail every time?
They go in a file, with the reason, and the loop keeps going.
{"id":"page-63","attempts":3,"codes":["missing-meta-description"],"at":"2026-07-26T10:14:00Z"}Two things matter here. One bad item doesn’t stop the other 79. And the failure is written down instead of quietly skipped, so at the end you get “79 done, 1 needs you” rather than “done.”
That one line is doing real work. Without it you get a run that looks complete and isn’t.
Why not just stop the whole thing when something fails?
Because then one weird item holds up everything, and you come back to a loop that died at item 4 of 200 an hour ago.
The other reason: the quarantined items are usually the interesting ones. A file that fails three times in a row generally has something structurally different about it. That’s information. It deserves your attention more than the 79 that worked.
Can a failed item ever get committed?
No. This is the one rule with no exceptions.
If it didn’t pass, it doesn’t get saved. Commit per item, so a bad item is revertible on its own and a good run isn’t tangled up with a bad one.
The reason this matters more than it sounds: the whole point of the loop is that you’re not reading every result. If failures can slip into the output, you’ve lost the only guarantee you had.
How big should a batch be?
Match it to the flakiest thing in the loop, not the fastest.
If every item is local computation, run hundreds. If every item edits a file and runs tests, twenty to fifty is comfortable. If every item hits a rate limited API, ten to twenty five, and not in parallel.
If every item drives a browser, go small and go serial. Browser automation is where loops break in the least obvious ways. Pages lazy load and then unload what you already scrolled past. Bot checks appear at request 30 but not at request 3. A page returns 200 with an empty body. These failures look like success, which is exactly the kind of failure a loop is worst at.
What’s the worst bug a loop can have?
Reporting success while doing nothing.
I hit this one directly. A runner template had the actual work step left as a comment for the user to fill in. Someone copies it, fills in the list and the check, forgets the middle, runs it. The loop walks all 200 items, does nothing to any of them, and prints a clean summary.
Every number in that summary is true. Nothing happened.
The fix is to make the unfinished state loud:
run_unit() {
echo "FATAL run_unit() has not been implemented" >&2
exit 2
# ... the real thing goes here
}Now the incomplete version stops on item one instead of lying about 200.
What about caps? If I run with a limit, is that fine?
Fine, as long as it says so.
57 passed, 3 quarantined, 200 in work list
NOT PROCESSED: 140 items beyond --limitA loop that silently truncates reads as “covered everything.” That’s the same class of bug as the one above: the report is technically true and completely misleading. If your loop drops anything, for any reason, that goes in the output.
Same for spend limits. If you cap it at a million tokens and it hits the cap, the report says where it stopped and what’s left, not “done.”
How does it end if there’s no list?
Depends on what you’re doing.
If you have a list, iterate it and stop. Easy.
If you’re searching for an unknown number of things, like bugs, run rounds until you get two in a row that find nothing new. Simple counters like “stop at 10” miss the tail. Just make sure you deduplicate against everything you’ve seen, not everything you kept, or the things you rejected come back every round and it never finishes.
If you’re spending money, stop at a budget, and keep a reserve so it can still write up what it found. A loop that spends everything searching and dies before reporting has bought you nothing.
What still needs me?
Name it in writing before you start, and be specific. “Use judgment” is not naming it.
Usually three things.
The judgment calls that pass every check. Data that agrees with itself and is still wrong, because the reason is outside the data. Nothing in the loop can see that.
Triaging quarantine. An item that failed three times usually has a structural problem worth understanding, not a fourth retry worth paying for.
Whether the list was the right list. The loop covers what you handed it. Nothing inside it can tell you the query was wrong. That one is entirely yours, and it’s the mistake that wastes the most time.