A conversation about why prompting stops working somewhere around item twenty.
I already write pretty good prompts. Why isn’t that enough?
Because the problem isn’t the prompt. It’s you.
Say you have 60 files calling a deprecated function, fetchUserV1(). They all need to move to fetchUserV2(), which takes different arguments. You open the first file, ask your AI to convert it, read the diff, notice it dropped the error handler, ask again, and now it’s right.
Two minutes, and you were paying attention the whole time.
Now do that 59 more times.
But the AI is doing the work. What am I actually doing?
You’re the part that notices when it goes wrong.
That’s the job you’ve quietly handed yourself. Every result comes back, you read it, you decide. Around file 20 you stop reading carefully and start skimming for shape. By file 40 you’re pattern matching on “looks like the others.” At file 47 it drops an await and you approve it, because it looked like the others.
You find out two weeks later when something times out.
A better prompt would have caught the await though.
Maybe. Say a really good prompt takes you from 90% correct to 95%.
You still have to look at all 60 results to find which three are in the 5%. The prompt improved the work. It changed nothing about the reading.
That’s the part worth sitting with. Making the AI better doesn’t shrink your job at all, as long as your job is checking.
Okay. What’s the alternative?
Write down what “done” means as a command, and let the command do the reading.
For this job, done means: the file doesn’t mention fetchUserV1 anymore, and the type checker is happy with it.
! grep -q 'fetchUserV1' "$FILE" && npx tsc --noEmit 2>&1 | grep -qv "$FILE"That’s the whole idea. Something other than you can now tell whether file 47 is finished.
How does that become a loop?
Four pieces.
The list. A command that names every item. Not a guess, not “the files that look old.” An actual command:
grep -rl 'fetchUserV1' ./srcRun it. It says 60. Now you know the size of the job, and at the end you can run it again to prove you’re done.
One at a time. One file per run, fresh each time. Fresh matters more than it sounds. A session that just did five files starts blending them, and by the sixth it writes a change that belongs to a file it already closed.
The check. The command above. It has to be able to say no.
When to stop. Three tries per file. If it still fails, write the name and the reason to a quarantine list and move on.
What does a failure actually look like?
The runner does file 12, runs the check, the check says no. So it hands the AI the reason, word for word:
src/api/orders.ts:41 Property 'userId' is missing in type
'{ id: string }' but required in type 'FetchUserV2Args'The AI reads that and fixes it. Second try passes.
Look at what that error is. Precise, true, and free. Compare it to what you’d have typed after skimming a diff, which is usually something like “hmm, check the args on that one.” The compiler is a better reviewer than you are at 4pm on the fortieth file, and it never gets bored.
And if a file fails all three times?
It goes in a file, with the reason:
{"file":"src/legacy/billing.ts","attempts":3,"reason":"tsc: 3 errors remain"}And the loop keeps going.
People skip this part, and it’s the part that makes the whole thing usable. One weird file shouldn’t stop the other 59. And you want the weird one written down instead of quietly skipped, because at the end you get “57 done, 3 need you,” and those 3 are usually the interesting ones.
Isn’t this just a bash script?
Often, yes. That’s a feature.
A lot of this work needs no AI at all once the check exists. If the check can also fix what it finds, you don’t need a model in the loop, you need a for-loop. Cheaper, faster, easier to trust.
The AI earns its spot when the fix takes judgment. Rewriting a call site with different arguments, yes. Renaming a variable consistently, not really.
When should I not bother?
Three cases.
You’re doing it once. Building the loop costs more than the job. Just do the job.
You can’t write the list command. Then the loop will do some of them, skip the rest, and tell you it finished. You’ll believe it. That’s worse than not running it.
You can’t write a check. Then you don’t have a loop, you have a way to produce wrong work faster than you can read it.
That last one is where most people get stuck, so the next post is entirely about finding a check when there isn’t an obvious one.
Give me the short version.
You are not going to out-prompt this. Sixty results is sixty results, and reading them is the cost.
Write the check once. Let it do the reading.