How Do You Know If a Prompt Is Actually Working?

A prompt is working when you act on its output without rewriting it — and it's failing when the output is plausible but ignored. The bar is not "did the AI produce something impressive"; it's "did the thing it produced change what you did next." Three tests settle it: the action test, the edit test, and the week-two test.

Why is "looks good" the wrong bar?

Because modern models clear it every time. Ask a frontier model for a morning briefing and you will get something briefing-shaped — confident, well-formatted, grammatical. That's table stakes, and it's also the trap: plausible output is the failure mode that doesn't announce itself. A prompt that errors out gets fixed. A prompt that produces respectable-looking text nobody acts on just quietly burns its slot, and you conclude — wrongly — that the agent "isn't that useful."

So the tests below deliberately ignore the output's quality as text. They measure the only thing that matters: whether the operation changed your behavior.

Test 1: The action test — did you do something because of it?

After each run, ask one question: what did I do that I wouldn't have done without this output? Replied to the flagged lead first. Killed the meeting the briefing showed was pointless. Pushed the deal the pipeline pulse said was going cold.

If the honest answer is "nothing," three runs in a row, the prompt is decorative. That doesn't always mean the prompt is broken — sometimes it's watching the wrong things, sometimes it's watching the right things at a threshold that never trips. But zero actions means zero value, whatever the output looks like.

Test 2: The edit test — how much do you rewrite?

This one is for drafting-style prompts — anything that produces words meant to leave the building in your name. Look at what you actually sent versus what the agent drafted. If you find yourself rewriting most of it every time, the prompt is failing: the model never received what it needed to sound like you, know your positions, and hit your reader.

The fix is almost never "write a cleverer prompt." It's context: the voice and preferences the agent should be accumulating through memory — the voice-lock and personal-index category of operation — so drafts converge on your register instead of restarting from generic every session. If edits aren't shrinking over weeks, the memory layer isn't being fed. (Which category does what is mapped in what you should actually use your AI agent for.)

Test 3: The week-two test — did it survive the novelty?

Everything gets read in week one, because week one is fun. The real test is the fourteenth day: is the briefing still opened? Does the triage flag still get a response inside the hour? Survival past novelty is the strongest signal a prompt has an actual job — and quiet abandonment is the strongest signal it never did.

Be honest about which side of that line each installed prompt sits on. An operation you've stopped reading isn't neutral; it's noise that trains you to ignore the agent entirely.

What do you fix when a test fails?

Note that every fix is a substitution or a feed — not a rewrite of the prompt's structure. If the skeleton was sound to begin with, working the fixes is the same five-swap exercise as the initial setup, covered in how to adapt a borrowed prompt to your business. If the skeleton was never sound — a 2023-era template dressed up as an operation — no amount of tuning saves it, and you're better off starting from a real drop-in prompt.

Keep receipts

One habit makes all three tests trivial: keep the outputs. A folder, a channel, anywhere the runs accumulate. After two weeks you're not relying on your impression of whether the prompt helped — you're looking at fourteen briefings and the actions they did or didn't produce. Claims are cheap; receipts decide. That's a house rule across everything Optimus ships — it's the entire premise of gimmetheproof.com — and it applies just as hard to your own prompt stack.

FAQ

How long should I run a prompt before judging it?

About two weeks of real runs for a recurring operation. One run tells you whether the plumbing works; two weeks tells you whether the output survives contact with your actual attention. Judge one-off prompts immediately — either you used the output or you didn't.

What if the output is good but I never read it?

Then the prompt isn't working, no matter how good the output is — but the fix is usually cadence or channel, not content. Move it to when you'll actually see it, or shorten it until reading it costs nothing. An unread briefing is inventory, not an operation.

Is more detailed output always better?

No. Length is often how a prompt hides its failure to prioritize. A briefing you skim in ninety seconds and act on beats a report you file unread. If the output keeps growing, tighten the definition of done — cap the length and force a "most important thing" at the top.

Should I A/B test my prompts?

For recurring operations, iterate serially instead: change one thing — a source, a threshold, the definition of done — and watch the next few runs. You don't have the volume for statistical testing, and you don't need it; a week of runs plus your own editor's eye is a sharper instrument.