AI agents

Reflection and Self-Correction Without Magical Thinking

Design reflection loops that store actionable lessons while recognizing correlated errors and memory contamination.

By the end you can

Define agent reflection and correction as an operational contract rather than a capability label
Contrast Retry without reflection with Verbal reflection in “A planning agent wrote a thoughtful postmortem and repeated the same failure”
Trace “Reflection can turn one error into persistent bad policy” through a concrete execution path
Produce “Run a controlled reflection experiment” with evidence for “Reflections include the evidence and task conditions that produced them”

Reflection changes the next prompt, not the model

Reflection converts feedback from a run into a concise hypothesis or instruction for a later attempt. It can improve behavior without changing model weights, especially when failures are observable and the stored lesson is relevant. Nothing about the model changes. What changes is what the next run is told.

A reflection is generated text, not ground truth.

It may misdiagnose the cause. It may overfit one episode, preserve malicious content, or fill context without altering the mechanism that failed. The rest of this lesson is that difference as other people measured it. Two coding benchmarks inside the Reflexion paper. The same 1,319 GSM8K problems solved with and without an oracle label. 100 graph-colouring instances. And 100 Blocksworld plans judged against a sound verifier.

Treat a reflection as a hypothesis on probation: it has earned nothing beyond the single run it came from until an external signal says otherwise.

Case

Reflexion: 91.0 against 80.1, on 164 hand-written problems

Reflexion reached 91% pass@1 on the HumanEval coding benchmark. The previous state of the art, GPT-4, reached 80%. The abstract leads with it: “For example, Reflexion achieves a 91% pass@1 accuracy on the HumanEval coding benchmark, surpassing the previous state-of-the-art GPT-4 that achieves 80%.” The paper was published at NeurIPS in 2023.

Table 1 is more specific than the abstract, and the extra digits matter. On HumanEval Python it reports pass@1 of 91.0 for Reflexion against a quoted GPT-4 state of the art of 80.1, and a previous state of the art of 65.8 for CodeT with GPT-3.5. HumanEval Rust: 68.0 against 60.0. Leetcode Hard Python: 15.0 against 7.5.

Those points are countable, because the benchmark is small and fixed. Chen and colleagues released HumanEval in 2021 and described it exactly: “We evaluate functional correctness on a set of 164 hand-written programming problems, which we call the HumanEval dataset. Each problem includes a function signature, docstring, body, and several unit tests, with an average of 7.7 tests per problem.” The released test split still contains exactly 164 examples. So 91.0 against 80.1 is a difference of roughly 18 problems. On that same set in 2021, Codex scored 28.8% and GPT-3 scored 0%.

One detail decides how far the result travels. The feedback Reflexion writes its lessons from is not the benchmark's hidden tests. It is unit tests the agent writes for itself, capped at six per problem.

Two zero-based bars on a 0 to 100 percent axis compare pass@1 on HumanEval: 80 percent for the previous GPT-4 state of the art as quoted by the paper, and 91 percent for Reflexion, which turned outcomes into written lessons. Drawn instead as failures, 20 problems in every 100 become 9 — both derived — which is 55 percent of the failures removed, also derived, and a change the 11-point gap between the two bars does not show. Reflexion's feedback came from outside the model, since HumanEval runs the tests and the lesson written after a failure reports what the tests said. Huang and colleagues tested the other case, a model revising its own answer with no external signal, and found that reasoning did not improve and at times degraded. pass@1 counts a single attempt per problem, and the 80 percent is a figure quoted by the Reflexion paper rather than an independent rerun. — Reflexion reaches 91 percent pass@1 on HumanEval against 80 percent before it — eleven points, or nine failures per hundred instead of twenty.

Visual

The retrieval rule decides whether a lesson returns

A reflection artifact is worth writing only once outcome feedback has been localized to a failure. A retrieval rule then decides when that artifact comes back. Keep ownership of the retrieval rule apart from ownership of validation, and keep their tests apart too.

The first box is the one every published result turns on. Given an oracle label that reported whether the answer was right, GPT-3.5 rose from 75.9 to 84.3 on GSM8K. Asked to revise with no such signal, the same model fell 75.9 to 75.1 to 74.7 over two rounds. The boxes downstream cannot recover what the first one never received.

FigureProcess · 5 steps

1
Outcome feedback
A test, environment state, human review, or reward signal.
2
Failure localization
Identify the decision, tool, state, or assumption that broke.
3
Reflection artifact
A bounded statement about what to change and why.
4
Retrieval rule
Select the lesson only when a later task matches its conditions.
5
Validation
Check whether the changed attempt actually improves the outcome.

Example

The evidence ledger for agent reflection and correction

Signals 2 and 3 have a published price tag. GPT-4 was run as both plan generator and verifier on 100 random Blocksworld instances. Every verdict was judged against VAL, a sound verifier. Valmeekam and colleagues reported the result in 2023. The LLM verifier was correct on 61 of the 100, at a false-positive rate of 84.45%: “Out of the 100 instances, the verifier LLM produces 54 true positives and 38 false positives (type-1 errors).” Thirty-eight invalid plans were signed off as valid by the same system that produced them.

End to end, the identical generator scored 40/100 with no backprompting at all, 55/100 with GPT-4 critiquing GPT-4, and 88/100 when VAL supplied the feedback instead. Signal 3 is that gap. What separated a hypothesis from a validated correction was which verifier was entitled to sign off, not how the critique read.

Signal 1: Reflections include the evidence and task conditions that produced them. Reflexion's HumanEval lessons were written after unit tests the agent wrote for itself failed, at most six per problem — a condition worth storing beside the lesson.
Signal 2: Stored lessons are tested before becoming durable policy. Tested means run again, not read again: GPT-4's own verdicts on 100 Blocksworld plans were correct 61 times, at a false-positive rate of 84.45%.
Signal 3: The runtime can distinguish a hypothesis from a validated correction. Swapping GPT-4's self-critique for the sound verifier VAL moved the same generator from 55/100 to 88/100.
Signal 4: Repeated failures trigger system redesign rather than endless reflective prompts. On 100 graph-colouring instances, binary "try again", first-error and all-errors feedback performed indistinguishably — more reflective prompting was not the missing ingredient.

Example

GPT-4 critiqued its own colourings fluently and fell from 16% to 1%

Direct prompting solved 16% of 100 graph-colouring instances. Backprompting GPT-4 with its own self-critique left 1 of the 100 correct. An external sound verifier, feeding back to the same generator, lifted performance to nearly 40%. Stechly and colleagues ran all three in 2023.

The critiques were not vague. They were specific, and they failed in the direction hardest to notice: “At some point in the backprompts of 40 instances, the generating model returned an optimal coloring. In none of those instances did the verifying GPT realize this.” Only 4 of the 100 runs were ever stopped by the LLM verifier. None of those 4 was correct.

The authors also varied how much the feedback said: binary "try again", the first error only, all the errors. The three performed indistinguishably. A postmortem that reads well is evidence about the prose. The run that follows it is evidence about the mechanism.

Decision at stake: Design reflection loops that store actionable lessons while recognizing correlated errors and memory contamination.
Hidden assumption: A detailed self-critique is reliable evidence of the true failure cause — the assumption that carried 16% down to 1%.
Primary control question: Reflection can turn one error into persistent bad policy, and a self-critique that missed an optimal colouring in all 40 instances where one was produced is the cheapest way to manufacture one.
Evidence to collect: Reflections include the evidence and task conditions that produced them, starting with who stopped the run: here the LLM verifier stopped 4 of 100, and none of the 4 was right.

Position

Reflection worked where an external signal existed

Self-correction is the feature that would make the surrounding engineering optional. So it is worth being exact about which setup was measured. Two setups, the same 1,319-problem GSM8K test set, one paper from ICLR 2024 whose title is the finding: Large Language Models Cannot Self-Correct Reasoning Yet. With oracle labels telling the model when it was wrong, GPT-3.5 rose from 75.9 to 84.3 and GPT-4 from 95.5 to 97.5. With intrinsic self-correction and no external signal, GPT-3.5 fell 75.9 → 75.1 → 74.7 and GPT-4 fell 95.5 → 91.5 → 89.0 over two rounds. Those three points cost 1, 3 and 5 model calls. Each round buys two extra calls and a decline. Same model, same problems: 75.9 to 84.3 with a label, 75.9 down to 74.7 without one.

The paper also reports the shape of the loss: “For GSM8K, 74.7% of the time, GPT-3.5 retains its initial answer. Among the remaining instances, the model is more likely to modify a correct answer to an incorrect one than to revise an incorrect answer to a correct one.” Revision is not neutral. Without a signal, it is a coin weighted against you, paid for in model calls.

That is why the graph-colouring agent is the ordinary case and not the unlucky one. Its critiques were specific, well phrased and descriptive of the symptom. The thing they never touched was the verifier. Reflexion's winning lessons were written after a test had failed and reported it. The losing ones were written after nothing outside the model had reported anything at all.

A written lesson can change the next prompt; only a change to the tool, the schema or the state contract changes the mechanism.

Key idea

Reflection can turn one error into persistent bad policy

A confident but incorrect lesson may be retrieved across many future tasks. If the memory system treats all reflections as trusted, local confusion becomes systematic behavior. One bad night becomes a rule — and the ratio at which that happens has been measured.

AgentPoison backdoored the long-term memory and RAG knowledge bases of an autonomous-driving agent, a QA agent and the healthcare EHRAgent. It was presented at NeurIPS in 2024, and its claim is blunt: “On each agent, AgentPoison achieves an average attack success rate higher than 80% with minimal impact on benign performance (less than 1%) with a poison rate less than 0.1%.” Retrieval succeeded 82% of the time, and the end-to-end attack 63%. The poisoning ratio stayed below 0.1% of stored entries. Benign performance fell by under 1%, and no model was retrained. In some settings that came from a single injected instance carrying a single-token trigger.

That is an adversary. An unvalidated reflection is the same object arriving by accident: a small number of stored entries that retrieval keeps selecting while benign metrics stay flat. Store provenance, scope, confidence, expiry, and validation status. Promote reflections only after repeated evidence.

A wrong lesson is cheapest to catch before the memory system starts handing it to every task that follows.

Steps

Run a controlled reflection experiment

Run the same failing task twice, once with the stored reflection retrieved and once without, and compare the outcomes rather than the prose. If the outcomes match, the note is decoration. Record what each run was given, and under what conditions it ran.

The Reflexion paper ran that comparison on itself and published the case where it lost. On MBPP Python, Reflexion scored 77.1 pass@1 against an 80.1 GPT-4 baseline. Reflecting made it worse. Table 2 gives the mechanism: a false-positive rate of 0.16 for MBPP Python against 0.01 for HumanEval Python — self-written unit tests passing while the implementation is wrong. Sixteen times in a hundred, the signal the lesson was written from was itself false. The lesson inherited the error with full confidence. The authors say so plainly: “We acknowledge that self-reflecting code-generation agents are bound to their ability to write diverse, comprehensive tests.”

So the experiment has two arms, not one. Compare reflected runs against unreflected runs. Then check, separately, whether the signal that produced the lesson was right. The same loop scored 91.0 on one benchmark and 77.1 on another. The difference was the quality of the tests, not the quality of the reflecting. It also shows how reflection can turn one error into persistent bad policy: the note that rescues this run may be the note that misfires on the next one.

FigureProcess · 5 steps

1
Capture the failure
Preserve inputs, state, actions, tool results, and outcome evidence.
2
Localize the mechanism
Separate model choice, missing context, tool design, and runtime failure.
3
Write a scoped lesson
State when the lesson applies and what action should change.
4
Validate on new cases
Compare reflected and unreflected runs on held-out tasks.
5
Promote or retire
Keep only lessons that improve outcomes without new regressions.

Store a lesson only with the conditions it came from

Use reflection as one candidate feedback mechanism. Prefer direct fixes when the failure comes from schema, authorization, state, or verification design.

That preference is not this course's private opinion. A 2024 critical survey of the self-correction literature found that prior studies often do not define their research questions in detail, and involve impractical frameworks or unfair evaluations that over-evaluate self-correction. It ran in the Transactions of the Association for Computational Linguistics. Its conclusion is threefold: “Our critical survey based on the newly categorized research questions shows that (1) no prior work demonstrates successful self-correction with feedback from prompted LLMs, except for studies in tasks that are exceptionally suited for self-correction, (2) self-correction works well in tasks that can use reliable external feedback, and (3) large-scale fine-tuning enables self-correction.”

The condition named there is the one to design against: reliable external feedback. A reviewer's first question about a stored reflection is whether it could turn one error into persistent bad policy. Accept the mechanism only when each stored note carries the evidence and the conditions of the task it came from, so its scope can be checked again later.

Asking a model to reflect its way past an authorization bug is a way of leaving the bug in place.

Key takeaways

Reflection converts feedback from a run into a concise hypothesis or instruction for a later attempt.
A reflection is generated text, not ground truth — backprompting GPT-4 with its own critique on 100 graph-colouring instances left 1 of the 100 correct, where an external sound verifier reached nearly 40%.
Reflexion reached 91.0 pass@1 on HumanEval Python against a quoted 80.1 — roughly 18 of 164 hand-written problems, each judged by an average of 7.7 unit tests. On MBPP Python the same loop scored 77.1 against the same 80.1 baseline, with a false-positive rate of 0.16 against 0.01.
On the same 1,319-problem GSM8K test set, an oracle label took GPT-3.5 from 75.9 to 84.3, while intrinsic self-correction took it 75.9 → 75.1 → 74.7 at 1, 3 and 5 model calls.
Store provenance, scope, confidence, expiry, and validation status; promote reflections only after repeated evidence — AgentPoison needed under 0.1% of stored entries to be retrieved 82% of the time and change the action 63% of the time.
Use reflection as one candidate feedback mechanism. Prefer direct fixes when the failure comes from schema, authorization, state, or verification design. The 2024 TACL survey found self-correction works well only where reliable external feedback exists.