Chain-of-thought prompting

The intervention is small enough to state in one sentence: in the few-shot exemplars, replace each answer with the worked reasoning that leads to it, and the model imitates the format on the new question. No weights move, no data is collected, and the paper's headline used eight exemplars.

The result was PaLM 540B on GSM8K: 17.9% with standard prompting, 56.9% with eight chain-of-thought exemplarssource, accessed 2026-08-28, which is a tripling from a formatting change. What made it a result rather than a curiosity is what it passed: the mark it passed was 55% on GSM8K, from a finetuned GPT-3 with a verifiersource, accessed 2026-08-28 — a prompt beat a fine-tune with a trained answer checker attached.

The condition the paper states about its own method. Chain of thought is an emergent ability of model scale: it does not positively impact performance for small models, and only yields gains with models of around 100B parameterssource, accessed 2026-08-28, and the smaller results in the same table show the slope: on GSM8K, LaMDA 137B moved from 6.5% to 14.3% and GPT-3 175B from 15.6% to 46.9%source, accessed 2026-08-28. Below that range the technique is not merely weaker, it is negative, and the authors name the mechanism: models of smaller scale produced fluent but illogical chains of thought, leading to lower performance than standard promptingsource, accessed 2026-08-28. Fluent and illogical is the pairing worth remembering — the visible reasoning is generated by the same process that generates the answer, so its readability is not evidence that it is what produced the answer.

Where the gains actually live. Sprague and colleagues (2024-09-18) checked the generalisation the technique had acquired by reputation: a meta-analysis over 100 papers plus fresh evaluation of 20 datasets across 14 models, finding strong benefits primarily on tasks involving maths or logic and much smaller gains elsewheresource, accessed 2026-08-28. The sharpest finding is a separator anyone can apply — on MMLU, generating the answer directly gives almost identical accuracy to chain of thought unless the question or the model's response contains an equals signsource, accessed 2026-08-28. The lift is not "the model thinks first"; it is the model executing symbol manipulation it would otherwise skip past. Follow that to its conclusion and the same study does: much of the gain comes from improving symbolic execution, and it underperforms relative to using a symbolic solversource, accessed 2026-08-28. If the value is execution, a program executes better.

Two practical consequences follow, and they point in opposite directions from the technique's reputation. On knowledge and reading tasks the extra generated reasoning is close to pure inference cost, and dropping it costs almost nothing. On arithmetic and symbolic work, where it does pay, the better move once a tool call is available is to stop asking the model to be the calculator.