Chain-of-thought prompting was formalised by Wei and colleagues at Google, in a paper first posted in 2022 and published at NeurIPS 2022 (Wei et al., 2022, arxiv.org/abs/2201.11903). The core idea is simple: instead of asking a model to jump straight to an answer, you ask it to show its working first, a series of intermediate reasoning steps that significantly improves a large model's ability to perform complex reasoning (research.google). You've seen this if you've ever written a prompt like "think step by step before answering." That's chain-of-thought in its most stripped-back form.
What makes it useful is that it's a single-pass process. The model reasons through a problem in one go, producing a chain of intermediate steps that lead to a conclusion (ibm.com). It's deterministic in structure, relatively fast, and because it doesn't require multiple calls to the model, it's cheap to run at scale. If you're handling a hundred thousand queries a day and each one needs structured reasoning, single-pass matters.
What it doesn't do is revisit its own work. Once the chain is laid out, it's laid out. The model doesn't go back, catch an error in step three, and correct the downstream logic. That's not a flaw, exactly. It's just a characteristic you need to know about.
Chain-of-thought tends to be the right fit when:
"Looped reasoning" isn't a single, agreed-upon term in the research literature. I'm using it as an umbrella for approaches where the model doesn't just reason once but iterates, critiques, and refines its own output across multiple passes. The clearest research example is Self-Refine, from Madaan and colleagues in 2023 (Madaan et al., 2023, arxiv.org/abs/2303.17651), where the model generates an initial output, then provides feedback on its own answer and uses that feedback to refine itself, repeating the loop iteratively until the output meets some stopping condition (openreview.net).
You also see this pattern in agentic systems, where a model takes an action, observes the result, and decides what to do next based on what it found. The loop is built into the architecture rather than the prompt.
The trade-off is obvious: multiple calls to the model, higher latency, more cost per task. But for the right kinds of problems, none of that matters against the quality improvement you get. Complex reasoning tasks, open-ended generation, situations where the model needs to catch and correct errors before the output reaches a human, these are where iteration genuinely earns its price.
Looped reasoning tends to be the right fit when:
Here's where I want to be direct with you, because there's a lot of confusion about this in the market.
Fine-tuning is frequently sold as the solution to reasoning problems. If your model isn't reasoning well, the sales pitch goes, train it on domain-specific data and it will get smarter about your domain. In my view, that's often the wrong diagnosis.
Fine-tuning improves consistency, tone, and format. It helps a model stay on-brand, respond in a specific structure, and avoid generic outputs. What it doesn't reliably do is improve fundamental reasoning capability. If your model is getting logical problems wrong, or failing to catch contradictions, or missing multi-step dependencies, fine-tuning on your company's documents isn't going to fix that. You'd be spending real time and money on a solution to the wrong problem.
In my experience, the better path is to fix the reasoning architecture first, whether that means better prompting, a chain-of-thought scaffold, or introducing a looped refinement step, and then consider fine-tuning for the surface-level consistency issues that remain. The order matters. I've signed off on fine-tuning projects where we spent weeks preparing training data and the improvement was negligible, because the real issue was that we hadn't designed the reasoning pattern correctly to begin with.
Fine-tuning is worth the cost when the problem is genuinely about style, structure, or domain vocabulary, and the underlying reasoning is already working. It's often not worth it when people are hoping it will make the model smarter in some fundamental sense.
Twenty years in, the most useful question I've found is this: what's the cost of a wrong answer, and how often will the model be wrong without intervention?
If a wrong answer is low-stakes and correctable by the user, chain-of-thought at scale is probably fine. If a wrong answer creates a bad clinical suggestion, a wrong financial figure in a report a client acts on, or a flawed decision in an automated pipeline with no human in the loop, you want the model checking its own work before it commits.
Three questions worth sitting with before you decide:
Pick one AI task in your current build or workflow where you've been frustrated by output quality. Don't reach for fine-tuning as the first option. Instead, ask whether the problem is that the model isn't reasoning through the task properly in the first place. Try adding a simple chain-of-thought scaffold to the prompt and see whether the output improves without changing anything else. If it does, you've just learned something important about where the actual problem was sitting. If it doesn't, you've ruled out the cheap fix and you have better information before you commit to something more expensive.
That's not a magic solution. It's just a habit of testing the simpler hypothesis before the costly one, which is, in my view, the most underrated skill in building with AI.
What's the core difference between chain-of-thought and looped reasoning? Chain-of-thought is a single-pass technique: the model produces a series of intermediate reasoning steps then an answer, without revisiting its work. Looped reasoning covers iterative approaches (like Self-Refine) where the model generates output, critiques it, and refines it across multiple passes.
Who introduced chain-of-thought prompting? Wei and colleagues at Google, in a paper first posted in 2022 (arxiv.org/abs/2201.11903) and published at NeurIPS 2022. It showed that generating intermediate reasoning steps significantly improves complex-reasoning performance in large language models.
When should I use looped reasoning instead of chain-of-thought? When the task is complex or ambiguous enough that a single pass will often be wrong, when error correction before output justifies the extra cost, when you're building agentic systems, and when your query volume keeps latency and API cost manageable.
Will fine-tuning fix my model's reasoning problems? Usually not. Fine-tuning improves consistency, tone, format, and domain vocabulary, not fundamental reasoning capability. Fix the reasoning architecture first (better prompting, a CoT scaffold, or a refinement loop), then fine-tune for surface-level issues that remain.
