The seminal development in this area was the introduction of Chain-of-Thought (CoT) prompting (Wei et al., 2022b). Prior to this, standard prompting involved presenting the model with an input (the question) and expecting a direct output (the answer). This direct input-to-output mapping proved brittle for tasks requiring sequential deduction or calculation. Wei et al. demonstrated that by providing a few examples (few-shot prompting) where the solution included the intermediate reasoning steps—a “chain of thought”—the model could adopt this strategy for novel problems.

The impact of this intervention was significant. On benchmarks requiring mathematical and commonsense reasoning (e.g., GSM8K), CoT prompting dramatically improved performance; for instance, it boosted the accuracy of the 540B parameter PaLM model on GSM8K from 18%

(standard prompting) to 57% (Wei et al., 2022b). The significance of CoT is twofold. First, it established that the reasoning process itself is a crucial output, serving as a cognitive scaffold that allows the model to decompose complexity and mitigate the risk of premature, incorrect conclusions. Second, the externalized reasoning chain offered a novel degree of interpretability, allowing researchers to diagnose where a model’s logic faltered, rather than just observing the final error.

The initial formulation of CoT required the manual creation of task-specific examples. This limitation was rapidly addressed by two critical innovations that enhanced the technique’s generality and robustness.

Kojima et al. (2022) introduced “Zero-Shot-CoT,” discovering that reasoning could be elicited without any examples. By simply appending a task-agnostic trigger phrase, such as “Let’s think step by step,” they induced the model to generate a coherent chain of thought. This finding was crucial, as it suggested that the reasoning abilities elicited by few-shot CoT were not merely a result of pattern mimicry but were an emergent property of scale, inherent to the model and awaiting the appropriate cognitive trigger.

To address the fragility of individual reasoning paths, Wang et al. (2022) introduced the concept of “Self-Consistency.” They observed that a single, greedily decoded chain of thought could be derailed by an isolated logical error. Their solution was an ensemble-like decoding strategy: generate multiple, diverse reasoning paths for the same problem by sampling from the model’s output distribution (e.g., using temperature sampling), and then select the final answer by majority vote. This approach of marginalizing out flawed reasoning through consensus substantially improved accuracy across a spectrum of benchmarks, yielding further significant gains (e.g., improving PaLM-540B accuracy on GSM8K from 57% to 74.4%).

TABLE 1

For problems characterized by significant compositional complexity, generating a correct, monolithic chain of thought can remain intractable. Addressing this, Zhou et al. (2022) introduced “Least-to-Most Prompting.” This strategy involves a multi-stage process. First, the model is prompted to decompose a complex problem into a sequence of simpler, ordered subproblems.

Subsequently, the model solves these subproblems sequentially, with the crucial feature that the solution to each preceding subproblem is injected into the context for the next. This methodical decomposition allows the model to manage cognitive load and tackle challenges that would be overwhelming if approached directly, proving especially effective for tasks requiring generalization far beyond the training data patterns.

Collectively, these foundational techniques transformed the perspective on LLMs, establishing that unlocking machine reasoning was not solely a matter of scale, but of designing interactions that effectively scaffold and guide their latent cognitive abilities. These methods are summarized and compared in Table 2.

A significant evolution beyond linear constraints was the “Tree of Thoughts” (ToT) framework (Yao et al., 2023). The authors argued that CoT is analogous to an intuitive, stream-ofconsciousness process, whereas complex problems often demand a more deliberate, systematic, and exploratory approach. This distinction explicitly draws upon the dual-process theory in cognitive science, contrasting “System 1” (fast, intuitive) thinking with “System 2” (slow, deliberate) reasoning (Kahneman, 2011). Many tasks require the exploration of different branches of thought and the ability to make strategic judgments—a process poorly modeled by the fixed structure of CoT.

The ToT framework operationalizes this deliberate process by structuring reasoning as a tree, where each node represents a “thought”—a coherent semantic unit serving as an intermediate step. The framework integrates three key components:

1. Thought Generation: At each node, the LLM acts as a “proposer,” generating multiple potential next steps, thereby creating distinct branches.

2. State Evaluation: The LLM is then utilized as an “evaluator,” assessing the progress and viability of each branch toward reaching a final solution. This evaluation (e.g., a numeric score or classification) provides the necessary feedback for guiding the search.

3. Search Algorithm: A systematic search algorithm (e.g., Breadth-First Search (BFS) or Depth-First Search (DFS)) navigates the tree, utilizing the state evaluations to prioritize exploration and prune unpromising branches.

This architecture enables the model to engage in deliberate problem-solving, incorporating lookahead and backtracking capabilities. Experiments on tasks requiring non-trivial planning or search demonstrated substantial performance gains over CoT. For example, in the Game of 24, a task requiring complex combinatorial search, GPT-4 achieved a success rate of only 4% using CoT, whereas the ToT framework boosted performance to 74% (Yao et al., 2023). This marked a critical transition from simple reasoning elicitation to systematic reasoning exploration.

While ToT expanded reasoning capabilities through branching, Besta et al. (2023) argued for an even more flexible topology. Human cognition often involves the synthesis of information from disparate reasoning paths and the iterative refinement of ideas. Their work, “Graph of Thoughts” (GoT), generalizes the reasoning structure into a directed acyclic graph.

In GoT, thoughts (nodes) and their dependencies (edges) allow for a richer set of cognitive operations than a tree structure permits:

• Aggregation: Thoughts from independent branches can be merged and synthesized into a new, more comprehensive thought, allowing the model to combine the strengths of different approaches (e.g., merging two different solution strategies into a superior hybrid).

• Refinement: Thoughts can be iteratively improved. The graph structure supports explicit “refinement loops,” where a thought can be critiqued and transformed multiple times until it meets a quality standard.

This graph structure inherently captures the complex, non-linear dependencies inherent in elaborate problem-solving. It enables an LRM not only to explore divergent paths but also to weave insights from those paths together, a capability essential for tasks involving complex planning, multi-faceted analysis, or solutions composed of many sub-solutions.

A crucial insight advanced by Zhou et al. (2024) is that the optimal reasoning structure is inherently task-dependent; no single pre-defined structure is universally optimal. Instead of imposing a structure (chain, tree, or graph), their SELF-DISCOVER framework empowers the LLM to autonomously formulate a bespoke reasoning plan.

The framework operates by allowing the model to select relevant atomic reasoning modules (e.g., “Critical Thinking,” “Decomposition,” “Analogy Generation”) and then structure these modules into an explicit reasoning plan tailored to the specific problem. This meta-reasoning capability—reasoning about how to reason—allows the model to dynamically adapt its problem-solving strategy on the fly. This self-discovered approach has shown improved performance on challenging benchmarks such as BIG-Bench Hard (Suzgun et al., 2022), demonstrating a trajectory toward more flexible, efficient, and autonomous reasoning systems. A comparison of these advanced reasoning topologies is provided in Table 3.

It is crucial to recognize that these advanced reasoning structures (Pillar II) do not operate in isolation but exhibit strong synergies with the other pillars. The effectiveness of exploratory frameworks like ToT and GoT often depends on the quality of the underlying reasoning elicited (Pillar I). Furthermore, the “State Evaluation” step in ToT frequently relies on the LRM’s ability to utilize external tools (Pillar III) for grounding or verification, or to apply self-critique mechanisms (Pillar IV) to assess the viability of a reasoning path. This interdependence (visualized in Figure 1) highlights the cohesive nature of the LRM framework, where advancements in one area potentiate the capabilities of the others.

TABLE 3

While the evolution from linear chains to complex graphs significantly enhances reasoning accuracy and robustness, this progression introduces substantial computational overhead. The different methodologies present distinct trade-offs between accuracy, latency, and computational cost.

Chain-of-Thought (CoT) is computationally efficient, typically requiring only a single forward pass (inference call) from the LLM. However, its accuracy is limited by the fragility of that single reasoning path. Self-Consistency increases robustness by sampling multiple paths, linearly increasing the computational cost (e.g., 10 samples equate to 10x the cost of CoT).

Frameworks like ToT and GoT introduce a multiplicative increase in cost. These methods require numerous LLM calls not only for generating potential thoughts (proposing branches) but also for evaluating the state of those thoughts to guide the search. In complex tasks, the search space can become vast, leading to hundreds or thousands of inference calls for a single problem. This high computational demand and increased latency currently limit the practical deployment of these advanced methods in real-time applications, highlighting the critical need for research into more efficient reasoning architectures (as discussed in Section 6).

Despite their linguistic fluency, LLMs are inherently poor at precise arithmetic and symbolic manipulation, as their processing relies on pattern matching rather than formal computation. To address this deficiency, Gao et al. (2022) introduced Program-Aided Language Models (PAL).

The core innovation is to reframe the task: instead of prompting the LLM to generate a final numerical answer, it is prompted to generate a program (e.g., in Python) that, when executed by a deterministic interpreter, yields the solution.

This approach leverages the complementary strengths of the two systems: the LLM excels at natural language understanding and decomposing the problem into logical, programmatic steps, while the external interpreter executes the computation with perfect fidelity. This synergy dramatically improves accuracy in mathematical reasoning and enhances interpretability.

4.2. Formal and Quantitative Reasoning

While PAL addresses computation via code, specialized efforts have focused on enhancing reasoning in formal and quantitative domains. Minerva (Lewkowycz et al., 2022), fine-tuned on extensive mathematical and scientific corpora, demonstrated significant improvements in solving complex quantitative problems by learning to utilize appropriate mathematical notation and concepts, even without relying on external execution.

Furthermore, the pursuit of truly verifiable reasoning has led to the integration of LLMs with formal verification systems and theorem provers. Frameworks such as LeanDojo (Yang et al., 2023) enable LLMs to interact with proof assistants (e.g., Lean). In this context, the theorem prover acts as an external environment that provides rigorous, deterministic feedback on the logical validity of the model’s proposed steps. This grounding in formal logic addresses the critical challenge of faithfulness, ensuring that the reasoning is not merely plausible but mathematically sound. Despite these advancements, significant limitations remain in formal reasoning. A primary hurdle is autoformalization—the process of accurately translating natural language problems into formal logical representations. Current models often struggle with the ambiguity and implicit context of natural language, leading to brittle translations that derail the subsequent proving process. Additionally, theorem proving itself faces the challenge of navigating a vast, combinatorial search space. While LLMs can suggest plausible steps, the search for a complete proof remains computationally intractable for complex theorems, often requiring sophisticated heuristics and human expertise that LRMs have yet to fully replicate.

The concept of tool use was generalized and scaled in “Toolformer” (Schick et al., 2023). This work introduced a self-supervised method for teaching an LLM to utilize external APIs (e.g., search engines, calculators, translation systems) without requiring massive human-annotated datasets. The model learns to autonomously decide when to call a tool, what arguments to pass, and how to integrate the tool’s output back into its generation process.

Complementing this, the ReAct (Reason and Act) framework (Yao et al., 2022) established a powerful paradigm for interleaving reasoning and action. In ReAct, the model generates alternating “thought” (internal reasoning about the plan and task updates) and “action” (interaction with an external tool, such as a knowledge base lookup) steps. The result of the action (an “observation”) is fed back into the context, prompting the next cycle. This tight feedback loop enables the model to construct dynamic, adaptable plans, closely mirroring human problem-solving in information-rich environments.

TABLE 4

The principles underlying PAL, Toolformer, and ReAct reflect a broader conceptual shift: positioning the LRM not as a monolithic problem-solver, but as the “reasoning engine” at the core of a larger ecosystem. This is evident in the design of contemporary flagship models like GPT-4

(OpenAI, 2023) and Gemini (Hassabis and the Gemini team, 2023).

This integration of reasoning and action is foundational to the rapidly growing field of Agentic AI. The LRM serves as the cognitive core for systems capable of executing complex, long-horizon tasks. This burgeoning field includes frameworks like AutoGPT and Voyager, as well as research into complex behavioral simulations, such as Generative Agents (Park et al., 2023), which utilize LRMs for planning, memory management, and reflection.

This paradigm is further supported by techniques such as Retrieval-Augmented Generation (RAG) (Lewis et al., 2020), where the LLM’s context is dynamically supplemented with information retrieved from external knowledge corpora. A retriever module first fetches relevant documents, which are provided to the LLM along with the prompt. This allows the model to ground its responses in specific, verifiable, and up-to-date information, significantly mitigating hallucination. Collectively, these approaches, summarized in Table 4, delineate a future where the LRM functions as a central orchestrator and autonomous agent, intelligently delegating sub-tasks to a suite of specialized, reliable external tools.

Evaluating the reasoning capabilities of LRMs remains a profound challenge. Standard benchmarks can sometimes be exploited by models that learn superficial statistical patterns without engaging in genuine, generalized reasoning. This has led to the development of more robust evaluation suites like MMLU (Hendrycks et al., 2020) and BIG-Bench (Suzgun et al., 2022).

Furthermore, comprehensive frameworks such as the Holistic Evaluation of Language Models (HELM) (Liang et al., 2022) emphasize the necessity of evaluating models across a broad spectrum of scenarios, metrics (including fairness and robustness), and potential biases.

A fundamental epistemological question persists: are these models reasoning from first principles, or executing a highly sophisticated form of pattern matching? Research from Microsoft (Research, 2023) explores this by systematically altering problem parameters to test a model’s adaptive reasoning capacity. Findings often indicate that while models excel at familiar patterns, performance degrades significantly as problems deviate from the training distribution. This suggests that robust, abstract reasoning remains an area requiring substantial progress.

A hallmark of advanced intelligence is the ability to recognize and rectify one’s own errors. A critical investigation by Huang et al. (2023), Large Language Models Cannot Self-Correct Reasoning Yet,” provided a sobering assessment of this capability. The study found that when an LLM is prompted to re-evaluate its own incorrect answer without external feedback, it frequently fails to identify its mistakes. In many cases, models express low confidence but arrive at the same incorrect conclusion, often reinforcing the flawed reasoning. This indicates that genuine, reliable self-correction is not an inherent emergent property of scale and necessitates dedicated mechanisms and training paradigms.

In response to this challenge, frameworks have been proposed to provide structured processes for iterative improvement.

These approaches generally fall into two categories: iterative refinement based on generated feedback, and principle-based supervision.

Iterative Refinement: “Self-Refine” (Madaan et al., 2023) establishes a structured, iterative loop that decomposes the refinement process into discrete steps. First, the model generates an output. Second, the model (often the same instance) is prompted to act as a critic, generating constructive feedback on its own output. Finally, the model uses this specific feedback to refine the initial solution. This cycle of “generate, provide feedback, refine” can be repeated. While this process remains externally scaffolded—requiring a designed prompting loop rather than intrinsic self-awareness—it demonstrates empirical improvements in output quality across various domains by forcing the model to consider its own deficiencies. It represents a concrete step toward developing the self-critical capabilities necessary for autonomous improvement, though its efficacy is contingent on the model’s ability to generate accurate critique.

Principle-Based Supervision: A distinct and more formalized approach, focused primarily on safety and alignment, is Constitutional AI (CAI) (Bai et al., 2022). Instead of relying solely on human feedback (Reinforcement Learning from Human Feedback - RLHF), which can be inconsistent and difficult to scale, CAI uses the AI model itself for supervision (Reinforcement Learning from AI Feedback - RLAIF). The process involves training a model to critique and revise its own responses based on an explicit set of guiding principles (a “constitution”).

This allows for the internalization of desired behaviors and norms in a transparent manner.

While developed for alignment (e.g., ensuring harmlessness), the underlying principle of using a formalized framework to guide self-correction offers a promising path toward making reasoning refinement more reliable and interpretable than ad-hoc refinement. These approaches are compared in Table 5.

TABLE 5