Select-Then-Decompose: From Empirical Analysis to Adaptive Selection Strategy for Task Decomposition in Large Language Models

AI summary

Overview

Research area: Task decomposition and planning in large language models (LLM prompting strategies for breaking complex problems into subtasks).

Technical level: Intermediate — readers should be familiar with prompting paradigms such as chain-of-thought, ReAct, and plan-and-execute, and with the idea of a performance-versus-cost trade-off.

Scope: The paper surveys and categorizes existing LLM task-decomposition approaches, empirically analyzes what drives their performance and token cost, and proposes a strategy that dynamically picks a decomposition approach per task.

Publication note: arXiv:2510.17922v1 [cs.CL], 20 Oct 2025; authors are affiliated with Beijing University of Posts and Telecommunications; the paper states it was accepted to the Main Conference of EMNLP 2025 (Oral).

What This Paper Is About

Existing task decomposition methods for LLMs — those incorporating memory, tool use, and feedback — have achieved strong results in specific domains, but the paper argues they largely overlook the trade-off between performance and cost. The authors want to know what factors actually drive both performance and token cost, and how to deliberately balance the two. To answer this, they first map the design space, then run controlled experiments, then build a strategy that adapts to each task instead of committing to one fixed decomposition paradigm.

Key Contributions

1. A comprehensive investigation of task decomposition in LLMs organized into six categorization schemes, plus empirical analysis of three factors that influence performance and cost (approach category, task characteristics, and decomposition/execution model configuration), yielding three insights and a set of practical principles.
2. Identification of five representative approaches (CoT, P&S, ReAct, P&E, P&E with DAG structure) covering three of the six categorization schemes, used as the experimental basis.
3. The Select-Then-Decompose (S&D) strategy, a closed-loop framework with selection, execution, and validation modules that dynamically chooses a decomposition approach to mitigate the performance–cost dilemma.
4. Experimental validation across multiple benchmarks showing S&D consistently lies on the Pareto frontier, balancing task performance and token cost. Code is released at https://github.com/summervvind/Select-Then-Decompose.

Main Findings

• The six categorization schemes: ❶ interleaving sequence between decomposition and execution (decomposition-first vs. interleaved), ❷ number of LLM calls required (implicit vs. explicit), ❸ topological structure of decomposition (DAG vs. linear), ❹ format of decomposition (code vs. text), ❺ range of subtask selection (limited vs. unlimited), and ❻ whether tools are used during execution (tool-augmented vs. pure LLM-based). The paper focuses on the first three because the latter three each include a category tailored to specific tasks.

• Overall benchmark performance (GPT-4o-mini, average across benchmarks): P&E (DAG) 69.40, P&E 67.81, ReAct 67.26, CoT 65.65, P&S 64.70, IO 48.60.

• Task-specific strengths: CoT scores 93.45 on GSM8K; P&E scores 52.13 on MATH; ReAct scores 89.85 on HumanEval and is described as the only approach achieving a score closest to 90 on code generation; P&E (DAG) scores 64.34 on Trivia Creative Writing (N=5), 63.88 on Trivia Creative Writing (N=10), and 65.15 on HotpotQA.

• Takeaway I — the performance–cost dilemma: Explicit approaches perform better on certain benchmarks but incur substantially higher cost. In Trivia Creative Writing (N=5), P&E's token consumption exceeds that of the implicit approach with the highest token usage by roughly 10×. P&E (DAG) performs consistently well but still uses approximately 4× more tokens than implicit approaches.

• Takeaway II — tasks determine the approach: Task characteristics determine the sequence, calling form, and topology of task decomposition. Math and reasoning tasks suit CoT's <implicit> and P&E's <linear> strategies; writing and comprehension tasks align with P&E (DAG)'s parallel decomposition; code generation, being iterative, benefits from ReAct's <explicit, interleaved> strategy. This is reinforced by MT-bench subjective evaluation using Claude-3.5-Sonnet as judge, where CoT leads on math (9.14 average) and reasoning (8.11), and P&E (DAG) leads on writing (8.53) and roleplay (8.29).

• Takeaway III — execution model scale matters most: In nine cross-model experiments using Qwen2.5-1.5B/7B/14B-instruct on MATH, accuracy improves with larger parameter size for both decomposition and execution models. The slope for scaling the execution model is noticeably steeper than for the decomposition model, and the execution-model line closely aligns with the overall line — indicating execution-model adjustment is the primary driver of the scaling effect. Separately, reasoning models (Deepseek-R1 distilled versions) consistently outperform non-reasoning counterparts across all parameter scales in the execution stage, though at the 14B scale reasoning models produced more invalid plans, suggesting increased reasoning ability may compromise format control.

• S&D results: S&D consistently lies on the Pareto frontier across five benchmark tasks. On HumanEval and HotpotQA it attains near-optimal performance using only 24.77% of the average token cost. On GSM8K and MATH (where candidate approaches show small performance gaps) it achieves higher accuracy with minimal additional cost. On Trivia Creative Writing it shows an approximately linear trade-off along the Pareto frontier.

• Approach selection distribution: Overall, implicit decomposition approaches dominate at approximately 85% of final selections, with explicit approaches at about 15%. CoT and P&E are more common in mathematical tasks, ReAct in code generation, and P&E (DAG) in writing and text understanding.

• Ablation (GPT-4o-mini, HumanEval): Full configuration scores 88.55 at 845.82 average tokens. Removing selection and using IO as the initial approach gives 86.59 (down 2.21%) at 542.35 tokens (down 35.89%). Removing selection and using a random initial approach gives 87.19 (down 1.53%) at 2782.34 tokens (up 228.99%). Removing the validation module gives 85.98 (down 2.90%) at 753.29 tokens (down 10.94%).

• Threshold sensitivity: On Trivia Creative Writing, raising the confidence threshold improves performance but increases token consumption, particularly beyond 0.9. A threshold of 0.7 minimizes token cost within the range 0.5 to 1.0 and yields performance even exceeding a threshold of 0.8, motivating T = 0.7 as the default.

• Generalization: On the DROP benchmark, S&D still maintains a Pareto advantage. The selection prompt contains no task- or dataset-specific information.

Methodology in Plain English

The authors work in three phases.

First, they survey existing task decomposition work and sort it along six dimensions, then pick five representative methods that span three of those dimensions (CoT, P&S, ReAct, P&E, P&E with DAG). They add direct LLM invocation (IO) as a baseline.

Second, they run experiments on five objective benchmarks — GSM8K and MATH (math), HumanEval (code generation), Trivia Creative Writing (creative writing), and HotpotQA (text understanding) — plus MT-bench as a subjective benchmark judged by Claude-3.5-Sonnet. GPT-4o-mini is the base model with temperature set to zero and the seed fixed at 42; they run five independent runs and report means with error bars. They also measure token consumption and API call frequency, and test how model scale and reasoning capability affect the decomposition versus execution stage using Qwen2.5 series and Deepseek-R1 distilled models on MATH.

Third, they build Select-Then-Decompose. The Selection Module uses an LLM with a prompt to analyze the question and return the most suitable decomposition approach plus reasoning. The Execution Module applies that approach to produce a candidate solution. The Validation Module has an LLM assign a confidence score C in [0,1]; if C is at or above a threshold T, the solution is accepted. Otherwise a staged switching mechanism moves through {IO}, then implicit approaches {CoT, P&S}, then explicit approaches {ReAct, P&E, P&E (DAG)}, sampling uniformly at random within each group. Experiments use T = 0.7 with up to 3 switching iterations.

Why This Matters

Impact on research: The paper reframes task decomposition as a selection problem rather than a fixed-paradigm problem. Its six-scheme taxonomy gives a shared vocabulary for comparing methods, and its empirical findings — especially that the execution model matters more than the decomposition model — challenge the common assumption that planning quality is the main bottleneck.

Real-world applications (drawn from the task categories the paper evaluates):

• Mathematical and reasoning assistance, where CoT-style implicit reasoning performs best at low cost.
• Code generation and iterative programming help, where ReAct-style interleaved reasoning and acting is favored.
• Creative writing and content synthesis, where P&E (DAG) parallel decomposition leads.
• Multi-hop question answering and document comprehension, where P&E (DAG)-style decomposition supports combining evidence from multiple sources.

Industry relevance: Token cost is a direct operating expense for deployed LLM systems. A strategy that reaches near-optimal accuracy on HumanEval and HotpotQA at 24.77% of average token cost, and that routes roughly 85% of queries through cheaper implicit approaches, is directly relevant to teams balancing quality and serving budget. The practical principles also give deployment guidance: for explicit decomposition, use a strong model for execution and a model with basic instruction-following ability for decomposition.

Future Directions

• Representation formats and tool use. The authors state they examined only the decomposition mechanism, not representation formats (code vs. text) or external tools. They note S&D may need adaptation for tasks requiring specialized tools.
• Training the selection capability. S&D relies solely on prompting, with no additional training; the authors note that a weak selection model may degrade overall performance.
• Finer-grained routing rules. The findings that different task categories favor different approaches (CoT for math, ReAct for code, P&E (DAG) for writing and comprehension) suggest room for more precise task-to-approach mapping.
• Understanding reasoning-model trade-offs. The observation that reasoning models produce more invalid plans at the 14B scale, while outperforming non-reasoning models in execution, raises an open question about where reasoning capability helps versus hurts.

Target Audience

Researchers and practitioners working on LLM reasoning, planning, and agentic systems who need to choose among decomposition strategies; engineers deploying multi-step LLM pipelines under cost constraints; and readers interested in a taxonomy-driven, empirically grounded alternative to picking a single fixed prompting paradigm. The paper assumes familiarity with methods like chain-of-thought, ReAct, and plan-and-execute, so it is best suited to an intermediate audience rather than complete beginners.

Authors’ abstract

Large language models (LLMs) have demonstrated remarkable reasoning and planning capabilities, driving extensive research into task decomposition. Existing task decomposition methods focus primarily on memory, tool usage, and feedback mechanisms, achieving notable success in specific domains, but they often overlook the trade-off between performance and cost. In this study, we first conduct a comprehensive investigation on task decomposition, identifying six categorization schemes. Then, we perform an empirical analysis of three factors that influence the performance and cost of task decomposition: categories of approaches, characteristics of tasks, and configuration of decomposition and execution models, uncovering three critical insights and summarizing a set of practical principles. Building on this analysis, we propose the Select-Then-Decompose strategy, which establishes a closed-loop problem-solving process composed of three stages: selection, execution, and verification. This strategy dynamically selects the most suitable decomposition approach based on task characteristics and enhances the reliability of the results through a verification module. Comprehensive evaluations across multiple benchmarks show that the Select-Then-Decompose consistently lies on the Pareto frontier, demonstrating an optimal balance between performance and cost. Our code is publicly available at https://github.com/summervvind/Select-Then-Decompose.