Task Decomposition and Workflow Prompting

A procurement assistant receives a contract. It has to identify obligations, compare policy, calculate deadlines, assign risk, and draft an email. One fluent response does all five at once. When the answer is wrong, there is nowhere to look.

The staged design costs more to run. In exchange you can see inside it and steer it. That trade has been measured, not merely argued.

"AI Chains", in 2022, chained LLM steps so that each step's output became the next step's input, then put the result in front of users. The abstract reports: “In a 20-person user study, we found that Chaining not only improved the quality of task outcomes, but also significantly enhanced system transparency, controllability, and sense of collaboration.” Transparency and controllability were not decorative by-products of the chain. They were what the participants used it for, debugging unexpected outputs by “unit-testing” sub-components of a chain. Intermediate contracts are testable because people actually tested them.

The middle option — asking one model to narrate a plan inside a single response — buys the appearance of that without the substance. A 2023 paper, "Language Models Don't Always Say What They Think", tested it. The authors introduced biasing features that the model never mentions, such as reordering multiple-choice options so that the answer was always "(A)". The abstract: “This causes accuracy to drop by as much as 36% on a suite of 13 tasks from BIG-Bench Hard, when testing with GPT-3.5 from OpenAI and Claude 1.0 from Anthropic.” The models went on generating explanations that rationalised the biased answers. A rationale that omits the real cause of the answer is not an audit trail, however well organised it looks. And nothing outside that single context window is in a position to contradict it.

A stage earns its place by producing an artifact that another component can check. The distance between doing that and not doing it can be the whole benchmark.

Least-to-most prompting decomposes each problem into ordered subproblems. The paper that introduced it, at ICLR 2023, records the result: “A notable finding is that when the GPT-3 code-davinci-002 model is used with least-to-most prompting, it can solve the compositional generalization benchmark SCAN in any split (including length split) with an accuracy of at least 99% using just 14 exemplars, compared to only 16% accuracy with chain-of-thought prompting.” At least 99% against 16%. Same model, same benchmark, every split including the length split. The specialised neuro-symbolic models the paper compares against had been trained on the full set of over 15,000 examples. Least-to-most used 14 exemplars. The ordering of subproblems was the intervention.

The first stage, acquiring evidence, carries its own measurement. ReAct interleaved reasoning traces with tool actions, and its abstract reports: “On two interactive decision making benchmarks (ALFWorld and WebShop), ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.” Acting against a Wikipedia API, the authors reported, overcame the hallucination and error propagation seen in pure chain-of-thought. The external call is not an ornament on the reasoning. It is the thing that stops the reasoning drifting. That is exactly why the boundary between acquiring evidence and generating interpretation is worth paying for.

A workflow stage takes one artifact in and puts a different one out. The schema should record source identifiers, confidence or validation status, policy version, and unresolved fields. Passing only natural-language summaries between stages can erase provenance and uncertainty. Typed state makes it easier to prevent an unverified claim from becoming a trusted fact downstream.

What happens when provenance travels as prose has been counted in shipped commercial products. A 2023 audit had humans check the answers of Bing Chat, NeevaAI, perplexity.ai and YouChat against the citations attached to them. The abstract: “We find that responses from existing generative search engines are fluent and appear informative, but frequently contain unsupported statements and inaccurate citations: on average, a mere 51.5% of generated sentences are fully supported by citations and only 74.5% of citations support their associated sentence.” These are systems whose entire product promise is the citation. On average barely half of their sentences were fully supported by the ones they carried. A citation rendered into text is a claim about provenance made by the same component that wrote the sentence. A source identifier held in a typed field and checked at the stage boundary is provenance, because something other than the generator has to sign for it.

Use the smallest number of stages that creates useful control boundaries. List the distinct claims. Mark which source each one requires. Assign each to code, a tool, a model or a person. Define the stage schemas so they carry provenance and unresolved fields. Specify what is retried and what escalates. Then evaluate the components as well as the whole.

That last step is the one teams treat as optional, and the field report on treating it that way is specific. Barnett and colleagues drew on three deployed case studies — in research, education and biomedicine — and catalogued seven distinct failure points. Their conclusion: “The two key takeaways arising from our work are: 1) validation of a RAG system is only feasible during operation, and 2) the robustness of a RAG system evolves rather than designed in at the start.” Stage metrics are not a pre-launch checklist item to be discharged and closed. They are the instrument you keep running. On this evidence, the failure points reveal themselves only once real traffic is passing through the stages.

Splitting a task into more model calls is not automatically better. Each stage should exist because it isolates a failure, permission, data source, or validation rule.

The next lesson examines reasoning and test-time computation. Letting the model deliberate internally can improve some tasks, while external workflow structure makes intermediate evidence easier to inspect.