Doing Less with More: A First-Principles Exploration of the Suitability of Agentic Computing over Alternative Architectural Choices

AI summary

Overview

Research area: Artificial intelligence systems architecture — specifically, comparing agentic LLM workflows against conventional deterministic invocation mechanisms (local function calls, REST, SOAP, MCP) for exact computation.

Technical level: Intermediate. The paper assumes familiarity with microservice protocols (REST, SOAP, MCP) and LLM tool-calling, but its argument is built on a deliberately simple arithmetic task.

Scope: A controlled benchmark of 25 configurations across seven invocation methods on 1,000 arithmetic expressions from the SVAMP dataset, measuring accuracy, wall-clock time, processor cost and lines of code, from which the authors derive an Architectural Decision Framework (ADF).

What This Paper Is About

Agentic AI is increasingly the default choice for automating business activities, but it is rarely compared on equal footing against older, cheaper mechanisms that do the same job. The authors isolate one narrow, exactly verifiable task — evaluating arithmetic expressions while respecting operator precedence (PEMDAS/BODMAS) — and measure whether LLMs and LLM-driven agents actually beat function calls, REST, SOAP and MCP on accuracy, speed and cost. The goal is not to argue about arithmetic per se, but to establish that there is no one-size-fits-all architectural answer and that invocation choices should be made against explicit trade-offs.

Key Contributions

1. A controlled, apples-to-apples evaluation of seven invocation strategies across four architectural families — native function calling, service-oriented architectures (REST, SOAP, MCP), direct LLM answering (one cloud API plus five local Ollama models), and agentic LLMs calling MCP tools — all run on the same 1,000 SVAMP equations with identical prompts and parsing rules.

2. A four-dimensional measurement scheme covering accuracy, total wall-clock time, processor runtime cost, and development effort in lines of code, reported per configuration and as slowdown/overhead multipliers against a native function-call baseline.

3. A decomposition of LLM failure modes into wrong answers versus format failures (non-numeric responses), plus tracking of tool misuse, tool-call counts and turn-limit overruns for the agentic configurations.

4. An Architectural Decision Framework (ADF) that maps task characteristics — exactness requirements, protocol rigor, latency sensitivity, cost structure — onto a recommended invocation mechanism.

Main Findings

• Deterministic methods are perfect and near-free. Function Calling, REST, SOAP and MCP all reach 100% accuracy. Function Calling completes all 1,000 equations in 6.9 ms (7 µs per equation) using 0.0104 s of processor time.

• Protocol overhead is bounded but large relative to in-process calls. REST and MCP are 326× slower than the baseline (2.25 s total) and SOAP 347× (2.40 s); processor cost is 1.08–1.20 s, or 104–116× baseline.

• Direct LLM answering tops out at 92.9% accuracy. Granite4-3B is the best local model at 92.6–92.9%, followed by Phi4-mini-3.8B (81.8–82.0%), Llama3.2-3B (73.1–74.3%) and Llama3.2-1B (42.5–44.6%). The cloud API (gpt-oss-20b on Groq) scores 15.5% and costs 39,180 tokens total (15,571 prompt, 23,609 completion).

• Latency spreads over six orders of magnitude. Direct local LLMs are 8,800× to 29,752× slower than Function Calling; the remote API is 44,399× slower; Qwen2-Math-1.5B is the slowest direct model at 240,320–256,929× because it writes out full step-by-step solutions.

• Tools rescue accuracy only for models that use them reliably. Qwen3.5-4B reaches 99.9% (T=0) and 98.1% (T=0.7) with MCP tools, at 990,497–992,683× baseline latency (6.8 s per equation). Granite4-3B improves from 92.6–92.9% to 96.5–98.0% but slows to 163,303–163,311×.

• Tools hurt weaker models. Phi4-mini-3.8B never calls a tool and drops to 63.1–66.3%, below its direct accuracy. Llama3.2-1B falls to 4.3–8.0% and Llama3.2-3B to 17.5–21.2%, despite making 1.44–1.73 tool calls per equation with 110–189 tool errors per 1,000 equations, and then answering in prose.

• Domain fine-tuning did not guarantee deployability. Qwen2-Math-1.5B, fine-tuned for mathematics, scores 0.0% at both temperatures — all 1,000 responses are format failures. Qwen2-Math-1.5B does not support tool calling at all, so it was replaced by Qwen3.5-4B in the agentic condition.

• Temperature is not a meaningful lever. Moving from T=0 (greedy decoding) to T=0.7 changed accuracy by at most 2.1 percentage points for direct invocation and 3.7 points with tools, with little effect on time or cost.

• Development effort does not track rigor or performance. Function Calling needs 23 lines, direct Ollama 40, Groq 44, REST 136 (75 client, 61 server), MCP 165 (104 client, 61 server), SOAP 258 (87 client, 171 server), and the LLM + MCP agent 259 (198 client, 61 server). Relative to Function Calling: REST 5.9×, MCP 7.2×, SOAP 11.2×; the agent is 6.5× the direct Ollama integration.

• Failure modes shift with architecture. Answering directly, models mostly miscalculate rather than break format (e.g., Granite4-3B: 66 wrong versus 5 format failures out of 71 at T=0). With tools, failures become interaction failures — Llama models produce 641–755 format failures per 1,000, and Llama3.2-1B exceeds the six-turn limit on 88–97 equations.

Methodology in Plain English

The researchers took 1,000 equation strings from the SVAMP benchmark (train and test splits combined) and asked every method to return the exact numerical answer. Because the correct answer is mechanically checkable and depends on applying operator precedence correctly — for example, (78.0*(186.0-172.0)) yields 1092, while multiplying first would give 14336 — the task isolates computation from open-ended language understanding.

They built seven ways to answer the same question. One is a native Python eval() call, the baseline. Three are small services that decompose each equation into pairwise operations and dispatch them one call at a time over HTTP: a Flask REST server on localhost:5001, an XML SOAP service on localhost:8000, and an MCP server on localhost:8100 exposing add, subtract, multiply and divide. Two are LLMs answering from a prompt with no tools — one cloud model via the Groq API and five local models via Ollama, each run at T=0 and T=0.7. The last is an agent: local models given the four MCP tools as function definitions, allowed up to six LLM turns with each reply capped at 256 tokens, ending when the model replies without a tool call.

Scoring used exact match on the parsed numeric output, treating any explanatory text as a format failure and therefore incorrect. Time was total wall-clock duration divided by 1,000 samples. Cost was measured two different ways, which the authors are explicit about: CPU-seconds from psutil for the deterministic pipelines, Ollama's self-reported inference duration (prompt evaluation plus generation, summed across all turns) for local models, and token counts for the cloud API. Development effort was counted with wc -l, excluding shared harness code.

Why This Matters

For research: The paper supplies a clean, reproducible control setting for a question usually argued anecdotally — when is an agent worth its overhead? Its result that greedy decoding (T=0) buys repeatability but not correctness, and that math-specialized models can fail an instruction-following requirement that general models of similar size satisfy, are direct challenges to common assumptions in model selection.

Real-world applications:

• Financial calculations and safety-critical logic, where the authors explicitly recommend deterministic methods as the only ones reaching 100% accuracy.
• Regulated, multi-party or auditable exchanges, where SOAP's WSDL-upfront disclosure and fixed fault format are argued to be preferable to REST's implementation-defined contracts.
• High-volume, latency-sensitive services, where any LLM route costs 8,800× to 992,683× the baseline and REST or MCP are marginally faster than SOAP (326× versus 347×).
• Exploratory prototyping against natural-language specifications, where direct LLM calls are the fastest to build at 40–44 lines and the recommended path is to migrate to REST once the task is understood.

Industry relevance: The lines-of-code data reframes "agents are easier to build" as a claim that does not survive measurement — the agentic route (259 lines) is as costly to build as SOAP (258 lines) and 6.5× the direct Ollama client, while still being slower and less accurate than a 23-line function call. Cost structure also differs in kind: cloud APIs are metered and forecastable but recurring, local models trade that for fixed hardware and energy costs at orders of magnitude more compute, and function calls have effectively no marginal cost.

Future Directions

• Extending the comparison to tasks that genuinely mix computation with language understanding, where the deterministic-method advantage may not hold as cleanly as it does for pure arithmetic.
• Testing richer semantics and multi-step workflows beyond single arithmetic expressions and a fixed prompt template.
• Broadening the temperature sweep beyond the two settings (T=0 and T=0.7) evaluated per model, and probing whether agent reliability changes with turn budgets other than the six-turn limit used here.
• Addressing the protocol overhead the authors flag as acceptable only for low-frequency services — batching or persistent connections for high-volume REST, SOAP and MCP deployments.

Target Audience

Software architects and engineering leads deciding whether to build an agentic system or a conventional service; AI practitioners selecting models for tool-calling workflows who need evidence that documented capabilities are not the same as measured behavior; and researchers studying agent reliability, cost modeling or evaluation methodology. Readers new to microservice protocols will still follow the argument, but will get more from the protocol-by-protocol comparisons with some background in REST, SOAP and MCP.

Authors’ abstract

There is growing interest in automating business activities with Agentic Artificial Intelligence (AI) due to latter's seeming ease of use. Never has it been easier, or costlier, to do less with more. However, little is known about when agents are preferable to established alternatives such as local computation, Representational State Transfer (REST), the Simple Object Access Protocol (SOAP), and the Model Context Protocol (MCP), particularly when development speed, performance, and operational cost are considered. We investigate this question using a controlled mathematical task that compares seven methods on a benchmark of 1,000 arithmetic expressions where semantics of operator precedence has to be preserved. We ran this setup across a native Function Calling baseline (Python library), REST, SOAP and MCP microservice architectures, direct responses from large language models (LLMs) using a Groq cloud API and five locally deployed Ollama models, and agentic systems in which five local LLMs invoke MCP tools. The deterministic methods achieve 100% accuracy with negligible latency. Direct LLM evaluation achieves at most 92.9% accuracy at over 22,000x the latency of a native function call. With MCP tools, the best agent reaches 99.9% accuracy, but incurs nearly 1,000,000x the baseline latency. These results demonstrate that LLMs approaches are poorly suited for exact arithmetic evaluation and motivate an informed architectural decision framework (ADF) for choosing an architecture based on accuracy and throughput requirements. We recognize that an actual interaction with an agent may span both reasoning and linguistic tasks. Hence our recommendation is to explore multiple architectural choices (based on task type) rather than one-size-fits-all solution (LLMs). Code is available at - https://github.com/Ritvik-G/adf/