SOLID: a Framework of Synergizing Optimization and LLMs for Intelligent Decision-Making

AI summary

Overview

Research area: Artificial intelligence and operations research — specifically the integration of mathematical optimization models with large language models (LLMs) for automated decision-making.

Technical level: Intermediate. The framework builds on ADMM-style dual decomposition, which requires some familiarity with convex optimization, but the paper presents the algorithm, case study, and prompt design in an accessible way.

Scope: The paper proposes SOLID, a distributed-optimization-inspired framework in which an optimization agent and an LLM agent iteratively exchange proposals mediated by dual prices and deviation penalties, and evaluates it on a 2024 NASDAQ stock portfolio case study.

What This Paper Is About

Optimization models are good at numerical, structured problems but cannot directly digest unstructured text such as news, clinical narratives, or analyst reports. LLMs are good at interpreting context but are unreliable at precise numerical optimization and structured reasoning at scale. This paper asks whether an automated pipeline can let the two kinds of agents work together — each handling what it does best — to produce better joint decisions than either alone. SOLID is the proposed answer: a coordinator passes a shared public decision and per-agent "prices" back and forth until the agents agree.

Key Contributions

1. A framework that combines the numerical precision of an optimization model with the contextual understanding and reasoning of an LLM, enabling decisions that draw on both structured and unstructured data.
2. A coordinated interaction mechanism inspired by dual decomposition, in which a coordinator uses dual prices and deviation penalties to drive iterative collaboration between the optimization agent and the LLM agent toward a consensus solution, with convergence under appropriate conditions.
3. A formal treatment (drawing on ADMM) showing that the algorithm converges to an optimal solution under stated convexity assumptions, plus discussion of what happens when those assumptions are violated by discrete or non-convex LLM behavior.
4. An empirical demonstration on a portfolio optimization task with alternative data (financial news), reporting improved annualized returns relative to an optimizer-only baseline, and showing robustness across three LLM backends.

Main Findings

• Portfolio values: Starting from $10,000 over 12 months, the SOLID variants (LLM+OPT and LLM_sparse+OPT) consistently achieved the highest portfolio values, followed by OPT, while the pure LLM strategies produced more moderate returns.
• Risk behavior: OPT showed pronounced volatility spikes around months 4–5, while LLM_sparse maintained consistently lower volatility. SOLID strategies achieved intermediate risk levels, balancing return potential against risk control.
• Allocation patterns: OPT frequently made extreme allocations to specific stocks (the paper names MDT as an example), corresponding to elevated risk, whereas LLM-based approaches distributed weights more evenly.
• Convergence in practice: Using the AMZN stock as an illustration, the two agents achieved exact convergence at certain time points (months 1 and 7); at other times decisions progressively aligned, indicating a tendency toward agreement despite LLM complexity.
• Model robustness: Across ChatGPT-4o-mini, GPT-4o, and o1-mini, SOLID pipelines consistently achieved the best risk-return balance. With 4o-mini, both LLM+OPT and LLM_sparse+OPT outperformed the other methods; with 4o, LLM_sparse+OPT achieved exceptionally high returns.
• Headline result: The abstract reports improved annualized returns compared to a baseline optimizer-only method, validating the synergy of the two agents. Specific annualized return or variance figures are not reported in the provided content.

Methodology in Plain English

The authors borrow the structure of ADMM, a distributed optimization method that splits a big problem into sub-problems and reconciles them. In SOLID, a "coordinator" plays that reconciling role between two agents:

• Each iteration, the coordinator sends both agents a current shared plan plus a price signal (a dual variable) and a penalty for drifting away from the shared plan.
• The optimization agent maximizes its utility, which now includes a payment term from its price and a quadratic penalty for deviating from the shared plan. In the case study this agent is a Markowitz mean-variance portfolio optimizer, minimizing variance subject to a target return and weights summing to one.
• The LLM agent does the same conceptually, but because LLMs handle semantics better than numbers, its decisions are discretized into seven confidence levels — "Very High," "High," "Somewhat High," "Neutral," "Somewhat Low," "Low," and "Very Low" — mapped to numbers from 0.6 down to 0 in 0.1 increments. It receives its own price signal and the current public decision in its prompt.
• The coordinator updates the prices based on how far each agent deviated, then computes a new shared decision, and the loop repeats until the iteration limit or small primal/dual residuals.

For the case study, the authors used Yahoo Finance for historical prices and Perplexity.ai to gather news. They selected 60 NASDAQ stocks across 10 industries and rebalanced month by month through all of 2024 (12 periods). The LLM agent used a four-step chain-of-thought prompt: read news and reflect on each stock, incorporate the shared plan and decision price to collaborate, state a confidence level, and then commit to a final allocation. Temperature was set to 0. The system prompt explicitly told the LLM that the optimizer had consistently outperformed by a significant margin (10–20%) and instructed it to adjust toward the optimizer's proposal unless it had strong data-backed reasons not to.

Compared strategies were pure optimization (OPT), pure LLM (LLM), the SOLID framework (LLM+OPT), a simple average of the two (AVG), and sparse variants (LLM_sparse, LLM_sparse+OPT, AVG_sparse) created by instructing the LLM to aim for sparsity and zero out low-confidence stocks. Primary metrics were total portfolio value and total variance.

Why This Matters

Impact on research: The paper offers a concrete bridge between prescriptive optimization and LLM reasoning, showing that the two can be coupled through an established distributed-optimization mechanism rather than one simply serving the other. It also contributes to the literature on prompt design by making economic signals (prices, penalties) part of the LLM's input, and it is transparent about where theoretical guarantees hold and where they break down.

Real-world applications:

• Portfolio management and asset allocation that combines quantitative risk models with qualitative market news and commentary.
• Supply chain and production planning, where structured cost/constraint models must be reconciled with unstructured supplier or logistics intelligence.
• Healthcare resource allocation, where numerical capacity models must account for narrative clinical information.
• Any enterprise decision workflow needing modular components that can be developed separately on separate datasets without sharing raw data.

Industry relevance: The modular design means each component can be developed independently on its own data, which preserves data privacy while allowing collaboration — attractive for organizations that cannot pool data. Removing the need for manual intervention between the optimizer and the LLM also points toward deployable automated decision pipelines.

Future Directions

• Extending convergence guarantees beyond the current convexity assumptions, since real-world problems involve discrete variables and non-convex objectives, and the paper notes that convergence cannot always be guaranteed when an agent behaves non-convexly.
• Further study of prompt design for the LLM agent, which the authors describe as a source of insight for effective prompting strategies.
• Testing SOLID in domains beyond finance, given the paper's claim that the framework is adaptable to a wide range of decision-making systems in critical domains.
• Determining how results scale to larger stock universes, longer horizons, and other LLM architectures, given that comparisons here cover three models and a 12-month, 60-stock setting.

Target Audience

Researchers and practitioners at the intersection of optimization and LLMs — particularly those in operations research, financial engineering, and applied AI — as well as anyone designing multi-agent decision systems that must combine numerical solvers with language-model reasoning. Readers without a background in convex optimization can still follow the case study and prompt design, but will get more from the theoretical sections with some familiarity with ADMM and dual decomposition.

Authors’ abstract

This paper introduces SOLID (Synergizing Optimization and Large Language Models for Intelligent Decision-Making), a novel framework that integrates mathematical optimization with the contextual capabilities of large language models (LLMs). SOLID facilitates iterative collaboration between optimization and LLMs agents through dual prices and deviation penalties. This interaction improves the quality of the decisions while maintaining modularity and data privacy. The framework retains theoretical convergence guarantees under convexity assumptions, providing insight into the design of LLMs prompt. To evaluate SOLID, we applied it to a stock portfolio investment case with historical prices and financial news as inputs. Empirical results demonstrate convergence under various scenarios and indicate improved annualized returns compared to a baseline optimizer-only method, validating the synergy of the two agents. SOLID offers a promising framework for advancing automated and intelligent decision-making across diverse domains.