Runtime Uncertainty Monitoring for LLM-Based Multi-Agent Systems Using Bayesian Networks

AI summary

Overview

Research area: Artificial Intelligence, LLM-based multi-agent systems, uncertainty quantification, actuarial risk modelling, Bayesian networks, and runtime safety monitoring. Technical level: Advanced. Scope: The paper proposes a runtime uncertainty-monitoring framework for LLM-based multi-agent systems in actuarial risk modelling, using calibrated token-level log-probabilities and a Bayesian Network to model uncertainty propagation across workflow stages.

What This Paper Is About

LLM-based multi-agent systems can automate actuarial workflows, but they are opaque, can hallucinate, vary across runs, and propagate errors from one agent to later stages. The paper’s goal is to provide structured runtime uncertainty propagation and safety monitoring for high-stakes insurance workflows, while reproducing baseline actuarial performance and giving additional insight into workflow stability.

Key Contributions

1. A centralised multi-agent framework with a Central Hub coordinating four specialised agents: Data Preparation, Modelling, Reviewing, and Explanation. The framework supports data preparation, modelling, review, explanation, and guardrail functions for actuarial workflows.
2. A novel uncertainty-propagation approach that uses token-level log-probabilities and a Bayesian Network. Log probabilities are not treated as direct probabilities of correctness or task success; instead, length-normalised log-probability summaries are transformed into calibrated task-level confidence estimates before being incorporated into the Bayesian Network.
3. A Bayesian Network representation of the actuarial workflow in which nodes represent successful task execution, directed edges capture dependencies, and conditional probability tables quantify node dependencies. The paper gives a conditional probability table example where WorkflowOK behaves as an “AND” gate and succeeds only if all parent nodes pass.
4. An evaluation across three open-source decoder-only LLM backends: Llama 2 7B, Llama 3.1 8B, and Qwen2.5 7B. The evaluation uses the original dataset (10 runs) and 20 systematically perturbed datasets, assessing agent behaviour, consistency, error detection, adaptability, safety, and uncertainty.

Main Findings

• Baseline actuarial performance is reproduced: Baseline model RMSE was 0.8853 for all backends. Mean RMSE was 0.8856 for Llama 2 7B, 0.8839 for Llama 3.1 8B, and 0.8671 for Qwen2.5 7B. Task Success Rate was 0.8000 for Llama 2 7B, 0.9000 for Llama 3.1 8B, and 0.8000 for Qwen2.5 7B. The paper reports at least 80% of runs completed successfully across all backends.

• Workflow uncertainty varies by backend: Mean workflow uncertainty was 0.7345 for Llama 2 7B, 0.5506 for Llama 3.1 8B, and 0.6012 for Qwen2.5 7B. Agent-level uncertainty scores were higher individually: Data Preparation Agent 0.9226, 0.8604, 0.8648; Reviewing Agent 0.9086, 0.8107, 0.8710; Modelling Agent 0.9097, 0.8292, 0.8410; Explanation Agent 0.8819, 0.8429, 0.8052; Overall Workflow 0.7345, 0.5506, 0.6012 for Llama 2 7B, Llama 3.1 8B, and Qwen2.5 7B respectively. The paper states that propagation through the workflow can significantly reduce the score even when individual agents have high uncertainty scores.

• Error detection differs substantially across backends: On perturbed datasets, Error Detection Rate was 0.4500 for Llama 2 7B, 0.6500 for Llama 3.1 8B, and 0.9000 for Qwen2.5 7B. Perturbed mean RMSE was 1.0655, 1.0728, and 1.0157, and perturbed mean workflow uncertainty was 0.7342, 0.5573, and 0.5833, for Llama 2 7B, Llama 3.1 8B, and Qwen2.5 7B respectively. Qwen2.5 7B achieved a 90% error detection rate, outperforming Llama 2 7B at 45% and Llama 3.1 8B at 65%.

• Temperature affects probability of successful workflow execution: Scenario A used low temperature (0.2) for all agents and P(WorkflowOK) was 87.1%. Scenario B used low temperature (0.2) for the Data Preparation and Modelling agents and medium temperature (0.7) for the Reviewing and Explanation agents, with P(WorkflowOK) at 69.0%. Scenario C used medium temperature (0.7) for all agents as the default, with P(WorkflowOK) at 59.2%. Scenario D used high temperature (1.2) for one agent, with P(WorkflowOK) at 42.8%.

• The Bayesian Network example shows workflow-level uncertainty propagation: In the presented example, most steps in the workflow show a certainty around 80%, but combining multiple agent outputs to finish the workflow results in a final certainty of 55%. This is significantly less than the individual agent certainties.

• Agent judgement and adaptability vary: Llama 2 tended to approve workflows more easily, while Llama 3.1 and Qwen more frequently included critical notes. Qwen2.5 7B showed a stronger tendency to adapt by retraining models or changing strategies. The Llama models often failed to detect severe issues and rarely adjusted their approach. The MAS was particularly sensitive to distributional changes (population shifts), while other perturbations such as missing values or sparsity were less reliably detected.

• Safety guardrails are included: The Explanation Agent acts as a guardrail by evaluating belief consistency, interpretability via Testing with Concept Activation Vectors (TCAV), and fairness through group-level comparisons between predicted and observed outcomes for sensitive attributes such as age and population density. The Bayesian Network is described as a runtime uncertainty-propagation monitor, not as proof of output correctness.

Methodology in Plain English

The system uses a centralised star topology. A Central Hub coordinates

Authors’ abstract

This paper investigates how multi-agent systems (MAS)-based on large language models (LLMs) can support actuarial risk modelling, with a particular focus on uncertainty quantification. Actuarial workflows represent a high-stakes decision-support setting where unreliable outputs may lead to incorrect risk assessment, unfair pricing, and regulatory non-compliance. To address uncertainty introduced by the probabilistic nature of LLMs and dependencies between agents, a multi-agent framework is proposed in which specialised agents perform data preparation, modelling, review, and explanation tasks under a central hub. The main contribution is a novel approach to uncertainty propagation using token-level log-probabilities and a Bayesian Network. Importantly, log probabilities are not treated as direct probabilities of correctness or task success. Instead, length-normalised log-probability summaries are transformed into calibrated task-level confidence estimates before incorporation into the Bayesian Network. Results show that the framework reproduces baseline actuarial performance while providing additional insight into workflow stability and runtime uncertainty propagation.