CausalTrace: A Neurosymbolic Causal Analysis Agent for Smart Manufacturing

AI summary

Overview

• Research area: Neurosymbolic artificial intelligence, causal inference, and agentic AI for smart manufacturing.
• Technical level: Intermediate. The paper combines causal discovery algorithms, knowledge graphs and ontologies, and LLM-based agents; readers need some grounding in machine learning and causal reasoning concepts, but the system architecture is described in largely plain terms.
• Scope: The paper presents CausalTrace, a neurosymbolic causal analysis agent embedded in the SmartPilot industrial CoPilot, and evaluates it on an academic rocket assembly testbed using causal assessment methods and the C3AN framework.

What This Paper Is About

Manufacturing AI systems often work as isolated black boxes that predict anomalies but cannot explain why they occurred, what caused them, or what would happen under a different intervention. CausalTrace aims to close that gap by combining data-driven causal discovery with structured industrial knowledge (ontologies and knowledge graphs) so that operators receive transparent, explainable decision support. The authors build the module into the existing SmartPilot multi-agent system and test it on real rocket assembly data.

Key Contributions

1. Design and implementation of CausalTrace, a neurosymbolic causal analysis agent performing real-time causal discovery, root cause analysis (RCA), causal effect estimation, and counterfactual reasoning.
2. Integration of data-driven causal methods with structured knowledge (ontologies, knowledge graphs), semantic user interfaces, and evaluation pipelines into a neurosymbolic agent workflow, enabling human-interpretable reasoning and a deployable decision-support system.
3. A comprehensive evaluation methodology grounded in the C3AN framework (Custom, Compact, Composite AI with Neurosymbolic Integration), covering robustness, intelligence, and trustworthiness, including counterfactual effect estimation, RCA validation, and comparisons against baseline and ablation variants.
4. Demonstration of deployment and practical utility on an academic rocket assembly testbed, highlighting readiness for real-world manufacturing environments.

Main Findings

• Root cause analysis accuracy: CausalTrace achieved MAP@3 of 94%, PR@2 of 97%, MRR of 0.92, and a Jaccard index of 0.92, indicating close agreement with expert-defined causes.
• Ontology-grounded explanations: Full CausalTrace scored ROUGE-1 of 0.91 in ontology QA, compared to 0.56 for the variant without knowledge graph and ontology grounding.
• Correlation baseline is weak: A correlation-based root cause ranking produced high false positives due to spurious associations and failed to capture causal directionality (Jaccard 0.33, MAP@3 44%, PR@2 51%, MRR 0.50).
• C3AN evaluation: CausalTrace attained 4.59/5 in the C3AN evaluation, demonstrating precision and reliability for live deployment. Averaged across a subset of 10 C3AN principles, GPT-4o-mini scored 4.19, LLaMA3-70B-8192 scored 4.32, and human evaluators scored 4.59.
• Evaluator agreement patterns: Inter-model agreement between GPT-4o-mini and LLaMA3-70B-8192 was reported as κ = 0.56 in the text (Table 3 lists 0.61 for LLM1–LLM2). Agreement with humans was lower (κ = 0.52 for GPT-4o-mini and κ = 0.58 for LLaMA3-70B-8192), while human evaluators showed moderate internal agreement (κ = 0.58). Both LLMs and humans aligned more closely with their own group.
• Strongest and weakest principle scores: Explainability received the maximum score of 5 from both LLMs and humans. Consistency scored 3.5 from GPT-4o-mini, and Abstraction and Interpretability each scored 3.3 from LLaMA3-70B-8192.
• Causal graph structure: Causal discovery was run on eight key variables identified through XGBoost feature selection and validated by domain experts. ICA-based LiNGAM produced 20 directed causal edges; DiffAN yielded 15.

Methodology in Plain English

The researchers added a causal analysis agent to SmartPilot, an existing multi-agent industrial CoPilot whose other agents are PredictX (anomaly prediction using multimodal sensor data), ForeSight (throughput forecasting with a Knowledge-Infused LSTM), and InfoGuide (conversational question answering). SmartPilot's natural language understanding and explanation are powered by LLaMA3-70B-8192.

CausalTrace ingests data either directly from programmable logic controllers for real-time streaming or through batch upload of historical sensor data, then lets users pick variables automatically or manually. Its Causal Discovery Engine builds directed acyclic graphs from multivariate sensor data using two methods, ICA-based LiNGAM and DiffAN. To make the graphs more trustworthy, the authors added bootstrap-based edge stability analysis: causal graphs are re-estimated on bootstrap-resampled datasets, edge stability scores are computed from how frequently edges appear, and low-confidence edges are down-weighted or removed. Edges are retained only if their stability score is at least 0.6; scores at or above 0.9 are treated as very strong, 0.8 to 0.9 as reliable, and 0.6 to 0.8 as moderately stable and to be used with caution. For each retained edge, the total causal effect is computed from the structural equation model using the matrix (I − B)⁻¹, which aggregates direct one-hop effects and indirect multi-hop effects.

For root cause analysis, the module combines expert-defined, cycle-aware sensor tolerance ranges with the learned causal topology to generate a ranked list of candidate root causes based on causal path effect strengths and sensor deviation analysis. Counterfactual validation lets a user select an edge, apply an intervention on one variable, and compare the predicted change in the other against observed changes in held-out data; small error supports the causal effect, while large error suggests misspecification or unobserved confounding.

Knowledge is injected in three ways: a smart manufacturing knowledge graph in RDF representing sensors, machines, parts and anomalies (accessed via rdflib), a dynamic process ontology implemented in Neo4j queried with Cypher for explanations, tolerance ranges and sensor-function mappings, and a causal graph prompt injection mechanism that serializes the total causal effect matrix into LLM prompts. The interface visualizes causal graphs with semantic metadata and lets users add or remove nodes and edges, with edits validated against the ontology. A memory module stores episodic (time-stamped interactions), semantic (structured annotations), and procedural (user preferences) memory in JSON format and injects these into InfoGuide responses.

Evaluation used the publicly available Future Factories Lab dataset from the McNair Aerospace Research Center at the University of South Carolina: 166K records sampled at 1.95 Hz over 30 hours, covering 285 complete rocket assembly–disassembly cycles, each segmented into 21 distinct operational states. The annotated version includes cycle state labels and ground-truth anomaly types indicating missing rocket components, including six types such as NoNose, NoBody2, and combined cases like NoBody2,NoBody1.

For the C3AN assessment, the framework defines 14 principles; a subset of 10 was selected based on operational relevance. Ten targeted questions per principle were crafted, with ground truth from domain experts and official manufacturing manuals. An LLM-as-a-Judge pipeline using GPT-4o-mini and LLaMA3-70B-8192 scored answers from 1 to 5 and explained each score. Six human evaluators—three manufacturing experts and three computer scientists—also scored responses on the same 1 to 5 rubric.

Why This Matters

Research impact: The paper demonstrates a concrete way to fuse causal discovery, symbolic knowledge structures, and LLM-based agents into one deployable pipeline, and it proposes C3AN-based evaluation with both LLM judges and human experts for measuring robustness, intelligence, and trustworthiness. The reported gap between LLM-judge scores and human scores, and between some individual evaluators, is itself evidence relevant to ongoing debates about automated evaluation.

Real-world applications:

• Shop-floor root cause diagnosis: ranking likely causes of an anomalous sensor reading so operators can act quickly.
• Proactive maintenance: using causal topology and sensor deviation analysis to anticipate failures.
• Counterfactual "what if" planning: estimating how intervening on one process variable would change another before making a change.
• Conversational operator assistance: natural language questions about causal graph structure, causal strength, and why a variable is or is not a likely root cause.

Industry relevance: The work targets high-stakes, safety-critical manufacturing where black-box predictions are insufficient for adoption. Its phased deployment path—first virtual validation through the Testbed as a Service framework with MQTT data replay, then real-world integration using an OPC UA Server, camera feeds, and Neo4j, RDF and HTTP SSE for knowledge and causal graphs—shows an explicit route from testbed to live production. The acknowledged constraints are relevant to practitioners: the centralized architecture has scalability limits, and direct camera connections impose spatial constraints due to cabling range. A second deployment iteration plans to replace the OPC UA Server and direct camera connections with a public MQTT broker, with SmartPilot publicly hosted, accepting added latency that the authors intend to address with edge computing and asynchronous communication.

Future Directions

1. Extending the agent to intervention planning beyond analysis, so it can recommend and evaluate actions rather than only diagnosing causes.
2. Continual causal graph learning, so discovered structures keep updating as processes and data drift rather than being learned once.
3. Safety-aware, instruction-following features for the agent in operational settings.
4. Architectural scaling: transitioning from the current centralized deployment to a decentralized processing model, and mitigating the latency introduced by the public MQTT broker approach using edge computing and asynchronous communication.

Target Audience

This paper is most valuable to researchers and practitioners working on neurosymbolic AI, causal inference, explainable AI, and agentic systems for industrial settings. It is also directly useful to manufacturing engineers, automation specialists, and smart-factory architects evaluating whether causal and knowledge-grounded AI can be deployed on live production lines, as well as to teams designing evaluation protocols that combine LLM-as-a-Judge scoring with human expert review.

Authors’ abstract

Modern manufacturing environments demand not only accurate predictions but also interpretable insights to process anomalies, root causes, and potential interventions. Existing AI systems often function as isolated black boxes, lacking the seamless integration of prediction, explanation, and causal reasoning required for a unified decision-support solution. This fragmentation limits their trustworthiness and practical utility in high-stakes industrial environments. In this work, we present CausalTrace, a neurosymbolic causal analysis module integrated into the SmartPilot industrial CoPilot. CausalTrace performs data-driven causal analysis enriched by industrial ontologies and knowledge graphs, including advanced functions such as causal discovery, counterfactual reasoning, and root cause analysis (RCA). It supports real-time operator interaction and is designed to complement existing agents by offering transparent, explainable decision support. We conducted a comprehensive evaluation of CausalTrace using multiple causal assessment methods and the C3AN framework (i.e. Custom, Compact, Composite AI with Neurosymbolic Integration), which spans principles of robustness, intelligence, and trustworthiness. In an academic rocket assembly testbed, CausalTrace achieved substantial agreement with domain experts (ROUGE-1: 0.91 in ontology QA) and strong RCA performance (MAP@3: 94%, PR@2: 97%, MRR: 0.92, Jaccard: 0.92). It also attained 4.59/5 in the C3AN evaluation, demonstrating precision and reliability for live deployment.