Intelligent Human-Machine Partnership for Manufacturing: Enhancing Warehouse Planning through Simulation-Driven Knowledge Graphs and LLM Collaboration

AI summary

Overview

Research area: Applied AI for manufacturing and logistics — specifically the combination of Discrete Event Simulation (DES), Knowledge Graphs (KGs), and Large Language Model (LLM) agents for warehouse planning.

Technical level: Advanced. The paper assumes familiarity with knowledge graphs, graph query languages (Cypher), simulation modeling, retrieval-augmented and iterative reasoning agent architectures, and the pass@k evaluation metric.

Scope: The paper presents and evaluates a two-stage framework that converts discrete-event warehouse simulation output into a knowledge graph and then lets a reasoning-capable LLM agent answer both factual operational queries and open-ended bottleneck investigations through natural language.

What This Paper Is About

Warehouse and manufacturing planners rely on Discrete Event Simulation to model how suppliers, workers, automated vehicles, forklifts, and storage interact, but the resulting output data is voluminous and highly granular, so extracting actionable insight traditionally requires manual inspection of aggregate statistics or custom scripts. This creates a barrier between the human decision-maker and the operational data. The paper's goal is to remove that barrier by converting simulation logs into a semantically rich Knowledge Graph and giving planners a natural-language LLM agent that can retrieve facts and iteratively diagnose bottlenecks while keeping the human in the loop.

Key Contributions

1. A simulation-to-knowledge-graph pipeline. A specialized warehouse ontology and a five-step automated construction pipeline that parses DES event logs, extracts and validates timestamps, creates resource nodes with properties, establishes timestamped package-flow relationships, validates data integrity, and optimizes graph structure with indexing for efficient Cypher querying.

2. A dual-path query architecture. A query classification module routes incoming natural-language questions either to a four-module QA Chain (Step Generation, Cypher Generation, Query Execution and Self-Reflection, Answer Synthesis) for operational queries, or to an Iterative Reasoning Chain (Reasoning, Evidence Collection and Integration, Self-Reflection and Validation, Sufficiency Assessment) for investigative queries.

3. A human-machine collaborative diagnostic workflow. The LLM agent decomposes a diagnostic problem into sub-questions informed by accumulated evidence and by human expert input, generating its own question sequence while the human expert supplies iterative investigative questions and retains decision authority.

4. Quantitative evaluation on operational and investigative tasks. Benchmarking against two baselines across 25 operational questions and three deliberately perturbed investigation scenarios, plus a multi-dimensional human evaluation of more than 12 investigation analysis logs.

Main Findings

• Near-perfect operational QA accuracy. The framework achieved an average Pass@1 of 0.92 and a perfect average Pass@2 of 1.00 on operational questions, compared with 0.59 Pass@1 / 0.71 Pass@2 for the single-pass baseline and 0.70 Pass@1 / 0.82 Pass@2 for the baseline with self-reflection added.

• Per-category robustness gains. Under the framework, Pass@1/Pass@2 scores were 1.00/1.00 for Supplier, 0.75/1.00 for Worker, 0.92/1.00 for AGV, 1.00/1.00 for Forklift, and 0.95/1.00 for Package. The baseline scored as low as 0.33/0.33 on the Worker category, which the self-reflection baseline only improved to 0.42/0.50.

• Monolithic queries are brittle. The paper reports that even when the baseline was given two independent chances to generate a monolithic query, it failed to produce a single correct answer 22% of the time (0.71 Pass@2), whereas the framework's second attempt succeeded in the rare 8% of cases where its first attempt failed.

• Bottleneck diagnosis in Scenario 1 (CamelCargo discharge delay). The agent found that CamelCargo's total discharge time of 6,870 seconds exceeded the global average of 4,955 seconds, a 1.39x ratio described as a 39% performance degradation. Supplier waiting time was ruled out (22.0 seconds versus a 21.6-second global average).

• Downstream imbalance detected through utilization. AGV utilization for CamelCargo was 54.8% against a 41.7% global average, while forklift utilization was 23.7% against a 40.1% global average, indicating an imbalance downstream of the real problem.

• Root cause localized to the worker stage. Stage-by-stage package analysis isolated the "Wait to Worker" phase as the bottleneck, with one package waiting 3,654 seconds versus a 1,375-second global average, while waiting times for AGV, forklift, and storage remained at or below average.

• Strong human expert ratings on investigative quality. Over 12 investigation analysis logs drawn from three scenarios, expert scores on a 10-point scale ranged from 7.96 to 9.0: Necessity 9.0, Completeness 8.58, Factuality 8.46, Logical Coherence 8.21, Clarity 8.04, Sufficiency 8.0, and Query Quality 7.96.

• Simulation scale used for evaluation. The DES model included twelve employees organized into teams of four, twenty AGVs at 3.5 km/hr averaging 140 meters per transport operation, five forklifts at 5 km/hr with stochastic storage operations of 60-90 seconds, suppliers moving at 20 km/hr carrying 30-35 packages each, a maximum of three simultaneous unloading operations, and storage of five blocks with fifteen bays of three shelves each, giving a capacity of 225 packages.

Methodology in Plain English

The researchers first built an in-house Discrete Event Simulation of a warehouse that unloads, internally transports, and stores incoming packages, tracking suppliers, workers, AGVs, forklifts, and packages with detailed timestamps, utilization metrics, and waiting times.

They then wrote a warehouse-specific ontology and an automated pipeline that reads the simulation's event logs and turns them into a knowledge graph, where resources become nodes and package movements become timestamped relationships. This graph is stored in Neo4J, a graph database queried with the Cypher language.

On top of the graph they placed an LLM agent implemented with OpenAI's GPT-5 through the LangGraph framework, using temperature settings between 0.0 and 0.3 so that diagnostic work stays accurate. When a question arrives, a classifier decides whether it is operational (a factual lookup) or investigative (a diagnosis). Operational questions are broken into individual steps, each step is converted to its own Cypher query, errors are caught and corrected through self-reflection, and the results are synthesized into an answer. Investigative questions trigger a longer loop: the agent writes a sub-question, queries the graph for evidence, reflects on whether the evidence holds up, and keeps generating follow-up sub-questions until it judges the evidence sufficient, then produces a summary. Human experts participate by supplying their own iterative questions — for example, comparing one supplier against the global average, checking equipment utilization for imbalance, examining package-level waiting times per stage, and asking whether any stage deviates enough to explain the delay.

Evaluation used 25 operational questions drawn roughly uniformly across entities and stages, scored with pass@k, and three investigative scenarios where a specific inefficiency was deliberately injected: a stage transfer delay for one supplier, degraded performance of one forklift, and a supplier-specific processing delay. Investigation quality was additionally scored by human experts along seven dimensions.

Why This Matters

Impact on research. The paper targets a gap it explicitly identifies: knowledge graphs are increasingly used in industrial and supply chain settings, and KG-LLM integration is a mature pattern, but applying KG-LLM systems to analyze DES output data remained largely unexplored. It also frames the problem as a human-machine partnership question rather than a purely technical one, and shows that decomposing queries into guided, self-reflected steps substantially outperforms single-pass query generation — a finding relevant to anyone building graph-querying agents.

Real-world applications.

• Warehouse and distribution center planners querying simulation results in natural language during daily and tactical planning cycles without writing Cypher or custom scripts.
• Root cause analysis when a supplier, worker team, or piece of equipment underperforms, isolating whether the cause is upstream waiting, equipment imbalance, or task allocation.
• Digital twin enrichment, where the KG-backed agent makes an existing simulation model more interpretable and diagnostic rather than merely descriptive.
• Democratizing access to simulation analytics for operations analysts, industrial engineers, and manufacturing planners who lack specialized query expertise.

Industry relevance. The setting is Industry 4.0 and smart manufacturing, where DES is already a core component of digital twins. The framework positions the LLM agent as an intelligent partner that translates human intent into precise analytical operations while the human retains oversight, validation, and decision authority — directly addressing the concern that AI systems either replace expertise or remain locked behind technical silos.

Future Directions

• Integrating advanced reasoning LLM architectures to improve the agent's diagnostic depth and efficiency.
• Extending the framework beyond unloading to a wider array of warehouse operations such as slotting, picking, loading, and inventory management.
• Combining simulated and on-field data, and developing rigorous benchmarking methodologies to formally quantify performance as the scope expands.
• Reducing the upfront burden of knowledge graph schema design through standardized schema templates and automated generation tools for common warehouse configurations.
• Establishing continuous monitoring procedures and confidence scoring mechanisms for critical diagnostic conclusions, since Cypher query generation and synthesized explanations still require ongoing validation for novel operational scenarios.
• Demonstrating generalizability beyond the warehouse unloading context and across different simulation frameworks.

Target Audience

This paper benefits manufacturing and logistics researchers working on digital twins and simulation analytics; AI researchers building knowledge-graph-grounded LLM agents and iterative reasoning pipelines; warehouse and supply chain practitioners evaluating natural-language decision support; and industrial engineers who need to diagnose bottlenecks from simulation output without specialized query expertise. Prerequisites for full value include basic familiarity with graph databases and Cypher, DES concepts, and modern LLM agent design patterns.

Authors’ abstract

Manufacturing planners face complex operational challenges that require seamless collaboration between human expertise and intelligent systems to achieve optimal performance in modern production environments. Traditional approaches to analyzing simulation-based manufacturing data often create barriers between human decision-makers and critical operational insights, limiting effective partnership in manufacturing planning. Our framework establishes a collaborative intelligence system integrating Knowledge Graphs and Large Language Model-based agents to bridge this gap, empowering manufacturing professionals through natural language interfaces for complex operational analysis. The system transforms simulation data into semantically rich representations, enabling planners to interact naturally with operational insights without specialized expertise. A collaborative LLM agent works alongside human decision-makers, employing iterative reasoning that mirrors human analytical thinking while generating precise queries for knowledge extraction and providing transparent validation. This partnership approach to manufacturing bottleneck identification, validated through operational scenarios, demonstrates enhanced performance while maintaining human oversight and decision authority. For operational inquiries, the system achieves near-perfect accuracy through natural language interaction. For investigative scenarios requiring collaborative analysis, we demonstrate the framework's effectiveness in supporting human experts to uncover interconnected operational issues that enhance understanding and decision-making. This work advances collaborative manufacturing by creating intuitive methods for actionable insights, reducing cognitive load while amplifying human analytical capabilities in evolving manufacturing ecosystems.