Towards general embodied intelligence: integrating large language models, knowledge bases, and reasoning capabilities to build the next generation of AI agents

AI summary

Overview

Research area: Artificial intelligence — specifically the intersection of large language models (LLMs), structured knowledge bases (KBs), reasoning ability (RA), and embodied AI, framed around a target called "general embodied intelligence" (GEI).

Technical level: Intermediate to Advanced. The paper assumes familiarity with Transformer architectures, pre-training/fine-tuning pipelines, knowledge graphs, and robotic perception-control loops, though it explains each from a historical starting point.

Scope: A survey and roadmap paper that reviews the technical foundations of LLM-centered embodied agents, proposes a unified framework connecting language, knowledge, reasoning, and physical embodiment, and names five key challenges for reaching general embodied intelligence.

What This Paper Is About

LLMs such as GPT-4, PaLM, Gemini, and Claude are strong general-purpose language learners, but they operate purely in the symbolic domain and "remain fundamentally disembodied," lacking grounded understanding of the physical or perceptual world. Meanwhile, embodied AI systems can navigate, manipulate objects, and act in real or simulated environments, but rely on rigid task-specific pipelines with limited high-level reasoning and semantic comprehension.

This paper surveys how to combine the two — using LLMs as "neural executors" inside embodied control loops — and adds structured knowledge bases and reasoning mechanisms into the picture, proposing a unified architecture and a roadmap toward general embodied intelligence. It is a review, not an experimental study, and the visible text reports no quantitative benchmark results.

Key Contributions

1. A comparative positioning against prior reviews. Table I compares five earlier surveys — Yao et al. (2025), Dong et al. (2025), Tan et al. (2025), Jeong et al. (2024), and Wang et al. (2025) — summarizing their core contributions and limitations (e.g., weak Sim2Real transfer, costly VLN training, hallucination and bias, prompt bias, closed-source APIs).

2. A unified GEI technical architecture. The authors state they build, "for the first time," a unified architecture from the perspective of coordinating language models, knowledge bases, reasoning mechanisms, and physical interaction systems, supporting perception, reasoning, and action. The framework is illustrated to show synergy among LLMs, KBs, RA, and embodiment.

3. Five key research directions. The abstract names them as efficient LLM deployment, closed-loop knowledge integration, hybrid symbolic-neural reasoning, perception-action grounding, and continual learning; the introduction and Section 4 framing names them as lightweight deployment, knowledge closure, reasoning enhancement, perception alignment, and safety and control. The paper presents these as a systematic technical roadmap.

4. A systematic technical grounding of the four components. Section 2 reviews LLMs, KBs, RA, and embodied intelligence from the perspectives of technical architecture, development history, and integration methods.

Main Findings

• LLM capability boundaries: LLMs hold billions to hundreds of billions of parameters and show four notable properties as listed — large scale, context-awareness, multimodal capabilities, and emergent capabilities. GPT-3 is cited as having more than 175 billion parameters.

• Historical trajectory of language modeling: From 1990s statistical methods, to Bengio's 2003 "A Neural Probabilistic Language Model," to CNNs and RNNs, LSTM (which alleviated gradient vanishing and long-term dependency issues but limited parallelization), to the 2017 Transformer with self-attention, to pre-trained models like BERT (masked language modeling, bidirectional) and GPT (autoregressive).

• Model taxonomy (Figure 2, 2018–2024): Encoder-only models (BERT, ALBERT, RoBERTa, DistillBERT) suit natural language understanding tasks; encoder-decoder models (BART, T5, mT5) suit natural language generation tasks; decoder-only models (GPT series, LLaMA, BLOOM, ChatGLM) perform strongly across a broad range of NLP tasks. The figure shows a trend toward decoder-only architectures dominating recent state-of-the-art models, with open-source (BERT, BLOOM, LLaMA) and closed-source (GPT-3, GPT-4, InstructGPT, ERNIE 3.0) development coexisting.

• Multimodal architecture: A unified multimodal LLM uses modality-specific encoders (Image, Video, Speech Encoder), aligned into the language model's latent space through adapter modules — I-Adapter with Q-Former for image features, V-Adapter with Q-Former for video, S-Adapter with C-Former for speech.

• Five-phase LLM construction pipeline: data collection and preprocessing; model architecture design; pre-training; fine-tuning and alignment; evaluation and iteration. Pre-training uses self-supervised objectives such as masked language modeling (MLM), causal language modeling (CLM), or span prediction, often across hundreds or thousands of GPUs or TPUs over weeks.

• Evaluation benchmarks named: GLUE and SuperGLUE for language understanding, HELM and MMLU for generation, BIG-Bench for reasoning, and BLEU, ROUGE, and Exact Match for task-specific metrics. The paper supplies no scores on any of these.

• Knowledge base composition: The KB is described as a dynamic system of multiple interrelated modules, not a monolithic component, spanning knowledge extraction, fusion, representation, reasoning, retrieval, and access to common knowledge bases. Sources include unstructured text (books, articles, social media) and structured knowledge such as knowledge graphs, domain databases, and tables. Pre-training corpora include Common Crawl, BooksCorpus, and Wikipedia.

• Selected LLM research mapped in Table II: Cummins et al. (Meta Large Language Model Compiler, compiler optimization), Hu et al. (context-aware prompting for VQA), Hu et al. (LLM-TIKG threat intelligence knowledge graphs), Qian et al. (Alibaba's HPN network for large-scale LLM training), Sun et al. (UniCoder with a universal code intermediate representation), Zhang et al. (BB-GeoGPT for Geographic Information Science), Wang et al. (VideoAgent for long-form video), Xia et al. (LLMGA for image generation and editing with precise control), Zhou et al. (weak-to-strong search for compute-efficient alignment), and Land et al. (glitch token detection for tokenizer robustness).

• Persistent LLM limitations identified: high computational resource consumption in training and inference, ethical issues from data bias and harmful or inaccurate generated content, and limited interpretability of the reasoning process.

• Embodied-side limitations: modern embodied agents perform real-time navigation, visual servoing, object manipulation, and task execution, but lack high-level cognition and struggle with open-ended, multi-step scenarios. The symbol grounding problem, hallucination, limited memory, and the absence of explicit world models are named as core obstacles.

• Stated limitations of this survey itself: limited empirical examples, and insufficient discussion of multilingual settings and real-world deployment.

Methodology in Plain English

This is a literature review and framework-proposal paper rather than an experimental one. The authors first survey the building blocks, each from a historical and architectural angle: how language models evolved from statistical n-grams to Transformers; how knowledge bases are built from unstructured text and structured graphs; how reasoning mechanisms aim to supply logical consistency and long-range inference; and how embodied systems close the loop between perception and control.

They then synthesize these strands into a proposed unified architecture in which the LLM sits at the center, drawing on external knowledge sources and structured reasoning, and driving perception and action in physical or simulated environments. To situate their work, they build a comparison table of five prior reviews with their contributions and limitations, then finish by naming five challenge areas as a forward research agenda. No datasets were collected, no models were trained, and no experiments were run for this paper.

Why This Matters

Impact on research. The paper argues that although LLMs, embodied intelligence, and neuro-symbolic reasoning have each been actively explored, there is a gap in systematizing their synergistic integration. It claims the literature lacks a unified perspective on how these fields can be combined into autonomous, language-driven, knowledge-augmented embodied agents capable of multimodal interaction, long-term planning, and adaptive reasoning. Its contribution is organizational: a shared architecture and a named set of open problems for the field to coordinate around.

Real-world applications named or implied in the text:

• Assistive and collaborative robots in human-machine symbiotic manufacturing, where LLMs improve interaction, collaboration, and execution.
• Customized manufacturing, where multi-source data and LLM reasoning are fused for semantic integration and decision-making.
• Agriculture and medical care, cited as future application areas for LLM/VLM-based planning and reasoning.
• Knowledge-intensive service pipelines, exemplified by the paper's document processing and QA workflow that ingests Word, Excel, PDF, JSON, and email files, indexes them, and answers user queries.
• Domain-specific expert systems, such as medical diagnosis, legal analysis, and geographic information science, where structured knowledge bases supply factual grounding.

Industry relevance. The challenges the paper highlights map directly onto deployment constraints: inference cost and hardware limits for embedding models on robots, integration of proprietary knowledge graphs with neural models, reliability and hallucination in safety-critical control, and the difficulty of transferring from simulation to the real world. The named infrastructure work (Alibaba's HPN for large-scale training) and compiler-domain LLM work (Meta's LLM Compiler) show commercial-scale interest. The comparative table flags closed-source APIs and fixed action sets as practical limitations for robotics adoption.

Future Directions

1. Efficient LLM deployment (lightweight deployment). Reducing compute, memory, and latency so models can run under the resource constraints of embedded and robotic hardware. The paper notes practical deployment requires compression, quantization, and latency control.

2. Closed-loop knowledge integration (knowledge closure). Resolving mismatches in representation, update mechanisms, and interfacing protocols between large neural architectures and external structured sources such as knowledge graphs, ontologies, and commonsense databases.

3. Hybrid symbolic-neural reasoning (reasoning enhancement). Combining explicit logical and structured reasoning with neural inference to supply logical consistency and long-range inference for agents, addressing LLMs' lack of causal reasoning.

4. Perception-action grounding (perception alignment). Solving the symbol grounding problem — translating abstract language instructions into concrete sensorimotor actions — and improving Sim2Real generalization and multimodal sensor fusion.

5. Continual learning and safety/control. The abstract lists continual learning as the fifth challenge, while Section 4 frames the fifth direction as security control, implying both persistent adaptation over time and safe, controllable behavior. A related open question the paper raises about itself is the under-explored multilingual setting and real-world deployment evidence.

Target Audience

This survey is most useful for graduate students and researchers entering the intersection of LLMs and robotics who need a single orienting document covering architectures, history, and open problems. It also suits robotics and embodied-AI engineers evaluating where LLMs, knowledge graphs, and reasoning modules might fit into a control stack, and technical strategists in manufacturing, healthcare, agriculture, and document-intelligence industries assessing feasibility and known bottlenecks.

Readers seeking quantitative comparisons, benchmark leaderboards, or reproducible experimental protocols will not find them here: the visible content reports no datasets, no measured results, and no performance numbers, and the authors themselves list "limited empirical examples" as a limitation. Note also that the manuscript text provided is truncated partway through Section II-C, so the detailed treatments of reasoning ability, embodied intelligence, the fusion architecture, and the challenge section are only partially visible; the corresponding claims above rest on the abstract, introduction, table of contents, and Section 2 framing.

Authors’ abstract

The convergence of large language models (LLMs), structured knowledge bases (KBs), and reasoning ability (RA) presents a promising trajectory toward general embodied intelligence (GEI). This paper reviews the evolution of LLM-centered intelligent systems, emphasising their integration with knowledge representation, logical reasoning, and physical embodiment. We analyse LLM architectures, pre-training methods, and inference mechanisms, along with their interaction with external knowledge sources and structured reasoning frameworks. Furthermore, we examine embodied intelligence (EI) paradigms wherein agents learn and act in physical environments. To synthesise these dimensions, we present a conceptual framework that illustrates the synergy among LLMs, KBs, RA, and embodiment, serving as a guiding model for perception, reasoning, and action rather than an implemented engineering architecture. To advance toward GEI, we identify five key challenges: efficient LLM deployment, closed-loop knowledge integration, hybrid symbolic-neural reasoning, perception-action grounding, and continual learning. This survey provides a comprehensive roadmap for developing adaptive, multimodal agents capable of operating in complex, dynamic settings.