Concept-RuleNet: Grounded Multi-Agent Neurosymbolic Reasoning in Vision Language Models

AI summary

Overview

Research area: Neuro-symbolic AI and interpretable computer vision, sitting at the intersection of vision-language models (VLMs), large language model (LLM) agents, and first-order logic reasoning.

Technical level: Intermediate. The paper assumes familiarity with VLMs, LLM prompting, and basic symbolic logic (conjunction, disjunction, first-order rules), but the core idea is explained through worked examples.

Scope: The paper proposes a three-agent system that mines visual concepts from training images to ground the symbols and logical rules used in neuro-symbolic reasoning, and evaluates it on five benchmarks across four vision-language models.

What This Paper Is About

Modern VLMs are accurate but opaque — they often hallucinate and give little insight into why a prediction was made, especially on data outside their pre-training distribution. Existing neuro-symbolic approaches generate interpretable logical rules, but they derive their symbols only from the task label, so the symbols are weakly grounded in the actual images and can be hallucinated or unrepresentative. Concept-RuleNet's goal is to restore visual grounding by conditioning symbol and rule generation on concepts mined directly from real training images, while keeping the reasoning pathway transparent.

Key Contributions

1. A three-agent neuro-symbolic framework (Concept-RuleNet) in which a Visual Concept Extraction agent, a Symbol Exploration and Neurosymbolic agent, and a Verifier agent collaborate to produce grounded, representative symbols and logical rules.

2. Empirical benchmarking across 5 datasets and 4 models — BloodMNIST, DermaMNIST, UC-Merced Satellite, WHU, and iNaturalist — tested on InstructBLIP-XXL, LLaVA-1.5, LLaVA-1.6, and (for medical data) LLaVA-Med.

3. Evidence that conditioning on visual concepts yields better grounding and representativeness, with reduced hallucination of symbols in generated rules.

4. An extension, Concept-RuleNet++, which adds counterfactual symbols and forms rules in a mixed normal form (combining disjunctive and conjunctive structures) for higher prediction performance.

Main Findings

• Average accuracy gain of about 5% over the strongest neuro-symbolic baseline. The paper reports that Concept-RuleNet augments state-of-the-art neuro-symbolic baselines by an average of 5% across the five benchmarks.

• Hallucinated symbols in rules reduced by up to 50%. Grounding symbol generation on real images cuts the appearance of symbols that never occur in the data.

• Statistical significance against Symbol-LLM. Across the evaluation, Concept-RuleNet vs. Symbol-LLM showed a mean difference of +4.99 percentage points, SD 2.95, t(4) = 3.79, p = 0.019 (two-tailed), 95% CI [1.3, 8.7], Cohen's d = 1.69, marked significant. (The paper's comparison table is truncated after this row in the available content.)

• Largest single improvement: 9.33% on UCMerced-Satellite with InstructBLIP-XXL (System-1 alone: 41.33; Symbol-LLM: 48.0; Concept-RuleNet: 57.33).

• Consistent gains on medical imaging. On BloodMNIST, Concept-RuleNet reached 18.09 (InstructBLIP), 14.57 (LLaVA-1.5), and 19.35 (LLaVA-1.6), versus System-1-only scores of 11.55, 11.55, and 10.05. On DermaMNIST it reached 8.54, 47.73, and 48.74 respectively.

• A domain-specific verifier helps. Swapping in LLaVA-Med as the verifier raised DermaMNIST accuracy to 7.86 (InstructBLIP), 34.21 (LLaVA-1.5), 66.33 (LLaVA-1.6), and 12.56 (LLaVA-Med).

• LLaVA-Med is a good verifier but a weak reasoner. Using LLaVA-Med as both the System-1 model and the verifier produced poor results (BloodMNIST 12.06, DermaMNIST 12.56), which the authors read as evidence that such models lack deep understanding of images but can still score symbol presence.

• Concept-RuleNet++ adds another 1–2% on average. With InstructBLIP, scores moved from 18.09 to 21.43 (BloodMNIST), 8.54 to 14.23 (DermaMNIST), 57.33 to 58.12 (Satellite), 20.40 to 21.52 (WHU), and 53.21 to 54.15 (iNaturalist). The caption states the improvement holds except on iNaturalist, attributed to extreme diversity in the training samples, while the table lists a 53.21 to 54.15 increase there.

• Ablation: visual context helps at every stage. On UC-Merced with InstructBLIP, accuracy was 48.00 with no context (the Symbol-LLM setting), 49.50 after initialization, 55.10 after exploration, and 57.33 after entailment.

• Better grounding, measured quantitatively. Symbols from Concept-RuleNet were more likely to be present in both train and test images. On Satellite and WHU, Symbol-LLM's symbol occurrence rate fell below 0.5.

• Better representativeness. When symbols were formatted as the question "How likely are {symbol1, symbol2,..} in predicting {class} for a {task}?", Concept-RuleNet's symbols averaged 0.54 likelihood versus 0.49 for Symbol-LLM.

• λ matters and is dataset-dependent. Too high or too low λ degraded performance (44.00 at λ=0.3, 48.00 at λ=0.7, 38.66 at λ=0.9 for Symbol-LLM; 49.33, 57.33, 48.00 for Concept-RuleNet). Using 90 images instead of 50 slightly reduced Concept-RuleNet's score (57.33 to 56.28), attributed to overfitting on obscure concepts.

Methodology in Plain English

The system runs in three sequential stages, each handled by a distinct agent:

1. Grounded visual concept extraction. A VLM agent (LLaVA-Med for medical data, LLaVA-1.6 otherwise) looks at a representative sample of training images for each class label and writes out short, human-readable visual descriptions — a "bag of visual attributes" rather than complex relationships. Duplicates are discarded, producing a set of observed concepts per label.

2. Conditional symbol exploration and rule formation. A strong LLM agent (GPT-4o-mini) starts from a set of initial symbols derived from the task label, then iteratively explores additional symbols using the mined visual concepts as context. Candidate rules are written as conjunctions of symbols implying a class. The LLM also scores entailment — how plausible each rule is given the visual concepts — and only rules above a threshold are kept. Rule length is capped to avoid overfitting.

3. Verification and prediction. At inference time, a vision verifier agent estimates the probability that each symbol in a rule is present in the test image, framed as a binary yes/no question and using the "yes" token probability as confidence. A whole rule's score is the minimum across its symbols (conjunction), and the best rule per class is selected. The final prediction is a weighted sum of the black-box System-1 score and the neuro-symbolic System-2 score, controlled by λ.

Concept-RuleNet++ extends this by adding counterfactual symbols — symbols from other classes verified inversely (probability of not being present) — forming rules that mix disjunction and conjunction. The authors also note the theoretical intuition that conditioning symbol entropy on the image reduces uncertainty: H(S|x) < H(S).

Why This Matters

Impact on research. The paper pins down a specific weakness in prior neuro-symbolic work — conditioning symbol discovery on the label alone — and shows empirically that this leads to hallucinated and unrepresentative symbols on out-of-distribution data, partly due to benchmark dataset leakage during LLM pre-training. Grounding symbol generation in images offers a general recipe that other neuro-symbolic pipelines can adopt.

Real-world applications:

• Medical imaging triage, where clinicians need to know which visual features drove a classification, not just the class label.
• Remote sensing and land-use monitoring, where satellite classes are underrepresented in web-scale VLM pre-training.
• Biodiversity and species identification, represented here by the iNaturalist-21 benchmark.
• Any regulated or high-stakes deployment where hallucinated reasoning steps are unacceptable and predictions must come with an auditable rationale.

Industry relevance. The approach improves zero-shot performance without fine-tuning, which the authors argue removes dependence on large-scale labeled data curation and expensive training runs. It also lets organizations layer an interpretable, verifiable reasoning stage on top of an off-the-shelf VLM rather than replacing it.

Future Directions

• Stronger domain-specific verifiers. The authors explicitly call for designing more powerful medical VLMs as verifiers, since LLaVA-Med as a verifier improved results while LLaVA-Med as a System-1 model did not.
• Handling extreme intra-class diversity. The stated exception for iNaturalist — where counterfactual symbols remain highly likely because training samples are so diverse — points to a need for better counterfactual selection in diverse domains.
• Better balancing of λ and image sampling. Since both too much and too little System-2 weight hurt, and too many extraction images cause overfitting, adaptive or learned policies for these hyperparameters are an open problem.
• Extending grounding measures. The paper's grounding and representativeness metrics use VLM likelihoods and an LLM-as-judge question; more rigorous or human-validated grounding measures would strengthen the claims.

Target Audience

Researchers and practitioners in neuro-symbolic AI, interpretable machine learning, and multimodal reasoning who already understand VLMs and LLM prompting — particularly those working on out-of-distribution or underrepresented domains such as medical imaging and remote sensing. It is also relevant to applied engineers evaluating whether to layer a transparent reasoning stage on top of an off-the-shelf VLM without fine-tuning.

Authors’ abstract

Modern vision-language models (VLMs) deliver impressive predictive accuracy yet offer little insight into 'why' a decision is reached, frequently hallucinating facts, particularly when encountering out-of-distribution data. Neurosymbolic frameworks address this by pairing black-box perception with interpretable symbolic reasoning, but current methods extract their symbols solely from task labels, leaving them weakly grounded in the underlying visual data. In this paper, we introduce a multi-agent system - Concept-RuleNet that reinstates visual grounding while retaining transparent reasoning. Specifically, a multimodal concept generator first mines discriminative visual concepts directly from a representative subset of training images. Next, these visual concepts are utilized to condition symbol discovery, anchoring the generations in real image statistics and mitigating label bias. Subsequently, symbols are composed into executable first-order rules by a large language model reasoner agent - yielding interpretable neurosymbolic rules. Finally, during inference, a vision verifier agent quantifies the degree of presence of each symbol and triggers rule execution in tandem with outputs of black-box neural models, predictions with explicit reasoning pathways. Experiments on five benchmarks, including two challenging medical-imaging tasks and three underrepresented natural-image datasets, show that our system augments state-of-the-art neurosymbolic baselines by an average of 5% while also reducing the occurrence of hallucinated symbols in rules by up to 50%.