ReGAIN: Retrieval-Grounded AI Framework for Network Traffic Analysis

AI summary

Overview

Research area: Network security and machine learning — specifically retrieval-augmented generation (RAG) and large language models (LLMs) applied to network traffic analysis and intrusion detection.

Technical level: Intermediate. Readers will benefit from familiarity with LLM pipelines, vector databases, and standard network security concepts (PCAPs, 5-tuples, SYN and ICMP floods), though the paper explains its architecture concretely.

Scope: The paper presents ReGAIN, a multi-stage LLM framework that converts network traffic into natural-language summaries, indexes them in a multi-collection vector database, and retrieves cited evidence to ground attack/no-attack verdicts — evaluated on ICMP ping flood and TCP SYN flood traces from MAWILab.

What This Paper Is About

Traditional traffic analysis tools either rely on hand-written rules that produce many false alarms with little explanation, or on machine learning models that detect accurately but behave as opaque black boxes analysts cannot interrogate. ReGAIN's goal is to close that gap by grounding an LLM's reasoning in retrieved, citable traffic evidence, so that every verdict comes with the records and heuristics that support it — and the system abstains rather than guessing when evidence is insufficient.

Key Contributions

1. The ReGAIN framework, which unifies structured traffic representation, semantic embedding, vector retrieval, and LLM reasoning into a single pipeline for network traffic analysis.
2. A retrieval design with quality controls: multi-collection retrieval across three specialized vector databases, adaptive context selection through automated metadata filtering and Maximal Marginal Relevance (MMR) diversity sampling, a two-stage bi-encoder-then-cross-encoder reranking mechanism, and a pre-generation abstention mechanism that returns "undecidable" to mitigate hallucinations.
3. A dual-mode evaluation that validates results against two independent sources — dataset ground truth and blind human expert adjudication — on real-world traffic containing ICMP ping flood and TCP SYN flood attacks.
4. A benchmark comparison against five baselines (Snort-style rule-based heuristics, SVM with RBF kernels, Random Forest with 100 decision trees, 1D CNN, and two-layer LSTM), showing ReGAIN achieves higher performance while adding citation-backed explainability.

Main Findings

• SYN flood, ground truth: ReGAIN recorded 4,075 true positives, 609 true negatives, 0 false positives, and 56 false negatives, yielding 98.82% accuracy, 100.00% precision, 98.64% recall, 99.32% F1, and AUC 0.99.
• SYN flood, expert labels: Under blind expert review the numbers shifted to 3,960 true positives, 587 true negatives, 114 false positives, and 78 false negatives — 95.95% accuracy, 97.20% precision, 98.07% recall, 97.63% F1, AUC 0.98. The paper attributes the new errors to edge cases such as partial SYN floods or rate-limited attempts and to some automated labels being overly permissive.
• Ping flood, ground truth: Perfect recall (100.00%) on all 356 true attack instances with zero false negatives, but precision of 74.48% because 122 benign ICMP cases were misclassified — overall accuracy 97.56%, F1 85.37%, AUC 0.99.
• Ping flood, expert labels: Recall remained 100.00% over 365 expert-validated attacks (nine more true positives than the automated labels captured), while false positives dropped from 122 to 113, raising precision to 76.36%, accuracy to 97.74%, and F1 to 86.60%, with AUC holding at 0.99.
• Overclassification of diagnostic ICMP: Most ping flood false positives came from short-lived or diagnostic ICMP bursts — reachability checks, latency probes, automated monitoring — that briefly resemble flood patterns, a distinction MAWILab annotations do not always make.
• Outperformance of baselines (SYN): Against the strongest baseline (LSTM), ReGAIN improved accuracy by 3.7 percentage points, precision by 14.5 points, and recall by 3.8 points.
• Outperformance of baselines (Ping): ReGAIN reached 100.0% recall versus 95.6% for LSTM, achieving zero false negatives while trading away precision (74.5%) — described as a deliberate design choice favoring detection sensitivity.
• Qualitative advantages: ReGAIN produces human-readable natural-language explanations with citations and supports conversational, iterative query refinement, which static classifiers cannot do.

Methodology in Plain English

The pipeline has four stages. First, traffic records are normalized into a fixed schema — timestamp, source and destination IP, port, protocol, and anomaly label — and a deterministic function converts each record into a short natural-language sentence, such as a description of a host sending an ICMP request flagged as a possible DoS anomaly. This summarization keeps LLM context windows from filling with raw log noise.

Second, each summary is embedded into a 384-dimensional vector and stored alongside structured metadata derived from the 5-tuple, labels, and timestamps. Entries live in three separate ChromaDB collections — telemetry, anomaly, and heuristic — so the system can pull context selectively or in parallel.

Third, when an analyst asks a question, the query is embedded into the same space and compared using cosine similarity. Named entity and IP extraction builds a metadata filter that narrows the search, a similarity threshold of τ = 0.3 discards weak candidates, MMR selects a diverse subset, and a bi-encoder followed by a cross-encoder reranks the survivors. If too few high-confidence items pass, the system abstains and returns "undecidable" with the missing evidence listed. Otherwise the top results go to GPT-4.1-nano at temperature 0, which returns a verdict (attack, no-attack, or undecidable), a justification citing retrieved record IDs or heuristic codes, and one or two mitigations, displayed through a command-line interface.

Evaluation used MAWILab v1.1 PCAPs from January 1, 9, and 10, 2022, focusing on ICMP ping floods and TCP SYN floods. Ground truth for SYN floods came from Zeek TCP connection states (S0 meaning SYN sent with no reply, versus SH, SF, RSTR, RSTO, and OTH for normal or benign traffic); ping floods were flagged by a sliding-window detector marking ten or more ICMP echo requests (type 8) from one source to one destination within 20 seconds, cross-validated against MAWILab heuristic-20 anomalies. A subset of logs was then blindly adjudicated by human experts using attack signatures, connection rates, IP diversity, and timing; logs with 5–9 requests were flagged for further review and those with fewer than five were deemed benign. The final evaluation covered 10,000 labeled instances.

Why This Matters

Impact on research: The paper argues that most prior LLM-for-networking work targets either detection accuracy or interpretability in isolation. ReGAIN demonstrates a hybrid evaluation strategy — automated ground truth plus human expert labels — and shows that the two disagree in informative ways, which is a methodological contribution beyond the specific system.

Real-world applications:

• Retrospective forensic investigations, where analysts reconstruct attack campaigns and validate alerts against historical PCAPs and flow records.
• Security operations center triage, where analysts need an explanation and supporting evidence behind an alert rather than an opaque score.
• Incident response and threat hunting, using the conversational interface to iteratively refine queries — for example, moving from "Show anomalies involving 203.0.113.5" to comparing TCP SYN activity in the same interval.
• Environments requiring auditable decision trails, since every verdict cites retrieved record IDs or heuristic codes.

Industry relevance: The paper positions ReGAIN as a decision-support tool that transforms traffic analysis from passive alerting into an interactive analytical tool. It also acknowledges a practical constraint: reliance on GPT-4-class remote APIs and lightweight 384-dimensional embeddings makes inference latency API-bound, suiting retrospective analysis rather than real-time monitoring.

Future Directions

• Reduce latency and improve privacy by moving to on-premise models such as LLaMA-3-8B, Mistral-7B, or Phi-3 with local embedding models, enabling fully offline operation in air-gapped or sensitive environments.
• Close the ping flood precision gap (roughly 74–76%) using dynamic similarity thresholds that vary by protocol type and traffic volume.
• Add temporal and rate-based filters to distinguish short benign ICMP bursts from sustained flooding.
• Evaluate the generated natural-language output itself, assessing whether summaries and explanations are not only accurate but genuinely clear and interpretable to analysts.

Target Audience

Network security researchers and practitioners — especially those working on intrusion detection, traffic forensics, and SOC tooling — will get the most from this paper, along with applied machine learning researchers interested in RAG pipelines, retrieval quality controls, and hallucination mitigation. Graduate students entering the intersection of LLMs and network security will find the architecture description and baseline comparison a useful reference point. The paper is less suited to readers seeking production-ready real-time deployment guidance, since the authors explicitly scope it to retrospective analysis.

Code, data, and model artifacts are available at https://github.com/270771/llm-traffic. The work was accepted to the 2026 IEEE International Conference on Computing, Networking and Communications (ICNC 2026) and was partially supported by NSF grants 2113945, 2200538, 2416992, and 2230610.

Authors’ abstract

Modern networks generate vast, heterogeneous traffic that must be continuously analyzed for security and performance. Traditional network traffic analysis systems, whether rule-based or machine learning-driven, often suffer from high false positives and lack interpretability, limiting analyst trust. In this paper, we present ReGAIN, a multi-stage framework that combines traffic summarization, retrieval-augmented generation (RAG), and Large Language Model (LLM) reasoning for transparent and accurate network traffic analysis. ReGAIN creates natural-language summaries from network traffic, embeds them into a multi-collection vector database, and utilizes a hierarchical retrieval pipeline to ground LLM responses with evidence citations. The pipeline features metadata-based filtering, MMR sampling, a two-stage cross-encoder reranking mechanism, and an abstention mechanism to reduce hallucinations and ensure grounded reasoning. Evaluated on ICMP ping flood and TCP SYN flood traces from the real-world traffic dataset, it demonstrates robust performance, achieving accuracy between 95.95% and 98.82% across different attack types and evaluation benchmarks. These results are validated against two complementary sources: dataset ground truth and human expert assessments. ReGAIN also outperforms rule-based, classical ML, and deep learning baselines while providing unique explainability through trustworthy, verifiable responses.