Signed but Steerable: Measuring and Closing the Text-Vector Integrity Gap in Offline Retrieval-Augmented Generation

AI summary

Overview

Research area: Security of retrieval-augmented generation (cs.CR), specifically the integrity of signed "knowledge packs" and vector databases used for fully offline RAG on edge devices.

Technical level: Advanced. The paper combines a formal geometric argument (an "angular margin" law), empirical security measurement across five corpora and two embedding models, numerical-reproducibility measurements across three hardware platforms, and on-device latency and answer-quality measurements.

Scope (one sentence): The paper measures how many queries an attacker who controls the embedding vectors shipped inside a validly signed knowledge pack can hijack under tolerance-based verification, quantifies the same exposure for compressed vector stores such as FAISS product quantisation, and proposes and evaluates a code-level verification scheme that removes it.

What This Paper Is About

Offline RAG systems on edge devices are updated with signed packages that ship text chunks together with precomputed embedding vectors (recomputing them on a small board is slow: about 29 minutes for a 950-passage pack on a Raspberry Pi 5 CPU in the authors' earlier study). A signature proves who published the pack, not that the vectors actually encode the text next to them, so a compromised publisher or insider can ship review-passing text with vectors chosen to steer retrieval. The paper quantifies exactly how much a verifier that recomputes vectors and accepts them within a cosine tolerance concedes to such an attacker, shows the same weakness in compressed vector stores, and closes the gap with verification of quantisation codes plus verifier-side canonicalisation.

Key Contributions

1. The margin law. For an optimal attacker, the hijack rate under a cosine tolerance τ equals the empirical distribution of angular retrieval margins evaluated at arccos τ. The law matches attack simulation exactly (maximum absolute difference 0.0 on the 300-question set) and lets an operator measure exposure from honest vectors and a query log without running attacks.
2. Measured exposure of compressed vector stores. Tolerance-based audits against FP16, SQ8, SQ4, product quantisation and binary storage leave 0–5.2%, 2.0–14.5%, 55–89%, 99.7–100% and 100% of queries hijackable respectively; an executed attack on a real FAISS product-quantisation index over 57,638 documents passes the tolerance check for all 60 tampered vectors and puts the attacker's document first for 25% of targets.
3. Exact code-level verification. A verifier that recomputes the embedding, compares quantisation codes rather than vectors, decides ambiguity itself with a calibrated guard band, and rewrites accepted entries with its own codes (canonicalisation) accepted every honest vector in the samples on three platforms and four compute backends, without special hardware.
4. On-device cost, end-to-end impact, and pre-registered negative results. Verification on the device CPU costs the concurrently running language model 23–31% of decoding speed (22.9–31.4% in the periodic regime), the attack lowers correct answers from 0.75 to 0.39, layered pack verification rejects all twelve structural update attacks tested, and the paper reports pre-registered negative results on abstention, context pruning and semantic screening along with code, data and a 34-entry deviation log.

Main Findings

• Tolerance is an attack budget. On SQuAD 2.0 with 3,648 BGE-M3 passage queries and 3,301 MiniLM 128-token-window queries, a tolerance of τ = 0.999 (2.56°) leaves 24.6% and 16.8% of queries hijackable; τ = 0.99 leaves 74.9% and 54.7%; τ = 0.95 leaves 99.6% and 91.9%.
• Margins are small everywhere. Median margins are 5.2° (BGE-M3) and 7.4° (MiniLM) on SQuAD; 5% of queries have margins below 0.6° and 0.8°. Across corpora the medians are 5.1–7.2° on SciFact, 3.5–6.3° on NFCorpus, 3.3–4.7° on FiQA and 2.1–3.2° on ArguAna, where 5% of queries are decided by less than 0.02° for BGE-M3.
• Even float-noise tolerances leak. At the FP32 x86-versus-ARM noise level (τ = 0.99999976, 0.04°), 0.49% (BGE-M3) and 0.24% (MiniLM) of SQuAD queries are exposed; across the five corpora 0–5.5% are exposed. Excluding exact duplicate documents, exposure falls to at most 1.2% (5/420 on ArguAna with BGE-M3), with 0/141 and 1/136 on NFCorpus, 18/3,646 (0.49%) and 7/3,148 (0.22%) on SQuAD, 2/269 (0.74%) and 0/225 on FiQA, and 0 on SciFact.
• A real INT8 tolerance is catastrophic. Calibrating at the empirical 0.1% quantile of cosines between dynamic-INT8 and FP32 outputs of all-MiniLM-L6-v2 in ONNX Runtime on the Raspberry Pi CPU over 1,745 windows gives τ = 0.904 (25.31°). Two of the 1,745 honest windows fall below it, the lowest at 0.898, and it leaves 100% (BGE-M3) and 98.4% (MiniLM) of queries hijackable, or 81.2% and 44.0% for a transfer attacker who does not know the exact query wording.
• Compression makes tolerance audits useless. Half-precision storage has the lowest measured exposure (0.03° maximum honest reconstruction angle, 0–5.2% hijackable); product quantisation (23–40°) and binary sign codes (41–47°) leave 99.7–100% and 100% exposed. On FAISS over 57,638 FiQA documents, the tolerance-based audit exposes 8.3% of queries for IndexScalarQuantizer (SQ8) and 100% for IndexPQ and IndexBinaryFlat, against 0.0% for IndexFlatIP.
• The executed store attack is an existence result. For 60 target queries, tampered product-quantisation codes pass the verifier's tolerance check in 100% of cases and place the attacker's carrier document at rank 1 under the index's own scoring for 25% of targets (95% CI 16–37%); code-level verification with canonicalisation blocks all 60.
• Common pipelines are not bit-reproducible. PyTorch FP32 changes with batch size on GPU and CPU (maximum absolute coordinate difference about 10⁻⁷; FP16 on GPU 10⁻⁴); PyTorch on an Apple M2 CPU and Metal GPU differs by 1.4·10⁻⁷ and 4.5·10⁻⁸; ONNX Runtime FP32 is batch- and padding-invariant on one x86 machine but differs between x86 and ARM by 1.9·10⁻⁷; dynamic INT8 changes by up to 0.03 depending on the other inputs in the batch.
• Coarse cells make exact code checks portable. Exact equality alone already accepts 98.99–99.03% of MiniLM SQ8 vectors (91.3–91.5% for BGE-M3); the smallest tested guard width that accepted every honest vector was 3·10⁻⁷ for MiniLM and 3·10⁻⁶ for BGE-M3 with SQ8, and at most 3·10⁻⁷, or none, for binary and product-quantisation codes, identical on the desktop and the Apple M2.
• Canonicalisation removes the publisher's choice. At the calibrated widths, a compromised publisher choosing among admissible ambiguous codes hijacks no MiniLM query (SQ8, SQ4 or PQ) and at most 0.16% for BGE-M3 SQ8; a needlessly wide band of 10⁻⁵ reaches 0.41%, against 2–100% for tolerance-based audits. These guard widths are empirical calibrations, not guarantees for other data.
• Verification competes with generation on a Pi 5. Re-embedding the 1,745-window corpus takes 72.9 s in FP32 and 41.6 s in dynamic INT8 on four CPU cores. Against a reference of 5.14 tokens/s and 210 ms p99 inter-token latency, periodic verification at 8 pairs/s costs 22.9–31.4% of decoding speed and raises p99 latency by 108–167%; saturating re-indexing at 15–25 items/s costs 64.5–73.9%.
• End-to-end harm on the device. Without attack the local Qwen2.5-3B model answers 0.75 correctly and refuses 0.13; with an FP32 verifier at τ = 0.999 it answers 0.74 and refuses 0.13, but at the INT8 reference threshold correct answers fall to 0.39 and refusals rise to 0.50 (wrong answers 0.11). The instruction-bearing payload reached the top-3 context for 100% of questions without a vector check, 2% at τ = 0.999 and 94% at the INT8 threshold.
• Pack-level results and negative results. Of 19 update cases (12 structural attacks, three legitimate updates, three content-level attacks, one vector desynchronisation), 17 of 19 decisions matched the expected outcome, and layered verification rejects all twelve structural attacks. Pre-registered experiments on abstention, context pruning and semantic screening produced negative results.

Methodology in Plain English

The authors start from a geometric observation. For each query, look at the angle between the query and the correct (gold) passage, and the angle to the nearest competing passage; the difference is the "margin". If a verifier accepts any shipped vector that is within some angle α of the honest one, an attacker can rotate the nearest competitor by α, so any query whose margin is smaller than α can be stolen. Counting how many queries have margins below α gives the expected hijack rate without running a single attack. They check this prediction against simulated attacks and then measure the margin distribution on SQuAD 2.0 and four BEIR corpora (SciFact, NFCorpus, ArguAna, FiQA) with two embedding models, BGE-M3 and all-MiniLM-L6-v2.

They then repeat the analysis for compressed vector stores, where any audit must tolerate the storage format's own reconstruction error, and they carry out a concrete attack on a FAISS product-quantisation index over 57,638 FiQA documents, editing stored codes until the decoded vector passes the verifier's tolerance check.

For the defence, they measure how much the same text's embedding drifts between hardware platforms, batch sizes and precision modes, then test a verifier that recomputes the embedding, quantises it with the store's codebook, and compares codes rather than vectors. Where a tiny guard band marks an entry as ambiguous, the verifier accepts either the exact code or its neighbour but then overwrites the stored value with its own code, so the publisher gains nothing from the ambiguity. Finally they measure on a Raspberry Pi 5 what this verification costs the concurrently running Qwen2.5-3B-Instruct model and how an end-to-end attack changes answer accuracy and refusal rates. Hypotheses, metrics and decision gates were written and dated before measurement; the authors call this pre-registered but note the protocol was not deposited in a public registry, and all deviations are logged.

Why This Matters

Impact on research. The paper reframes a question that supply-chain signing, model-signing specifications and embedding-provenance schemes such as VectorPin leave open: signatures and per-embedding signatures bind bytes to keys, but not a vector to the text it is supposed to encode, and they fail when the signer is the adversary. The margin law turns "how much tolerance is safe" into a measurable, corpus- and model-specific quantity, and transfers the tolerance-versus-exactness debate from verifiable inference to retrieval.

Real-world applications:

• Disconnected field, industrial and defence deployments that receive knowledge through update packages and cannot re-embed large corpora on-device.
• Any organisation operating a vector database whose stored vectors are audited by recomputation, where the audit must tolerate product quantisation or binary codes.
• Insider-risk scenarios where a holder of a legitimate content-signing key ships review-passing text with manipulated vectors.
• Defence against indirect prompt injection, since a strategically placed chunk can carry instructions into the top-k context.

Industry relevance. The measurements give concrete numbers for operators of FAISS-style stores and for edge-AI vendors: SQ8 leaves 2.0–14.5% of queries exposed, an INT8-calibrated tolerance leaves 98–100%, and code-level verification costs the co-running language model 23–31% of decoding speed, which argues for scheduling verification when the model is idle or moving it to a separate processor.

Future Directions

• Extending the measurements to accelerators and index families. The paper explicitly leaves dedicated neural processing units, IVF and graph indexes, and text-and-vector attackers to future work; how far vendor-compiled quantised graphs deviate, and whether they are bit-reproducible, is unmeasured.
• Turning guard widths into guarantees. The reported widths (3·10⁻⁷ for MiniLM and 3·10⁻⁶ for BGE-M3 with SQ8, and so on) are empirical calibrations on the authors' samples and a grid of tested values; the paper states that a deployment should measure the deviation between its own publisher and verifier platforms, which invites a principled calibration procedure.
• Making the exact path cheap enough to run alongside generation. Verifying on the device CPU competes with decoding, so moving verification to a separate processor or designing integer-only, bit-identical embedding paths are open engineering directions.
• Replacing the negative results. The pre-registered experiments on abstention, context pruning and semantic screening did not work as hoped, leaving open what complementary defences could catch a vector-level manipulation that content review cannot see.

Target Audience

Security researchers and practitioners working on RAG and vector databases; engineers building offline or edge AI systems that distribute signed knowledge packs; teams responsible for auditing vector-store integrity; and readers interested in numerical reproducibility of embedding and quantised inference pipelines. The paper assumes familiarity with cosine similarity, quantisation schemes such as SQ8 and product quantisation, and floating-point numerics, so it is best suited to an advanced audience, though the margin-law argument itself is accessible to anyone comfortable with vectors and angles.

Authors’ abstract

Offline retrieval-augmented generation (RAG) on edge devices is updated through signed knowledge packs that carry text and precomputed embedding vectors. A signature proves who published a pack, not that its vectors encode its text. We study this text-vector integrity gap. A verifier that recomputes vectors and accepts them within a cosine tolerance concedes a predictable share of queries to an attacker who controls the vectors: the hijack rate equals the distribution of angular retrieval margins at the tolerance (the margin law). Margins are small in five corpora and two embedding models, so an idealised tolerance at the level of floating-point noise leaves up to 5.5% of queries exposed (at most 1.2% in any corpus once exact duplicate documents are excluded), and a threshold calibrated from our CPU INT8-FP32 comparison of a small embedding model (at the empirical 0.1% quantile of honest cosines) leaves 98-100% exposed; on a Raspberry Pi 5 running a local language model, such an attack lowers correct answers from 0.75 to 0.39. Integrity audits of compressed vector stores fare no better: on a FAISS product-quantisation index over 57,638 documents, tampered vectors that all pass the audit put the attacker's document first for 25% of targets. Verifying quantisation codes instead, with ambiguity decided by the verifier, a calibrated guard band and canonicalisation, accepted every honest vector in our samples on three platforms and left the publisher no choice. We also show that common floating-point and dynamic-INT8 inference pipelines are not bit-reproducible across batch sizes or architectures, and report pre-registered negative results on abstention, context pruning and semantic screening. We release code and data.