Valori: A Deterministic Memory Substrate for AI Systems

AI summary

Overview

• Research area: Systems for AI — specifically deterministic memory and vector storage, sitting at the intersection of AI infrastructure, numerical computing, and reproducible systems engineering.
• Technical level: Intermediate. The paper is readable without deep systems expertise, but benefits from familiarity with embeddings, floating-point arithmetic, approximate nearest neighbor search, and basic state-machine notation.
• Scope: The paper argues that AI memory should be a deterministic, replayable state machine built on fixed-point arithmetic rather than floating-point, and supports that argument with cross-platform bit-level divergence experiments, a snapshot-transfer test, and a recall comparison.

What This Paper Is About

AI systems store and search vector embeddings using floating-point (IEEE 754) arithmetic, which behaves differently across hardware and compilers — so the same model, input, and code can produce different memory states on x86 versus ARM machines. This non-determinism is introduced at embedding generation, before indexing or retrieval even happens, and it silently breaks replayability, audit trails, and safe deployment. The paper introduces Valori, a deterministic memory substrate that replaces floating-point memory operations with Q16.16 fixed-point arithmetic and models memory as a replayable state machine, aiming to guarantee bit-identical memory states, snapshots, and search results across platforms.

Key Contributions

1. A formal framing of deterministic memory. The paper defines memory as a state machine with transition function S_{t+1} = F(S_t, C_t) and states the determinism requirement formally: for any initial state S_0 and command sequence C_1...C_N, the final state must be identical regardless of environment (CPU architecture, OS, compiler version).
2. Empirical evidence that non-determinism enters before indexing. A controlled experiment on two machines encoding the same five texts with the same model shows bit-level differences in every dimension examined, establishing that the "forking path" begins at embedding generation.
3. A Q16.16 fixed-point memory kernel. Valori is implemented as a no_std Rust kernel using 32-bit signed integers with a 16-bit fractional part, integer accumulation during dot products, deterministic command serialization, and exact snapshot/restore.
4. Precision as a configurable "memory contract." Rather than fixing one numeric representation, the paper proposes selectable precision layers (Q16.16 today; Q32.32, Q64.64/Q128 as future options) with the claim that determinism is preserved independently of the precision choice because integer operations remain hardware-agnostic.

Main Findings

• Bit-level divergence in every inspected dimension. Using sentence-transformers v2.6.1 and the sentence-transformers/all-MiniLM-L6-v2 model on a Windows x86_64 PC versus a MacBook Pro ARM64 (Apple Silicon), the first five dimensions of vector 0 differed in raw hex: dimension 0 was 0xbd8276f8 (x86) versus 0xbd8276fc (ARM); dimension 1 was 0x3d6bb481 versus 0x3d6bb470; dimension 2 was 0x3d1dcdf1 versus 0x3d1dcdf9; dimension 3 was 0xbd601d21 versus 0xbd601d16; dimension 4 was 0x3b761ffb versus 0x3b762229.
• Cosine similarity masks the problem. Despite identical inputs, the cosine similarity between the divergent vectors remained greater than 0.9999, while the bit-level difference still breaks state replicability guarantees and can alter distance rankings and the retrieved k-NN set.
• Cross-platform snapshots are bit-identical. In a snapshot-transfer test, the kernel was initialized on an x86 machine with 10,000 inserted vectors, the state was hashed to H_A, transferred to an ARM machine, loaded, and hashed again to H_B. The result was H_A ≡ H_B, and k-NN result ordering remained identical after restore across platforms.
• Latency stays low. In local benchmarks on a MacBook Pro M3, raw retrieval latency was less than 500 μs for typical k-NN queries, which the paper describes as sufficient for real-time agentic applications.
• Recall barely moves under fixed-point quantization. Two indices with identical parameters, insertion order, and dataset partition were built — one standard f32 and one Valori Q16.16 — and compared on Top-10 nearest neighbors. Float32 HNSW achieved Recall@10 of 1.000 and Valori Q16.16 HNSW achieved 0.998, a mean Recall@10 of 99.8%.
• Determinism is achieved at the memory boundary, not at inference. Valori explicitly does not make neural inference deterministic; it normalizes non-deterministic model outputs into a deterministic memory state once they cross the kernel boundary.
• Stochastic index construction is made deterministic. HNSW-style index building is adapted by processing batch inserts in verified sorted order (usually by ID), fixing entry points to the first inserted node (ID 0), and using fixed-point distance metrics so graph topology is identical across runs.

Methodology in Plain English

The work proceeds in three stages. First, the authors isolate the problem empirically: they run the exact same Python script encoding a fixed list of five sentences on an x86 Windows machine and an ARM MacBook Pro using the same sentence-transformers model, save the embeddings to disk, then inspect the raw hexadecimal bits of the stored floats rather than their printed decimal values, because printing rounds away the differences that matter.

Second, they build an alternative memory kernel. Instead of floats, everything inside the kernel is a 32-bit signed integer in Q16.16 format — the lower 16 bits hold the fractional part, giving a range of [-32768, 32767] and a resolution of roughly 0.000015. Floating-point values are converted to fixed-point immediately at the kernel boundary. Dot products accumulate in i64 (or wider) intermediates to avoid overflow before narrowing back down. The kernel is written in no_std Rust so it can run on bare metal, WASM, or a standard OS, and is wrapped by a separate "Node" layer (Axum/Tokio) that handles HTTP, persistence, and networking without altering kernel logic. Because all operations are integer ALU instructions, results are meant to be bit-identical on x86, ARM, RISC-V, and WASM.

Third, they validate. They transfer a snapshot containing 10,000 vectors from an x86 machine to an ARM machine and compare hashes and k-NN ordering; they benchmark retrieval latency on a MacBook Pro M3; and they measure semantic fidelity by building a floating-point HNSW index and a Q16.16 HNSW index with identical parameters, insertion order, and dataset partition, then comparing Top-10 neighbor overlap (Recall@10).

Why This Matters

Impact on research. The paper reframes floating-point non-determinism from a negligible numerical nuisance into a correctness failure mode for AI memory, and proposes determinism as a primitive that should be enforced at the memory layer rather than left to the model. It explicitly positions itself against throughput-optimized vector databases (FAISS, Milvus, Qdrant) that accept floating-point non-determinism as a speed tradeoff, and it imports reproducibility standards from scientific computing and game simulation into high-dimensional vector search. It also deliberately scopes the claim: fixed-point determinism is offered as an existence proof for semantic fidelity in one widely-used configuration, not a universal performance theorem.

Real-world applications.

• Robotics and drones: a system trained in simulation can load the exact same memory kernel onto embedded hardware without behavior shift.
• Regulatory compliance: financial and medical AI agents can be audited by replaying their entire command log to verify why a decision was reached.
• Decentralized AI: nodes in a distributed network can verify they hold the same "truth" by comparing memory state hashes.
• Consensus systems: blockchains and decentralized AI networks, where all nodes must converge to an identical state after processing the same inputs — a requirement floating-point memory violates and a deterministic state machine satisfies by construction.

Industry relevance. For regulated sectors such as finance, healthcare, defense, and robotics, bit-level reproducibility is presented as a correctness requirement rather than an optional feature, and the absence of it is described as silently compromising audit trails. The precision-as-contract design gives architects a way to trade precision for performance without giving up replayability, which is directly relevant to anyone deploying agentic or RAG systems across heterogeneous fleets of x86 servers and ARM edge devices.

Future Directions

• Higher-precision contracts. Extending the kernel beyond Q16.16 to Q32.32 (framed for enterprise AI agents needing higher dynamic range and auditability) and Q64.64/Q128 (framed for scientific computing and defense systems needing long-horizon numerical stability).
• Faster deterministic arithmetic. Exploring the use of SIMD integer instructions, where it can be done safely, to close the performance gap between software fixed-point arithmetic and hardware-accelerated floating-point operations.
• Quantization error at the extremes. Understanding and mitigating quantization error for vector components that are extremely small or extremely large, since Q16.16 has limited dynamic range compared to f32 — the paper argues this is not an issue for normalized embeddings typically in [-1, 1].
• Deterministic inference. The non-determinism of the embedding model itself remains explicitly an open problem for the broader field; deterministic inference or model-side quantization are named as possible directions, though described as orthogonal to the memory guarantees Valori provides.

Target Audience

This paper is most useful to systems and infrastructure engineers building AI memory, vector databases, or agentic/RAG platforms; to architects in regulated industries (finance, healthcare, defense) who need auditable, replayable AI pipelines; to distributed-systems and blockchain engineers who need nodes to converge on identical state; and to robotics and edge-AI developers deploying the same model across x86 servers and ARM hardware. Researchers working on numerical reproducibility, approximate nearest neighbor search, or trustworthy AI will also find the problem framing useful, as will readers who want a concise, empirically grounded argument for treating determinism as a memory-layer property rather than a model-layer one.

Authors’ abstract

Modern AI systems rely on vector embeddings stored and searched using floating-point arithmetic. While effective for approximate similarity search, this design introduces fundamental non-determinism: identical models, inputs, and code can produce different memory states and retrieval results across hardware architectures (e.g., x86 vs. ARM). This prevents replayability and safe deployment, leading to silent data divergence that prevents post-hoc verification and compromises audit trails in regulated sectors. We present Valori, a deterministic AI memory substrate that replaces floating-point memory operations with fixed-point arithmetic (Q16.16) and models memory as a replayable state machine. Valori guarantees bit-identical memory states, snapshots, and search results across platforms. We demonstrate that non-determinism arises before indexing or retrieval and show how Valori enforces determinism at the memory boundary. Our results suggest that deterministic memory is a necessary primitive for trustworthy AI systems. The reference implementation is open-source and available at https://github.com/varshith-Git/Valori-Kernel (archived at https://zenodo.org/records/18022660).