Learning Grouped Lattice Vector Quantizers for Low-Bit LLM Compression

AI summary

Overview

Research area: Machine learning, specifically efficient inference and model compression for large language models (post-training weight quantization).

Technical level: Advanced. The paper builds on lattice theory, vector quantization, companding transforms, and constrained bit-allocation optimization, and assumes familiarity with LLM quantization baselines such as GPTQ, QuIP#, AQLM and QTIP.

Scope: The paper introduces Grouped Lattice Vector Quantization (GLVQ), a post-training quantization framework that learns a group-specific lattice codebook plus a group-specific companding function for each group of LLM weights, targeting compression below 3 bits per weight.

What This Paper Is About

Large language models are expensive to store and run, so post-training quantization (PTQ) compresses their weights into low-bit formats. Standard uniform quantization degrades badly at very low bit-widths (2 or 3 bits per weight), and existing vector-quantization methods either use one fixed lattice for the whole model (QuIP#) or learn free-form codebooks that require a costly lookup at decode time (AQLM). GLVQ's goal is to learn a separate structured lattice codebook for every weight group so that each group's unique parameter distribution is matched, while keeping decoding as a cheap matrix-vector multiplication.

Key Contributions

1. A grouped lattice vector quantization approach that dynamically adapts the lattice structure to fit the unique distribution of each weight group, rather than imposing a single fixed lattice (such as the E8 lattice used by QuIP#) across the model.
2. A group-specific companding mechanism that reshapes each parameter distribution before lattice quantization, using an independently learned curvature parameter per group to reduce distortion in low-magnitude regions.
3. The use of Babai rounding to approximate the nearest-lattice-point search during training, which makes the generation matrix optimizable by gradient descent, plus salience-determined bit allocation (SDBA, taken from Slim-LLM) to assign bit-widths per group under a global budget.
4. Experiments on post-training quantization benchmarks for LLMs (Llama 1 and Llama 2, 7B to 70B) showing a better accuracy-versus-inference-efficiency trade-off than existing baselines under extreme compression. Source code is released at https://github.com/xzhang9308/GLVQ.

Main Findings

• Perplexity at 2 bits: On Llama 2-70B, GLVQ-32D reaches a perplexity of 3.36 on Wikitext-2, versus QTIP (3.78) and QuIP# (3.91).
• Cross-scale improvement: On Llama 1-13B at 2-bit on Wikitext-2, GLVQ-32D improves on QuIP# by 0.41 perplexity points (5.38 vs. 5.79) and by 0.53 points on C4 (6.95 vs. 7.48).
• Larger lattices help: GLVQ-32D consistently outperforms GLVQ-8D, indicating that larger lattice dimensions give a more expressive codebook.
• Zero-shot accuracy: At 4-bit, GLVQ-8D reaches 51.2% on ARC-Challenge and 81.6% on PIQA on Llama 2-70B, slightly surpassing QTIP and QuIP#. At 2-bit on Llama 2-13B it reaches 40.0% on ARC-Challenge and 78.0% on PIQA, versus QuIP# (39.5%, 77.3%) and QTIP (39.2%, 77.8%).
• Fractional and sub-2-bit rates: At 1.5 bits on Wikitext-2, GLVQ reaches perplexity 7.01 (7B), 6.11 (13B) and 4.99 (70B), outperforming PV-Tuning at matched rates and far surpassing PB-LLM (69.20 / 151.09 / 28.37 at 1.70 bits). At 1.0 bit, GLVQ attains 7.83 (7B), 7.59 (13B) and 6.11 (70B), outperforming BiLLM and OneBit by large margins on 7B and 13B and closely matching PV-Tuning.
• Throughput and memory bandwidth: On an NVIDIA RTX 4090 with batch size 1 and 2-bit quantization on Llama 2-7B, GLVQ-32D-u runs at 100.2 TOK/s with 608 GB/s and perplexity 5.55, against QTIP at 105.2 TOK/s, 628 GB/s and 5.91. The full mixed-precision GLVQ-32D runs at 82.0 TOK/s, 521 GB/s and perplexity 5.41; on Llama 2-70B it runs at 18.7 TOK/s, 702 GB/s and perplexity 3.36 (QTIP: 25.8 TOK/s, 840 GB/s, 3.78).
• Storage overhead: With the default configuration (d = 16, 4-bit weights, m_g = 4096, n_g = 128), side information adds only 0.2% to the weight codes; over the whole Llama 2-7B model this is roughly 2 MB on top of a 1.1 GB 4-bit payload, versus 13.4 GB in FP16.
• Decoding cost: On-the-fly decoding cuts peak activation-plus-weight memory by more than 10x versus pre-decompressing an entire layer, adds d² + d multiplies per sub-block, and increases end-to-end latency by only 2–3% relative to a standard 4-bit uniform PTQ baseline.
• Bit allocation ablation: Replacing salience-driven allocation with a uniform bit-width raises perplexity from 5.64 to 5.79 on Llama 1-13B at 2-bit, and from 3.62 to 3.72 on Llama 2-70B; the gap persists at 3-bit and remains visible at 4-bit.

Methodology in Plain English

The paper breaks the problem into two stages because jointly optimizing codebook size and codebook structure for every group is intractable.

First, bit allocation: given a global bit budget, each group is assigned an integer bit-width. The authors reuse the Salience-Determined Bit Allocation mechanism from Slim-LLM, which minimizes the KL divergence between the original and quantized layer outputs subject to a constraint that the number of groups given N+1 bits equals the number given N−1 bits. With output channels m and group size g = 128, a double-pointer search needs only O(log m) iterations.

Second, lattice codebook learning: each group's weight matrix is reshaped into d × ℓ blocks, and a group-specific generation matrix G_g (d × d) is learned, with d chosen from {8, 16, 32, ...}. The integer lattice indices Z_g are never directly optimized; they come from Babai rounding of G_g⁻¹W_g, which makes the process differentiable in practice and costs O(d³). Training alternates between fixing G_g to refresh the integer indices and fixing the indices to update G_g by gradient descent on the layer-output reconstruction error. Spectral normalization keeps the singular values of G_g in a stable range, and a Frobenius regularizer with λ = 0.1 penalizes drift from the initial lattice, which is initialized by the Cholesky decomposition of the group's covariance matrix.

On top of this, each group gets a companding transform: the μ-law function with its own learned curvature μ_g, which compresses the heavy tails of the weight distribution before quantization and expands it afterwards. μ_g is initialized as 100·tanh(κ_g/10), where κ_g is the sample kurtosis of the group's weights, so heavier-tailed groups start with stronger companding; after each update it is projected into the range [10, 255] for numerical stability. Only G_g and μ_g are learned parameters; the integer codes are refreshed by rounding each iteration. Decoding is then simply an inverse companding applied to G_g times the stored integer indices.

Why This Matters

Impact on research. The paper shows that per-group adaptivity in vector quantization is a viable substitute for a single universal lattice at a fixed bit budget, and that combining adaptive lattices with companding addresses the heavy-tailed nature of LLM weights. It positions GLVQ between QuIP# (fixed, fast, less adaptive) and AQLM (flexible, but with expensive lookup decoding), and it opens a line of enquiry into the relationship between parameter distribution and quantization fidelity.

Real-world applications (implied by the paper's framing):

• Deploying LLMs on edge devices and other resource-limited hardware, where hundreds of gigabytes of inference memory are unavailable.
• Cost reduction in memory-bound inference serving, since the paper's stated motivation is that state-of-the-art models with billions of parameters demand expensive hardware.
• Accuracy-sensitive generation tasks such as summarization or language generation, where the mixed-precision GLVQ configuration is described as particularly advantageous.
• Aggressive compression settings (1.0–1.5 bits per weight) for scenarios with severe memory and energy constraints, without relying on retraining or fine-tuning.

Industry relevance. The efficiency claims are concrete: roughly 2 MB of side information on top of a 1.1 GB 4-bit payload for Llama 2-7B, more than 10x reduction in peak memory from streaming decode, and a 2–3% latency penalty relative to a 4-bit uniform PTQ baseline. Those are the numbers that determine whether a compression method can be integrated into an existing serving stack, and the fact that decoding is a matrix-vector multiply rather than a codebook lookup makes it more hardware-friendly than free-form vector quantization.

Future Directions

1. Extending the grouped lattice and companding design to activations, as well as weights; the paper quantizes only weights, and cites NestQuant as prior work on weights and activations.
2. Systematic study of lattice dimension choice (d = 8 versus d = 32 versus other values) and its interaction with bit-width, given that the paper reports consistently better fidelity at larger d but does not fully characterize the trade-off against complexity.
3. Investigating companding functions beyond the μ-law family, and whether the kurtosis-based initialization of μ_g can be improved.
4. Evaluating on more recent model families (the introduction names GPT, BERT, Llama, Qwen and DeepSeek, but experiments cover only Llama 1 and Llama 2 from 7B to 70B), and reporting the remaining ablation results referenced in Appendix Tables 6–13, since the content available here only details the bit-allocation ablation.

Target Audience

Researchers and engineers working on model compression, efficient inference, and low-bit quantization for large language models; practitioners who need to deploy LLMs under tight memory, energy, or bandwidth budgets; and readers with a background in signal processing, lattice quantization, or numerical optimization who are interested in how structured codebooks can be adapted to neural network weight statistics.

Authors’ abstract

Large Language Models (LLMs) have demonstrated remarkable capabilities but typically require extensive computational resources and memory for inference. Post-training quantization (PTQ) can effectively reduce these demands by storing weights in lower bit-width formats. However, standard uniform quantization often leads to notable performance degradation, particularly in low-bit scenarios. In this work, we introduce a Grouped Lattice Vector Quantization (GLVQ) framework that assigns each group of weights a customized lattice codebook, defined by a learnable generation matrix. To address the non-differentiability of the quantization process, we adopt Babai rounding to approximate nearest-lattice-point search during training, which enables stable optimization of the generation matrices. Once trained, decoding reduces to a simple matrix-vector multiplication, yielding an efficient and practical quantization pipeline. Experiments on multiple benchmarks show that our approach achieves a better trade-off between model size and accuracy compared to existing post-training quantization baselines, highlighting its effectiveness in deploying large models under stringent resource constraints. Our source code is available on GitHub repository: https://github.com/xzhang9308/GLVQ.