The Pulse
Linkup Releases 149M-Parameter SPARSEUP Retrieval Model
Linkup Research has open-sourced SPARSEUP, a 149-million-parameter sparse embedding model under the Apache 2.0 license. The model scores 56.4 nDCG@10 on BEIR-13 and reaches more than 97% recall in about 380 microseconds per query with Seism
AI.info Team ·
Linkup Research released SPARSEUP on September 17 as its first open-source model: a 149-million-parameter sparse retriever built on ModernBERT and licensed under Apache 2.0. The company reports an average score of 56.4 nDCG@10 on BEIR-13, calling it the strongest public vocabulary-based sparse encoder it knows of below 150 million parameters.
Unlike dense retrieval systems, which represent each passage as a single continuous vector, SPARSEUP assigns weights to vocabulary terms. That output can be stored in an inverted index and inspected when a search result fails. Linkup is positioning the model as the sparse counterpart to LightOn's DenseOn and LateOn releases, using the same backbone family and fine-tuning data to make the three retrieval approaches easier to compare.
SPARSEUP Completes LightOn's Retrieval Lineup
Linkup starts with the LateOn-unsupervised checkpoint, a ModernBERT-based model originally trained for late-interaction retrieval. Because that checkpoint no longer included a masked-language-modeling head, the team grafted ModernBERT's original head back onto it before fine-tuning.
Training uses LightOn's fine-tuning mixture with contrastive learning, seven hard negatives sampled from a pool of 50 for each query, and in-batch negatives. Linkup says the training run fit on a single H100 GPU and did not use cross-encoder distillation. Queries and documents receive [Q] and [D] prefixes before the model produces a vocabulary-sized sparse vector scored with a dot product.
Three Changes Keep the Vocabulary Vector Sparse
A standard SPLADE-style implementation on the same backbone produced large bags containing stopwords and unrelated terms. Linkup attributes the problem to ModernBERT's high MLM logits and the freedom for every input token to expand across roughly 50,000 vocabulary dimensions.
SPARSEUP first shifts the logits before applying the sparse activation: log(1 + ReLU(x - 15)). Linkup selected 15 after probing MS MARCO documents, saying values between 10 and 20 worked. The shift moves weak activations below the point where they would otherwise create dense output bags.
The second change keeps only the 12 strongest vocabulary dimensions for each input token before max pooling. That limit controls expansion per token rather than imposing a fixed limit on the final vector. The third change folds byte-level BPE variants together. Forms such as heat, Heat, Ġheat and ĠHeat map to one canonical vocabulary entry, reducing the output space from about 50,000 dimensions to roughly 34,000.
A 56.4 Score With Clear Trade-Offs
SPARSEUP reaches 56.4 average nDCG@10 on BEIR-13, excluding MS MARCO. Linkup says the result is the first score above 56 from a sparse encoder below 150 million parameters that it knows of. The model performs well on ArguAna and Touché and beats DenseOn on HotpotQA, but DBPedia is a weak point and the largest gap against DenseOn appears on FiQA.
The controlled comparison with LightOn's models is less favorable to the sparse system. Using the same backbone family and training data, LateOn scores 58.9 and DenseOn 57.9 on BEIR-13, compared with SPARSEUP's 56.4. Linkup reports that SPARSEUP uses approximate Seismic search for the evaluation, while the LightOn figures use exact search, so the comparison does not hold search conditions constant.
On decontaminated BEIR, SPARSEUP comes within 0.17 points of DenseOn. Linkup cautions that the decontaminated NQ and MS MARCO subsets contain only 21 and 46 queries, respectively, making those averages sensitive to small changes.
Seismic Search Keeps Query Latency Below a Millisecond
Linkup reports that SPARSEUP produces an average of 47 non-zero terms per query and 190 per document on MS MARCO. SPLADE-v3 produces 25 and 170 in the same comparison, leaving SPARSEUP with somewhat denser queries while keeping document representations in a similar range.
With a Seismic inverted index, SPARSEUP reaches more than 97% recall against exact search in about 380 microseconds per query in a single-threaded test. The figure covers the retrieval operation rather than the full pipeline, and Linkup does not compare it directly with dense or late-interaction systems running on different hardware.
Linkup says expanding the sparse vectors could improve BEIR performance by roughly one to two points, but the release keeps the representations smaller instead. That choice makes the model less competitive on some benchmark sets while preserving the storage, inspection and inverted-index properties that distinguish sparse retrieval from dense embeddings.
Weights Are Available Under Apache 2.0
SPARSEUP's weights are available through the Linkup-Platform/linkup-sparseup-embed-v1 model repository. Developers can load it through Transformers or Sentence Transformers with trust_remote_code=True. Linkup's post also exposes tools for converting outputs into term dictionaries, rendering weighted terms and tracing which input subtokens produced the strongest dimensions.
The release is not presented as a universal replacement for dense retrieval. Linkup describes it as an open baseline with different strengths and failure modes: strong lexical matching, readable output and sub-millisecond approximate search, alongside weak expansions for some entities and number queries. Its immediate value is giving retrieval teams a permissively licensed sparse model that can be tested beside dense and late-interaction alternatives under similar training conditions.