E2Rank: Unifying Text Embedding and Listwise Reranking for Effective and Efficient Search

AI summary

Overview

Research area: Information retrieval and natural language processing, specifically dense text embedding models and LLM-based document reranking.

Technical level: Advanced. The paper assumes familiarity with contrastive embedding training, learning-to-rank losses, and the retrieve-then-rerank pipeline.

Scope: The paper introduces E²Rank, a single embedding model trained to serve as both first-stage retriever and listwise reranker by converting the listwise reranking prompt into a pseudo-relevance-feedback query embedding (arXiv:2510.22733).

What This Paper Is About

Modern search systems typically run two stages: an embedding model retrieves candidates quickly, then an LLM-based listwise reranker reorders them with much finer relevance judgment but at high computational cost. The paper asks whether those listwise interaction signals can be transferred into an embedding model itself, so that reranking becomes a cheap cosine-similarity comparison instead of slow autoregressive decoding. The goal is one model that retrieves, reranks, and preserves strong general embedding quality.

Key Contributions

1. A unified formulation of retrieval and listwise reranking. The authors reinterpret the listwise prompt (instruction plus top-K candidate documents plus query) as a pseudo-relevance feedback (PRF) query, so that both retrieval and reranking use the same scoring function: cosine similarity between embeddings.

2. Decoding-free listwise reranking. The listwise prompt is encoded into a single PRF-enhanced query embedding, which is then compared against document embeddings that remain context-independent and reusable across both stages, removing the need for autoregressive generation or output-token logits.

3. A two-stage training recipe. Stage I applies contrastive learning (InfoNCE) to build the embedding model; Stage II continues training with a joint InfoNCE plus RankNet objective so the same model gains ranking ability without losing embedding ability.

4. Empirical evaluation across reranking, reasoning-intensive retrieval, embedding, and latency benchmarks, including a controlled comparison against generative listwise baselines (RankQwen3) fine-tuned on the same Qwen3 backbones.

Main Findings

• Ranking gains over same-backbone generative rerankers. E²Rank improves the average over RankQwen3 at every model scale, including +2.71 NDCG@10 at 0.6B under matched training data. With the mixed training data, E²Rank-0.6B gains +4.07 average, E²Rank-4B +1.76, and E²Rank-8B +0.96. Gains concentrate on out-of-domain BEIR datasets; TREC-DL differences are mixed and several at 4B/8B are not statistically significant.

• Strong headline BEIR numbers. In the broader baseline comparison, E²Rank-8B attains the top DL20 score (71.16) and the highest BEIR average (54.35), outperforming the larger zero-shot RankGPT-4o (DL20 69.52, BEIR average 53.09). E²Rank-4B reaches a BEIR average of 54.14, and E²Rank-0.6B reaches 52.09.

• Competitive on reasoning-intensive BRIGHT without RL. E²Rank-8B scores an average of 33.4, surpassing RankQwen3-8B (32.0) and most reasoning rerankers listed, and underperforming only ReasonRank (7B, 35.7), which is trained on synthetic reasoning data.

• Much lower latency. Measured on DL20, TREC-COVID, DBPedia, and Robust04 on a single NVIDIA A100 80G GPU using vLLM, E²Rank achieves up to roughly 5 times speedup over RankQwen3 at 8B on TREC-COVID. E²Rank-8B reranks a query in 2.55 s (DL20), 3.40 s (COVID), 2.51 s (DBPedia), and 3.33 s (R04), versus 6.44, 16.93, 6.40, and 15.36 for RankQwen3-8B. It is also faster than the single-token-decoding FirstZephyr-7B on every dataset and requires no decoding.

• Embedding quality is preserved and even improved on retrieval. On MTEB (English v1, 56 datasets), E²Rank-8B reaches 65.03 average, 4B reaches 64.47, and 0.6B reaches 61.25. Ranking supervision improves retrieval by +1.58 for E²Rank-8B relative to contrastive learning alone, with gains also in pair classification, classification, and STS, while clustering shows a small trade-off.

• End-to-end unified pipeline works across retriever types. E²Rank improves every first stage tested, including BM25 and dense retrievers such as BGE-base, Contriver, SPLADE++ED, and Qwen3E-0.6B. On DL20, E²Rank-8B self-reranking reaches 78.02 with 55.08 on BEIR and 31.00 on BRIGHT.

• The RankNet loss is the most critical component. In the 0.6B ablation, removing RankNet in Stage II drops DL20 from 70.15 to 66.50 and BRIGHT from 30.96 to 22.40; removing Stage I contrastive learning drops MTEB(v2) from 63.41 to 60.61. Retaining RankNet but removing the listwise prompt also degrades ranking (66.29 DL20, 22.69 BRIGHT), showing the PRF prompt, not richer labels alone, drives the reranking ability.

• About 20 feedback documents is the sweet spot. Ranking improves consistently as the listwise prompt includes more documents up to fewer than 20, then plateaus around 20. Top-20 retrieved documents perform best; Random-20 and Last-20 selections hurt performance, falling below the query-only baseline.

• Supervision is necessary for using PRF signals. Classical text-based PRF (56.57 DL20, 46.52 BEIR, 29.62 BRIGHT) and Rocchio-style vector PRF (63.96, 49.20, 21.85) both fall well short of E²Rank-0.6B (70.15, 52.09, 30.96), and the text-based variant even hurts ranking quality.

• Backbone generalization. Applying Stage-II training to GTE-Qwen2-1.5B improves reranking while preserving embedding performance, showing the recipe transfers beyond Qwen3. GTE-Qwen2 uses bidirectional attention, unlike the decoder-only Qwen3 models.

• Teacher labels are noisy. Qwen3-32B generates the ranking permutations; gold-positive-first agreement is only slightly above 50 percent on MS MARCO. The zero-shot teacher reaches 53.37 average NDCG@10 on the BEIR subset and 23.8 on BRIGHT. The authors state they did not conduct a separate human-label study.

Methodology in Plain English

The researchers start with an LLM-based embedding model that encodes a document or query by taking the final hidden state at the end-of-sequence token. Stage I trains this model with standard contrastive learning on roughly 1.5 million public samples from the E5 dataset, using an InfoNCE loss with a temperature of 0.03, batch size 512, learning rate 5e-6, and one epoch.

Stage II continues training with two objectives at once. The first is the same contrastive loss, which keeps the model good at plain retrieval. The second is a RankNet pairwise loss, which compares document pairs from the candidate set and penalizes incorrect relative ordering, using a temperature of 0.1 and a weight λ = 2.0 (performance varies by less than 1.0 NDCG@10 across λ values of 0.5, 1.0, 2.0, and 3.0). The paper reports about 700 steps at batch size 128 with 15 negatives.

The training data for Stage II comes in two versions: MS MARCO data from prior work with 40K samples, or a mixture of retrieval datasets drawn from the intersection of the E5 and BGE-M3 datasets plus two Chinese retrieval datasets. Section 4.1 describes this mixture as about 87k samples, each with one query, one positive, and 15 negatives, while Appendix A describes a mixed dataset of approximately 157k samples. Relevance orderings are labeled by Qwen3-32B and treated as noisy supervision.

At inference, instead of asking the model to generate a ranking, the system builds a listwise prompt from the instruction, the top-20 candidate documents, and the query. It encodes that prompt into one embedding and ranks the top-100 candidates by cosine similarity against precomputed document embeddings. Because document embeddings do not depend on the query or the candidate set, they can be reused, and no autoregressive decoding happens.

Why This Matters

Impact on research. The paper argues that the autoregressive generation used by RankGPT-style rerankers is not what makes listwise reranking effective; the contextual interaction in the listwise prompt is. It bridges the classical PRF literature and modern LLM listwise reranking, and it reframes reranking as a query-embedding problem rather than a text-generation problem. That framing could reshape how retrieval and ranking components are trained and deployed, since it replaces a two-model pipeline with one model and one scoring function.

Real-world applications:

• Enterprise and web search systems that need sub-second reranking on commodity GPU hardware, where RankGPT-style latency is prohibitive.
• Retrieval-augmented generation pipelines, where better candidate ordering improves the context passed to a generator; the reusable document embeddings also fit vector-index architectures.
• Domain search over scientific, legal, or medical corpora, where reasoning-intensive queries matter; the BRIGHT results indicate robustness without reinforcement learning or explicit reasoning traces.
• Multilingual and Chinese-language search, since the mixed Stage II training data includes cMedQAv2 and MMarco Chinese.

Industry relevance. The efficiency numbers matter commercially: the 0.6B model reranks a query in 0.45 to 0.63 seconds across the four measured datasets while beating RankQwen3 baselines, and the 8B model avoids the 15 to 17 second COVID latencies of its generative counterpart. Batch inference and reusable document embeddings reduce serving cost for large indexes, and keeping embedding quality on MTEB means one model can replace separate retrieval and reranking deployments.

Future Directions

• Encoder-only and other architectures. The authors state that validation is limited to decoder-only LLM-based embedding models and that broader testing on encoder-only architectures remains future work, though GTE-Qwen2-1.5B with bidirectional attention is one demonstrated extension.

• Long documents and large candidate sets. The listwise prompt grows with the number of candidate documents, which the limitations section flags as a challenge for very long documents or large candidate sets.

• Robustness under weak first-stage retrieval. Performance depends on feedback quality; the authors note that controlled corruption degrades gracefully on dense-relevance datasets but can fall below the first stage when relevant evidence is extremely sparse.

• Better supervision. Stage II relies on LLM-generated ranking labels with only slightly above 50 percent gold-positive-first agreement on MS MARCO and no separate human-label study, leaving label quality and domain variation as open questions.

Target Audience

This paper is most useful to information retrieval researchers and engineers working on dense retrieval, reranking, and retrieval-augmented generation, particularly those who weigh ranking quality against serving latency. It also suits practitioners who maintain two-stage search pipelines and want a single-model alternative, and graduate students studying unification of retrieval and ranking objectives. Readers need a working understanding of contrastive embedding training and learning-to-rank losses to follow the methodology.

Authors’ abstract

Text embedding models deliver competitive retrieval performance with high efficiency, but their ranking fidelity remains limited compared to LLM-based listwise rerankers, which capture fine-grained query-document and document-document interactions at high computational cost. We propose E2Rank (Efficient Embedding-based Ranking), a unified framework that extends a single text embedding model to perform both retrieval and listwise reranking via continued training under a listwise ranking objective. The key insight is to treat the listwise prompt---constructed from the query and its top-K candidates---as a pseudo-relevance feedback (PRF) query, enabling reranking via cosine similarity against precomputed document embeddings without autoregressive decoding. Empirically, E2Rank achieves state-of-the-art results on BEIR, competitive performance on the reasoning-intensive BRIGHT benchmark, significantly lower latency than existing LLM-based rerankers, and improved embedding performance on MTEB---all within a single model.