Are LLMs Reliable Rankers? Rank Manipulation via Two-Stage Token Optimization

AI summary

Overview

Research area: Natural Language Processing, specifically the security and robustness of LLM-based information retrieval and reranking systems.

Technical level: Advanced. The paper assumes familiarity with adversarial prompting, gradient-based token optimization, autoregressive language modeling, and standard IR evaluation metrics (rank, perplexity).

Scope: The paper introduces RAF (Rank Anything First), a two-stage token optimization method that generates short, natural-sounding text injections to systematically raise a target item's position in LLM-generated rankings, and evaluates it against two baselines across four open-source LLM rerankers and three product categories.

What This Paper Is About

LLMs are increasingly deployed as rerankers that take a query and a candidate list of products and output a final ranked order. The paper shows this pipeline is manipulable: an attacker can append a small amount of carefully optimized text to a target product's description and reliably push that product up the ranking.

The goal is to build an attack that is simultaneously effective and hard to detect. Prior methods trade one for the other, either manipulating ranks with text that looks obviously adversarial, or producing fluent text that barely shifts the ranking. RAF is designed to achieve both at once.

Key Contributions

1. Method. RAF, an interpretable token-by-token prompt optimization attack for LLM-based reranking that couples a rank-target loss with an entropy-guided readability weight and temperature-based token sampling.

2. Evaluation protocol. A reranking-aligned evaluation pipeline that randomly shuffles candidate order per run (rather than always placing the target last, as prior work did) and repeats every experiment with 10 different random seeds, comparing against two strong baselines.

3. Empirical findings. RAF achieves lower average ranks and markedly lower perplexity than SRP and STS, with comparable bad word ratios, and shows cross-model transfer.

Main Findings

• Lower rank across the board: With a fixed adversarial sequence length of 30 for all methods, RAF attains the lowest average rank across all four rerankers and three categories. Examples: on Llama3.1-8B, RAF reaches rank 4.43 on Book and 3.26 on Coffee Machine; on Mistral-7B, rank 2.54 on Camera; on DeepSeek-7B, rank 2.36 on Coffee Machine; on Vicuna-7B, rank 4.13 on Book. The paper notes the no-injection baseline should intuitively be 5.5.

• Much lower perplexity: On Llama3.1-8B, RAF yields perplexity 15.90 versus 76.02 for SRP and 92.41 for STS on Book, and 10.89 versus 50.16 and 151.27 on Coffee Machine. The paper states DeepSeek-7B and Vicuna-7B occasionally favor SRP in specific settings, but RAF generally sustains strong performance.

• Comparable bad word ratios: RAF generally matches or beats SRP on the proportion of flagged words, with differences described as marginal in cases where SRP is slightly lower (e.g. Camera under Mistral-7B).

• Both objectives are necessary (ablation): On Llama-3.1-8B over all STSData categories, dual objectives give rank 3.69 and perplexity 14.10; target only gives rank 5.01, perplexity 75.07; readability only gives rank 5.81, perplexity 13.14. Removing readability makes the algorithm hard to converge and multiplies optimization time; removing the ranking objective removes manipulation effectiveness.

• Cross-model transfer on open-source models: Prompts optimized on Llama-3.1-8B shift rank by only +0.12 on Mistral-7B, +0.55 on Deepseek-7B, and −0.05 on Vicuna-7B. SRP degrades more, by up to +1.51 on Deepseek-7B.

• Limited closed-source transfer: On GPT-5.1 (gpt-5.1-2025-11-13), RAF's rank degrades from 3.37 on the source model to 5.76, versus 5.95 for SRP. The authors explicitly state they do not claim RAF reliably compromises closed-source rerankers, calling robust closed-source attacks an open problem.

• Token efficiency: With only 10 tokens, RAF already exceeds the performance of other methods using 30 tokens, indicating more efficient use of the token budget. The paper argues gains are not purely length bias, since appending unoptimized tokens does not improve ranking.

• Discrete optimization advantage: SRP's best prompts often occur at early optimization steps rather than at convergence, because it optimizes a continuous soft prompt that must be discretized; RAF optimizes directly in discrete token space and improves steadily.

• Human evaluation: In a fully anonymous A/B-style study comparing RAF against SRP on fluency/coherence, persuasiveness, and manipulation detectability, the paper reports RAF produces higher-quality, more persuasive, and less adversarial-appearing prompts. STS was excluded due to obviously unnatural language. Specific participant numbers and percentages are not reported in the content.

• Two failure modes identified: (1) "Fluent but ineffective" — when the model is highly confident (low entropy), readability dominates and the suffix reads naturally but does not shift relevance judgments. (2) "Mode collapse" — the ranking gradient drives distinct products (e.g. an espresso machine and a cappuccino maker) toward near-identical generic phrases like "stainless steel body" or "user-friendly interface," which is detectable at the corpus level via n-gram overlap or duplicate-phrase detection.

Methodology in Plain English

The attacker picks a target product and appends a short adversarial text sequence to its description. The sequence is built one token at a time, left to right. For each new token position, the method runs two stages.

In Stage 1, gradients of two losses are combined: a ranking loss (how likely the model is to output the desired target product text) and a readability loss (how likely the candidate token is under the model given the description and existing attack text). A fixed weight (w1) controls the tradeoff, and the top B tokens by this combined signal form a candidate shortlist.

In Stage 2, each shortlisted candidate is scored exactly on both losses. Rather than fixed weights, RAF uses entropy-based dynamic weighting: when the model's next-token distribution is confident (low Shannon entropy), the readability weight increases; when the model is uncertain, the ranking objective dominates. The readability weight is computed as beta times (H_max − H(p_read)) / H_max, where H_max = log|V|. The final token is drawn by temperature-controlled sampling from a softmax over the combined loss, rather than always taking the greedily best token, to avoid local optima.

A random initialization at each position is refined by alternating the two stages until the top-scoring candidate repeats or the combined loss stabilizes; the finalized token is appended and the process moves to the next position.

For evaluation, the researchers used STSData (books, cameras, coffee machines), converted JSON-like product fields into natural-language reranker inputs, and tested on Llama-3.1-8B-Instruct, Mistral-7B-Instruct-v0.3, DeepSeek-LLM-7B-Chat, and Vicuna-7B. Baselines were Strategic Text Sequence (STS, a greedy coordinate gradient method) and StealthRank Prompt (SRP, energy-based optimization with Langevin dynamics). All methods were tuned under the same trial budget; RAF's final configuration was target weight 300 and candidate list size 512 in Stage 1, and target weight 40 with beta = 2 in Stage 2. Three metrics were used: average rank (mean over ten independent trials), perplexity over the concatenation of adversarial prompt and original description, and bad word ratio against a fixed inventory of flagged words and phrases.

Why This Matters

Research impact: The paper reframes LLM reranking as an attack surface rather than a purely accuracy question. It also questions the fairness of prior evaluation protocols, showing that always placing the target last introduces positional bias, and that randomized ordering with multiple seeds gives more honest comparisons. The transfer results suggest that vulnerability to natural-sounding prompts is partly a property of language, not just of one model's token preferences.

Real-world applications:

• E-commerce and product search, where sellers could embed optimized text in listings to lift their items in LLM-driven recommendations.
• Retrieval-augmented generation pipelines, where reranked passages feed downstream answers, so manipulated ranking propagates into the generated content.
• Recommendation systems that rank items by contextual reasoning over natural-language descriptions.
• Content platforms and marketplaces that need detection tooling for adversarial listing text at corpus scale.

Industry relevance: Any organization that has replaced or augmented a traditional ranker with an LLM reranker inherits this attack surface. The paper's finding that only 10 optimized tokens can beat baselines using 30 tokens means the cost of a realistic attack is low, while the closed-source experiment suggests that scaling the target model reduces but does not eliminate the effect. The corpus-level mode-collapse failure mode also gives defenders a concrete detection signal.

Future Directions

• Corpus-aware diversity constraints. The per-instance objective does not penalize redundancy across items, which is exactly what makes mode collapse detectable. The authors name this as future work.
• More effective closed-source attacks. RAF's rank degraded from 3.37 to 5.76 on GPT-5.1, and the authors explicitly frame robust attacks against proprietary rerankers as an open problem.
• Validation on realistic pipelines. The limitations section notes the method was developed on a simplified LLM reranking pipeline and that effectiveness in practical LLM-driven retrieval, with more sophisticated workflows and defenses, remains to be validated.
• Defenses and evaluation protocols. The conclusion calls for systematic defenses and evaluation protocols that explicitly address adversarial robustness, and the conclusion also raises trustworthiness and fairness concerns in LLM-driven retrieval.

Target Audience

Security and adversarial-ML researchers working on LLM robustness; information retrieval and recommender-system engineers who deploy LLM rerankers; trust-and-safety teams responsible for marketplace or search content integrity; and NLP researchers interested in gradient-based discrete prompt optimization and the effectiveness-versus-naturalness tradeoff. Readers without a background in gradient-based token optimization or IR evaluation metrics will find the method section demanding, but the results tables and failure-mode analysis are accessible on their own.

Authors’ abstract

Large language models (LLMs) are increasingly used as rerankers in information retrieval, yet their ranking behavior can be steered by small, natural-sounding prompts. To expose this vulnerability, we present Rank Anything First (RAF), a two-stage token optimization method that crafts concise textual perturbations to consistently promote a target item in LLM-generated rankings while remaining hard to detect. Stage 1 uses Greedy Coordinate Gradient to shortlist candidate tokens at the current position by combining the gradient of the rank-target with a readability score; Stage 2 evaluates those candidates under exact ranking and readability losses using an entropy-based dynamic weighting scheme, and selects a token via temperature-controlled sampling. RAF generates ranking-promoting prompts token-by-token, guided by dual objectives: maximizing ranking effectiveness and preserving linguistic naturalness. Experiments across multiple LLMs show that RAF significantly boosts the rank of target items using naturalistic language, with greater robustness than existing methods in both promoting target items and maintaining naturalness. These findings underscore a critical security implication: LLM-based reranking is inherently susceptible to adversarial manipulation, raising new challenges for the trustworthiness and robustness of modern retrieval systems. Our code is available at: https://github.com/glad-lab/RAF.