The Pulse
MiniRep Combines Agent Reputation With Current-Answer Checks
A paper introduces MiniRep, which weighs agents’ past records alongside the quality and risk signals in their current answers. Tests show strong results on math tasks, though the evaluation replayed cached proposals without live debate roun
AI.info Team · Written with AI ·
A strong reputation can help an AI agent sway a group’s answer—and give a malicious agent a way to exploit the group’s trust. A paper submitted to arXiv on September 30 by Jiaming Zhang, Yuwan Liu, Yue Huang and Sisi Duan introduces MiniRep, a system designed to weigh an agent’s past record against the answer it gives now. As Zhang and colleagues wrote in the paper’s abstract:
“However, reputation derived from past performance may not reliably predict an agent’s behavior on new tasks, particularly when malicious agents can adapt their behavior and influence other agents during collaboration.”
Jiaming Zhang, Yuwan Liu, Yue Huang and Sisi Duan, Tsinghua University, authors of the MiniRep paper
Multi-agent debate typically lets several AI agents propose answers, exchange arguments and produce a combined result. MiniRep focuses on that final step: deciding which agents’ answers should count, and how much. Its central wager is that historical reputation alone is not enough when an agent can first earn trust and then change its behavior.
Past performance meets current evidence
MiniRep has three parts. A Response Analyzer checks each current proposal for quality and signs of manipulation. It combines patterns across answers with a semantic check from a 1.5-billion-parameter language model; a separate behavioral model looks for signals such as attempts to override instructions and unexpected changes between debate rounds.
A Reputation-Aware Aggregator then adjusts agents’ historical weights using those current-task signals, penalizes suspicious or unusable proposals and limits the collective influence of agents built on the same underlying model. A State Updater records quality and suspicious behavior for later tasks. In the authors’ design, a high historical score does not guarantee that an agent’s answer will carry weight on the current task.
The tested attacks include coordinated agents submitting a shared wrong answer, agents offering different answers that support the same false conclusion, and agents that behave honestly long enough to build a reputation before attacking. The researchers also tested small changes to otherwise plausible proposals, such as altering an arithmetic operation or a boundary value.
Math results lead; other tasks are mixed
The evaluation covers GoEmotions, a dataset for emotion classification; MATH, a collection of competition-level problems; and HumanEval Pro, a Python coding benchmark. The researchers used 100 fixed tasks per dataset and seven API-accessible language models, arranged in four agent compositions. Their setup tested seven attack types and four placements of compromised agents, or 112 attack conditions per dataset.
On MATH, MiniRep scored 66.75% accuracy in clean runs, compared with 59.25% for the best evaluated baseline. Under attack, it scored 61.95%, against 54.37% for the best baseline, and achieved a higher score than every baseline in 84 of the 112 MATH attack conditions. In a 10-agent mix of seven stronger and three weaker models, it beat all baselines in all 28 MATH attack conditions.
The results are not uniformly better. On clean GoEmotions tests, MiniRep’s sample F1 score was 34.42%, below the best baseline’s 36.42%; on clean HumanEval Pro, its Pass@1 score was 77.75%, below 79.50%. The paper reports that MiniRep had the highest average attacked score on three of the four task metrics, but not on GoEmotions sample F1.
The test isolates the aggregation step
The strongest results come with an important boundary: the researchers replayed cached agent proposals with zero debate rounds. That choice let them compare aggregation systems using the same responses, but it did not test whether MiniRep holds up in a live, multi-round debate where agents influence one another as they argue.
The study also assumes agents keep persistent identities and that the system knows which agents share an underlying model. Its results are descriptive point estimates, not evidence of statistical significance; the authors note that the 112 conditions are not independent task samples or generation-seed repetitions. They identify transfer to more attack types, multi-round debate and attacks aimed directly at the screening models as untested.
MiniRep therefore demonstrates a narrower claim than its name might suggest: combining current-answer checks with past performance improved several benchmark results, especially on math under attack. Whether the same method improves a debate in which agents actually respond to one another remains outside this evaluation.