Autodiscover: A reinforcement learning recommendation system for the cold-start imbalance challenge in active learning, powered by graph-aware thompson sampling

AI summary

Overview

Research area: Machine learning for systematic literature review (SLR) screening — specifically active learning, reinforcement learning / multi-armed bandits, graph representation learning, and human-in-the-loop annotation.

Technical level: Advanced. The abstract assumes familiarity with active learning, heterogeneous graph neural networks, attention mechanisms, and Thompson sampling; the underlying problem (screening papers for reviews) is accessible to a general reader, but the proposed solution is technically dense.

One-sentence scope: A thesis proposing AutoDiscover, a graph-aware, reinforcement-learning-driven framework that replaces fixed active-learning query strategies with an adaptive agent that selects which documents a human expert should screen next.

What This Paper Is About

Systematic literature reviews require experts to manually screen large numbers of documents, and this is becoming a bottleneck as scientific output grows. The situation is made harder by two conditions: very few documents are actually relevant, and expert labels are scarce and expensive to obtain. Standard active learning systems address this with fixed rules for choosing the next document to label, but those rules never adapt and ignore the network structure connecting papers, authors, and metadata. The goal of this work is to treat the choice of "what to screen next" as an ongoing decision problem solved by an agent that learns and adapts during the review itself.

Key Contributions

1. AutoDiscover framework — reframes active learning as an online decision-making problem driven by an adaptive agent rather than a static query strategy.
2. Heterogeneous graph modeling of literature — documents, authors, and metadata are represented as a heterogeneous graph, with a Heterogeneous Graph Attention Network (HAN) learning node representations that inform the agent's choices.
3. Graph-aware Discounted Thompson Sampling agent — a DTS agent dynamically manages a portfolio of query strategies, balancing exploration and exploitation as the review unfolds under non-stationary dynamics where a strategy's usefulness can change over time.
4. TS-Insight — an open-source visual analytics dashboard for interpreting, verifying, and diagnosing the agent's decisions.

Main Findings

• Improved screening efficiency: On the 26-dataset SYNERGY benchmark, AutoDiscover achieves higher screening efficiency than static active learning baselines. The abstract does not report specific performance figures, effect sizes, or per-dataset breakdowns.
• Cold-start mitigation: The agent is specifically credited with mitigating the cold-start problem — it bootstraps discovery from only a minimal number of initial labels, a regime where the abstract states static approaches fail.
• Adaptivity under non-stationary conditions: Because the agent reallocates among multiple query strategies in real time using incoming human labels, it can respond to changing strategy utility during a review rather than committing to one fixed rule.
• Interpretability is treated as part of the system: TS-Insight is presented as a means of inspecting and diagnosing agent behavior, implying that transparency of the agent's choices is a first-class concern rather than an afterthought. No quantitative usability or interpretability results are given in the abstract.

Methodology in Plain English

The approach rests on three linked ideas.

First, the scientific literature being screened is not treated as a flat list of documents. Instead it is built into a heterogeneous graph in which documents, authors, and metadata are different types of nodes connected by different types of relationships. A graph attention network then learns a numerical representation for each node that captures both its content and its position in that network.

Second, instead of hard-coding one rule for picking the next document to screen, the system keeps a set of candidate query strategies — a portfolio — and lets an agent decide which one to use at each step. The agent uses Discounted Thompson Sampling, which naturally trades off trying strategies that might be good (exploration) against relying on strategies that have worked so far (exploitation), while discounting older evidence so that it can track strategies whose usefulness shifts over the course of the review.

Third, the human expert stays in the loop. Each label they provide is fed back immediately as the reward signal that updates the agent's beliefs, so the system improves as the review proceeds. Evaluation is carried out on the 26-dataset SYNERGY benchmark against static active learning baselines, with the additional Tooling of TS-Insight for inspecting the agent's decision-making after the fact.

Why This Matters

Impact on research: Systematic literature reviews underpin evidence-based research across medicine, social science, and beyond, yet they scale poorly with publication volume. If the choice of what to screen next can be learned rather than fixed, reviewers can reach the same conclusions with fewer expensive expert decisions — directly addressing the low-prevalence, label-scarce conditions the abstract identifies as the core obstacle.

Real-world applications:

• Evidence synthesis in healthcare and clinical guideline development, where review teams must screen thousands of citations under tight time and expert-availability constraints.
• Research groups and graduate programs conducting literature surveys, where a small number of annotations must stretch across a large candidate pool.
• Journal and conference peer review triage, where relevant submissions are a small minority of what arrives.
• Patent and prior-art searching, where the relevant-document rate is low and relationships among documents, inventors, and classifications form a natural graph.

Industry relevance: Any organization that maintains a curated knowledge base from a large, continuously growing document stream — pharma and medical affairs, legal discovery, competitive intelligence, and enterprise search — faces the same low-prevalence plus scarce-expert-label structure. The framework's reliance on graph structure and metadata means it fits domains where documents already sit inside rich relational networks. TS-Insight also matters commercially: a screening agent whose choices cannot be inspected is difficult to justify to a regulator or a review team, so a diagnostic dashboard lowers the barrier to adoption.

Future Directions

• Broader benchmarking beyond SYNERGY: The abstract reports results on a single 26-dataset benchmark; generalization to other review corpora, domains, and annotation protocols is untested here.
• Quantifying the cold-start advantage: The claim that the agent succeeds where static approaches fail warrants precise characterization of how few initial labels are required and under what prevalence conditions, none of which the abstract reports.
• Portfolio composition and strategy design: Which query strategies belong in the portfolio, and whether new strategies can be added or learned, remains an open design question.
• Human factors and workflow integration: TS-Insight is introduced as a diagnostic tool, but how reviewers actually interpret and act on agent explanations — and whether that trust improves or degrades screening accuracy — is not addressed in the abstract.
• Scaling and cost: The abstract does not discuss the computational cost of maintaining a heterogeneous graph and training a HAN agent during a live review, which is a practical constraint for very large corpora.

Target Audience

Researchers working on active learning, human-in-the-loop machine learning, and reinforcement learning or bandit-based decision systems will find the core methodological contribution most relevant. Those building tools for evidence synthesis, systematic review automation, and scholarly information retrieval are the primary applied audience, and the TS-Insight dashboard speaks directly to human-computer interaction and visual analytics researchers interested in interpretability for adaptive systems. Practitioners in domains with low-prevalence, expert-label-scarce document triage — pharma, legal, and enterprise knowledge management — will benefit from the problem framing even if the technical details of graph attention networks and Thompson sampling are outside their background. A reader without prior exposure to active learning or bandit methods should expect to consult background material, since the abstract itself is written at an advanced level.

Authors’ abstract

Systematic literature reviews (SLRs) are fundamental to evidence-based research, but manual screening is an increasing bottleneck as scientific output grows. Screening features low prevalence of relevant studies and scarce, costly expert decisions. Traditional active learning (AL) systems help, yet typically rely on fixed query strategies for selecting the next unlabeled documents. These static strategies do not adapt over time and ignore the relational structure of scientific literature networks. This thesis introduces AutoDiscover, a framework that reframes AL as an online decision-making problem driven by an adaptive agent. Literature is modeled as a heterogeneous graph capturing relationships among documents, authors, and metadata. A Heterogeneous Graph Attention Network (HAN) learns node representations, which a Discounted Thompson Sampling (DTS) agent uses to dynamically manage a portfolio of query strategies. With real-time human-in-the-loop labels, the agent balances exploration and exploitation under non-stationary review dynamics, where strategy utility changes over time. On the 26-dataset SYNERGY benchmark, AutoDiscover achieves higher screening efficiency than static AL baselines. Crucially, the agent mitigates cold start by bootstrapping discovery from minimal initial labels where static approaches fail. We also introduce TS-Insight, an open-source visual analytics dashboard to interpret, verify, and diagnose the agent's decisions. Together, these contributions accelerate SLR screening under scarce expert labels and low prevalence of relevant studies.