OwlerLite: Scope- and Freshness-Aware Web Retrieval for LLM Assistants

AI summary

Overview

Research area: Human-Computer Interaction, with close ties to information retrieval, retrieval-augmented generation (RAG), web crawling and indexing, and browser-based LLM assistants. The paper is published in the Companion Proceedings of the ACM Web Conference 2026 (WWW Companion '26), Dubai, United Arab Emirates, April 13–17, 2026, under a CC BY 4.0 license (DOI: https://doi.org/10.1145/3774905.3793140).

Technical level: Intermediate. Readers need basic familiarity with RAG pipelines, dense retrieval, ranking metrics such as NDCG, and concepts like near-duplicate detection (SimHash, Hamming distance).

Scope: The paper proposes and describes OwlerLite, a browser-based RAG system that makes user-defined retrieval scopes and semantic data freshness first-class parts of retrieval, and reports a small-scale offline evaluation on MS MARCO V2.1 data from the TREC 2024 RAG track.

Authors and affiliations: Saber Zerhoudi, Michael Dinzinger, and Jelena Mitrović at the University of Passau, Germany; Michael Granitzer at the University of Passau and IT:U Austria, Linz. The project repository is listed as https://github.com/searchsim-org/owlerlite.

What This Paper Is About

Browser-based LLM assistants that use RAG typically search a single fixed, periodically refreshed index, so users cannot say which sources an answer should come from and cannot tell whether a page has been meaningfully updated since it was indexed. This can produce answers that blend trusted and untrusted content or rely on stale snapshots, even when the live page contains an important edit. OwlerLite's goal is to give users reusable named scopes over curated web pages and to detect only semantically meaningful page changes, re-indexing just the affected text so answers stay in scope and up to date.

Key Contributions

1. Problem formalization. The paper formalizes scope- and freshness-aware RAG over web resources, defining retrieval scopes, semantic freshness, scope fidelity (SF@k), scope leakage (SL@k), and stale-answer risk R(q,t), and deriving three design principles: scope fidelity (P1), semantic freshness (P2), and transparency (P3).

2. Architecture. It proposes a freshness-aware crawler with two-stage semantic change detection (SimHash fingerprints with an embedding-based second check) and a scope-aware retrieval objective that can be added to LightRAG as metadata filters plus an added scoring feature.

3. Implemented system and positioning. OwlerLite is implemented as a TypeScript browser extension plus a Python FastAPI backend wrapping LightRAG, a vector store, and the change detector, with an explanation layer; the paper positions it against large-scale crawlers (OWLer), knowledge-graph RAG, personal knowledge systems, web-search-enabled assistants, and explainable IR work.

4. Offline evaluation. It reports a minimal offline evaluation protocol using the TREC 2024 RAG segmented MS MARCO V2.1 corpus with synthetically generated scopes.

Main Findings

• Scope fidelity improves. The scope-aware system raises scope fidelity SF@10 from 0.64 to 0.83 on the evaluated TREC 2024 RAG topics.

• Scope leakage is roughly halved. SL@10 drops from 0.36 to 0.17 with the scope-aware re-ranking.

• Relevance cost is small. NDCG@10 falls only slightly, from 0.503 (baseline) to 0.495 (scope-aware), suggesting the scope prior guides retrieval toward the intended scope while largely preserving topical relevance.

• Negligible loss when scopes are well aligned. For queries where most relevant items already fall in the target scope (≥ 80 %), the NDCG@10 drop is < 0.005.

• Two-stage change detection filters noise. Chunks with hash-based similarity above τ₁ = 0.97 are treated as unchanged; those below τ₂ = 0.90 as updated; the intermediate band is resolved by an embedding-based semantic deduplication check similar to SemDeDup. Only chunks classified as changed are re-ingested with a new version identifier.

• Evaluation is explicitly limited. The authors state the experiment uses synthetic scopes and only a subset of topics, so the results are initial empirical support rather than a definitive validation. No user study, latency measurement, or direct measurement of stale-answer risk is reported.

Methodology in Plain English

The system centers on scopes — named, reusable sets of web pages or sources that a user defines, for example through URL patterns or page selections. At query time the user picks one or more scopes, and the system restricts retrieval to those pages and re-ranks results with a preference for the selected scope.

For freshness, a crawler periodically refetches the URLs in each scope, respecting robots.txt and using HTTP validators such as ETag and Last-Modified where available. Each fetched page goes through a three-step ingestion: main article text is extracted with the Trafilatura extractor, the cleaned text is split into 200–400 token chunks along paragraph and heading boundaries with a small overlap, and each chunk receives a 64-bit SimHash fingerprint computed over 5-gram shingles. To detect updates, chunks from the new and old page versions are aligned and compared by Hamming distance, converted into a similarity score by dividing the Hamming distance by 64. Changed chunks get re-indexed; unchanged ones do not.

Each indexed chunk is annotated in LightRAG with scope identifiers, source URL, a version identifier, an update timestamp, and derived freshness features. Retrieval happens in two stages: candidate generation with a metadata filter restricted to the selected scopes, then scoring. The final score combines four weighted components — vector similarity, a graph-based evidence score from LightRAG, a scope prior, and a recency feature — with weights α, β, and δ controlling the trade-off. A browser extension exposes the score breakdown, scope labels, and a version lineage "diff" view.

For the offline evaluation, the authors used a local OpenSearch index of the TREC 2024 RAG segmented corpus (MS MARCO V2.1 segments with URL, title, headings, and segment text fields) together with the associated 1024-dimensional embeddings from Cohere's embed-english-v3.0. They sampled 200 development topics from the TREC 2024 RAG Retrieval task, keeping only topics with at least one relevant segment in the judged pool. To simulate user scopes, they partitioned the corpus into K = 20 synthetic groups by applying k-means clustering to document-level embeddings (averages of a document's segment embeddings). For each query, the cluster containing the largest number of judged relevant segments was designated the target scope. Two systems were compared: a dense baseline retrieving the top-100 segments by cosine similarity, and a scope-aware variant re-ranking those same top-100 candidates with the scoring function using γ = 0.1, α = 0.8, β = 0.2, δ = 0. Metrics were NDCG@10 with official TREC RAG graded labels, plus SF@10 and SL@10.

Why This Matters

Impact on research. The paper argues that scope and freshness deserve explicit modeling rather than being treated as implementation details, and it introduces SF@k and SL@k as metrics to report alongside standard ranking metrics such as NDCG@k. It also provides a browser-scale counterpart to the OWLer crawler from the OpenWebSearch.eu project, which targets large-scale distributed crawling for a general open web index. Because scopes are implemented as metadata filters and freshness as a scoring feature, existing LightRAG deployments could adopt the approach with minimal changes.

Real-world applications:

• Organizational knowledge assistants that must answer only from approved internal or official documentation, where scope leakage means citing untrusted sources.
• Documentation and developer tooling, where an assistant should track a specific product's official docs and notice when a page's content meaningfully changes.
• Research and reading workflows, where users maintain bookmark lists or reading corpora and want answers drawn only from those sources.
• Policy- and compliance-sensitive settings, where the intended corpus must match the corpus actually used to generate an answer — the paper notes that it surfaces scope and version data while intentionally leaving content verification and policy decisions to system operators.

Industry relevance. Commercial LLM assistants with web search offer broad coverage but are described as stateless and opaque: they do not maintain persistent user-defined corpora, do not support reusable named scopes for project-specific collections, tie freshness to the search engine, and do not track semantic versions of content. OwlerLite targets that gap as a complementary persistent layer over curated web resources, which is relevant to any product where answer provenance and version control matter.

Future Directions

1. Expand the evaluation beyond 200 TREC 2024 RAG topics and 20 synthetic k-means clusters to setups that more closely match real user-defined scopes and organizational policies.

2. Replace parts of the heuristic pipeline — SimHash over n-grams with fixed thresholds τ₁ and τ₂, simple chunk-level recency features — with learned or domain-specific models of change, scope, and stale-answer risk.

3. Improve the combination of scope and freshness signals, exploring better ways to weight and integrate them than the current fixed-weight scoring function.

4. Integrate with large-scale crawlers such as OWLer and study collaborative scope management, where scopes are shared or maintained by multiple users.

Target Audience

This paper is most useful to researchers and practitioners working on retrieval-augmented generation, browser-based or personal AI assistants, and web crawling and index maintenance. It also suits information retrieval researchers interested in evaluation metrics beyond relevance, engineers building RAG systems over curated corpora who want to add scope filtering and change detection, and HCI researchers studying transparency and user control in LLM assistant interfaces. Readers looking for a full-scale empirical validation will not find one here — the authors themselves describe OwlerLite as primarily a system contribution with a small-scale offline evaluation.

Authors’ abstract

Browser-based language models often use retrieval-augmented generation (RAG) but typically rely on fixed, outdated indices that give users no control over which sources are consulted. This can lead to answers that mix trusted and untrusted content or draw on stale information. We present OwlerLite, a browser-based RAG system that makes user-defined scopes and data freshness central to retrieval. Users define reusable scopes-sets of web pages or sources-and select them when querying. A freshness-aware crawler monitors live pages, uses a semantic change detector to identify meaningful updates, and selectively re-indexes changed content. OwlerLite integrates text relevance, scope choice, and recency into a unified retrieval model. Implemented as a browser extension, it represents a step toward more controllable and trustworthy web assistants.