Enhancing Stock Market Information Retrieval with Retrieval-Augmented Generation

AI summary

Overview

Research area: Information retrieval and natural language generation for finance (cs.IR), specifically Retrieval-Augmented Generation applied to Vietnamese-language stock market news.

Technical level: Intermediate. Readers should be comfortable with embeddings, vector similarity search, and the retriever–generator split in RAG; the paper explains its components but assumes this background.

Scope: The paper describes and evaluates a three-stage RAG pipeline (LaBSE embeddings stored in FAISS, hybrid FAISS + BM25 retrieval, Gemini-1.5-Flash generation) over 30 Vietnamese stock market articles from VnExpress, scored with ROUGE and latency on 100 queries.

What This Paper Is About

General-purpose large language models hallucinate, go stale, and lack domain expertise, which is a particular problem in stock markets where conditions change constantly and accuracy matters. The authors build a RAG system that retrieves relevant Vietnamese financial news articles before generating an answer, so the model's output is grounded in retrieved text rather than parametric memory. The goal is to measure how closely those generated answers match reference texts (via ROUGE) and how fast the system responds.

Key Contributions

1. A concrete, end-to-end RAG architecture for Vietnamese stock market news, combining LaBSE sentence embeddings, FAISS vector search, BM25 re-ranking, and Gemini-1.5-Flash generation, with each stage documented in detail.
2. A hybrid retrieval design that applies BM25 re-ranking on top of FAISS L2-distance nearest-neighbor candidates, intended to capture both semantic similarity and keyword-level relevance.
3. A Vietnamese-specific preprocessing pipeline using the Underthesea library for word segmentation and stop word removal, addressing the fact that Vietnamese is multi-syllabic and not space-separated like English.
4. An empirical evaluation reporting ROUGE-1, ROUGE-2, ROUGE-L and latency statistics (average, minimum, maximum) over 100 stock market queries.

Main Findings

• High lexical overlap on ROUGE-1: F-measure 0.9555, with precision 0.9616 and recall 0.9597, which the authors interpret as strong capture of individual financial keywords.
• Stronger-than-expected bigram retention: ROUGE-2 F-measure 0.9314 (precision 0.9374, recall 0.9356), read as evidence that two-word financial phrases survive generation intact.
• Structure preserved at sentence level: ROUGE-L F-measure 0.9436 (precision 0.9496, recall 0.9478), indicating the longest common subsequence between generated and reference text is long.
• Low average latency: 1.1626 seconds across 100 queries, with a minimum of 0.917 seconds and a maximum of 2.455 seconds. The abstract rounds this to "1.16 seconds."
• Small, categorized corpus: 30 VnExpress articles split into Market Trends & Index Performance (8), Corporate Stocks & Sectors (9), Foreign Investment & Liquidity (7), and Regulatory & Corporate Actions (6).
• No baseline comparison is reported: the paper presents ROUGE and latency figures for its own system only, without comparing against a non-RAG LLM, an alternative retriever, or an ablation of the BM25 re-ranking step. It also does not describe how the 100 reference responses used for ROUGE scoring were constructed.

Methodology in Plain English

The authors collected 30 stock market news articles from VnExpress, a Vietnamese news site, covering trading volume, price swings, corporate earnings, and macroeconomic factors. They cleaned the text by trimming whitespace, stripping URLs and special characters with regular expressions, lowercasing everything, segmenting Vietnamese words with Underthesea, and removing common Vietnamese stop words so that only content-bearing words remained for embedding.

Each cleaned document was converted into a numerical vector with LaBSE, a multilingual sentence embedding model chosen for its Vietnamese support, and stored in a FAISS database. (The paper also states that documents were tokenized with Jieba, a tool built for Chinese, and describes it as adapted for Vietnamese, alongside the Underthesea step.)

When a user asks a question, the question is embedded with the same LaBSE model, and FAISS returns the closest documents by L2 (Euclidean) distance between vectors. Those candidates are then re-scored with BM25, a classic keyword-ranking formula that weighs term frequency, document length, and inverse document frequency, to prioritize results that match on actual query terms. The re-ranked documents are inserted into a structured prompt with the user's question, and Gemini-1.5-Flash generates the answer from that prompt. Quality was measured by comparing 100 generated responses against reference texts with ROUGE-1, ROUGE-2, and ROUGE-L, and responsiveness was measured by timing each query.

Why This Matters

The work sits at the intersection of two active threads: applying RAG to reduce hallucination in high-stakes domains, and extending NLP tooling to Vietnamese, a language with comparatively few financial NLP resources. It contributes a documented recipe rather than a new model, which is useful for practitioners who need a starting point for a non-English financial retrieval system.

Real-world applications:

• Investor and analyst assistants: answering natural-language questions about recent market movements with answers grounded in cited news articles.
• Newsroom and research summarization: condensing Vietnamese financial reporting on trading volume, foreign capital flows, and regulatory changes.
• Risk and compliance monitoring: surfacing relevant regulatory and corporate-action news when policies change, a category the corpus explicitly covers.
• Retail brokerage chat interfaces: delivering fast (sub-1.2-second average) contextual answers inside trading apps.

Industry relevance: the average latency of 1.1626 seconds and the use of Gemini-1.5-Flash, chosen for real-time performance and long-context handling, position the system for interactive deployment rather than batch analysis. The multilingual LaBSE choice matters for firms operating in Southeast Asian markets where English-only retrieval pipelines fall short.

Future Directions

• Add baselines and ablations: comparing against an un-augmented LLM, a FAISS-only retriever, and a BM25-only retriever would establish whether the hybrid retrieval step actually contributes to the reported ROUGE scores.
• Move beyond ROUGE: ROUGE measures n-gram overlap, so high scores may partly reflect generated text that echoes the retrieved source documents. Human evaluation, faithfulness or groundedness metrics, and factual-consistency checks would test whether answers are correct as well as lexically similar.
• Scale the corpus and clarify the evaluation set: 30 articles and 100 queries is small, and the paper does not report how reference responses were produced. A larger corpus with documented ground-truth construction would make results more credible.
• Resolve the tokenization discrepancy and extend to complex queries: the paper describes both Underthesea and Jieba in its preprocessing and embedding stages; consolidating on a Vietnamese-appropriate tokenizer, and testing multi-hop or comparative financial questions, are natural next steps. The authors themselves name handling complex financial queries, integrating additional external knowledge sources, and reducing latency as future work.

Target Audience

Graduate students and applied researchers starting work on retrieval-augmented generation who want a fully described, reproducible pipeline; NLP engineers building non-English (specifically Vietnamese) financial question-answering systems; and financial technology teams evaluating whether a hybrid vector-plus-keyword retriever with a hosted LLM generator meets their latency and quality bar. Readers seeking rigorous benchmarking or novel modeling contributions will find the evaluation thin and should treat the ROUGE figures as an initial signal rather than a validated result.

Authors’ abstract

Natural Language Processing (NLP) has been widely applied in financial domains, enabling automated analysis and retrieval of stock market information. However, traditional NLP-based systems often struggle with maintaining contextual accuracy and generating relevant responses in dynamic financial environments. To address these challenges, Retrieval-Augmented Generation (RAG) models have emerged as a powerful solution, combining retrieval-based methods with generative capabilities to enhance response quality. In this study, we present a Retrieval-Augmented Generation (RAG) system designed to generate responses related to stock market information. The system integrates retrieval-based and generative approaches to enhance the accuracy and relevance of financial text generation. To evaluate its effectiveness, we employ the ROUGE metric, specifically ROUGE-1, ROUGE-2, and ROUGE-L, to measure lexical and structural similarity between generated and reference texts. Experimental results on a dataset of 100 stock market-related queries demonstrate high ROUGE scores, indicating strong alignment with reference documents. Additionally, we assess latency to evaluate the system responsiveness, achieving an average response time of 1.16 seconds. These findings confirm that our RAG model is capable of providing accurate and timely stock market insights, making it a promising approach for financial information retrieval.