MiNER: A Two-Stage Pipeline for Metadata Extraction from Municipal Meeting Minutes

AI summary

Overview

Research area: Natural Language Processing / Information Retrieval, specifically domain-specific named entity recognition and metadata extraction from government documents.

Technical level: Intermediate. The paper assumes familiarity with transformer models, question answering formulations, and NER tagging schemes, but its core ideas are explainable in plain terms.

Scope: The paper introduces MiNER, a two-stage pipeline that locates metadata-bearing sections of Portuguese municipal meeting minutes with a question answering model and then labels the individual metadata entities with fine-tuned transformer models, benchmarking the result against general-purpose LLMs.

What This Paper Is About

Municipal meeting minutes are heterogeneous, free-form documents that record who attended a meeting, when and where it happened, and what type of session it was, but these details are almost never formatted consistently. Existing named entity recognition models recognize general categories such as person or organization, but not the domain-specific fields that minutes actually contain. The paper's goal is to build and evaluate an automatic pipeline that finds the metadata-bearing regions of a minute and extracts each metadata field from them, then to compare that pipeline against large general-purpose language models on quality, speed, and emissions.

Key Contributions

1. A two-stage pipeline (MiNER) that combines question-answering-based metadata boundary detection with transformer-based named entity recognition, applied to municipal meeting minutes rather than parliamentary or scholarly text.
2. Explicit boundary detection before entity extraction, in contrast to prior metadata-extraction work that assumes the metadata section is already identified or pre-segmented.
3. Release of a dataset and fine-tuned models, including the CitiLink-derived metadata annotations and the fine-tuned models, establishing what the authors describe as the first benchmark for metadata extraction from municipal meeting minutes.
4. A multi-dimensional comparison against open-weight (Phi) and closed-weight (Gemini) LLMs, measuring predictive performance, inference cost, and estimated carbon footprint, plus cross-municipality and incremental-learning evaluations.

Main Findings

• Boundary detection works well with a fine-tuned QA model. XLM-RoBERTa outperformed BM25 and a dense retriever on all metrics and in both languages, reaching F1 0.826 and EM 0.792 in Portuguese, and F1 0.714 and EM 0.604 in English. BM25 scored F1 0.094 (pt) and 0.093 (en), and the dense retriever scored F1 0.077 (pt) and 0.111 (en); both unsupervised baselines had EM of 0.0 in Portuguese.

• Fine-tuned transformers achieve strong in-domain entity extraction. On the global split, BERTimbau obtained the best Portuguese result (F1 0.96) and XLM-RoBERTa+Deslex the best English result (F1 0.94). Adding a CRF layer did not improve over the base models (XLM-RoBERTa+CRF dropped to F1 0.90 in Portuguese and 0.89 in English).

• Some categories are near-perfect, others are hard. Date, start time, end time, and meeting number are extracted with F1 = 1.00. "Councilors present" reaches F1 0.98, while absent and replaced members range between 0.28 and 0.70, with frequent confusion among presence states. "Ordinary meetings" are robustly detected (F1 0.95–1.0), whereas "extraordinary meetings" underperform because only six instances exist.

• Most residual errors are boundary errors. The models usually identify the correct entity type but struggle to delimit the exact span.

• Generalization across municipalities is limited. Leave-one-out evaluation dropped to F1 0.80 (BERTimbau, base) and F1 0.72 (XLM-RoBERTa, base) in Portuguese, with English lower still (0.70–0.72). Deslexicalization slightly reduced these numbers in the leave-one-out setting (BERTimbau 0.78, XLM-RoBERTa 0.70). In Portuguese the drop appears mainly as lower recall; in English it appears as overgeneralization and less precise boundaries.

• Very little local supervision is needed. In incremental evaluation, most municipalities exceeded F1 0.95 by the third added annotated minute, and several reached near-ceiling F1 after one example (Campo Maior 0.995, Porto 0.971, Fundão 0.980).

• The pipeline beats general-purpose LLMs on quality, speed, and emissions. Gemini achieved F1 0.27 with precision 0.83 and recall 0.16, producing fragmented outputs. Phi could not be reliably evaluated because of inconsistent JSON outputs. The pipeline required over 1,800 times less inference time (approximately 0.4 s versus 737 s) and emitted nearly 400 times less carbon (6×10⁻⁶ versus 2.2×10⁻³ kg CO₂e).

• The segmentation stage is mostly a noise-reduction and efficiency win. Removing Stage 1 reduced F1 only slightly, from 0.965 to 0.945, but increased NER training time from approximately 1 minute to approximately 50 minutes.

• Dataset composition. The CitiLink-based dataset contains 120 Portuguese municipal meeting minutes from six municipalities. The abstract refers to 180 metadata segments, while the dataset section reports 32,364 metadata segments; entity counts are meeting number (120), date (120), location (100), start time (120), end time (100), meeting type (101), president (119), and councilors (966), totaling eight categories. An automatically translated English version was aligned with LinguAligner and the English test set was human-reviewed.

Methodology in Plain English

The pipeline splits the problem into two steps. In the first step, a question answering model — trained in a SQuAD v2-style setup, where the model either returns a span of text or predicts "no answer" — is asked to locate the opening and closing passages of each minute, which is where metadata lives. The model used was deepset/xlm-roberta-large-squad2, trained for 3 epochs with learning rate 3e-5, batch size 8, weight decay 0.01, max length 512, and stride 128. The identified sections are concatenated into a much smaller text region.

In the second step, transformer models classify every metadata mention inside that reduced region. The authors use BERTimbau-large for Portuguese and XLM-RoBERTa-large for English, each with and without a CRF layer, trained for 15 epochs with early stopping (patience 3), learning rate 2e-5, batch size 2 per device with gradient accumulation over 4 steps, and weight decay 0.01. Data was split at the document level into 60% training, 20% validation, and 20% testing.

To help models transfer between municipalities, the authors apply deslexicalization: participant and location names are replaced with synthetic values drawn from the Faker library with 60% probability, dates and times are varied in format or content with 30% probability, and municipality mentions are replaced with a @MUNICIPIO placeholder. Personal information was anonymized in the dataset, for example replaced with "***".

For evaluation, the authors measure Exact Match and F1 for boundary detection and precision, recall, and F1 for entity recognition, plus inference cost and estimated carbon footprint via the Code Carbon library. They run a leave-one-out protocol (train on five municipalities, test on the sixth) and a simulated low-resource deployment adding one annotated minute at a time. LLM baselines (Gemini and Phi) were accessed through the LangExtract library.

Why This Matters

This work matters because local government transparency depends on citizens, journalists, and researchers being able to search and analyze municipal records, yet those records are published in formats no standard tool can parse. The paper also shows that small, domain-tuned models can outperform much larger general-purpose LLMs on a structured extraction task while being dramatically cheaper and cleaner in carbon terms, which is a meaningful practical result for public-sector deployments with limited budgets.

Real-world applications:

• Searchable civic archives that let citizens query meetings by date, location, participants, or session type.
• Automated indexing and linking of municipal records into open-data portals and information retrieval systems.
• Downstream analytics such as voting identification, meeting summarization, and text segmentation, which the authors note can reuse the extracted segments.
• Journalism and civic-monitoring workflows that need to track attendance patterns or decision timelines across many municipalities.

Industry relevance: The findings apply to any organization extracting structured fields from messy, semi-structured documents — legal, regulatory, and administrative pipelines especially — and provide evidence that task-specific fine-tuning with modest annotation budgets can be a better operational choice than calling a general-purpose LLM.

Future Directions

• Improve cross-municipality generalization, since stylistic and structural variability still hinders transfer, potentially via adaptive fine-tuning strategies or domain-agnostic representations.
• Strengthen the segmentation stage, which currently yields only marginal end-to-end accuracy gains despite its efficiency benefits.
• Address boundary precision and imbalanced categories, particularly presence states for absent or substituted members and the extraordinary-meeting class that has only six examples.
• Extend the pipeline into broader information management tasks, such as integration with summarization or document-linking pipelines, and broaden evaluation beyond the six municipalities currently covered.

Target Audience

Researchers and practitioners in natural language processing and information retrieval who work on domain-specific NER and metadata extraction; public-sector digital government and open-data teams that need to make municipal records machine-readable; and applied machine learning engineers evaluating whether fine-tuned transformer models are a better fit than general-purpose LLMs for structured extraction under cost and carbon constraints. The paper is accessible to readers with an intermediate background in NLP.

Authors’ abstract

Municipal meeting minutes are official documents of local governance, exhibiting heterogeneous formats and writing styles. Effective information retrieval (IR) requires identifying metadata such as meeting number, date, location, participants, and start/end times, elements that are rarely standardized or easy to extract automatically. Existing named entity recognition (NER) models are ill-suited to this task, as they are not adapted to such domain-specific categories. In this paper, we propose a two-stage pipeline for metadata extraction from municipal minutes. First, a question answering (QA) model identifies the opening and closing text segments containing metadata. Transformer-based models (BERTimbau and XLM-RoBERTa with and without a CRF layer) are then applied for fine-grained entity extraction and enhanced through deslexicalization. To evaluate our proposed pipeline, we benchmark both open-weight (Phi) and closed-weight (Gemini) LLMs, assessing predictive performance, inference cost, and carbon footprint. Our results demonstrate strong in-domain performance, better than larger general-purpose LLMs. However, cross-municipality evaluation reveals reduced generalization reflecting the variability and linguistic complexity of municipal records. This work establishes the first benchmark for metadata extraction from municipal meeting minutes, providing a solid foundation for future research in this domain.