CitiLink: Enhancing Municipal Transparency and Citizen Engagement through Searchable Meeting Minutes

AI summary

Overview

Research area: Natural Language Processing and Information Retrieval, applied to civic technology (e-government transparency).

Technical level: Intermediate. The paper assumes familiarity with LLM prompting, full-text search ranking (BM25), faceted filtering, and standard evaluation metrics (F1, ROUGE-L, BLEU), but explains the system at a demo level rather than a deep methodological level.

Scope: A demonstration paper presenting CitiLink, an end-to-end platform that uses an LLM (Gemini 2.0 Flash) to convert Portuguese city council meeting minutes into structured, searchable records, with usability testing by municipal staff and quantitative evaluation of extraction accuracy.

What This Paper Is About

City council minutes are public by law, but they are written in dense bureaucratic language that makes it impractical for most citizens or journalists to find specific decisions, votes, or discussions. CitiLink attacks this problem by building a pipeline and interface that automatically extract structured facts — who attended, what was discussed, and how each councilor voted — from raw minutes, so users can search and filter those records instead of reading entire documents. The paper reports both how the system was built and how well it performed in usability sessions and on a manually annotated benchmark.

Key Contributions

1. An LLM-based extraction pipeline that converts unstructured municipal minutes into three structured layers: metadata (participants, location, date, meeting type), subjects of discussion, and individual voting outcomes (in favor, against, abstention).

2. A deployed, searchable platform combining BM25 full-text search with faceted filtering by municipality, topic, party, or participant, plus multiple navigation views (municipality overview, list view, timeline view, per-minute page).

3. A Portuguese municipal minutes corpus of 120 documents contributed by six municipalities (Alandroal, Campo Maior, Covilhã, Fundão, Guimarães, and Porto), anonymized and annotated by two annotators with curator validation, with an English version produced via DeepL for multilingual demonstration.

4. A dual evaluation covering real-user usability (eight think-aloud sessions with municipal personnel) and extraction quality of Gemini 2.0 Flash across all three extraction layers.

Main Findings

• Metadata extraction is reliable: Gemini 2.0 Flash achieved a macro F1 of 0.84 across all defined metadata fields when averaged over the six municipalities, indicating consistent performance across categories.

• Subject extraction is partially captured: Because the LLM does not output text offsets, ground-truth subjects were matched to generated candidates using BERTimbau embeddings and cosine similarity. Scores of ROUGE-L 0.31 and BLEU 0.21 suggest the model captures content and structure reasonably but paraphrases heavily, which the authors attribute to inherent task variability rather than pure error.

• Voting extraction is the hardest layer: Voting position identification (favor, against, abstained) reached a macro F1 of 0.67, noticeably lower than metadata, which the authors attribute to the greater linguistic complexity of the task.

• Scale of extracted data: From 120 minutes the system produced 115 unique metadata elements (86 participants, 16 locations, 111 dates, 2 minute types), 3,079 subjects of discussion, and 24,040 votes — 22,702 in favor, 161 against, 1,177 abstentions, a distribution reflecting the typical consensus-driven nature of Portuguese council voting.

• Strong usability signals: Across eight approximately 50-minute think-aloud sessions with administrative staff, all participants found navigation intuitive and six said they would use the platform again. Feedback led to concrete refinements of the demo.

Methodology in Plain English

The team fed each minute to Gemini 2.0 Flash as plain text and used carefully written prompts to pull out three kinds of information: administrative details about the meeting, the topics discussed, and how each councilor voted. Extracted metadata were cross-checked against predefined database lists of participants, municipalities, and topics to keep names and places consistent. Everything was stored in MongoDB Atlas, which supports both keyword search and filtering by category, and a React front-end backed by a Flask API lets users browse by municipality, scan a timeline, or search across all documents.

To test accuracy, the researchers compared the model's output against a manually annotated gold standard for all 120 Portuguese minutes. Metadata and votes were scored with F1, while subjects were scored by first pairing each gold subject with its most similar model-generated subject using BERTimbau sentence embeddings, then computing ROUGE-L and BLEU. Separately, eight municipal staff members used the system in guided think-aloud sessions, narrating their reasoning while completing ten tasks designed to test navigation and retrieval. The system was also wired into a restricted back-office so municipalities can upload minutes and validate extracted data, keeping a human in the loop.

Why This Matters

Impact on research: Most prior work on council meeting analysis focuses on video recordings, and European Portuguese is an under-resourced language for this task. CitiLink shows that general-purpose LLMs with prompt engineering can produce usable structured data from legal-administrative text without task-specific model training, and it provides a public benchmark dataset and annotation protocol for others to build on.

Real-world applications:

• Citizen oversight: Residents can search for decisions affecting their neighborhood, such as traffic regulation changes, without reading hundreds of pages of formal minutes.
• Local journalism: Reporters can quickly locate votes, track a councilor's voting record, or monitor a topic over time across multiple municipalities.
• Municipal administration: Staff can generate indexed, cross-referenced archives automatically and validate outputs through the back-office rather than maintaining manual records.
• Civic transparency monitoring: Organizations ranking municipalities on openness could use the extracted vote and attendance data as machine-readable transparency indicators.

Industry relevance: The architecture is a transferable template for any organization sitting on large volumes of formal, repetitive documents — regulatory filings, board minutes, legislative records, court proceedings — where searchability currently depends on manual indexing. The paper also gives a concrete case study of using a commercial LLM for extraction with human validation, including honest reporting of where accuracy falls short.

Future Directions

• Integrate open-source language models across the extraction layers, reducing reliance on a single commercial API and enabling comparison against Gemini's scores.
• Broaden evaluation beyond municipal staff to citizens and journalists through questionnaires and focus groups, testing whether the platform delivers value to the audiences it was designed for, not just the people who write the minutes.
• Scale to larger archives: The 120-minute corpus was a validation benchmark; the open question is how the pipeline and search quality hold up across full historical municipal archives or nationwide collections.
• Improve voting outcome extraction, the weakest layer at 0.67 F1, where complex sentence structures describing individual councilor positions remain a bottleneck.

Target Audience

Researchers and practitioners in information retrieval, NLP, and civic technology, particularly those working on low-resource languages, legal-administrative text, or LLM-based information extraction with human-in-the-loop validation. It is also relevant to e-government policymakers, municipal digital-transformation teams, and journalists interested in tools for accountability reporting. Readers seeking deep technical novelty should look elsewhere; the value here is in the complete, evaluated, publicly available system and its honest reporting of extraction limits.

Authors’ abstract

City council minutes are typically lengthy and formal documents with a bureaucratic writing style. Although publicly available, their structure often makes it difficult for citizens or journalists to efficiently find information. In this demo, we present CitiLink, a platform designed to transform unstructured municipal meeting minutes into structured and searchable data, demonstrating how NLP and IR can enhance the accessibility and transparency of local government. The system employs LLMs to extract metadata, discussed subjects, and voting outcomes, which are then indexed in a database to support full-text search with BM25 ranking and faceted filtering through a user-friendly interface. The developed system was built over a collection of 120 minutes made available by six Portuguese municipalities. To assess its usability, CitiLink was tested through guided sessions with municipal personnel, providing insights into how real users interact with the system. In addition, we evaluated Gemini's performance in extracting relevant information from the minutes, highlighting its effectiveness in data extraction.