CitiLink-Minutes: A Multilayer Annotated Dataset of Municipal Meeting Minutes

AI summary

Overview

• Research area: Natural Language Processing and Information Retrieval — specifically, resource construction (annotated corpora) for civic and governmental text.
• Technical level: Intermediate. No novel model architecture is proposed, but the paper assumes familiarity with named entity recognition, span extraction, multi-label classification, and standard evaluation metrics.
• Scope: This paper presents CitiLink-Minutes, a manually annotated multilayer corpus of 120 European Portuguese municipal meeting minutes from six Portuguese city councils, released with baseline results for three downstream tasks.

What This Paper Is About

City council meeting minutes record local decisions, policy discussions, and voting outcomes, but they have seen almost no computational research compared to parliamentary records, largely because no annotated datasets exist. Prior work on municipal governance relies mostly on English-language video transcripts that lack linguistic annotation, leaving structured written minutes unstudied. This paper builds a high-quality, privacy-safe, multiply annotated corpus of Portuguese municipal minutes and establishes baseline task definitions and results so that future systems can be trained and compared on this domain.

Key Contributions

1. A manually annotated corpus of 120 municipal minutes from six Portuguese municipalities (Alandroal, Campo Maior, Covilhã, Fundão, Guimarães, and Porto), covering the 2021–2024 administrative term. The documents contain over one million tokens, 20,375 entities, 11,162 relations, and more than 38,000 individual annotations, spanning four layers: personal information, metadata, subjects of discussion, and voting.
2. A four-layer annotation framework grounded in the ISO 24617 SemAF specification, modeling annotations as entities (text span plus semantic label, optional attribute, optional value) and links (relations between entities), with a double-annotation and curator-validation protocol and reported inter-annotator agreement.
3. An interactive dashboard for exploring the dataset's structure and annotations, plus a FAIR-compliant release through GitHub and the INESC TEC repository under a persistent DOI, with all personal identifiers de-identified.
4. Task definitions and published baselines for three activities — metadata identification (new), voter/vote identification (redefined), and multi-label topic classification (redefined) — with fine-tuned BERTimbau models and few-shot Gemini-2.5-Pro prompts released for reproducibility.

Main Findings

• Encoder models dominate generative models on structured extraction. For metadata identification, fine-tuned BERTimbau reached macro-F1 of 0.752 and micro-F1 of 0.959, while Gemini-2.5-Pro scored 0.273 on both. The generative model showed high precision (0.833) but very low recall (0.163), meaning it extracted correct spans but missed most of them.
• Vote identification behaves similarly. BERTimbau achieved macro-F1 of 0.814 and micro-F1 of 0.705; Gemini-2.5-Pro reached 0.556 and 0.584. The wide gap between macro and micro scores indicates that frequent categories such as Voting and Voter-Favor are recognized well, while rare or lexically variable categories such as Subject remain difficult.
• Multi-label topic classification favors the fine-tuned encoder. BERTimbau scored macro-F1 of 0.642, micro-F1 of 0.822, Hamming Loss of 0.029, and micro Average Precision of 0.870. Gemini-2.5-Pro scored 0.496, 0.525, 0.070, and 0.317 respectively, showing substantially weaker handling of the imbalanced 22-label space.
• Voting behavior varies sharply by municipality. Across the corpus, 67.6% of explicit voting positions were in favor. Covilhã recorded an unusually high 53.7% abstention rate, and Porto had the largest share of votes against (13.2%), illustrating differing council decision-making styles.
• Annotation agreement is strong overall but uneven by layer. Subject and voting layers showed consistent agreement under Krippendorff's alpha, but the personal information layer was only moderate and the free-text theme attribute was the weakest, attributed to undefined entity subtypes and the open-ended format. The topic attribute also scored lower because 22 label values are possible and subjects can legitimately hold multiple topics.
• LLM pre-annotation sped up the workflow. A pilot using LLM-based metadata pre-annotation reduced annotation time by roughly two hours per document, echoing prior work on human-AI collaborative annotation.
• The corpus is semantically dense despite modest document counts. Alandroal, with only about 52,000 tokens, still contains nearly 3,000 entities; Covilhã leads with 4,518 entities and 2,585 relations.

Methodology in Plain English

The team partnered with six Portuguese municipalities to obtain 479 official minutes from the 2021–2024 term. They narrowed this down using four criteria: only minutes after October 2021 (when the mayors took office), a mix of session types (ordinary, extraordinary, public, private), meetings with 15 to 40 agenda items to balance topic coverage against annotation effort, and a preference for public meetings that address matters of broad civic relevance. This produced 120 minutes, exactly 20 per municipality.

Four linguistics students did the annotation on the INCEpTION platform. Before the main effort, they ran a pilot of ten minutes per municipality to train annotators and to test whether a large language model could pre-annotate metadata and save time, which it did.

The annotation scheme separates information into two kinds of structures. An entity is a text span with a label — for example, the span "Rui Moreira" labeled as Participant with the attribute Mayor and the value RM for his party. A link connects two entities with a relation — for example, connecting a participant entity to the span "deliberated" through a Voting relation with the attribute "in favour." These structures are applied across four layers: personal information, meeting metadata, subjects of discussion, and voting outcomes.

Each minute was independently annotated by two annotators across all layers, then reviewed by a curator who resolved disagreements, consulting annotators and domain experts when needed. The metadata layer was the exception, annotated by one person because the task was straightforward, though still curator-checked. Agreement was measured with Krippendorff's alpha.

For the baselines, the researchers split the data chronologically into 72 training, 24 validation, and 24 test minutes, reserving the most recent meetings for testing to simulate deployment on future data. They fine-tuned BERTimbau Large with a token-classification head for the two span extraction tasks and a multi-label output layer with binary cross-entropy for topic classification, and compared against few-shot prompted Gemini-2.5-Pro.

Why This Matters

Impact on research. Municipal governance has lagged far behind parliamentary research in computational linguistics, largely because no annotated written-minute corpora existed. This dataset opens a new domain, provides a non-English resource where earlier efforts were almost entirely English and video-based, and supplies reproducible baselines so new methods can be measured against a common yardstick. The four-layer schema and the linked entity-relation formalism also give researchers a template for annotating comparable documents in other languages and jurisdictions.

Real-world applications:

• Civic transparency portals. Searchable, structured access to council decisions so journalists and citizens can find how a specific topic was debated or how a particular councilor voted.
• Automated monitoring and summarization. Generating per-meeting summaries, tracking agenda items across sessions, and flagging decisions that affect a given neighborhood or policy area.
• Public accountability and journalism. Detecting voting patterns, abstention trends, or party-line discipline across municipalities and time periods.
• Legal and administrative record-keeping. Producing consistent metadata (dates, participants, locations, meeting types) and standardized records from heterogeneous municipal formats.
• Privacy-preserving document release. The de-identification protocol offers a model for publishing government records without exposing personal identifiers.

Industry relevance. GovTech vendors, legal-tech companies, and firms building document intelligence products can use the corpus to train and evaluate extraction pipelines for long, inconsistently structured administrative text. The finding that fine-tuned encoder models beat few-shot generative models on structured span extraction is directly actionable for teams deciding between fine-tuning and prompting. The demonstrated time savings from LLM pre-annotation also speaks to real annotation-cost economics.

Future Directions

• Extend the annotation layers. The authors plan deeper semantic annotation, including finer-grained personal information types (so identifiers can be labeled by category rather than replaced with asterisks) and subject summaries.
• Add an English-translated version with aligned annotations to enable cross-lingual exploration and comparative studies.
• Handle sub-subjects in voting. The current voting layer links votes only to the overarching subject, so when a subject contains multiple separate votes, the specific sub-subjects are not represented. Resolving this would require a more granular schema.
• Broaden municipal coverage. Six municipalities may limit how well models trained here generalize to others; expanding the corpus would test and improve robustness.
• Address open research questions. Does encoder superiority over few-shot generative models hold as prompting and long-context models improve? Can the party-level voting records be resolved to individual councilors without external knowledge bases? How transferable are these annotations to other Romance languages or to municipal systems with different document conventions?

Target Audience

Researchers in NLP and IR working on domain-specific corpora, information extraction, and civic or legal text will get the most from this paper, particularly those interested in non-English resources and low-resource domains. Computational social scientists and political scientists studying local governance and voting behavior will find the corpus directly usable. Annotation specialists and project managers can learn from the double-annotation, curator-validation, and LLM pre-annotation workflow. Practitioners in GovTech and legal-tech evaluating extraction approaches for administrative documents will find the comparative baselines practically relevant.

Authors’ abstract

City councils play a crucial role in local governance, directly influencing citizens' daily lives through decisions made during municipal meetings. These deliberations are formally documented in meeting minutes, which serve as official records of discussions, decisions, and voting outcomes. Despite their importance, municipal meeting records have received little attention in Information Retrieval (IR) and Natural Language Processing (NLP), largely due to the lack of annotated datasets, which ultimately limit the development of computational models. To address this gap, we introduce CitiLink-Minutes, a multilayer dataset of 120 European Portuguese municipal meeting minutes from six municipalities. Unlike prior annotated datasets of parliamentary or video records, CitiLink-Minutes provides multilayer annotations and structured linkage of official written minutes. The dataset contains over one million tokens, with all personal identifiers de-identified. Each minute was manually annotated by two trained annotators and curated by an experienced linguist across three complementary dimensions: (1) metadata, (2) subjects of discussion, and (3) voting outcomes, totaling over 38,000 individual annotations. Released under FAIR principles and accompanied by baseline results on metadata extraction, topic classification, and vote labeling, CitiLink-Minutes demonstrates its potential for downstream NLP and IR tasks, while promoting transparent access to municipal decisions.