A Human Behavioral Baseline for Collective Governance in Software Projects

AI summary

Overview

• Research area: Computational social science / institutional analysis of open source software governance, with implications for AI governance and participatory AI design. Submitted under the Natural Language Processing category (arXiv:2510.08956v2 [cs.CL], 17 Nov 2025).
• Technical level: Intermediate. The methods (institutional grammar parsing, sentence embeddings, topic clustering, entropy and divergence metrics, bootstrap confidence intervals) are standard NLP and information-theoretic tools, but they are applied to a policy-text domain that requires some conceptual background.
• One-sentence scope: The paper builds a large-scale, longitudinal, pre-AI baseline of how open source projects encode participation and control in version-controlled GOVERNANCE.md files, so future evaluations of AI-mediated workflows can measure whether authority concentrates or redistributes.

What This Paper Is About

AI tools are increasingly embedded in software project management, drafting pull requests, triaging issues, and enforcing release gates, which shifts decision rights toward sociotechnical pipelines. Debates about whether this concentrates or redistributes authority have outpaced empirical evidence, largely because there is no historical baseline of how governance evolved in primarily human-governed settings. The authors address this gap by parsing governance documents from hundreds of open source repositories into structured institutional statements and measuring how governance changed from the earliest to the latest snapshot of each project.

Key Contributions

1. A pre-AI governance corpus and baseline. The study captures several years of open source projects before the widespread adoption of AI-managed project tools and management suites, providing a neutral reference point for future evaluations of AI-mediated governance.
2. An account of governance as encoded and renegotiated collectively. Rather than inferring governance only from individual behaviors or outcomes, the work traces how roles, responsibilities, and decision rights evolve in the written constitution of each project.
3. A replicable, artifact-agnostic text-based analytical framework. The pipeline is designed to extend beyond markdown files to other governance records (e.g., prompts used to steer agentic workflows), supporting future studies of power structures under AI agent-human co-production.
4. A three-question framing linking the baseline to participatory AI. The authors ask how communities distribute authority over time, how norms and decision rights are encoded, and whether open source governance evolution can serve as a model for participatory AI design.

Main Findings

• Governance breadth grows for roles and actions. Projects defined more distinct constructs over time, with a mean paired change of +0.484 for Roles (95% CI [0.258, 0.713], n=244; Initial K of 3.46 rising to 3.95) and +0.602 for Actions (95% CI [0.417, 0.793], n=266; Initial K of 3.86 rising to 4.46). With rarefaction (equal statements from each snapshot, cap 100), the gains remain positive: +0.224 for Roles [0.092, 0.352] and +0.228 for Actions [0.134, 0.326].
• No breadth effect for deontics or objects. Deontics moved +0.008 (95% CI [-0.038, 0.055], n=236) and the rarefied estimate was -0.024 [-0.062, 0.012]. Objects moved +0.062 (95% CI [-0.062, 0.186], n=97), with a rarefied estimate of +0.075 [-0.009, 0.162]. The authors state that object results are underpowered in the paired subset and reported for completeness.
• Attention becomes more evenly distributed for roles and actions. Mean change in Shannon entropy was +0.092 for Roles (95% CI [0.011, 0.173], n=169; Initial H 1.775 to Latest H 1.866) and +0.074 for Actions (95% CI [0.017, 0.134], n=213; Initial H 1.905 to Latest H 1.979). The authors describe these intervals as excluding zero.
• Prescriptive force is comparatively stable. Under the standard deontic inventory, the mean change in entropy was -0.005 (95% CI [-0.066, 0.056], n=144; Initial H 1.057 to Latest H 1.052). Under a binary recode of deontics into enabling versus restricting, entropy showed a small decrease of -0.032 (95% CI [-0.066, -0.001], n=149; Initial H 0.108 to Latest H 0.076).
• There is within-project drift for all features. Jensen-Shannon divergence was 0.202 [0.172, 0.234] for Roles, 0.126 [0.107, 0.146] for Actions, 0.062 [0.048, 0.079] for Deontics, and 0.009 [0.005, 0.014] for the binary deontic recode. Drift is larger for roles and actions than for deontics.
• Edits look punctuated rather than steady. The change distribution has many near zeros with a minority of large moves, which the authors describe as consistent with punctuated edits rather than steady drift.
• Overall pattern: maturation by accretion. Catalogs of who acts and what is done broaden and rebalance, while prescriptive polarity changes slowly. The authors also report that robustness checks across alternative minimum per-version thresholds remained consistent.

Methodology in Plain English

The authors started with GitHub repositories that contain a governance file at the repository root, ending up with 710 repositories with at least one such file, spanning a corpus from 2013 to 2022 with governance commits recorded through June 2022 (earliest commit 2013-05-09; latest 2022-05-19). GOVERNANCE.md dominates, appearing in 673 of 710 projects (94.8%), alongside 37 filename variants. Markdown structure was detectable in 498 repositories (70.1%), with a median of 5 sections and a range of 2 to 25. Across the corpus the authors recorded 3,889 governance commits corresponding to repository-by-commit pairs and 2,890 unique commit OIDs, covering 107,869 line-level edits with 82,076 additions and 25,793 deletions.

For the longitudinal analysis they built a paired subset: for each repository they traversed the Git history to recover the earliest valid governance snapshot ("initial") and paired it with the most recent ("latest"). This produced paired data for 637 repositories over an observation period from 2014-03-26 to 2022-05-18, with GOVERNANCE.md again dominant at 601 of 637 (94.3%). Inclusion required at least two recoverable snapshots falling on different calendar days. The gap from earliest to latest commit had a median of 0 days with interquartile range 247, minimum 0, and maximum 2616; the authors call 0-day gaps within-day revisions. The across-day change subset comprised 279 of 637, or 43.8%. Projects with only one usable snapshot were excluded from longitudinal analysis but retained for descriptive statistics.

Documents were preprocessed by removing badges and images, converting tables to lists, normalizing headings, stripping markup, and segmenting text into paragraph blocks and sentences. Coreference resolution was applied to reduce pronoun ambiguity while maintaining a reversible mapping to original offsets.

Each document was then parsed with the Institutional Grammar framework into institutional statements with four components: a Role (the grammatical subject, such as "project lead" or "contributor"), an Action (the verb, such as "commit," "assign," or "review"), a Deontic (the modal force, such as "may," "should," "must," "can," or "cannot"), and an Object (the grammatical object, either another actor or a resource). The authors illustrate this with "The technical committee must ratify the development roadmap." Component extraction used the NLP4Gov toolkit, which combines dependency parsing with semantic role labeling. Modal verbs were canonicalized into a closed set of deontic types and role names were normalized into a controlled vocabulary manually; Actions were manually categorized using a Typology of Rules adapted from the institutional analysis literature. To test reliability of the manual categorization, two authors coded the same sample of 50 Actions and achieved a percent agreement of 82% and Cohen's kappa of 0.92 over 9 labels including a null label.

Each canonical tuple was rendered as a short governance statement, encoded with a Sentence-BERT encoder, and clustered with BERTopic at both snapshots using the library's standard hyperparameters without custom tuning. Three families of metrics were computed: richness K, the count of distinct clusters per repository with a presence threshold of at least two statements; normalized Shannon entropy H in bits (base 2) over cluster proportions, with paired change ΔH equal to H_latest minus H_initial; and Jensen-Shannon divergence in bits between the aligned initial and latest cluster distributions as a measure of drift. A rarefied ΔK sampled an equal number of statements from both snapshots (cap 100) and averaged paired differences over repeated draws to control for document length. Entropy and JSD were computed only for repositories with at least five labeled statements in each snapshot. All repository-level estimates were aggregated as equal-weight means with percentile confidence intervals from a repository bootstrap resampling repositories with replacement, B = 10,000, with 95% intervals unless noted otherwise.

Why This Matters

Impact on research. The paper supplies what the authors describe as the first large-scale longitudinal baseline of open source governance prior to the widespread adoption of AI-managed tools. It converts governance prose into measurable quantities (breadth, evenness, drift), which turns claims about AI-induced shifts in authority into falsifiable pre/post comparisons. The framework is also explicitly designed to be transferable to other governance records, including prompts used to steer agentic workflows.

Real-world applications.

• Evaluating AI coding and project-management assistants by comparing paired changes in evenness and construct counts for roles and actions against this human baseline.
• Designing governance-aware authoring tools that help communities name and rebalance roles and actions, surface uneven coverage, and propose governance-ready statements identifying actor, deontic force, action, and object.
• Review workflows that expose governance edits as structured deltas showing which dimension changed, with explicit acknowledgement required before restricting language is intensified.
• Foundation-level and platform-level policy design for projects drafting constitutions, contributing guides, or maintainership documents.

Industry relevance. As AI capabilities are embedded in team tooling, decision rights migrate from human maintainers toward sociotechnical pipelines in which algorithms and people jointly govern workflows. The paper argues that when multiple groups interact with the same algorithmic system, their strategies can interfere unexpectedly, so a campaign that nearly succeeds in isolation may lose efficacy when a second group acts simultaneously. For organizations deploying such tooling, the baseline offers a way to detect whether authority is becoming more concentrated or participation more uneven after assistants are introduced.

Future Directions

• Extending the baseline to cohorts that adopt AI-assisted governance and development tools, enabling before-and-after comparisons of concentration, breadth, and drift.
• Linking governance change to outcomes such as review latency, newcomer acceptance, and workload distribution.
• Examining causal pathways by pairing textual change with event data from code review and release processes.
• Addressing the stated limitations: governance text rather than behavior, rules that live outside GOVERNANCE.md (for example CONTRIBUTING.md, CODEOWNERS, CI settings, and issue templates) or in informal channels, a predominantly English corpus, survivorship and timing effects from requiring two recoverable snapshots, and the direction-agnostic nature of Jensen-Shannon divergence, which can be affected by cluster relabeling across snapshots.

Target Audience

Researchers in computational social science, AI governance, and participatory design; NLP practitioners working on policy and institutional text; open source community managers and foundation staff; and software engineering researchers studying team power structures under AI agent-human co-production. The paper is also relevant to engineers and product teams building AI assistants into collaborative development platforms, who need measurable criteria for whether those tools concentrate or redistribute authority.

Authors’ abstract

We study how open source communities describe participation and control through version controlled governance documents. Using a corpus of 710 projects with paired snapshots, we parse text into actors, rules, actions, and objects, then group them and measure change with entropy for evenness, richness for diversity, and Jensen Shannon divergence for drift. Projects define more roles and more actions over time, and these are distributed more evenly, while the composition of rules remains stable. These findings indicate that governance grows by expanding and balancing categories of participation without major shifts in prescriptive force. The analysis provides a reproducible baseline for evaluating whether future AI mediated workflows concentrate or redistribute authority.