Executable Governance for AI: Translating Policies into Rules Using LLMs

AI summary

Overview

Research area: AI governance and policy compliance — natural-language processing applied to converting policy documents into machine-executable rules.

Technical level: Intermediate. The core ideas (rule extraction, human annotation, evaluation metrics) are accessible, but the paper assumes familiarity with LLM prompting, JSON schemas, and formal verification concepts such as SMT solvers and runtimes like NeMo Guardrails.

Scope in one sentence: The paper introduces Policy → Tests (P2T), a pipeline plus compact domain-specific language (DSL) that turns prose AI policies into normalized, machine-readable rules, and evaluates both the quality of those rules against human annotators and their downstream effect on an AI agent's violation rate.

What This Paper Is About

AI policies — laws, standards, and corporate frameworks — are written as prose, so anyone who wants to actually test or enforce them must first hand-convert sentences into concrete, executable checks. That manual conversion is slow, error-prone, and hard to scale, which delays safeguards reaching real deployments. The paper addresses this by building an automated pipeline that extracts obligation-bearing clauses from policy documents and outputs structured, provenance-tagged rules that downstream enforcement and evaluation systems can consume directly.

Key Contributions

1. A policy-to-rule pipeline with a compact DSL. P2T ingests policy documents, mines candidate clauses, uses LLMs under a strict JSON schema to emit atomic rules, validates them, repairs them, de-duplicates them, tags testability, and generates examples. The DSL encodes scope (actor, data_domain, context), hazard, conditions, exceptions, requirement, severity, evidence, and testability with evidence signals drawn from a closed set of eight channels (io_check, log_check, config_check, ci_gate, data_check, repo_check, access_check, attest_check).

2. A corpus spanning heterogeneous governance sources. The authors run the pipeline over the EU AI Act (Regulation (EU) 2024/1689, Articles 8–15), NIST AI RMF Profiles (MAP 1.1–5.3 and MEASURE 1.1–2.7), the HIPAA Privacy Rule (45 C.F.R. Part 164), and the Microsoft Responsible AI Standard v2, producing 522 extracted rules and 427 unique rules after de-duplication.

3. A human-annotated gold set and agreement audit. Five source-specific gold sets totaling n = 427 rules (per-document n = {51, 117, 42, 77, 140}) were annotated by two independent raters with senior adjudication, with reported agreement using Cohen's κ and Krippendorff's α.

4. A downstream safety case study plus open-source release. Three HIPAA-derived, I/O-testable rules were encoded as NeMo output rails and applied to a generative agent, with an LLM judge measuring violation rates on 60 prompts. The paper releases the codebase, DSL, prompts, and rule sets.

Main Findings

• Rule yield and testability vary sharply by document. Across the corpus, 276 candidate spans produced 522 extracted rules and 427 unique rules, with 58.8% of unique rules flagged as testable and 50 testable rules carrying input/output evidence signals. Testable rates ranged from 33.3% for the EU AI Act to 85.7% for the HIPAA Privacy Rule (NIST AI RMF 43.6%; Microsoft Responsible AI Standard v2 75.0%).

• Span- and rule-level quality against gold sets. Per document, coverage ranged from 0.55 ± 0.20 (EU AI Act) to 0.76 ± 0.11 (HIPAA); span F1 ranged from 0.75 ± 0.08 (HIPAA) to 0.84 ± 0.06 (Microsoft); span AUPRC ranged from 0.70 ± 0.13 (HIPAA) to 0.91 ± 0.06 (Microsoft). Structured extraction (SE) slot similarity was much lower — 0.24 ± 0.09 for the EU AI Act up to 0.30 ± 0.04 for Microsoft — as was evidence-signal similarity, from 0.24 ± 0.10 to 0.36 ± 0.06.

• Robust inter-annotator agreement on the gold set. Macro-averages were span κ = 0.83 ± 0.03, testable κ = 0.76 ± 0.04, scope-actors α = 0.63 ± 0.05, and hazard κ = 0.64 ± 0.05. Quality control included a 30-item calibration set and seeding 5% of assignments with previously adjudicated hidden-gold items.

• Judge and repair stages help; de-duplication trades recall for redundancy reduction. With GPT-5-mini, adding Judge + Repair raised coverage from 0.9296 to 0.9437, span F1 from 0.9714 to 0.9784, and SE slot similarity from 0.3641 to 0.3886. Adding de-duplication moved coverage to 0.8873, span F1 to 0.9640, and SE slot similarity to 0.3624 while removing duplicates (DupIdx 0.1525). The same pattern held for GPT-4o-mini, where Judge + Repair raised span F1 from 0.8342 to 0.8824 and SE slot similarity from 0.2528 to 0.2944.

• Guardrails reduced measured violations. On 60 prompts (20 clean, 20 obfuscated, 20 compositional), the baseline assistant had an overall violation rate of 0.34, versus 0.05 for the guarded version. By bucket: clean 0.02 versus 0.00, obfuscated 0.58 versus 0.08, compositional 0.42 versus 0.06. Relative to each system's own clean bucket, the baseline degraded by +0.56 (obfuscated) and +0.40 (compositional), while the guarded system degraded by only +0.08 and +0.06.

• Paragraph-level chunking was the best of three strategies tested. Sliding context windows produced reasonable results but increased duplicates, while single-sentence spans minimized duplication but yielded under-specified rules when clauses depended on surrounding qualifiers. Paragraph-level spans became the default.

• Reported pipeline cost is far below manual effort. Across four documents the pipeline processed 42,465,118 input tokens for about $20 total, running roughly 30 minutes to 3 hours per document. Per-document pipeline cost ranged from $3.5 (EU AI Act) to $6.5 (Microsoft), compared with estimated manual annotation of 4.3–6.8 hours for the EU AI Act and 11.7–18.7 hours for Microsoft at 5–8 minutes per atomic rule.

• Known failure modes persist. Recurrent errors include softened or dropped qualifiers, scope misassignment when cues are nonlocal or cross-referenced, nested exceptions and negations, overlapping or multi-party scopes, implicit conditions outside the extracted span, and over-normalization that blurs distinctions between "may," "should," and "shall."

Methodology in Plain English

The authors built a six-step pipeline that keeps deterministic code where possible and uses LLMs only where interpretation is needed.

1. Ingestion and chunking strips boilerplate and splits documents into addressable spans, each carrying text, location, and a stable identifier for traceability.
2. Clause mining (optional) uses deterministic cues — deontic markers, exception cues, actor mentions, numbers, dates, cross-references — to filter down to spans likely to contain real obligations, tagging clause type and extracting thresholds and deadlines.
3. Structured extraction has an LLM emit atomic rules under a strict JSON schema with few-shot examples. A judge LLM then flags missing hazards, empty scope, unverifiable evidence, or conflicts; a repair LLM makes minimal, provenance-preserving edits. Optional checks include an evidence gate requiring appropriate evidence fields, an SMT (Satisfiability Modulo Theories) consistency check that encodes rules as logical constraints to detect contradictory requirements over overlapping scopes, and counterfactual probing that paraphrases source clauses to test polarity sensitivity.
4. De-duplication first applies a canonical structural signature, then a semantic pass that embeds rules and merges high-similarity pairs within the same document and scope.
5. Testability tagging asks an LLM to decide whether an objective pass/fail oracle exists and which evidence channels apply.
6. Example generation produces benign and adversarial prompts for testable rules that have an I/O evidence signal.

Evaluation compared AI-extracted rules against a human-annotated gold set. Because no established benchmarks exist for policy-to-rule translation, the authors deliberately relied on internal consistency, human agreement, and controlled ablations rather than model-to-model comparisons. Two decoding seeds were run per document, and reported values are means across seeds with 95% bootstrap confidence intervals. Ablations varied the extraction model (GPT-5-mini and GPT-4o-mini) and stacked safeguards in sequence (LLM → +judge+repair → +deduplication). For the safety study, three HIPAA-derived rules were encoded as NeMo output rails and an LLM judge scored responses pass/fail against a JSON rule description.

Why This Matters

Impact on research. The paper defines and attacks a gap that sits between existing work on training-time alignment (RLHF, Constitutional AI), runtime guardrails (NeMo Guardrails, Guardrails AI), evaluation toolkits (OpenAI Evals, PromptFoo), and rules-as-code engines (OpenFisca, DMN, OPA/Rego). Those efforts either assume rules already exist or assume tests already exist. P2T positions itself as the missing translation layer, and by releasing code, prompts, DSL, and rule sets, it gives the field a reproducible starting point and a template for auditing policy-to-rule fidelity.

Real-world applications:

• Regulated industries handling sensitive data. The HIPAA-derived rules and the guarded-versus-baseline experiment show a concrete path to encoding permitted-use and disclosure constraints as runtime rails.
• Compliance and audit teams. Rules carry provenance (document, citation, span_id) and testability flags with evidence channels, so reviewers can trace each check back to its source clause and know what artifact would prove compliance.
• AI assurance and evaluation vendors. Companies packaging process checks and technical evaluations into toolkits can feed extracted rules into NeMo Colang snippets, Guardrails validators, OPA/Rego policies, or Evals-style batch prompts.
• Enterprises maintaining internal AI standards. The pipeline can be re-run on updated documents to track policy evolution without retraining, which suits organizations whose standards (like the Microsoft Responsible AI Standard v2 used here) change over time.

Industry relevance. The cost comparison is the headline for practitioners: roughly $20 in tokens and 30 minutes to 3 hours per document, versus tens of hours of expert annotation at 5–8 minutes per rule. Combined with the measured reduction in violation rates under obfuscated and compositional prompts, that economics is what makes continuous, audit-ready compliance checking plausible rather than aspirational.

Future Directions

• Interactive validation loops. The authors propose incorporating user feedback into extraction and repair, so domain experts can correct ambiguous clauses rather than relying purely on automated judging.
• Richer semantic modeling. The current DSL is deliberately compact and does not capture all nuances, such as temporal or probabilistic constraints; the paper flags this and calls for richer representations.
• Expansive, evolving benchmarks. Because no established benchmarks for policy-to-rule translation exist, the authors used internal consistency, human agreement, and ablations instead. Building shared benchmarks is presented as necessary to stress-test real-world scenarios and enable direct comparisons.
• Robustness to the residual failure modes. Softened qualifiers, nonlocal scope cues, nested exceptions and negations, implicit conditions, and over-normalization of "may" versus "shall" remain unsolved. The paper notes that SMT checks, evidence gating, and de-duplication target contradictions, verifiability, and redundancy rather than ambiguity, and that human oversight remains essential in high-stakes rule extraction.

Target Audience

This paper is most valuable to AI governance and compliance practitioners who need to operationalize policy text, machine-learning engineers building guardrails and evaluation harnesses, and policy or legal technologists interested in how regulatory prose can be formalized. Researchers working on LLM-based information extraction, document-grounded generation, or verifiable AI systems will also find the DSL design, the ablation methodology, and the safety case study relevant. Readers should be comfortable with LLM pipelines, JSON schemas, and basic evaluation metrics; no deep formal-methods background is required to follow the argument, though the SMT consistency check is described only at a high level.

Authors’ abstract

AI policy guidance is predominantly written as prose, which practitioners must first convert into executable rules before frameworks can evaluate or enforce them. This manual step is slow, error-prone, difficult to scale, and often delays the use of safeguards in real-world deployments. To address this gap, we present Policy-to-Tests (P2T), a framework that converts natural-language policy documents into normalized, machine-readable rules. The framework comprises a pipeline and a compact domain-specific language (DSL) that encodes hazards, scope, conditions, exceptions, and required evidence, yielding a canonical representation of extracted rules. To test the framework beyond a single policy, we apply it across general frameworks, sector guidance, and enterprise standards, extracting obligation-bearing clauses and converting them into executable rules. These AI-generated rules closely match strong human baselines on span-level and rule-level metrics, with robust inter-annotator agreement on the gold set. To evaluate downstream behavioral and safety impact, we add HIPAA-derived safeguards to a generative agent and compare it with an otherwise identical agent without guardrails. An LLM-based judge, aligned with gold-standard criteria, measures violation rates and robustness to obfuscated and compositional prompts. Detailed results are provided in the appendix. We release the codebase, DSL, prompts, and rule sets as open-source resources to enable reproducible evaluation.