GAVEL: Towards Rule-Based Safety Through Activation Monitoring

AI summary

Overview

• Research area: AI safety and interpretability, specifically activation-based monitoring of large language models, with a design borrowed from rule-sharing practice in cybersecurity.
• Technical level: Advanced. The paper assumes familiarity with transformer internals (hidden states, attention outputs, residual streams), linear probes, steering vectors, and sparse autoencoders.
• Scope: The paper proposes and evaluates GAVEL, a framework that decomposes model activations into interpretable "cognitive elements" (CEs) and enforces user-written Boolean rules over them in real time, in place of coarse misuse-category detectors.

What This Paper Is About

Existing activation-based safeguards are trained on broad misuse datasets (for example generic "cybercrime" or "hate speech" collections), which makes them imprecise, inflexible, and opaque: they fire on benign lookalike content, cannot be reconfigured without new data and retraining, and do not say which part of a generation triggered an alarm. GAVEL replaces those coarse categories with fine-grained, composable cognitive elements such as "making a threat," "payment tools," or "masquerade as human," then lets practitioners express safety or policy constraints as human-readable Boolean rules over the presence of those elements.

Key Contributions

1. Cognitive Elements (CEs): The paper introduces CEs as interpretable, token-level activation primitives capturing mid-level model activity, task, or behavior, and releases an initial vocabulary of 23 CEs grouped into directives to users, LLM tasks/behaviors, and topics. CEs are individually curated so they stay modular and composable rather than being trained on overlapping misuse categories.
2. The GAVEL detection framework: A pipeline that collects CE activations, trains a multi-label detector over them, aggregates detection over a temporal window, and evaluates logical predicates over CE presence. Practitioners can configure or update constraints without retraining the model or the detector, and rule violations expose the specific triggering tokens.
3. Open resources: The authors open source GAVEL, release code and datasets for constructing CEs, collecting activations, composing rules, and detecting violations, and provide GAVEL Studio, an interactive rule-authoring and management tool, plus an agentic tool that generates CEs, rules, and excitation datasets from a natural-language description of a policy.
4. An evaluation across nine misuse categories: Nine scenarios grouped into cybercrime, psychological harm, and scam automation, compared against finetuning baselines, reading-vector projection, moderation APIs, and a per-category activation classifier.

Main Findings

• Precision advantage: On nine misuse categories with Mistral-7B, GAVEL reports an average AUC of 0.99, average balanced accuracy of 0.96, and average FPR of 0.00, with AUC scores above 0.98 and near-zero false positives even against deliberately close benign conversations.
• Granularity beats category classifiers: A classifier trained to detect activations for each misuse category (rather than individual CEs) overgeneralizes and produces higher false positives (average FPR 0.07), whereas GAVEL's per-CE definitions reduce irrelevant pattern capture.
• Concerned with attention outputs: Ablations found attention outputs outperform MLP outputs for detection, with an average TPR of 95.5% versus 82.3%.
• CEs co-occur: 54% of detected malicious dialogues involved tokens with multiple simultaneously active CEs, showing that a detector trained on isolated CEs still generalizes to overlapping ones.
• Elicitation method matters: The authors' excitation method, which wraps each exemplar in the prompt "Think about ⟨c⟩ while revising the following: ⟨s⟩" (labeled ERI), produced higher classification accuracy than naive prefilling or a directive with no contextualization on the target CE.
• Robustness to obfuscation: A GPT-4 judge given explicit instructions matched GAVEL on plain-text violations, but under Misdirection attacks its detection collapsed to 12% on Delusional Thinking and 15% on Election Interference, while GAVEL maintained over 85% recall across all categories.
• Deceptive marketing case: On a Deceptive Marketing dataset where self-promotion is woven into medical advice, GPT-4 achieved only a 24% detection rate while GAVEL reached 73% by detecting the co-occurrence of Build Trust and Healthcare topics.
• Automated adaptation: The automated CE-generation tool produced "out-of-the-box" performance on external benchmarks, including 0.97 TPR on Political Risk from Reasoning Shield and an average 0.945 TPR on ToxiGen, with no manual intervention.
• Multilingual transfer: Using excitation data written only in English, GAVEL was evaluated on four categories in English, Spanish, and Mandarin and achieved TPRs such as 1.0 for Anti-LGBTQ in English, 0.97 for Tax Authority in Spanish, and 0.98 for Tax Authority in Mandarin, with FPRs at or below 0.02.
• Cross-model generalization: The same text-only CE datasets and rules were used to protect Llama3-8B, Mistral-7B, Qwen3-8B, and Gemma-4B; smaller or weaker models produced noisier activations, which the compositional rule structure partly compensated for by acting as an ensemble.
• Low deployment false positives: On a benign background set of 1,000 natural conversations split between UltraChat and DialogueSum, GAVEL with Mistral-7B achieved FPRs of 0.088 and 0.008 respectively.

Methodology in Plain English

The approach starts from text, not from activations. For each cognitive element, the authors write or generate an excitation dataset of short sentences that illustrate that element, for example threatening sentences for "making a threat." Each sentence is wrapped in an explicit directive telling the model to think about the target CE while revising the sentence, and the internal activations of the generated tokens are captured.

From those captured activations, the authors stack attention outputs across a selected set of layers into a per-token representation, then train a multi-label classifier over all CEs so that a single token can be marked as carrying several elements at once. In this paper the detector is a lightweight multi-label RNN with 3 GRU layers and 256 units that processes 5-token segments, trained on 300 samples per CE with an 80:20 split using Adam at 3e-4 and binary cross entropy; the framework is described as compatible with other classifier architectures.

At inference, each token's activations are classified into CE probabilities. These are aggregated into a presence vector over a sliding window, typically spanning the whole conversation, and each user-written rule is evaluated as a Boolean formula over that vector. A rule fires when its predicate holds, and the associated action (interject, override, or steer) is executed; the evaluation in this paper is limited to the detection/alert outcome, since the paper treats response and mitigation methods as already established.

For evaluation, the authors generated datasets with GPT-4.1, validated with GPT-5, covering nine misuse categories as multi-turn dialogues of 7–18 user–assistant exchanges, with 150 misuse conversations per category (50 held out for calibrating CE thresholds) and 500 closely related benign conversations

Authors’ abstract

Large language models (LLMs) are increasingly paired with activation-based monitoring to detect and prevent harmful behaviors that may not be apparent at the surface-text level. However, existing activation safety approaches, trained on broad misuse datasets, struggle with poor precision, limited flexibility, and lack of interpretability. This paper introduces a new paradigm: rule-based activation safety, inspired by rule-sharing practices in cybersecurity. We propose modeling activations as cognitive elements (CEs), fine-grained, interpretable factors such as 'making a threat' and 'payment processing', that can be composed to capture nuanced, domain-specific behaviors with higher precision. Building on this representation, we present a practical framework that defines predicate rules over CEs and detects violations in real time. This enables practitioners to configure and update safeguards without retraining models or detectors, while supporting transparency and auditability. Our results show that compositional rule-based activation safety improves precision, supports domain customization, and lays the groundwork for scalable, interpretable, and auditable AI governance. We open source GAVEL and introduce GAVEL Studio, an interactive rule authoring and management tool. Code and datasets are available at github.com/Offensive-AI-Lab/gavel.