LMSM: LLM Security Framework Inspired by Linux Security Modules

AI summary

Overview

Research area: LLM security and runtime enforcement (cs.CR), combining ideas from systems security (Linux Security Modules) with model-internal interpretability artifacts such as sparse autoencoders, transcoders, and linear probes.

Technical level: Advanced. The paper assumes familiarity with LLM serving internals (continuous batching, packed forwards, KV cache), interpretability artifacts, and reference-monitor concepts from operating-system security.

Scope: The paper proposes LMSM, an LSM-inspired framework that separates calibrated model-internal evidence, versioned policy rules, and a distinct enforcement gate for authorizing buffered LLM output release, and evaluates it on Qwen3-4B with SAE, transcoder, and dense-probe backends.

What This Paper Is About

Layered defenses around LLMs (alignment, prompt controls, external guards) reduce risk but do not provide a trusted path from model-internal evidence to enforcement before an output is released. Existing interpretability-based safety methods couple each artifact to its own calibration, policy logic, and intervention code, so every new artifact becomes a fresh integration project rather than a stronger shared defense. LMSM borrows the separation behind Linux Security Modules: a selected backend exposes calibrated evidence, a versioned policy evaluates active rules over trusted per-request context, and a separate gate authorizes buffered output release.

Key Contributions

1. Reframing model-internal safety as a runtime-mediation problem. The authors separate the integrity of the release path from the effectiveness of installed evidence and rules, rather than treating interpretability signals as complete defenses in themselves.

2. The LMSM design. Stable interfaces among a selected backend (which exposes calibrated evidence), versioned rules over request-keyed state, and a gate that authorizes buffered output release. Backend, rule-set, and schedule changes stay local instead of forcing a rebuild of request handling or enforcement. The framework targets five goals: G1 mediated release, G2 request isolation, G3 separation, G4 independent evolution, and G5 intervention attribution.

3. A prototype implementation. LMSM is implemented for Hugging Face Transformers and continuously batched vLLM (with buffered_output() as the supported release boundary) hosting artifact-backed SAE and transcoder deployments and task-fitted dense probes, without changing base-model parameters.

4. An evaluation across serving conditions. The paper evaluates scheduler churn, backend substitution, rule composition, temporal behavior, policy effectiveness, and serving cost, including the result that LMSM-Checkpoint reduces HarmBench attack success rate from 39.20% to 3.32% on Qwen3-4B while retaining 98.14% of the throughput of a matched serving path with monitoring disabled at width 32.

Main Findings

• Mediation survives scheduler churn. All 32 duplicate request pairs preserve their action, selected category, intervention step, and complete vector of per-rule threshold-crossing outcomes as the scheduler moves and reuses batch slots.

• Backend substitution is local. Changing the backend binding does not require changes to request handling, the evaluator interface, or enforcement. The paper reports a dense-probe–transcoder substitution (Table 4(b)) and cross-model Gemma/SAE and Qwen/transcoder realizations (Table 4(a)) as evidence of this locality.

• Policy effectiveness on Qwen3-4B. Task-fitted policies reduce matched harmful-output rates by 82.5%–91.5%. LMSM-Checkpoint reduces HarmBench attack success rate from 39.20% to 3.32%, with XSTest false refusals rising from 2.40% to 4.40%.

• Serving cost is modest. Relative to the same serving path with monitoring disabled, the Checkpoint policy retains 98.14% throughput at width 32, and in a separate composition campaign retains 96.89% with all 15 rules active. Rule-composition cost is measured against the Matched Empty Extension baseline with 1, 6, and 15 active rules.

• Two enforcement schedules coexist. LMSM-Checkpoint retains fixed prefill and early-generation summaries and evaluates its 15 rules once after generated token 64; LMSM-Anytime maintains a running-prefix representation, evaluates its separately fitted rules after every decode step, and acts at the first threshold crossing. Both use the same fixed-OR composition and active-rule-order tie-breaking.

• Composition is two-layered. A rule's condition c_i combines evidence within one rule, while fixed OR resolves triggers across active rules; any triggered rule causes an intervention, with declared active-rule order providing deterministic action and attribution when several rules trigger together.

• Only the gate changes state. The policy evaluator returns a per-request decision but cannot mutate the model, terminate the request, or authorize output release; only the enforcement gate applies the decision and authorizes the response released by the runner. Three decisions are supported: allow, terminate, and refuse.

• Scoped security claims. The authors explicitly distinguish mediation correctness (an architectural and systems property) from policy effectiveness (an empirical property of a particular deployment), and state that LMSM does not inherit kernel isolation or tamper resistance, that identifier authentication and collision management are the trusted host's responsibility, and that arbitrary token streaming is not covered.

Methodology in Plain English

The authors start from an analogy. In an operating system kernel, the reference-monitor principle says every security-relevant operation must cross a trusted mediator before it changes protected state, and Linux Security Modules turned that principle into a stable interface: the kernel supplies trusted context, a pluggable module supplies a policy decision, and the kernel enforces the result. LMSM ports that division of labor to LLM serving, with buffered output release as the protected transition.

Before serving starts, the deployment operator installs three things into the control plane: backend bindings (which artifact or method to use, on which hook and activation site), a versioned library of atomic rules (each pairing one target with evidence channels, a calibrated condition, and a candidate action), and one active policy bundle (which rules are active, the temporal schedule, the rule order, the resolution mapping, and a version). Untrusted requests cannot select their own hook, backend, threshold, schedule, or action.

During generation, the data plane works in one batched pass: the serving runtime performs one packed model forward over all active requests, a batched backend call maps the configured activations to one aligned batch of evidence vectors, per-request state is updated under the trusted host's request identifier, and the evaluator returns an aligned decision vector. Because the state is keyed by request identity rather than by transient packed-row position, a request's evidence follows it as rows shift, is deleted when it finishes, and starts empty for newly admitted requests. Only the enforcement gate then applies decisions, so stopping one request does not cancel or recompute the rest of the batch. Interventions synchronously create records linking the request, rule and target, evidence, policy metadata, action, and token views.

Backends come in two provenances: artifact-backed (an existing released representation such as an SAE or transcoder is bound and calibrated) and task-fitted (the backend is learned or substantially adapted for the deployment target), where the dense probes are topic-specific, class-balanced logistic probes over Qwen3-4B layer-24 MLP-input activations of dimension 2,560. The paper reports that the truncation of the paper's short summary includes the statement that the evaluated prototype runs on one GPU; further hardware detail is not reported in the provided content.

Why This Matters

Impact on research. The paper argues that interpretability advances currently arrive as standalone guards, each bringing its own decision and action code, so that adding a backend recreates the guard stack. LMSM offers a common path from interpretability and model-internal analysis to runtime enforcement, which reframes progress in that area as an incremental improvement to a deployed defense rather than a new integration project. It also makes the mediation/effectiveness distinction explicit, so a correct release path is not mistaken for a correct policy.

Real-world applications:

• Regulated advisory services. The paper's own telehealth example: a service deploys an already aligned model but must keep it from issuing patient-specific dosage instructions, a restriction set by its jurisdiction rather than by the model provider. The operator binds a released SAE or transcoder if it exposes a representation aligned with the target, or fits a backend from in-domain examples otherwise.
• Deployments with operator-specific obligations. Because the active-rule set is a deployment choice rather than a fixed property of the backend or rule library, operators can reuse the same bindings and library while activating different rule subsets, and can leave rules inactive for targets the base model already covers.
• Multi-tenant or high-throughput serving.* The evaluation targets continuously batched vLLM with per-request selective enforcement, which is the setting where one request's intervention must not disturb others sharing a packed batch.
• Audit and compliance workflows. Each intervention produces a record identifying the request, selected rule and target, active rules, policy and rule-library versions, scalar score and threshold, decode step, action, and token views, supporting intervention attribution (goal G5).

Industry relevance. Serving cost is the practical gate for any runtime monitor, and LMSM reports retaining 98.14% of unmatched-path throughput at width 32 and 96.89% with all 15 rules active, on top of a strongly reduced HarmBench attack success rate (39.20% to 3.32%) at a modest false-refusal increase on XSTest (2.40% to 4.40%). The design targets the operational concern that interpretability artifacts, model targets, and enforcement policies change at different rates, and that backend, rule-set, and schedule changes should remain local.

Future Directions

• Trust-boundary hardening. LMSM trusts the serving process, the model, backend artifacts, policy bundle, and enforcement implementation, and does not authenticate host-supplied request identifiers or resolve collisions. The authors place these responsibilities with the trusted host; extending guarantees beyond that boundary is left open.
• Coverage beyond the current release path. The paper states its claims do not cover arbitrary token streaming, and the evaluated prototype implements three decisions (allow, terminate, refuse) for the single protected transition of buffered output release.
• Better evidence and better calibration. Because policy effectiveness is empirical and deployment-specific, follow-up work could ask how much of the measured effect comes from backend choice versus rule calibration, and whether better artifacts can be dropped in without touching the runtime.
• Generalizing mediation points. The current prototype mediates one transition. Whether the same substrate can host additional protected transitions in a longer serving path, while preserving the separation goals, is not answered by the reported content.

Target Audience

Researchers and practitioners in LLM security, systems security, and ML serving infrastructure. It is most useful to readers who already understand continuous batching and interpretability artifacts, since the paper's distinctive move is architectural rather than a new detection method. Deployment operators, platform engineers, and safety reviewers responsible for configuring what a deployed model may release will also find the policy/evidence/enforcement separation and the intervention-record schema directly relevant.

Authors’ abstract

Large language models (LLMs) are increasingly deployed with layered defenses, yet malicious prompts can still bypass them. Interpretability methods can expose model-internal signals along the generation path that could inform enforcement, but these signals are not security controls by themselves. Deployments that adapt them for safety typically couple each signal to its own calibration, policy logic, and intervention code, so each new artifact creates integration work instead of strengthening a shared defense. We present Language Model Security Modules (LMSM), a security framework that adapts the separation behind Linux Security Modules (LSM) to LLM serving. In LMSM, a selected security backend exposes calibrated evidence, a versioned policy evaluates active rules over trusted per-request context, and a separate gate authorizes buffered output release. This design separates mediation correctness from policy effectiveness, and it allows backend, rule, or schedule changes without rebuilding request handling or enforcement. Our prototype shows the separation working in practice: with Hugging Face Transformers and continuously batched vLLM, the same substrate hosts artifact-backed sparse autoencoder (SAE) and transcoder deployments and task-fitted dense probes, preserves request-specific decisions under scheduler churn, and selectively enforces and composes multiple rules per request. On Qwen3-4B, LMSM-Checkpoint reduces HarmBench attack success rate from 39.20% to 3.32%, with XSTest false refusals rising from 2.40% to 4.40%, while retaining 98.14% of the throughput of a matched serving path that performs no monitoring work at 32 active sequences. LMSM gives advances in interpretability and model-internal analysis a common path to runtime enforcement.