Breaking Up with Normatively Monolithic Agency with GRACE: A Reason-Based Neuro-Symbolic Architecture for Safe and Ethical AI Alignment

AI summary

Overview

Research area: AI safety, machine ethics, value alignment, and neuro-symbolic AI architectures — specifically the design of agent architectures that separate normative (moral) reasoning from instrumental (goal-seeking) decision-making.

Technical level: Advanced. The paper is a conceptual and architectural proposal that leans on formal logic, defeasible/default reasoning, deontic logic, and action theory. Readers without a background in symbolic AI or normative philosophy will find the formalism demanding, though the paper's motivating argument (the "flattening problem") is stated in accessible terms.

Scope: The paper proposes a modular reason-based "governor" architecture called GRACE, formalizes its three modules and their interfaces, and illustrates it conceptually through a single running example (an LLM therapy assistant called TherapAI) — without reporting empirical results.

What This Paper Is About

Most current AI agents compress instrumental decision-making and normative constraints into one opaque policy function, which the authors call the "flattening problem." This erases accountability, blocks auditing, and makes it hard for stakeholders to contest or revise an agent's decisions in morally charged contexts. The paper's goal is to describe GRACE (Governor for Reason-Aligned Containment), an architecture that structurally decomposes decision-making into a symbolic moral reasoner, the original (possibly neural) decision-making agent, and an enforcement guard, so that moral reasoning stays explicit, interpretable, and contestable while the underlying agent stays intact.

Key Contributions

1. A diagnosis of "flattening." The paper names the flattening problem — the collapse of instrumental rationality, normative constraints, and their integration into a single policy π — and enumerates four resulting pathologies: opacity, brittleness, contestability deficit, and verification challenge.

2. GRACE, a three-module containment architecture. The Governor for Reason-Aligned Containment decouples normative reasoning from instrumental optimization via a Moral Module (MM) that derives permissible macro action types, a Decision-Making Module (DMM) that encapsulates the target agent and proposes instrumentally optimal primitive actions, and a Guard (G) that enforces moral compliance. The architecture is described as able to contain AI agents of "virtually any design" without modifying the encapsulated core agent.

3. A reason-based foundation for the Moral Module. Rather than fixed rules or reward feedback alone, the MM is built on normative reasons, refining Horty's default-logic reason theory ⟨W, D, <⟩ into a parametrized reason theory ⟨R, D, <⟩ that is instantiated into a situation-specific reason model ⟨R↓, D↓, <↓⟩. This gives the system interpretable justification, generalizability, defeasibility, and formal tractability, and lets the architecture function as a deontic logic that determines which macro action types are justified.

4. The Moral Advisor (MA) and case-based moral learning. A Moral Advisor — a human overseer, expert committee, domain-specific normative system, or mixture — supplies case-based, reason-based feedback of the form "action a was (not) permissible in state s for reason P; instead φ was expected." This lets the MM incrementally build a coherent reason theory and remain adaptable to evolving normative standards.

Main Findings

• No empirical results are reported. The paper states explicitly that its conceptual validity is "illustrated through concrete scenarios, laying the groundwork for future empirical evaluation." Empirical demonstrations are referenced only as separate, prior/companion proofs of concept applying GRACE to (a) reinforcement learning and (b) large language models, attributed to other works (Dargasz 2026a, 2026b) rather than presented here.

• The flattening problem has four named pathologies. Opacity (grounds for decisions inaccessible, hindering auditing and trust), brittleness (implicit norms fail under distribution shift or novel moral contexts), contestability deficit (no meaningful intervention points for stakeholders), and verification challenge (alignment checks must target the whole policy rather than separable components).

• Action abstraction is the core interface problem. The paper distinguishes primitive actions a ∈ A, macro actions (temporally extended sequences s₀a₁s₁a₂…), and macro action types (MATs) φ ∈ L, which are decidable predicates over macro actions. Normative reasoning applies to MATs, not to primitive token-level operations; the containment challenge is to ensure a ⊧ₛ φ for some permissible φ in state s.

• GRACE's decomposition is a multi-agent system, not just a pipeline. Each module m ∈ {MM, DMM, G} maintains its own internal state Bₘ, observes its own observation space Oₘ, and has its own action function πₘ, forming specialized, sequentially acting agents that compete yet cooperate.

• The Guard provides the enforcement layer. The DMM may be sub-symbolic, making hard guarantees difficult; the Guard is symbolic and formal at its core, monitoring and verifying permissibility of primitives, possibly supported by neural components for interpreting observations. Depending on the application, the monitor may be synthesized from the logical specifications — which, depending on agent and scenario, "may collapse into shielding," the Safe RL technique for preventing constraint-violating actions.

• TherapAI runs through five formalized cases. Case 1 introduces the LLM therapist's competing moral tensions (therapeutic effectiveness vs. patient safety, confidentiality vs. disclosure duty, autonomy vs. intervention, professional boundaries across cultures). Case 2 sets up two defaults — δ₁ = D(X) ⟶ φ_P(X) (protect privacy-sensitive data) and δ₂ = W(X) ⟶ φ_F(X) (follow expressed patient wishes) — with δ₁ < δ₂. Case 3 has the MM infer "protect data l" after a patient messages "I'm going to the park and will hurt myself," producing idle behavior that the MA corrects with feedback (idle, φ_R(X), F(X)), prompting a new default δ₃ = F(X) ⟶ φ_R(X) with δ₃ > δ₁. Case 4 has the patient add "please do not intervene in my plan!", producing two conflicting permissible MATs, φ_R(h) and φ_F(¬i), left unresolved by the priority ordering and pushed to the DMM's judgment under uncertainty. Case 5 has the Guard check whether the proposed primitive action call_number N satisfies a ⊧ₛ φ_R(h), and possibly reject it if N is not an emergency number.

• Reason-based framing is presented as a middle path. The paper argues current approaches present a false dichotomy: pure rule-based top-down systems (Neufeld et al.) are explicit and verifiable but struggle with context-sensitivity and dynamic adaptation; learning-based bottom-up approaches lack transparency and can exhibit value misalignment and reward hacking. A cited data point for the limits of bottom-up moral learning is the Delphi experiment, trained on 1.7M crowdsourced moral judgments, which achieved high accuracy on moral predictions but exhibited systematic biases.

• The framework is claimed to be domain-agnostic. Because normative reasons apply across normative domains, the paper argues GRACE can also address legal compliance, cultural conventions, organizational policies, and institutional requirements, not only ethics.

Methodology in Plain English

The paper is an architecture and design paper, not an experimental study. The authors proceed in four steps.

First, they build an argument: they characterize current agents as stateful systems with an update function and a policy function π, then argue that folding moral constraints into π conflates two fundamentally different kinds of reasoning ("oughts" that cannot be unified in one decision procedure) and diagnosing the resulting failure modes.

Second, they choose a normative foundation. Instead of hardcoding a fixed rule set or training on reward signals, they adopt normative reasons — facts that count in favor of courses of action — and implement them using Horty's default logic (a defeasible reasoning system), which they adapt by replacing plain factual premises with parametrized reasons inferred from observations. From a general reason theory, the system derives a situation-specific reason model for the current context, then applies defeasible inference to determine which action types are justified.

Third, they decompose the agent. The Moral Module applies the reason theory to observations and outputs a set of permissible macro action types, with optional justifications. The Decision-Making Module keeps the original agent's instrumentation and neural expressiveness, choosing among permissible macro action types and proposing primitive actions. The Guard checks whether the proposed primitive action actually falls under a permissible macro action type, blocking it and triggering re-decision if not. A Moral Advisor outside the system supplies correction when impermissible behavior slips through.

Fourth, they walk the design through a single extended scenario — an LLM therapy assistant — showing how feedback flows between modules and how the reason theory is revised after a mistake. The authors note that parts of a previous version of the architecture were described elsewhere in greater formal detail.

Why This Matters

Impact on research. The paper positions GRACE as an extension of the Guaranteed Safe AI research agenda (formal world models, safety specifications, verifiable control mechanisms) from the safety domain into the ethical domain. It argues for replacing monolithic policy architectures with structurally decomposed ones, and it offers a concrete alternative to both pure rule-based machine ethics and purely data-driven value learning. It also explicitly does not commit to a unique "ethical ground truth," which is significant because it means the same architecture can host pluralistic or changing normative views rather than freezing one.

Real-world applications:

• AI therapy and mental health support — the paper's running example; balancing therapeutic effectiveness against confidentiality, non-maleficence, autonomy, and professional boundaries.
• Legal compliance — the paper states the framework applies across normative domains, including legal requirements.
• Organizational policy and institutional requirements — the authors list organizational policies and institutional requirements as domains where reason-based governance applies.
• - Cultural conventions — also named as an applicable normative domain, relevant to agents deployed across diverse cultural contexts.
• Robotics and general autonomous agents — the paper defines AI agents broadly, "from future robots to LLM-based therapeutic assistants," and claims GRACE can contain agents of virtually any design without modifying the core agent.

Industry relevance. The architecture's selling point is containment: a deployed agent — for example a reinforcement learning system or an LLM — does not need to be retrained or redesigned. Instead, a symbolic governor wraps it, supplies permissible action types, and blocks non-compliant outputs. For organizations facing audit, certification, or regulatory requirements, the paper's claim is that this yields separable components that can be validated individually, plus formal verification and statistical guarantees of alignment enforced by the Guard, rather than an unverifiable whole-policy check.

Future Directions

• Empirical validation. The paper explicitly frames itself as laying groundwork for future empirical evaluation and reports no results of its own. The open question is how GRACE behaves in practice.

• Designing and validating the proof-of-concept implementations. Two proof-of-concept implementations — one for reinforcement learning and one for large language models — are cited as separate works (Dargasz 2026a, 2026b) and are not evaluated here, leaving their results as a distinct line of inquiry.

• Formal analysis of the feedback–theory relationship. The paper notes that the quality of the reason theory depends on the quality of Moral Advisor feedback, and flags this as "opening avenues for formal analysis."

• Conflict resolution among system components. A footnote describes a possible advanced version of the architecture in which the Moral Advisor resolves conflicts between system components arguing over the interpretation of norms and macro action types — a capability not developed in the main text.

• Working out when guarantees are achievable. The paper acknowledges that the exact methodology and nature of the guarantees possible depend heavily on the application, and that depending on agent and scenario, the Guard's monitor may collapse into shielding, or the permissibility check may be "easy and exact or probabilistic at best." Delineating which scenarios yield which level of guarantee remains open.

Target Audience

Researchers and practitioners working on AI safety, machine ethics, value alignment, and neuro-symbolic systems will get the most from this paper, particularly those interested in the Guaranteed Safe AI agenda or in architectures that keep normative reasoning symbolic and auditable. It should also interest AI governance and policy audiences concerned with auditability, contestability, and oversight of autonomous systems. Readers with a philosophical background in normative reasons, defeasible reasoning, or deontic logic will find the formal machinery familiar; readers seeking empirical benchmarks or deployed-system evaluations will not find them here, since the paper presents a conceptual architecture and an illustrative case study rather than measured results.

Authors’ abstract

As AI agents become increasingly autonomous, widely deployed in consequential contexts, and efficacious in bringing about real-world impacts, ensuring that their decisions are not only instrumentally effective but also normatively aligned has become critical. We introduce a neuro-symbolic reason-based containment architecture, Governor for Reason-Aligned ContainmEnt (GRACE), that decouples normative reasoning from instrumental decision-making and can contain AI agents of virtually any design. GRACE restructures decision-making into three modules: a Moral Module (MM) that determines permissible macro actions via deontic logic-based reasoning; a Decision-Making Module (DMM) that encapsulates the target agent while selecting instrumentally optimal primitive actions in accordance with derived macro actions; and a Guard that monitors and enforces moral compliance. The MM uses a reason-based formalism providing a semantic foundation for deontic logic, enabling interpretability, contestability, and justifiability. Its symbolic representation enriches the DMM's informational context and supports formal verification and statistical guarantees of alignment enforced by the Guard. We demonstrate GRACE on an example of a LLM therapy assistant, showing how it enables stakeholders to understand, contest, and refine agent behavior.