Anonpsy: A Graph-Based Framework for Structure-Preserving De-identification of Psychiatric Narratives

AI summary

Overview

• Research area: Clinical natural language processing, specifically privacy-preserving de-identification of psychiatric clinical narratives.
• Technical level: Advanced (assumes familiarity with LLM pipelines, semantic graphs, mixed-effects statistical modeling, and clinical de-identification concepts).
• Scope: The paper introduces Anonpsy, a framework that converts psychiatric case narratives into perturbable semantic graphs and regenerates de-identified text, evaluated against a strong LLM-only rewriting baseline using five board-certified psychiatrists and GPT-5-based automatic evaluation.

What This Paper Is About

Psychiatric case narratives reveal who a patient is not only through explicit identifiers like names and dates, but through idiosyncratic life events embedded in the structure of the clinical story. Existing de-identification methods — token-level PHI masking and LLM-based synthetic rewriting — work on raw text and give little control over which semantic elements survive rewriting. The authors reframe de-identification as a structure-preserving generation problem: represent each case as a temporal, typed semantic graph, perturb only the identifying context, then regenerate the narrative from the edited graph.

Key Contributions

1. Structured formulation. The authors formulate psychiatric narrative de-identification as a structured generation problem and introduce a dynamically editable semantic graph representation of clinical entities, temporal anchors, and typed relations.
2. Graph-constrained perturbation and generation framework. They propose a pipeline that selectively alters identifying narrative content (demographics, symptom-associated context, numeric test values) while preserving temporal, diagnostic, and causal structure, then regenerates text conditioned on the perturbed graph.
3. Expert and GPT-5 evaluations. They report evaluations with five board-certified psychiatrists on 20 cases together with a GPT-5-based automatic evaluation on all 90 cases, comparing Anonpsy against the original narratives and a strong prompt-engineered LLM-only rewriting baseline.
4. Component-level ablations. They isolate the roles of STEB (Situation-Thought-Emotion-Behavior) decomposition and retrograde perturbation ordering, showing these components play complementary roles in privacy protection and structural fidelity.

Main Findings

• Diagnostic fidelity is preserved under expert evaluation. For the soft-F1 scoring of diagnoses written by raters, group means were Anonpsy 0.719, LLM-only 0.743, and Original 0.725. Fixed-effect contrasts against Anonpsy were non-significant (LLM-only vs. Anonpsy β = 0.024, p = 0.699; Original vs. Anonpsy β = 0.005, p = 0.932), and Wilcoxon tests on per-case and per-input-type averages also found no significant differences.
• Narrative–diagnosis congruence stays high. The mean probability of a "yes" response to whether the ground-truth diagnosis was consistent with the case was 0.900 for Anonpsy and 0.960 for both Original and LLM-only, with posterior 95% credible intervals for all narrative-type contrasts including zero.
• Anonpsy carries substantially lower re-identification risk. Across 50 rater–case evaluations, mean paired risk score was 2.800 for LLM-only versus 1.540 for Anonpsy, with medians of 3.000 and 1.000 respectively (Wilcoxon W = 84.5, p < 0.001). No Anonpsy narrative received a score above 3, and no case showed majority agreement among raters that the Anonpsy output was at moderate risk (score ≥ 3); only one case received a score of 3 by two raters.
• Risk differences concentrate in narratives that evoke the original. When a version was chosen as more similar to the original, mean risk was 3.122 for LLM-only (n = 41) versus 2.111 for Anonpsy (n = 9; Mann–Whitney U = 280.0, p = 0.012). For non-chosen versions, mean risks were nearly indistinguishable (LLM-only 1.333; Anonpsy 1.415; p = 0.731). LLM-only was selected as more similar in 41 of 50 trials (82%; binomial p < 0.001).
• Moderate-risk ratings were driven by rare clinical features, not narrative resemblance. In the only case where two raters gave a score of 3, both attributed their judgment to a rare diagnosis (Lyme disease) rather than preserved events, quotations, or locations. No rater identified an unchanged concrete anchor such as an exact date, location, or unique event in an Anonpsy output.
• GPT-5 evaluation on all 90 cases mirrors the human findings. GPT-5 showed no significant differences between Anonpsy, the original narratives, and the LLM-only baseline on diagnosis prediction and diagnosis acceptability, while embedding-based semantic similarity and a GPT-5 insider simulation favored Anonpsy (more semantically distant from originals, lower re-identification risk scores than LLM-only rewrites).
• STEB decomposition mainly buys privacy. Removing STEB (NoSTEB) left diagnostic predictability (Wilcoxon p = 0.981) and acceptability (McNemar p = 0.677) statistically unchanged, but significantly increased semantic cosine similarity (0.718 → 0.746, p < 0.001) and GPT-5 re-identification risk scores (p < 0.001).
• Retrograde perturbation ordering mainly buys structural coherence. Removing it (NoRetro) significantly reduced diagnostic predictability (soft-F1 0.575 → 0.525, p = 0.048) and acceptability (0.765 → 0.644, p = 0.032) with no significant change in cosine similarity (p = 0.305) or risk score (p = 0.748).
• Trade-off premise check positions Anonpsy on the recallability–structure plane. Across methods, PHI masking had the highest semantic cosine similarity to originals (mean = 0.902), SDC was moderately reduced (mean = 0.796), and Anonpsy was lowest (mean = 0.718). Original narratives scored a mean soft-F1 of 0.665 (SD = 0.276) on diagnosis preservation; the report of PHI masking and SDC soft-F1 values is cut off in the provided text, so those figures are not reported here.
• Errors are localized, not structural. The primary error source is conservative entity inclusion during graph conversion, where symptom-like descriptors (e.g., "she felt depressed") may be retained even when not explicitly diagnosed. Because perturbation and generation operate on validated graph structures, such inconsistencies remain localized.

Methodology in Plain English

The framework has three operators, all implemented as hybrid systems combining deterministic rules with schema-constrained prompting of a single locally deployed LLM, gpt-oss:120b, accessed through a ChatOllama backend. Lower decoding temperatures are used for schema-constrained extraction and narrative realization; a higher temperature is used for perturbation to encourage semantic diversity. All runs used a local machine with four NVIDIA RTX A6000 GPUs, each with 48 GB VRAM, and no proprietary or closed API systems.

1. Conversion (ℰ). A four-stage pipeline maps free text into a semantic graph: schema-guided entity extraction, temporal episode construction and normalization, deterministic temporal reconciliation, and typed relation construction. Node types include diagnoses, symptoms, treatments, past history, and a structured VisitEvent that anchors all temporal offsets. Symptom episodes are decomposed using the STEB schema (Situation-Thought-Emotion-Behavior), derived from the cognitive model of psychopathology. Durations carry onset offsets and spans relative to an index encounter at day 0, encoded as signed integers so ordering works without calendar dates. Relations include MANIFESTS_AS, TREATMENT_OF, PRESENTS_WITH, and INDUCES, with permissible source-target type pairs fixed in advance.
2. Perturbation (𝒫). The graph is edited only in attribute and context fields; the temporal backbone and typed relations are untouched. Demographic attributes are perturbed under clinically informed feasibility constraints (age perturbations must remain consistent with onset thresholds and duration requirements of DSM-5-TR diagnoses; sex perturbations respect sex-specific diagnostic labels), STEB contextual narratives are rewritten under constraint, and numeric test values are substituted within range. Policies are deterministic and diagnosis-informed rather than learned.
3. Generation (𝒟). Events are sorted by temporal anchor and associated diagnoses to build a deterministic timeline; symptoms are grouped by linked diagnosis, and treatments targeting the same diagnosis with overlapping durations are narrated in the same block. The perturbed graph is serialized into a structured prompt, and the LLM realizes the narrative following that structure without introducing new entities.

Evaluation setup. The corpus is DSM-5-TR Clinical Cases, a publicly available collection of clinician-authored psychiatric narratives published by the American Psychiatric Association. Of 92 available cases, two multi-person narratives (a couple-therapy case and a shared-trauma vignette) were excluded, leaving 90. Three variants were compared: the Original narrative, the Anonpsy output, and an LLM-only rewrite generated by the same backbone with the same decoding configuration, carefully prompt-engineered to preserve diagnoses, temporal ordering, causal relations, and symptom trajectories while modifying demographics, locations, interpersonal details, and distinctive life events, plus a self-critique refinement stage. Human evaluation involved five board-certified psychiatrists (recruited via social media, certification verified at enrollment, informed consent, nominal honorarium) on 20 cases in randomized, blinded order.

Why This Matters

• Impact on research. The paper argues that no prior work has used a temporal event graph with typed relations as an intermediate representation for controlled generation in psychiatric de-identification. Existing approaches either operate at the token level and ignore higher-level clinical structure, or rely on direct text-to-text rewriting without a controllable intermediate representation. This work makes "structure" an explicit, editable object in the de-identification loop.
• Real-world applications:
  • Sharing psychiatric case material for research or education while reducing the chance that a knowledgeable reader can link a narrative back to a specific person.
  • Building de-identified case repositories that still support downstream tasks such as diagnosis prediction and risk stratification.
  • Deploying privacy tooling in clinical environments where cloud LLM APIs are policy-restricted, since the whole pipeline runs on a local model.
  • Producing synthetic teaching cases that preserve DSM-5-TR diagnostic logic without reproducing distinctive patient life events.
• Industry relevance. Health systems, clinical NLP vendors, and research consortia that need to move narrative clinical text across institutional boundaries face both regulatory pressure (HIPAA, GDPR) and a need to keep the text diagnostically useful. This work suggests that graph-based intermediate representations with deterministic constraints can be a tractable design choice in privacy-sensitive deployments, and that a locally deployed open-weight model can serve as the backbone.

Future Directions

1. Test on messier real-world documentation. The evaluation uses a curated educational corpus with coherent narrative flow and fully specified diagnostic labels. Real documentation is more heterogeneous, and extremely brief or fragmented notes with limited narrative context or temporal structure may provide insufficient information for episode-level graph normalization, potentially requiring preprocessing or schema adaptation.
2. Broaden human expert evaluation. The human evaluation covers 20 cases because of the specialized expertise required; broader studies with more clinicians and more diverse datasets would strengthen generalizability.
3. Formalize perturbation as optimization. Perturbation policies are currently clinically informed heuristic rules rather than a formally optimized objective; the authors suggest recasting structure-preserving de-identification as a constrained optimization or invariance-preservation problem.
4. Compare against more systems. The empirical comparison focuses on PHI masking and one carefully controlled LLM-only rewriting baseline, isolating the effect of graph constraints; comparisons with additional de-identification systems or privacy-preserving text generation frameworks remain open.
5. Address residual risk. Risk may persist for cases involving rare diagnoses or highly distinctive symptom profiles, and the authors state de-identified narratives should not be treated as risk-free substitutes for real clinical records without additional safeguards.

Target Audience

Clinical NLP and health-informatics researchers working on de-identification, synthetic clinical data generation, or privacy-preserving text processing; psychiatrists and clinical informaticists involved in data-sharing governance; and engineers building local or on-premises LLM pipelines for regulated healthcare settings. Readers will benefit most if they are comfortable with graph representations, LLM prompt engineering, and mixed-effects statistical analysis, though the high-level design logic is accessible without that background.

Authors’ abstract

Psychiatric narratives encode patient identity not only through explicit identifiers but also through idiosyncratic life events embedded in their clinical structure. Existing de-identification approaches, including PHI masking and LLM-based synthetic rewriting, operate at the text level and offer limited control over which semantic elements are preserved or altered. We introduce Anonpsy, a de-identification framework that reformulates the task as graph-guided semantic rewriting. Anonpsy (1) converts each narrative into a semantic graph encoding clinical entities, temporal anchors, and typed relations; (2) applies graph-constrained perturbations that modify identifying context while preserving clinically essential structure; and (3) regenerates text via graph-conditioned LLM generation. Evaluated on 90 clinician-authored psychiatric case narratives, Anonpsy preserves diagnostic fidelity while achieving consistently low re-identification risk under expert, semantic, and GPT-5-based evaluations. Compared with a strong LLM-only rewriting baseline, Anonpsy yields substantially lower semantic similarity and identifiability. These results demonstrate that explicit structural representations combined with constrained generation provide an effective approach to de-identification for psychiatric narratives.