Improving Forecasts of Suicide Attempts for Patients with Little Data

AI summary

Overview

Research area: Machine learning for health, specifically time-series forecasting of suicide-related events from ecological momentary assessment (EMA) smartphone data.

Technical level: Advanced. The paper assumes familiarity with Gaussian processes, variational inference, ELBO derivations, kernel design, and latent-variable models, though the motivating clinical problem is described accessibly.

Scope in one sentence: The paper diagnoses why suicide-attempt forecasts fail for patients with sparse data and proposes a Latent Similarity Gaussian Process (LSGP) that lets data-poor patients borrow forecasting trends from similar patients in an inferred latent space.

What This Paper Is About

Ecological Momentary Assessment studies survey patients multiple times per day about suicidal urges, intent, and affect, creating an opportunity to forecast imminent suicide risk in time to intervene. Two problems make this hard: suicide attempts are extremely rare (so there is little positive data to learn from), and patients are heterogeneous (so a single model fit to everyone performs poorly). The authors show that per-patient ("idiographic") models do better than one global model but overfit when a patient has little data, and they propose modeling patient similarity as a continuous, learned latent space so that sparse-data patients can draw on trends from similar peers.

Key Contributions

1. Demonstrating that patient heterogeneity extends from suicidal ideation to suicide-related events (SREs). The authors show that idiographic (per-patient) models outperform a single model fit to all patients across nearly all baselines and metrics, implying patients have conflicting classification boundaries that degrade a pooled model.

2. Showing that discrete groupings are a poor fit for this heterogeneity. An experiment that randomly assigns patients to G groups and fits one model per group shows that performance improves as G grows — even with random assignment — and that demographic-based groupings perform worse than random ones on nearly all metrics.

3. Introducing Latent Similarity Gaussian Processes (LSGPs). The model places each patient's latent variable zₙ in a latent space where distance reflects similarity in forecasting trajectory, concatenates zₙ with the patient's EMA responses, and feeds the result into a GP classifier. The authors state that LSGPs (a variant of GP with latent covariate, multi-group GPs, and meta-learning GPs) have never previously been applied in this context.

4. Providing a graph-based view of learned patient similarity and preliminary forecast results. Patient covariance is treated as an adjacency matrix and visualized; without kernel design or hyperparameter search, the sparse variational LSGP (SV-LSGP) outperforms all baselines except one.

Main Findings

• Idiographic models beat single models, except on specificity. In Figure 1, across all baselines and metrics other than specificity (which stayed constant), the difference between idiographic and single models was almost always positive. In Table 1, for example, single LR achieves ROC-AUC 0.68 ± 0.01 and sensitivity 0.05 ± 0.01, while idiographic LR reaches ROC-AUC 0.85 ± 0.01 and sensitivity 0.36 ± 0.02.

• Patients with less data get worse forecasts from both model types. Figure 1's bottom row shows that the 30% of patients with the fewest data points consistently receive worse forecasts across most metrics and most models than the 30% with the most data (except for sensitivity). Single models are dominated by high-data patients and generalize poorly; idiographic models overfit the sparse ones. The single-model ED/EB-LR methods show the sharpest stratification in Table 1 (bottom third log-likelihood −1.87 ± 0.00 versus top third −1.09 ± 0.00).

• Even random grouping boosts performance. Splitting patients randomly into G groups, fitting a separate model per group, and increasing G improved all metrics except specificity (accounted for by increased sensitivity), repeated 10 times per value of G.

• Demographic groupings underperform random groupings. Models trained on demographic-grouped patients performed worse than randomly grouped patients on nearly all metrics, raising the question of whether demographics explain the heterogeneity at all.

• SV-LSGP nearly matches the best baseline without tuning. In Table 1, SV-LSGP achieves average log-likelihood −0.50 ± 0.01 (bottom third), −0.40 ± 0.01 (middle third), −0.29 ± 0.01 (top third), −0.35 ± 0.01 (all), ROC-AUC 0.85 ± 0.01, PPV 0.73 ± 0.01, sensitivity 0.37 ± 0.02, and specificity 0.97 ± 0.00. The only baseline it does not beat is idiographic VB-LR (log-likelihood −0.47 ± 0.01 bottom, −0.38 ± 0.01 middle, −0.28 ± 0.01 top, −0.33 ± 0.01 all; ROC-AUC 0.87 ± 0.00; PPV 0.73 ± 0.01; sensitivity 0.42 ± 0.02; specificity 0.96 ± 0.00).

• Demographics do not explain learned patient similarity. Modularity Q on the similarity graphs was 0.12 for gender, 0.13 for sexual orientation, 0.08 for age, and 0.15 for highest completed education — all close to 0, indicating balanced within/between-group connections. In nearly all graphs, the largest similarities were between patients of different groups.

• The model learns 3 latent dimensions. The reported configuration uses D_z = 3, meaning patient similarity is summarized by only three latent coordinates.

Methodology in Plain English

The researchers first ran a diagnostic comparison. They took EMA survey responses (0–10 Likert-scale ratings of affect, suicidal urge/intent, and behavior) and asked whether a patient would experience a suicide-related event (SRE) — a self-injurious behavior with non-zero intent to die, or a hospital presentation with suicidal thoughts to prevent an attempt — within the week following a given survey. They compared one model trained on everything ("single") against one model per patient ("idiographic"), using KNN, logistic regression (plain, empirical-Bayes, and variational), and Gaussian process classification, scored on average log-likelihood, PPV, ROC-AUC, specificity, and sensitivity.

To capture heterogeneity without splitting patients into discrete boxes, they give each patient a latent vector drawn from a standard normal prior. That vector is concatenated onto each of the patient's survey responses, so the model sees both what the patient reported and who the patient is in latent terms. A Gaussian process with a kernel over these concatenated inputs produces a latent score, which a Bernoulli likelihood with a sigmoid link converts to a probability of an SRE. Because the likelihood is non-Gaussian and the dataset is large, exact inference is impossible; the authors use sparse variational inference with 2,000 inducing points, a mini-batch size of 150, 15,000 gradient steps, learning rate 0.005, and a Monte Carlo approximation of the expectation in the ELBO, implemented in NumPyro and Jax. The kernel factorizes into an ARD kernel on the latent coordinates and a "state-dependent" linear kernel whose parameters are neural networks, allowing patients in different latent regions to have different priors.

Finally, they interpret the learned latent space as a graph: patients are nodes, and the covariance between their latent variables is the edge weight. Pruning weak edges and coloring nodes by demographic group visually shows whether similar patients share demographics, and modularity Q (ranging from −1 to 1) quantifies this numerically.

The data came from 623 recruited participants across two Boston-area hospitals — 315 adults (18+) from a psychiatric emergency service and 308 adolescents (12–19) from a psychiatric inpatient unit — surveyed 6 times per day for three months. After keeping only patients with at least 3 SREs and 3 non-SREs, the analysis used N = 77 patients and 14,763 complete EMA surveys, split 50%/25%/25% into train/validation/test across 5 different cuts, with each method run 5 times per cut using random seeds.

Why This Matters

Impact on research. The paper provides concrete evidence that suicide-risk prediction is not a single-population problem. It also offers a cautionary methodological result: because even random patient groupings improve performance, subgroup-discovery methods applied to this kind of data risk finding structure that is not clinically meaningful — and their conclusions could be misleading.

Real-world applications:

• Just-in-time adaptive interventions in smartphone mental-health apps, where forecasts for newly enrolled or low-engagement users must be made before enough personal data accumulates.
• Clinical decision support in emergency departments and inpatient psychiatric units, where the cohort studied here was recruited.
• Risk-monitoring triage, since the source study used a real-time risk-monitoring team to intervene when participants indicated high suicidal intent.
• Personalized care planning, by identifying which patients share underlying risk trajectories and might respond to shared mechanisms of intervention.

Industry relevance. Digital mental health platforms, wearable and smartphone-based monitoring products, and health systems that collect intensive longitudinal patient data all face the cold-start problem this paper targets. The latent similarity graph also suggests a route to auditing whether demographic features alone are sufficient for personalization — which matters for fairness and for avoiding misleading subgroup claims in regulated health products.

Future Directions

1. Kernel design and inductive bias. The authors expect that investigating the inductive biases of different kernels will let the LSGP outperform all baselines, since it generalizes both the GP and LR-based baselines evaluated here.

2. Searching for latent dimensions that larger numbers of patients might occupy. The reported D_z is only 3; exploring the latent space more richly is an open avenue.

3. Identifying non-demographic factors that define patient similarity. Since modularity was close to 0 for gender, sexual orientation, age, and education, the authors hope to explore similarity based on other factors to understand their clinical significance.

4. Handling the practical constraint on deployment. Patients were included only if they had at least 3 SREs and 3 non-SREs, and the authors state this means, for the method to be used in practice, patients must have at least one recorded SRE in their data — a substantial limitation for genuinely cold-start patients.

Target Audience

This paper is most valuable to machine learning researchers working on time-series or longitudinal health data (particularly the TS4H workshop audience), clinical researchers and statisticians studying suicide risk prediction from EMA, and methodologists interested in latent-variable Gaussian process models and subgroup-discovery pitfalls. Digital mental health product teams and psychiatric clinicians with data-science exposure will also find the framing useful, though the math is dense. Statistical ML practitioners will need comfort with variational inference and kernel methods to follow Section 3 in full.

Authors’ abstract

Ecological Momentary Assessment provides real-time data on suicidal thoughts and behaviors, but predicting suicide attempts remains challenging due to their rarity and patient heterogeneity. We show that single models fit to all patients perform poorly, while individualized models improve performance but still overfit to patients with limited data. To address this, we introduce Latent Similarity Gaussian Processes (LSGPs) to capture patient heterogeneity, enabling those with little data to leverage similar patients' trends. Preliminary results show promise: even without kernel-design, we outperform all but one baseline while offering a new understanding of patient similarity.