Dual-Scale Relational Graph Transformers for Ecosystem-Aware Fraud Detection

AI summary

Overview

• Research area: Machine learning for financial fraud detection, specifically graph-based deep learning applied to account takeover (ATO) fraud in digital banking.
• Technical level: Advanced. The paper assumes familiarity with graph transformers, attention mechanisms, heterogeneous graphs, and production fraud-model evaluation metrics.
• One-sentence scope: The paper introduces Hermes, a dual-scale graph transformer that jointly models a session's local heterogeneous relational neighborhood and the evolving global fraud ecosystem, evaluated on more than 130 million high-risk transaction sessions from a leading U.S. financial institution.

Publication details as stated: arXiv:2610.04138v1 [cs.LG], published 02 Oct 2026, prepared for the Geometric Distributional Deep Learning (GDDL) workshop, licensed CC BY 4.0. Authors are affiliated with the University of Kansas (Lawrence, KS, USA) and AI Foundations, Capital One (McLean, VA, USA).

What This Paper Is About

Account takeover fraud occurs when an attacker gains access to a legitimate user's account, and detecting it forces a difficult tradeoff: flagging more sessions catches more fraud but also challenges more legitimate customers, creating friction. Production systems typically score each session in isolation using tabular models, discarding the relational structure that connects sessions through shared accounts, devices, and network identifiers. The paper's goal is to build a single model that reasons over both the local relational neighborhood around a session and the broader, shifting state of the fraud ecosystem, while strictly using only information that would be available at the moment a session is scored.

Key Contributions

1. Hermes, a dual-scale architecture. Hermes (HEterogeneous Relational Micro–macro graph transformer Encoder for high-risk Sessions) fuses a local graph representation with a global ecosystem representation in a unified framework for session-level fraud prediction, trained end-to-end.
2. Micro-GT, a local relational graph transformer. It applies four structured attention masks in sequence (identity-self, relation-group, chain, and full-context) over a temporally safe multi-hop neighborhood, preserving the distinct semantics of account, device, and network relations while capturing higher-order interactions across heterogeneous paths.
3. Macro-GT, an ecosystem-scale transformer. It attends over non-anticipative daily "climate" tokens spanning global fraud activity, platform-specific behavior, cross-platform shifts, velocity and anomaly signals, ecosystem topology, and the current risk regime, augmented with adaptive class prototypes that track representative fraud and benign session patterns over time.
4. Production-scale evaluation. The paper reports a large-scale study on more than 130 million high-risk transaction sessions from a leading U.S. financial institution, including component ablations and temporal-stability analysis across four held-out test months.

Main Findings

• Best overall performance: Hermes achieves 85.12% overall AUC, 83.46% P1 AUC, 81.35% P2 AUC, 5.373% FR@50R, and 48.91% R@5FR, the best result across all reported metrics in the paper's ablation table.
• Gains over the production system: Relative to the production XGBoost system (80.26% overall AUC, 78.05% P1 AUC, 74.10% P2 AUC, 9.671% FR@50R, 39.24% R@5FR), Hermes achieves a 44.44% relative reduction in customer friction and a 24.66% relative improvement in fraud recall.
• AUC point improvements: Compared with production, Hermes improves overall, P1, and P2 AUC by 4.86, 5.41, and 7.25 points respectively.
• Tabular baselines plateau: MLP performs nearly identically to production XGBoost (80.29% AUC), and FT-Transformer underperforms XGBoost at 79.16% AUC overall and 66.54% on P2.
• Sequential context is insufficient alone: Transformer-Seq improves overall AUC to 81.95% but remains weak on P2 at 72.97%.
• Graph structure helps: GraphSAGE (GNN) reaches 82.00% overall AUC and 76.20% P2 AUC, reducing FR@50R by 10.03% relative to production.
• Label propagation is a strong baseline: GraphSAGE + LP (ATLAS) reaches 84.02% overall AUC, 82.45% P1 AUC, and 78.75% P2 AUC, reducing FR@50R by 34.59% and improving R@5FR by 19.62% relative to production.
• Complementary component gains: Each successive component improves results — Micro-GT (full attention) reaches 84.58% AUC; Micro-GT (relational masks) reaches 84.78% AUC and 83.10% P1 AUC; adding Macro-GT (climate tokens) reaches 84.93% AUC and 81.06% P2 AUC with FR@50R of 5.519% and R@5FR of 48.44%; adding adaptive class prototypes produces the final Hermes results.
• Temporal stability: Across four consecutive held-out test months, Hermes consistently outperforms all baselines with no degradation relative to the graph-based models in later months, indicating robustness across changing fraud regimes.
• Extreme imbalance context: Fraud accounts for less than 0.1% of all sessions in the dataset.

Methodology in Plain English

The approach has three main ideas.

Building a temporally safe graph. Each high-risk session is a node. Sessions that share an account, device, or network identifier are linked, but only backward in time: the model may look at earlier sessions, never future ones. Identifiers are treated as their own time-conditioned nodes representing "the state of this identifier at the moment the target session is scored," so their representations evolve as history accumulates. Because fraud labels are only confirmed after a delay (the "label lag," denoted ℓ), label-dependent features are computed only from predecessors whose labels are already adjudicated at the target's timestamp. The construction uses H = 2 session hops with fixed budgets of at most K1 hop-1 and K2 hop-2 predecessors selected by most-recent timestamp.

Micro-GT for local structure. The target session, its neighboring session tokens, and its directly attached identifier tokens are converted into a token sequence with feature, type, hop, temporal, and structural positional encodings, mixed through a two-layer MLP. Instead of one undifferentiated attention operation over all pairs, Micro-GT runs four masked attention passes in order: (1) identity-self, where the target and its own identifier tokens attend to each other; (2) relation-group, where session tokens interact when they share relation types with the target; (3) chain, where each hop-1 session forms a group with the target and its own hop-2 predecessors; and (4) full-context, where all non-padding tokens interact. These passes are stacked in NL transformer blocks, and a seed-conditioned attention pooling step produces the local representation h_micro.

Macro-GT for ecosystem context. For each calendar date, the system computes typed "climate" tokens summarizing the fraud ecosystem, grouped into global fraud activity (G), platform-specific behavior (P1, P2), cross-platform shifts (X), velocity and anomaly signals (V), ecosystem topology (E), and current risk regime (R). All statistics are computed non-anticipatively — label-dependent quantities use only sessions whose labels have matured by that date. Macro-GT attends over these climate tokens together with the target representation, its identifier states, and two sets of adaptive class prototypes (K_f fraud prototypes and K_nf benign prototypes). Prototypes are initialized with K-Means++ on class-stratified training embeddings, updated after each training batch by exponential moving average with momentum α outside the backward pass, and reinitialized if they remain unassigned past a fixed patience threshold.

Training. The micro and macro embeddings are concatenated and projected through a learned linear layer, followed by a two-layer MLP head with GELU activation and dropout producing a scalar logit. Training uses class-reweighted binary cross-entropy, with the positive-class weight set to the ratio of non-fraud to fraud counts in the training set.

Note: the specific values of K1, K2, NL, K_f, K_nf, α, the recency window T, and the label lag ℓ are written symbolically and are not reported numerically in the content provided. The appendix included in the provided content also cuts off partway through the cross-identifier session feature definitions, and Appendix A.6 (climate-feature definitions) is referenced but not shown.

Why This Matters

Impact on research. The paper argues that local graph message passing has known limitations (oversmoothing and oversquashing from repeated aggregation, blurring of distinct relation semantics), and that models restricted to local neighborhoods remain unaware of ecosystem-level regime changes. Hermes proposes a unified way to combine expressive local relational modeling with global context, and a production-scale evaluation protocol that enforces serve-time and label-maturity constraints throughout training, evaluation, and inference. It also builds on the tokenization framework of the Relational Graph Transformer (RelGT) while adding temporally structured local attention and class-stratified EMA prototypes.

Real-world applications.

• Account takeover detection in digital banking and financial services, where flagging a session triggers a step-up challenge or intervention.
• Reducing customer friction in authentication and login risk systems, since the paper's FR@50R metric is explicitly framed as an operational proxy for customer friction.
• Fraud ecosystems where attackers reuse infrastructure — shared devices, accounts, and network identifiers — across platforms and channels.
• Any non-stationary fraud problem with delayed label adjudication, where the paper's non-anticipative climate-token design could inform how global context is encoded safely.

Industry relevance. The system is evaluated on proprietary production data from a leading U.S. financial institution, and its baseline is the deployed XGBoost production system. Reported improvements over that deployed system — a 44.44% relative reduction in customer friction and a 24.66% relative improvement in fraud recall — are framed as deployment-relevant operating-point gains rather than purely academic metrics. The temporal-stability analysis is motivated by the production reality that fraud distributions shift continuously and frequent model recalibration is costly.

Future Directions

• Hyperparameter sensitivity and tuning. The paper presents K1, K2, NL, K_f, K_nf, α, and the recency window T symbolically without reported values, leaving open how sensitive the reported gains are to neighborhood budgets, transformer depth, and prototype momentum.
• Generalization beyond the evaluated setting. The evaluation is on a single institution's digital channels across one year; whether the dual-scale design transfers to other institutions, fraud typologies beyond ATO, or other geographic markets is not established in the provided content.
• Latency and computational cost. The paper does not report inference latency, throughput, or training cost at the 130-million-session scale, which matters for real-time scoring in production.
• Interpretability and operational trust. Beyond the ablation showing that each component contributes, the paper does not report whether the structured attention masks or climate tokens produce explanations that fraud analysts can act on.

Target Audience

This paper benefits most: machine learning practitioners and researchers working on graph neural networks and graph transformers, especially those applying them to financial fraud; fraud and risk data scientists at banks and payment platforms who need operating-point metrics like FR@50R and R@5FR; production ML engineers concerned with temporal leakage, label lag, and non-anticipative feature engineering; and researchers interested in how global "ecosystem" context can be encoded alongside local relational structure. Readers without a background in attention mechanisms and heterogeneous graphs will find the method sections dense, though the introduction and experiments are accessible to a broader fraud-analytics audience.

Authors’ abstract

Account takeover (ATO) fraud is a growing threat to digital banking, requiring effective detection while minimizing friction for legitimate customers. Production systems predominantly rely on tabular models that score sessions in isolation, discarding the relational structure of the underlying interaction network. Although graph-based models exploit relationships among sessions and network entities, they primarily reason over local neighborhoods and therefore capture only part of the problem: fraud risk depends jointly on the local relational structure surrounding a session and the evolving global state of the fraud ecosystem. We present HERMES (HEterogeneous Relational Micro--macro graph transformer Encoder for high-risk Sessions), a dual-scale architecture that jointly models these complementary scales of information. Micro-GT captures local heterogeneous graph structure through structured, relation-aware attention over a temporally safe session neighborhood. Complementing this local representation, Macro-GT models ecosystem-level context using non-anticipative climate tokens that summarize fraud dynamics, platform shifts, and infrastructure reuse, together with adaptive class prototypes that track representative fraud and benign session patterns over time. Evaluated on more than 130 million high-risk transaction sessions from a leading U.S. financial institution, HERMES consistently outperforms production and strong graph-based baselines, achieving a 44.44% relative reduction in customer friction and a 24.66% relative improvement in fraud recall over the production system. Ablation and temporal-stability analyses further demonstrate complementary gains from local relational modeling and global ecosystem context across changing fraud regimes.