Flexible Intensities Matter: A Comprehensive Re-evaluation of Hawkes Processes and Neural Temporal Point Processes

AI summary

Overview

Research area: Machine learning for continuous-time event sequences — specifically temporal point processes (TPPs), comparing classical Hawkes/Poisson models against neural TPPs.

Technical level: Intermediate. The paper uses point-process notation (conditional intensity, cumulative intensity, log-likelihood decomposition) but its central argument is accessible: prior comparisons were unfair because the classical baselines were too rigid.

Scope (one sentence): A systematic, unified re-benchmark of Poisson and Hawkes processes of increasing flexibility against neural TPPs on five standard datasets plus two large-scale medical tasks (UK Biobank disease histories and MIMIC-IV ICU records), under autoregressive next-event prediction.

What This Paper Is About

Neural TPPs have repeatedly been reported to substantially outperform classical Hawkes processes, and this claim has shaped how the field justifies new neural architectures. The authors argue those comparisons were rigged by construction: Hawkes baselines were almost always restricted to a constant baseline and a fixed-shape exponential kernel with no inhibitory effects, while the neural models were highly flexible. This paper re-runs the comparison with Hawkes processes given spline baselines, learned spline kernels, and inhibitive variants — and asks where the remaining differences actually live inside the likelihood.

Key Contributions

1. A re-specified Hawkes baseline. The authors introduce a family of Hawkes processes with learnable linear-spline baselines and spline triggering kernels (plus inhibitive variants using a Softplus-wrapped intensity), and show that the previously reported gap to neural TPPs largely disappears on all five standard benchmarks.

2. Flexible intensity heads for neural TPPs. They introduce NeuralSplineTPP, a Transformer encoder with a spline-based intensity head that keeps a closed-form cumulative intensity (enabling exact sampling) and outperforms softplus-headed intensity models (THP, SAHP, NHP) while being on par with the intensity-free IntensityFree.

3. A time/type decomposition of the likelihood. By splitting the joint log-likelihood into a time term log p(t) and a time-conditional type term log p(d | t), and sweeping spline knot counts from K = 10 to 400, they show that the benefit of flexibility accrues almost entirely to the time term — more knots do not improve the type term on any dataset.

4. Two large-scale medical evaluation tasks and a software release. They build a 1,257-type terminating disease-history task from UK Biobank and a 117-type ICU-event task from MIMIC-IV, and release TorchPP, an open-source PyTorch framework with consistent implementations of classical and neural TPPs supporting terminating processes, right-censoring, exact sampling, and analytical/numerical integration.

Main Findings

• The Hawkes–neural gap is mostly an artifact of weak baselines. With flexible baselines, learned kernels, and inhibition, the best Hawkes variant matches or beats the softplus-headed neural TPPs (THP, SAHP, NHP) on all five standard datasets. Inhib. Spline/Spline Hawkes is the best model overall on Retweet, Spline/Spline Hawkes ties NeuralSplineTPP on StackOverflow and Taobao, and the residual gap to the flexible neural models is small on Taxi (0.09 nats) and substantial only on Amazon.

• Intensity parametrization flexibility is a dominant performance axis for neural models too. NeuralSplineTPP outperforms THP, SAHP, NeuralConstTPP, and NHP by a considerable margin, tying with SAHP only on StackOverflow — consistent with the output head, not the encoder, being the decisive factor. Notably, the time-constant NeuralConstTPP performs comparably to THP while offering a closed-form cumulative intensity.

• Flexibility buys a sharper time density, not better type prediction. Raising the spline knot count from K = 10 to 400 raises log p(t) on Retweet and Amazon by 0.9–1.6 nats for both NeuralSplineTPP and Spline/Spline Hawkes, while log p(d | t) does not improve on any dataset. Across model families, history-aware models differ several times more in the time term than in the type term on Retweet, Amazon, and Taobao. The authors summarize this as: on these benchmarks, neural TPPs mostly model when events occur, not which.

• Taxi is the informative exception. Taxi event types (pick-ups and drop-offs in five New York City boroughs) strictly alternate, so the type-conditional term separates inhibition-capable models (inhibitive Hawkes and neural TPPs) from purely excitatory Hawkes processes.

• Discretized timestamps inflate the time term. Retweet timestamps are integers, and a continuous density can place narrow peaks on that grid, pushing the log-likelihood up without bound. When scored by the probability of the recorded time bin instead, the gain from more knots vanishes on Retweet but persists on Amazon, whose timestamps lie on no grid.

• UK Biobank: disease histories look additive. Spline/Spline Hawkes trails NeuralSplineTPP by only 0.07 nats in joint log-likelihood and 0.08 percentage points in time-conditional accuracy, while outperforming the other neural TPPs — suggesting much of the next-event structure of first-occurrence disease histories is captured by pairwise interactions. Inhibitive Hawkes were not evaluated on UKB, and NHP did not converge there.

• MIMIC-IV: the only dataset where neural TPPs retain a genuine edge. NeuralSplineTPP leads Spline/Spline Hawkes by 0.45 nats joint (0.22 time, 0.23 type) with time-conditional accuracy of 38.3% versus 33.5% for the most accurate Hawkes variant, Inhib. Spline/Spline Hawkes. The lead is concentrated on the 41.6% of events recorded in the same minute as the previous event; on all other events, NeuralSplineTPP and Spline/Spline Hawkes differ by only 0.02 nats in log p(d | t), and Spline/Spline Hawkes has the higher log p(t).

• Component value varies by dataset. A non-stationary baseline (spline vs. homogeneous Poisson) helped mainly on Retweet; inhibition helped on Taxi, Retweet, and Amazon, gave no gain on StackOverflow, and hurt on Taobao (attributed to harder optimization of numerically integrated inhibitive models). Replacing the exponential kernel with a spline kernel improved the spline-baseline Hawkes process on all datasets.

• Reproduction fixes. The authors identified a sampling configuration issue in the EasyTPP evaluation pipeline and adjusted sampling and integration settings; their improved training schedule yields stronger results than originally reported, particularly for SAHP.

• Standard benchmarks may be near saturation. A time-rescaling goodness-of-fit check finds the flexible models best calibrated in time on four of the five datasets, supporting the claim that pairwise interactions account for most observable structure.

Methodology in Plain English

The team builds one consistent codebase (TorchPP) and re-implements every model under a single protocol so that comparisons are apples-to-apples.

On the classical side, they take the Hawkes intensity — a baseline rate plus a sum of decaying responses from every past event — and progressively loosen its constraints. The baseline can be a constant or a learnable linear spline over absolute sequence time; the triggering kernel can be a fixed exponential or a learnable spline function of the time lag for each pair of event types. Because a standard Hawkes intensity is non-negative by construction and therefore cannot express inhibition, they add variants that pass the sum through a Softplus clamp (they tested ReLU and found Softplus more stable and better-performing), estimating the cumulative intensity numerically with a stochastic trapezoidal rule.

On the neural side, they hold the encoder roughly fixed (mostly a Transformer over joint time/type embeddings) and vary the output head: NeuralConstTPP (time-constant intensity), THP/SAHP (Softplus heads needing numerical integration), NHP (a continuous-time LSTM with a Softplus head), NeuralSplineTPP (a spline head with closed-form cumulative intensity), and IntensityFree (a log-normal mixture over inter-event times, no intensity at all).

They train every model on full-sequence log-likelihood with hyperparameters tuned on validation log-likelihood via a multi-stage search, run five random seeds, and report mean and standard deviation. Test metrics follow the EasyTPP convention (per event, conditioned on the first event, excluding the survival term after the last event): joint log-likelihood split into a time term and a type-conditional term, top-1 time-conditional accuracy, and time RMSE. Because some metrics require numerical integration over the next-event time, they compute test metrics on the first 1,000 test sequences per dataset and, for integrated metrics on MIMIC-IV, the first 500.

The key diagnostic is a knot-count sweep: increasing spline resolution and observing separately how much the time term and the type term improve, which localizes where flexibility pays off.

Why This Matters

Impact on research. The paper challenges a widely cited justification for new neural TPP architectures. If a well-specified Hawkes process with spline kernels matches flexible neural models on five standard benchmarks, then improvements reported against exponential-kernel Hawkes baselines may be measuring baseline weakness rather than architectural progress. The time/type decomposition also gives the field a sharper diagnostic: reporting only joint log-likelihood hides the fact that gains are concentrated in event timing.

Real-world applications:

• Electronic health records and disease trajectories — modeling the order and timing of diagnosis codes (the UK Biobank task uses 1,257 disease categories including death).
• Intensive care monitoring — sequences of procedures and medications during ICU stays (the MIMIC-IV task uses 117 item types).
• Social media and information diffusion — retweet cascades and their temporal dynamics.
• Seismology — earthquake aftershock sequences, the original motivating application for Hawkes processes, where an interpretable baseline-plus-triggering decomposition is valuable.

Industry relevance. For teams building event-sequence systems, the practical message is economic: a well-tuned, flexible Hawkes process is a sensible starting point because it is interpretable, has a closed-form cumulative intensity, and supports exact sampling — so the cost and complexity of a neural TPP is only justified by data with structure beyond pairwise excitation. The released TorchPP package lowers the barrier to running that comparison directly, and the caution about integer timestamps matters for anyone computing likelihoods on log or calendar data.

Future Directions

• Datasets with genuinely non-pairwise structure. The authors argue future benchmarks should target data plausibly featuring complex long-range dependence and interactions beyond pairwise ones, including richer modalities such as lab measurements, genomics, images, and reports.
• Handling discretized and simultaneous events. Integer timestamps allow flexible models to reach an unbounded time likelihood, so either dequantization or discrete-time models are needed; simultaneous events require discretized time, set-valued marks, or a deliberate tie-breaking strategy.
• Extending the comparison beyond the current scope. Cox-type models, flow-, diffusion-, and state-space-based TPPs, and recurrent/other neural families were left out but are natural next comparisons within TorchPP.
• Beyond next-event prediction. Long-horizon simulation, planning and decision-making use cases, calibration for models without closed-form cumulative intensity, and inclusion of sex and lifestyle covariates in the UK Biobank pipeline remain open.

Target Audience

Machine learning researchers working on temporal point processes and event-sequence modeling; computational epidemiologists and clinical data scientists evaluating models on electronic health records; and practitioners in industry who need to choose between an interpretable classical model and a neural one. Readers who want a rigorous, reproducible justification for the modeling choices in an event-sequence pipeline — rather than a new state-of-the-art architecture — will get the most from it.

Authors’ abstract

Temporal point processes (TPPs) are the canonical framework for event sequence data, with applications spanning seismology, social media, and electronic health records. Recently, neural TPPs have been reported to substantially outperform classical TPPs, such as Hawkes processes, but these comparisons pitted flexible neural models against Hawkes processes in their most restricted form, and did not consider large-scale medical data. Here, we revisit this comparison through a systematic evaluation of Poisson and Hawkes processes of increasing flexibility against neural TPPs, on five established benchmark datasets and on two large-scale medical evaluation tasks derived from UK Biobank disease histories and MIMIC-IV ICU records, under autoregressive next-event prediction. First, the previously reported gap between Hawkes processes and neural TPPs largely disappears once Hawkes processes are equipped with flexible, learnable baselines and kernels. Second, the flexibility of the intensity parametrization also limits prior neural TPPs: a Transformer with a spline-based intensity head outperforms previous intensity-based neural TPPs and is on par with the intensity-free model. Decomposing the log-likelihood into a time term $\log p(t)$ and a type term $\log p(d \mid t)$, we find that this flexibility advantage lies almost entirely in the time density: more flexibility does not improve the type term. On these benchmarks, the gains of neural TPPs thus lie mostly in modeling when events occur, not which. Third, our results underscore the value of complex datasets for evaluating TPPs: only on the MIMIC-IV ICU records do neural TPPs retain an advantage in the type term. Our results inform both future method development and model choice. We release our framework, TorchPP, as an open-source package with consistent implementations of classical and neural TPPs.