Dual-branch Spatial-Temporal Self-supervised Representation for Enhanced Road Network Learning

AI summary

Overview

Research area: Road network representation learning (RNRL) — a spatiotemporal machine learning problem at the intersection of graph self-supervised learning, hypergraph learning, and sequence modeling.

Technical level: Advanced. The paper assumes familiarity with Graph Neural Networks (GAT), hypergraph neural networks (HGNN+), contrastive learning with mutual information objectives, and causal Transformer encoders.

Scope: The paper proposes and evaluates DST, a dual-branch spatial-temporal self-supervised framework that learns road representations by combining a spatial branch (mix-hop transition matrix plus graph-hypergraph contrastive learning) with a temporal branch (Transformer-based next-token prediction regularized by a weekday/weekend discrimination task).

What This Paper Is About

Road network representation learning aims to turn each road segment into a low-dimensional vector that supports downstream traffic tasks. Existing self-supervised GNN methods rely on neighborhood smoothing, which breaks down when spatially distant roads are functionally similar (spatial heterogeneity) and when traffic patterns vary by hour and by weekday versus weekend (temporal dynamics). DST addresses both problems by learning spatial semantics and temporal traffic dynamics in two separately pre-trained branches that are then fused.

Key Contributions

1. A dual-branch road representation framework (DST) that models road networks from both a spatial view and a temporal view, targeting spatial heterogeneity and temporal dynamics.
2. For the spatial branch: a mix-hop transition matrix derived from trajectories, plus hypergraph-based contrastive learning using three newly designed hyperedge types to capture long-range and high-order functional relations.
3. For the temporal branch: a next-token prediction self-supervised task over traffic-dynamic sequences using a causal Transformer, with an auxiliary discrimination task that distinguishes weekday traffic modes from weekend traffic modes.
4. An empirical demonstration across three real-world datasets and three downstream tasks, including cross-city zero-shot transfer experiments (Beijing to Porto), showing state-of-the-art performance and strong transferability.

Main Findings

• Best overall performance: Averaged over five random seeds, DST outperformed all baselines on every reported task and dataset. On destination prediction it reached ACC@1 / MRR of 0.7288 / 0.8213 (Beijing), 0.6766 / 0.8101 (Porto), and 0.8335 / 0.8950 (Xi'an). On travel time estimation its MAE / RMSE were 236.6965 / 363.2039 (Beijing), 101.4223 / 143.6598 (Porto), and 202.8479 / 307.7553 (Xi'an).
• Speed inference results: DST achieved MAE / RMSE of 2.4595 / 3.2557 (Beijing), 3.4259 / 4.5538 (Porto), and 4.4987 / 5.6557 (Xi'an), the best in each case. The second-best model, JCLRNT, scored 2.8512 / 3.9013 (Beijing), 3.7475 / 4.9999 (Porto), and 4.5138 / 5.7294 (Xi'an).
• Baseline comparison: Classic graph methods (Node2Vec, GCN, GAE, TGCN) underperformed because they are not designed for road networks; SRN2Vec and Toast improved on them via random-walk road weighting; DyToast degraded notably on speed inference; JCLRNT and TrajRNE performed well due to trajectory information but were described as deficient in modeling temporal dynamics.
• Ablation (Beijing dataset): Removing the learnable transition matrix (w/o P_hop) or the same-type hyperedges (w/o hg2) caused the most severe degradation on speed inference. Removing temporal modeling (w/o tm) significantly degraded both trajectory-based tasks. Removing region hyperedges (w/o hg1) or unidirectional hyperedges (w/o hg3) produced moderate degradation across all tasks.
• Parameter sensitivity: Evaluated on destination prediction across the three datasets, DST maintained consistent performance despite parameter variation. Small mini-batch sizes reduce the number of negative samples (risking underfitting), while large sizes increase computational and memory cost. DST was described as not sensitive to traffic batch size, with a slight improvement at small traffic batches. Increasing the next-token prediction loss weight improved performance.
• Zero-shot transfer (Beijing to Porto): DST-transfer scored MAE / RMSE of 3.5126 / 4.6181 on speed inference, 108.0329 / 150.2018 on travel time estimation, and ACC@1 / MRR of 0.6424 / 0.7765 on destination prediction, compared with GAE (9.0482 / 10.1209; 112.1389 / 154.2606; 0.3119 / 0.4204) and JCLRNT (4.1047 / 5.3195; 109.7691 / 151.6060; 0.0167 / 0.0338).
• Case study: Using Beijing's highest-traffic road (ID=144) as an anchor, DST identified road ID=1748 as representationally close despite geographic distance. Both segments lie on Beijing's third ring and share functional role, unidirectional flow, and dual inflow/outflow characteristics.
• Not reported: The paper content provided does not report dataset sizes, the number of road segments per city, the embedding dimension used, specific hyperparameter values, or the numerical results of the fusion-method comparison (referenced to Appendix D.1).

Methodology in Plain English

Each road's static attributes (road ID, coordinates, type, length, number of lanes) are pulled from OpenStreetMap and embedded, and edge features are computed as angle and Haversine distance between connected segments. Trajectories are first map-matched onto the road network.

From the trajectories, the model builds a "mix-hop" transition matrix: for every pair of roads appearing in the same trajectory, weight is assigned as m − (j − i) where m is the number of roads in the trajectory and j − i is the hop distance. Closer pairs get larger weights, but distant reachable pairs are still included. This matrix is row-normalized (with a self-loop fallback when a row sums to zero), used to initialize a learnable weight matrix, and applied to the feature embeddings before graph convolution.

The spatial branch then runs two parallel encoders. A multi-layer Graph Attention Network processes the ordinary road graph using node features and edge features. A multi-layer HGNN+ processes a hypergraph built from three hyperedge types: (1) functional zones from spectral clustering, (2) roads of identical type regardless of location, and (3) geographically adjacent unidirectional roads, following Tobler's First Law of Geography. The two views are trained contrastively by maximizing mutual information between a road's graph-view representation and its hypergraph-view counterparts, optimized with a mini-batch strategy.

The temporal branch converts trajectories into traffic volume sequences — visit counts per road across 24 hours, split into weekdays and weekends (T = 24, C = 2). A Transformer encoder with sinusoidal positional encoding processes these sequences and takes the final hidden state as the temporal representation. Two losses are trained jointly: a regression loss predicting the next time-step value (next-token prediction), and a classification loss separating weekday from weekend sequences.

The two branches are pre-trained separately and then fused. The paper states that concatenation along the last dimension was used, and that summation and gating-mechanism fusion were also explored.

Why This Matters

Impact on research: The work argues that spatial heterogeneity and temporal dynamics are both fatal to the neighborhood-smoothing assumption underlying self-supervised GNNs for road networks, and it supplies a concrete architectural answer — mix-hop trajectory weighting, multi-type hyperedges, and sequence-level self-supervision — that outperforms trajectory-enhanced predecessors. The zero-shot cross-city result also contributes evidence that spatiotemporal pre-training can transfer across cities with different topology and mobility patterns.

Real-world applications:

• Traffic speed inference on urban road segments, supporting routing and congestion management.
• Travel time estimation for trip planning and logistics.
• Trajectory destination prediction for navigation, ride-hailing dispatch, and demand forecasting.
• Cross-city deployment, where a model pre-trained in a data-rich city is applied to a new city without target-city labels.

Industry relevance: The paper explicitly frames transferable models as a way to reduce deployment costs and resource expenditure across cities. The supporting code is publicly released at https://github.com/chaser-gua/DST. The authors are affiliated with Zhejiang University, Hangzhou City University, Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security, Bangsun Technology, and Nanyang Technological University, and the work was funded in part by provincial and municipal programs in Zhejiang and Hangzhou.

Future Directions

1. Extreme-event forecasting: The authors state they will next investigate emergency forecasting for road networks under extreme natural disasters such as typhoons and storms.
2. Handling city-specific components in transfer: The mix-hop transition matrix depends on urban nodes, so it was removed for the zero-shot experiment. Whether it can be adapted rather than dropped is left open.
3. Richer fusion strategies: Three fusion approaches (concatenation, summation, gating) are mentioned, but the detailed comparison is deferred to Appendix D.1 and is not included in the available content, leaving the optimal fusion design unresolved.
4. Scaling and robustness questions: Parameter sensitivity was studied only on destination prediction, and the baseline DyToast showed that weak dynamic modeling can degrade performance on some tasks — suggesting open questions about how to make temporal representation learning robust across heterogeneous downstream tasks.

Target Audience

This paper is best suited to researchers and graduate students working on spatiotemporal data mining, urban computing, and graph self-supervised learning, as well as machine learning engineers building production systems for traffic prediction, navigation, or mobility analytics who are comfortable reading GNN and Transformer architectures.

Authors’ abstract

Road network representation learning (RNRL) has attracted increasing attention from both researchers and practitioners as various spatiotemporal tasks are emerging. Recent advanced methods leverage Graph Neural Networks (GNNs) and contrastive learning to characterize the spatial structure of road segments in a self-supervised paradigm. However, spatial heterogeneity and temporal dynamics of road networks raise severe challenges to the neighborhood smoothing mechanism of self-supervised GNNs. To address these issues, we propose a $\textbf{D}$ual-branch $\textbf{S}$patial-$\textbf{T}$emporal self-supervised representation framework for enhanced road representations, termed as DST. On one hand, DST designs a mix-hop transition matrix for graph convolution to incorporate dynamic relations of roads from trajectories. Besides, DST contrasts road representations of the vanilla road network against that of the hypergraph in a spatial self-supervised way. The hypergraph is newly built based on three types of hyperedges to capture long-range relations. On the other hand, DST performs next token prediction as the temporal self-supervised task on the sequences of traffic dynamics based on a causal Transformer, which is further regularized by differentiating traffic modes of weekdays from those of weekends. Extensive experiments against state-of-the-art methods verify the superiority of our proposed framework. Moreover, the comprehensive spatiotemporal modeling facilitates DST to excel in zero-shot learning scenarios.