SMART: A Surrogate Model for Predicting Application Runtime in Dragonfly Systems

AI summary

Overview

Research area: Machine learning for high-performance computing (HPC) — specifically, data-driven surrogate models that replace or accelerate parallel discrete event simulation (PDES) of Dragonfly interconnect networks.

Technical level: Advanced. The paper assumes familiarity with graph neural networks, transformers, large language models, discrete-event network simulation, and HPC interconnect topologies.

Scope: The paper proposes SMART (Surrogate Model for Predicting Application RunTime), a model that fuses graph neural networks with a large language model to predict per-process application iteration times on a 1,056-node Dragonfly system, evaluated on two CODES-generated datasets under two job placement strategies.

What This Paper Is About

Large HPC systems use high-radix, low-diameter Dragonfly interconnects, but when multiple applications share the network, workload interference makes application runtime hard to predict. High-fidelity flit-level PDES (such as CODES, built on the ROSS engine) models this interference accurately but is computationally intractable at scale, requiring processing of billions of flit-level events across thousands of routers. The goal of this work is a lightweight surrogate model that predicts application iteration times accurately enough to support hybrid simulation, where PDES and the learned model trade off against each other.

Key Contributions

1. Stated objective. Develop an accurate yet lightweight surrogate model for predicting application iteration times in Dragonfly networks, to accelerate networking simulation and improve networking decisions.
2. A novel architecture. SMART combines graph neural networks, which capture the hierarchical and spatial connectivity of the Dragonfly topology, with large language models, which enhance temporal modeling via long-range pattern recognition and contextual prompting. The authors state this is the first effort to build a surrogate model for large-scale networking using LLMs and GNNs.
3. Extensive evaluation. Experiments on two datasets generated from a 1,056-node Dragonfly system, with multiple representative HPC workloads executed concurrently under contiguous and random job placement, showing that SMART significantly outperforms all baselines in predictive accuracy and achieves an inference time of 0.515 seconds, at least an order-of-magnitude speedup over the original simulation time.
4. Open artifacts and an online tuning strategy. The paper releases code (https://github.com/SPEAR-UIC/SMART) and datasets (https://zenodo.org/records/16667461), and introduces an online tuning scheme that updates model weights using intermittent PDES ground-truth feedback during inference.

Main Findings

• Accuracy on D1. On dataset D1 (MILC and LAMMPS), the best SMART results are MAPE 3.19 for MILC and 1.78 for LAMMPS, both under contiguous placement with TunGNN = 2, TinLLM = 8, and Ft = 8. SMART consistently beats LSTM, DCRNN, LAST, and MEAN across all column-wise comparisons in Table 3.
• Accuracy on D2. On dataset D2 (MILC, LAMMPS, NN), the best SMART results are MAPE 3.39 for MILC (random placement), 1.86 for LAMMPS (random placement, TunGNN = 2), and 2.79 for NN (random placement, TunGNN = 2) with TinLLM = 8 and Ft = 8. Baselines on D2 range from roughly 5.5 to 15.2 MAPE.
• Best hyperparameters. The best performance is obtained for TinLLM = 8 and TunGNN = 2 in all cases, which the authors interpret as the LLM capturing long-term temporal patterns while the GNN focuses on recent spatial dynamics.
• Online tuning helps. Incorporating online tuning (Ft = 8), in which model weights are updated every 8 test iterations, consistently reduces forecasting errors compared to a static model (Ft = ∞) on both datasets and both placement strategies.
• Robustness to iteration-time variability. Table 5 reports that LAMMPS has far larger iteration-time variance than MILC and NN (standard deviation of 807,331.12 under random placement versus 3,089.45 for contiguous MILC), yet SMART's MAPE for LAMMPS stays at 1.88 (contiguous) and 1.86 (random).
• Small placement gap. Accuracy differences between contiguous and random placement are minimal: MILC shows a MAPE difference of 0.12% and LAMMPS a gap of 0.02%, even though random placement introduces greater iteration-time variability from network interference.
• Low inference cost. SMART achieves an inference time of just 0.515 seconds, at least an order-of-magnitude speedup over the original simulation. Training each SMART model takes approximately 3–4 hours on two NVIDIA A100-PCIE GPUs with 40 GiB memory each and CUDA 12.4, described as a one-time cost.
• Ablation and sensitivity results not reported here. The introduction mentions ablation studies on the GNN and Time-LLM components and hyperparameter sensitivity analysis, but the truncated content does not include those results.
• Practical ceiling acknowledged. The authors note that achieving 100% accuracy is unrealistic in highly dynamic networking environments (discussion section truncated).

Methodology in Plain English

The researchers first run high-fidelity CODES simulations to collect two kinds of data: application data describing workload characteristics, and router data recording port-level features every 250 microseconds. They turn this into a temporal graph representation, a sequence of graphs G1 through GT where each graph corresponds to one iteration. Nodes are router ports (some also representing directly attached compute devices), and edges come from two rules: fully connecting ports of the same router, and connecting ports joined by global or local links.

Each node carries features describing the network state at that port during the iteration, aggregated using statistics such as minimum, maximum, average, or a quantile over the 250-microsecond snapshots. Aggregation bounds differ for active nodes (defined by the start and end timestamps of the iteration being predicted) and non-active nodes, whose bounds are derived from terminal-port activity or the nearest active node's timestamps.

The prediction problem is stated formally: given a look-back window of previous iteration times y and network characteristics x for a process rank, predict the next iteration time y at t+1.

SMART has three integrated components:

1. A GNN encoder that applies a graph convolutional network to each graph in the look-back window, producing node embeddings that capture spatial dependencies.
2. A temporal transformer that processes the sequence of GNN embeddings with positional encoding and multi-head attention to capture temporal dependencies, outputting temporal node embeddings Z.
3. An LLM-powered component that follows the Time-LLM mechanism. Historical iteration times are patched, normalized, embedded through a patch embedding module, and fed to a pre-trained LLM together with prompts built using the Prompt-as-Prefix (PaP) technique. The prompt includes the workload name, a task description ("Forecast the next step given the previous [TinLLM] steps information"), and input statistics such as minimum, maximum, and median values.

In the final integration step, a node mask filters the transformer output so only active nodes are kept, and for each active node the LLM-derived hidden state is concatenated with the corresponding GNN node embedding. The combined vector passes through a fully connected layer to produce the prediction. The paper emphasizes that this integration is learned end-to-end rather than manually tuned.

An online tuning strategy handles the evolving nature of network traffic: the first phase is offline learning on 30% of the available data, and the second phase is inference, where if the iteration index is a multiple of Ft, data from t−Ft to t is used to update weights by back-propagation. Ft = ∞ means online tuning is switched off.

Why This Matters

Impact on research. The work argues that high-fidelity PDES does not scale to large or real-time scenarios, and that surrogate models can be substituted into a hybrid simulation loop with a control mechanism that decides when to switch between PDES and the faster model. It also positions itself as the first integration of GNNs and LLMs for spatio-temporal modeling of application runtime in HPC interconnects, extending earlier graph-based work on a 72-node Dragonfly system and establishing benchmarks on a much larger 1,056-node system.

Real-world applications:

• Network design and provisioning for Dragonfly-based supercomputers, including the six of the top ten systems on the 6/2025 Top500 list that use Dragonfly interconnects.
• Routing and congestion management, where faster runtime forecasts support better online decisions.
• Resource allocation and job scheduling, since the model generalizes across contiguous and random job placement strategies.
• Workload co-scheduling studies for production systems such as Frontier and Aurora, whose Slingshot networks use the 1D Dragonfly variant studied here.

Industry relevance. Vendors and national laboratories operating exascale machines need fast, accurate predictions of how concurrent jobs will interfere on shared network links. Replacing multi-hour, multi-day simulations with a model that infers in 0.515 seconds is directly useful for operational planning, and the released code and datasets lower the barrier for others to build on the approach.

Future Directions

• Improving accuracy limits. The authors state that 100% accuracy is unrealistic in highly dynamic networking environments, leaving open how close a surrogate can get and under what conditions.
• Full reporting of ablations. The truncated content does not include the promised ablation study quantifying the individual contributions of the GNN and Time-LLM components, nor the hyperparameter sensitivity analysis.
• Scaling and generalization. The paper frames generalization across job placement strategies, workloads, and traffic patterns as a goal; how SMART behaves beyond the 1,056-node system and beyond the three studied workloads (MILC, LAMMPS, and a 27-point 3D stencil NN) is an open question.
• Tighter hybrid integration. The online tuning strategy assumes ground-truth data is available every Ft iterations during inference; how the model performs under sparser or irregular feedback, and how the PDES-versus-surrogate control loop should be designed, remains to be explored.

Target Audience

HPC network researchers and practitioners working on interconnect simulation, performance modeling, and workload interference; machine learning researchers interested in spatio-temporal modeling that combines graph neural networks with large language models; and engineers at supercomputing centers or vendors who need fast runtime prediction to inform scheduling, routing, and resource allocation on Dragonfly-based systems.

Authors’ abstract

The Dragonfly network, with its high-radix and low-diameter structure, is a leading interconnect in high-performance computing. A major challenge is workload interference on shared network links. Parallel discrete event simulation (PDES) is commonly used to analyze workload interference. However, high-fidelity PDES is computationally expensive, making it impractical for large-scale or real-time scenarios. Hybrid simulation that incorporates data-driven surrogate models offers a promising alternative, especially for forecasting application runtime, a task complicated by the dynamic behavior of network traffic. We present \ourmodel, a surrogate model that combines graph neural networks (GNNs) and large language models (LLMs) to capture both spatial and temporal patterns from port level router data. \ourmodel outperforms existing statistical and machine learning baselines, enabling accurate runtime prediction and supporting efficient hybrid simulation of Dragonfly networks.