PF$Δ$: A Benchmark Dataset for Power Flow under Load, Generation, and Topology Variations

AI summary

Overview

Research area: Machine learning for power systems — specifically benchmarking ML surrogate models for AC power flow calculation.

Technical level: Intermediate. Readers will benefit from familiarity with power systems (AC power flow, bus types, contingency analysis) and graph neural networks, though the paper's central argument is accessible without deep expertise in either.

Scope: The paper introduces PFΔ, a large-scale benchmark dataset of 859,800 solved AC power flow instances across six grid sizes with simultaneous load, generation, and topology perturbations, plus standardized evaluation tasks and a comparative study of a traditional solver and three GNN-based models.

What This Paper Is About

Power flow calculation — determining voltages and power flows from known loads and generator setpoints — is the computational backbone of grid operations, but running it repeatedly for contingency analysis or topology optimization is slow and expensive. Machine learning models, especially graph neural networks, promise speedups, yet research in this area is fragmented: papers use different data generation schemes and evaluation setups, and existing large-scale datasets capture only a subset of the variations real grids face. This paper builds a single benchmark that varies loads, generator setpoints, and grid topology together, then uses it to test how well current GNN approaches actually hold up.

Key Contributions

1. The PFΔ dataset. A benchmark of 859,800 solved AC power flow instances spanning six bus system sizes (IEEE-14, IEEE-30, IEEE-57, IEEE-118, GOC-500, and GOC-2000), three contingency regimes (N, N−1, and N−2), and three feasibility regimes (feasible, approaching infeasibility, and close-to-infeasible). The dataset provides 159,600 samples per bus system, except GOC-2000, which has 61,800 samples due to the computational cost of scaling to larger grids.

2. A data generation scheme. A pipeline combining the OPF-Learn load sampling method [9], the topological perturbation scheme used in OPFData [10], and a new approach for generating diverse generator profiles by randomly permuting generator cost parameters. Load sampling uses a custom ACOPF formulation with output-variable limits removed, solved via PowerModels.jl and Ipopt.

3. Standardized tasks and metrics. Four task groups covering in-distribution generalization to N−1 and N−2 perturbations, data efficiency, out-of-distribution generalization to different grid sizes, and performance on close-to-infeasible cases. Metrics are intentionally unsupervised (power balance mismatch mean and maximum, plus runtime) to avoid biasing evaluation toward the specific solution Ipopt converged to.

4. A comparative evaluation. A traditional Newton-Raphson solver and three GNN-based approaches — CANOS (adapted as CANOS-PF), PowerFlowNet (PFNet), and GraphNeuralSolver (adapted as GNS-S) — assessed under the proposed tasks, with failure-mode analysis.

Main Findings

• Topology training matters more than data volume. Models trained only on the base N topology generally perform poorly on N−1 and N−2 cases. Training on N−1 perturbations improves generalization to both N−1 and N−2 across all models. In data efficiency tasks, most models show consistent performance between high-data and low-data regimes, with GNS-S as the exception — in the low-data regime it performs comparably to when it is trained without any topological perturbations at all.

• No model matches the traditional solver's accuracy. All models fail to achieve power balance losses comparable to Newton-Raphson, even when they include physics-informed components. CANOS-PF achieves the best average performance across Task Group 1, but performs poorly on N−1 and N−2 cases when trained solely on N, which the authors attribute to its use of analytical formulas to derive branch flows and slack values from nodal predictions, amplifying error.

• GNNs are faster but less precise. Under Task Group 3 (training on a 118-bus system, testing on 57-bus and 500-bus systems), GNN-based models outperform Newton-Raphson in runtime, with the fastest model achieving approximately a 5× speedup. However, even the best models do not reach the 10⁻⁶ precision levels typical of Newton-Raphson.

• Newton-Raphson convergence degrades with grid size. In the Task 3.1 test set, NR convergence rates from a flat start were 95.2%, 65.7%, and 43.4% for bus sizes 57, 118, and 500 respectively. Its power balance loss values were on the order of 10⁻⁶. For the 2000-bus system, NR failed to converge from a flat start, so its runtime and PBL are not reported.

• Close-to-infeasible cases remain a weak spot. CANOS-PF achieves the lowest average error overall across Task 4, while GNS-S attains the lowest close-to-infeasible-specific loss on Task 4.1. Adding approaching-infeasible cases improves CANOS-PF's performance on the close-to-infeasible regime in Task 4.2. PFNet consistently performs worst on these cases, which the authors read as evidence of the difficulty of handling near-infeasibility through purely supervised learning. CANOS-PF and GNS-S maintain strong performance on feasible cases even when trained only on approaching-infeasible and close-to-infeasible samples (Task 4.3), whereas PFNet's performance drops drastically.

• Different failure modes. CANOS-PF is more consistent and stable across scenarios, while PFNet and GNS-S exhibit higher variability and occasional qualitatively different failure modes. GNS-S also shows high error variance, possibly due to its constrained architecture's sensitivity to structural perturbations.

Methodology in Plain English

The team started from standard power network data for each grid size and applied three perturbations simultaneously to a base case. First, loads were sampled using the OPF-Learn approach, which uniformly samples from a convex set containing the ACOPF feasible space and shrinks that set using infeasibility certificates whenever an infeasible point appears. The authors note that the common alternative of sampling active power demand within ±20% of the nominal base case did not produce sufficient load profile diversity. Second, topological perturbations were applied: each sample is subject to one of four equally probable events — removal of up to two randomly chosen generators, removal of up to two randomly chosen lines, removal of one line and one generator, or no removal. Third, generator cost parameters were randomly permuted between and among generators to generate diverse active power and voltage setpoints.

Each perturbed sample was then passed through a customized ACOPF formulation using PowerModels.jl and Ipopt, with limits removed on reactive power generation, voltage magnitude at PQ buses, and branch flows, making the ACOPF constraints equivalent to the power flow equations. Only samples with feasible solutions were accepted. Solutions the solver converged to were saved for supervised learning compatibility, but all benchmark evaluation metrics are unsupervised, based on how well a model's outputs satisfy the power balance equations.

Close-to-infeasible cases were generated separately using continuation power flow, tracing a path of solutions with a continuation parameter until the power flow Jacobian becomes singular at the steady-state stability limit. Additional "approaching infeasibility" samples just before that boundary were generated as data augmentation for training.

Evaluation used four task groups. In-distribution tasks used training sets of 54,000 samples, with test sets of 6,000 feasible and 600 close-to-infeasible samples per bus size. Data efficiency tasks used 36,000 and 18,000 training samples. Out-of-distribution tasks trained on one set of grid sizes and tested on others. Challenging-case tasks used combinations of 48,600 feasible plus 5,400 close-to-infeasible samples, 27,000 plus 5,400 plus 21,600, and 10,800 plus 43,200 samples. For GOC-2000, all sample counts are halved.

The primary analysis was conducted on the IEEE 118-bus system. Hyperparameter tuning used training data from Task 1.3, the most topologically diverse within the dataset. Each model was trained three times with different random weight initializations, with error bars as standard deviations across runs. GNS-S and PFNet were trained on NVIDIA V100 or NVIDIA RTX 8000 GPUs, CANOS-PF on NVIDIA RTX 6000 or L40S GPUs. Newton-Raphson runtimes were measured on an Intel Xeon Gold 6140 CPU and GNN runtimes on an NVIDIA RTX 8000 GPU.

Why This Matters

The paper's central impact is methodological: it gives the ML-for-power-flow field a shared dataset and evaluation protocol, replacing the current situation where different papers use different data generation processes and evaluation approaches and therefore cannot be meaningfully compared. It also shows concretely where current approaches fall short, which the authors frame as a set of open problems rather than a solved one.

Real-world applications the work speaks to:

• Contingency analysis. Assessing the impact of equipment outages requires solving thousands of power flow problems within operational timeframes, e.g., every 5 minutes. Faster, dependable approximators would directly support real-time decision-making.

• Topology optimization. Deciding which bus-splitting or line-switching actions relieve congestion involves searching a combinatorially large action space — the paper notes that a single substation on a 118-bus system can have 65,000 possible configurations — with a power flow evaluation at each search step.

• Climate resilience and uncertainty simulation. More frequent extreme weather is expected to cause more component outages, so operators must simulate outcomes across a wide range of generation, load, and outage scenarios.

• Grid planning and expansion. Operators evaluating a future grid where the network has grown would benefit from models that generalize to unfamiliar grid sizes rather than being tied to a fixed topology.

Industry relevance centers on grid operators and planners who currently rely on faster but less accurate linear approximations such as DC power flow — which the paper notes cause feasibility issues — and on ML researchers who need a realistic testbed before deployable solvers are possible. The paper notes that deployment requires scaling to realistic grid sizes of over 1000 nodes.

Future Directions

• Fast, scalable, accurate solvers. Achieving feasibility on par with Newton-Raphson and generalizing to grids with more than 1000 buses remain primary goals, ideally without relying on overly deep architectures that suffer from oversmoothing.

• Infeasibility detection. The dataset includes samples near the loadability limit, but there is currently no benchmark for assessing whether a model can detect that a solution does not exist. Since Newton-Raphson solvers do not indicate a reason for non-convergence, ML models could fill this gap.

• Higher-order contingencies. Extending robustness to N−k contingencies for k > 2, where the number of possible outages grows combinatorially, and ensuring performance across a meaningful subset of those scenarios.

• Out-of-distribution load profiles and scalable generation. Accurately representing the full feasible load space in large networks is computationally infeasible, and developing data generation methods that achieve high feature diversity for networks with thousands of nodes is a stated priority.

Target Audience

ML researchers working on surrogate models, physics-informed learning, and graph neural networks for physical systems; power systems researchers interested in the limits of learned power flow approximations; and grid operators or planners evaluating whether ML-based tools are ready for real-time deployment. The paper is most useful to readers who want both a ready-to-use benchmark and a clear-eyed account of where current methods fail.

Authors’ abstract

Power flow (PF) calculations are the backbone of real-time grid operations, across workflows such as contingency analysis (where repeated PF evaluations assess grid security under outages) and topology optimization (which involves PF-based searches over combinatorially large action spaces). Running these calculations at operational timescales or across large evaluation spaces remains a major computational bottleneck. Additionally, growing uncertainty in power system operations from the integration of renewables and climate-induced extreme weather also calls for tools that can accurately and efficiently simulate a wide range of scenarios and operating conditions. Machine learning methods offer a potential speedup over traditional solvers, but their performance has not been systematically assessed on benchmarks that capture real-world variability. This paper introduces PF$Δ$, a benchmark dataset for power flow that captures diverse variations in load, generation, and topology. PF$Δ$ contains 859,800 solved power flow instances spanning six different bus system sizes, capturing three types of contingency scenarios (N , N -1, and N -2), and including close-to-infeasible cases near steady-state voltage stability limits. We evaluate traditional solvers and GNN-based methods, highlighting key areas where existing approaches struggle, and identifying open problems for future research. Our dataset is available at https://huggingface.co/datasets/pfdelta/pfdelta/tree/main and our code with data generation scripts and model implementations is at https://github.com/MOSSLab-MIT/pfdelta.