Spiking Heterogeneous Graph Attention Networks

AI summary

Overview

Research area: Spiking neural networks (SNNs) applied to heterogeneous graph neural networks (HGNNs), specifically semi-supervised node classification on graphs with multiple node and edge types.

Technical level: Advanced. The paper assumes familiarity with graph neural networks, meta-paths, attention mechanisms, and the Integrate-Fire-Reset dynamics of spiking neurons including the IF, LIF, and PLIF models and surrogate-gradient training.

Scope: The paper proposes SpikingHAN, a heterogeneous graph attention network whose final representation is a binarized 1-bit spike sequence, and evaluates it against homogeneous GNNs, homogeneous graph SNNs, and heterogeneous GNNs on three node-classification datasets (DBLP, ACM, IMDB), measuring accuracy alongside parameters, memory, training time, and GPU energy.

What This Paper Is About

Heterogeneous graphs contain multiple types of nodes and relationships, and HGNNs handle them well but usually through complex designs that consume large amounts of memory, compute, and power. The authors bring the brain-inspired, event-driven efficiency of spiking neural networks into heterogeneous graph learning to cut those costs. Their goal is a model that stays competitive on node classification while using far fewer parameters, less memory, faster inference, and less energy than conventional HGNNs.

Key Contributions

1. A new model, SpikingHAN, that combines SNNs with HGNNs, using shared-parameter single-layer graph convolution for meta-path neighbor aggregation, semantic-level attention over meta-paths, and an SNN module whose Integrate, Fire, and Reset dynamics produce a sparse binary representation.

2. The first attempt to integrate SNNs into heterogeneous graph data (per the authors' claim), extending prior spiking work that had focused on homogeneous and dynamic graphs.

3. Shared-parameter aggregation instead of per-meta-path attention modules, which the authors argue reduces model complexity while retaining expressive power, in contrast to HAN's design of assigning a distinct node attention module to each meta-path.

4. Comprehensive experiments on three real-world datasets (DBLP, ACM, IMDB) showing competitive node classification together with advantages in parameters, inference speed, memory, and energy.

Main Findings

• Node classification is competitive but not uniformly best. On DBLP Micro-F1 at a 20% training ratio, SpikingHAN scores 93.7 ± 0.1, matching PHGT (93.7 ± 0.2) and edging HINormer (93.6 ± 0.1) and HAN (93.1 ± 0.1); at 40% PHGT leads at 94.6 ± 0.2 versus SpikingHAN's 93.8 ± 0.2.

• IMDB shows a SpikingHAN lead at higher training ratios. Micro-F1 at 60% is 65.2 ± 0.5 for SpikingHAN versus 65.0 ± 0.2 for PHGT and 64.8 ± 0.2 for HINormer; at 40% SpikingHAN is 64.2 ± 0.3 versus PHGT's 64.5 ± 0.1.

• Parameter counts are far smaller. SpikingHAN uses 15,201 parameters on DBLP, 128,385 on ACM, and 102,593 on IMDB, compared with 7,348,964 (HINormer, DBLP), 8,791,360 (PHGT, DBLP), 6,419,715 (HAN, IMDB), 8,577,159 (HINormer, IMDB), and 9,311,232 (PHGT, IMDB).

• Memory use is the lowest on every dataset. Maximum GPU memory allocation during training is 45.2 MB (DBLP), 137.32 MB (ACM), and 220.19 MB (IMDB) for SpikingHAN, against 7,156.53 MB for PHGT on DBLP and 6,361.18 MB for HINormer on DBLP.

• Energy consumption is lowest and stable. The paper reports that PHGT — the best classifier on some settings — consumes roughly 13 times the GPU energy per epoch of SpikingHAN, with an insignificant performance gain. Energy is described as higher in the first epoch (model initialization and weight setting) than in later epochs, which are mainly fine-tuning.

• Heterogeneous models generally beat homogeneous ones on these tasks, since homogeneous models treat all nodes and edges uniformly and lose heterogeneous semantics.

• Adding SNNs costs little accuracy. SpikingGCN and SpikeGCL have far fewer parameters than GAT and DAGNN yet show no significant performance gap.

• PLIF neurons work best among the tested spiking neurons. With a 20% training rate, the simple IF neuron already performs well; LIF improves on IF through a biologically reasonable leakage term; PLIF makes that leakage learnable and performs slightly better than LIF in most cases.

• The optimal number of time steps is a trade-off, not a global optimum. The reported best T per dataset balances compute cost against performance, since more time steps mean more Integrate/Fire/Reset operations, more runtime, and more memory. Specific T values are shown in figure form and are not reported numerically in the text.

Methodology in Plain English

SpikingHAN has three stages.

First, for each predefined meta-path, every node gathers information from its meta-path neighbors using a single graph convolution layer whose weights are shared across all meta-paths, rather than giving each meta-path its own module. This keeps the model small while still producing one embedding per meta-path.

Second, a semantic-level attention mechanism (inspired by HAN) scores how important each meta-path is. Each meta-path embedding is passed through a nonlinear transform plus a bias, compared against a learnable attention vector, averaged over all nodes, and normalized with Softmax. The resulting weights are used to combine the meta-path embeddings into one unified node representation.

Third, that representation passes through a fully connected layer that keeps only a linear transformation, with nonlinear activations and biases removed, following the LightGCN assumption that depth is not critical for graph label prediction. The output becomes the input current to a spiking neural network. Over T discrete time steps, neurons accumulate membrane potential (Integrate), emit a spike when the potential reaches the threshold V_th (Fire, via the Heaviside function), and then subtract the threshold from the membrane potential (Reset, subtractive form). Training uses the parametric LIF formulation, where the membrane time constant is learnable, and a surrogate gradient Θ′(x) ≜ σ′(αx) for backpropagation. Classification comes from average-pooling the spikes across time steps to get a class firing rate, optimized with cross-entropy loss over labeled nodes.

Reported complexity: the meta-path attention module is roughly O(P · |V| · d), and the SNN module adds roughly O(T · d²), with efficiency gained from binary spikes and no costly continuous activations. Experiments use 10 random seeds, training ratios of 20%, 40%, and 60%, and meta-path sets {APA, APVPA, APTPA} for DBLP, {PAP, PSP} for ACM, and {MDM, MAM} for IMDB.

Why This Matters

Impact on research: The work extends spiking graph learning from homogeneous and dynamic graphs into heterogeneous graphs, a more common real-world setting, and suggests that a binarized 1-bit representation can rival floating-point HGNNs while being much cheaper to run. It also challenges the assumption that each meta-path needs its own aggregation module.

Real-world applications:

• Social network analysis, where multiple entity and interaction types must be modeled cheaply.
• Recommendation systems built on heterogeneous user-item-relation graphs.
• Bioinformatics and academic literature mining, such as classifying papers by author, venue, and term relations (the DBLP and ACM setups).
• Deployment on resource-constrained or mobile devices, where memory and power budgets are tight.

Industry relevance: Lower parameter counts, lower GPU memory, and lower energy per epoch translate directly into cheaper training and inference, and into the possibility of running graph models on neuromorphic or edge hardware. The authors frame the energy savings as having economic and environmental value for large-scale deployment.

Future Directions

• Scaling to larger and more diverse heterogeneous graphs. The evaluation covers three datasets with modest node counts, so behavior at web-scale remains untested.

• Automating meta-path selection. SpikingHAN relies on predefined meta-path sets; relation-based HGNNs such as GTN, HetSANN, and ie-HGCN avoid manual meta-paths, and combining that with spiking neurons is an open question.

• Systematic tuning of time steps and neuron type. Since the best T is described as a local rather than global optimum, and PLIF only slightly outperforms LIF, the compute-performance trade-off needs a principled selection rule.

• Hardware-level validation. The energy claims come from GPU monitoring via NVIDIA's pynvml, averaged over 10 random seeds; measuring actual consumption on neuromorphic chips, where the paper's spiking advantages would be most pronounced, is a natural next step.

Target Audience

Researchers and graduate students working on graph neural networks, spiking neural networks, or efficient machine learning, plus practitioners who need heterogeneous graph models that fit within tight memory, latency, and power budgets. Readers without a background in SNN neuron dynamics or meta-path-based HGNNs will need to consult the cited foundational works (HAN, SpikingGCN, SpikeGCL, LIF/PLIF models) first.

Authors’ abstract

Real-world graphs or networks are usually heterogeneous, involving multiple types of nodes and relationships. Heterogeneous graph neural networks (HGNNs) can effectively handle these diverse nodes and edges, capturing heterogeneous information within the graph, thus exhibiting outstanding performance. However, most methods of HGNNs usually involve complex structural designs, leading to problems such as high memory usage, long inference time, and extensive consumption of computing resources. These limitations pose certain challenges for the practical application of HGNNs, especially for resource-constrained devices. To mitigate this issue, we propose the Spiking Heterogeneous Graph Attention Networks (SpikingHAN), which incorporates the brain-inspired and energy-saving properties of Spiking Neural Networks (SNNs) into heterogeneous graph learning to reduce the computing cost without compromising the performance. Specifically, SpikingHAN aggregates metapath-based neighbor information using a single-layer graph convolution with shared parameters. It then employs a semantic-level attention mechanism to capture the importance of different meta-paths and performs semantic aggregation. Finally, it encodes the heterogeneous information into a spike sequence through SNNs, simulating bioinformatic processing to derive a binarized 1-bit representation of the heterogeneous graph. Comprehensive experimental results from three real-world heterogeneous graph datasets show that SpikingHAN delivers competitive node classification performance. It achieves this with fewer parameters, quicker inference, reduced memory usage, and lower energy consumption. Code is available at https://github.com/QianPeng369/SpikingHAN.