HyperComplEx: Adaptive Multi-Space Knowledge Graph Embeddings

AI summary

Overview

Research area: Knowledge graph embeddings (KGE), link prediction, and geometric deep learning — specifically the use of mixed geometric spaces (hyperbolic, complex, Euclidean) to represent relational structure in large graphs.

Technical level: Advanced. The paper is written in a formal machine learning style with manifold geometry (Poincaré ball model, Möbius addition, complex/Hermitian bilinear scoring) and assumes familiarity with standard KGE baselines and the filtered link-prediction evaluation protocol.

Scope: The paper proposes HyperComplEx, a single embedding framework that learns per-relation weights over three geometric subspaces, and evaluates it on five computer-science research graphs (1K to 10M papers) plus three standard benchmarks (DBP15K, Hetionet, FB15K) against six baselines.

What This Paper Is About

Knowledge graphs contain many kinds of relationships at once — hierarchies (a paper belongs to a concept), asymmetries (one paper cites another), and symmetries (two authors collaborate) — but each existing embedding method commits to one geometry: Euclidean models struggle with hierarchies, complex/bilinear models handle asymmetry, and hyperbolic models struggle with symmetric relations. The authors build a single model that learns, per relation type, how much to rely on each geometry, and they test whether that adaptivity pays off as graphs grow from thousands to millions of entities.

Key Contributions

1. Adaptive space attention. A relation-specific attention vector α_r = [α_r^H, α_r^C, α_r^E] is produced by a softmax over three trainable parameters W_r ∈ ℝ³, and the final score is the weighted sum of the hyperbolic, complex, and Euclidean scores. Geometry selection is learned end-to-end rather than fixed by hand.
2. Multi-space consistency regularization. A loss term penalizes squared disagreement between the scores produced by different geometric spaces for the same triple, intended to keep predictions coherent while still letting each space specialize.
3. Scalable architecture design. Adaptive dimension allocation that redistributes the dimension budget across the three subspaces as graph size grows, CPU-resident sharded embeddings with GPU caching of active mini-batches for graphs above 10^6 entities, and FP16 mixed-precision training with dynamic loss scaling (described as accelerating tensor operations by up to 1.7×).
4. A released dataset family and implementation. Five computer-science research knowledge graphs built from OpenAlex and Semantic Scholar (CS-1K through CS-10M), with code and data released publicly on GitHub and Zenodo for reproducibility.

Main Findings

• Best MRR on every dataset tested. HyperComplEx scores 0.831 (CS-1K), 0.771 (CS-10K), 0.704 (CS-100K), 0.608 (CS-1M), 0.612 (CS-10M), 0.411 (DBP15K), 0.642 (Hetionet), and 0.692 (FB15K). The strongest baseline on the large graphs, ComplEx, reaches 0.789, 0.736, 0.688, 0.581, 0.584, 0.350, 0.608, and 0.657 respectively.
• The authors report a 4.8% relative gain over the best baseline on the 10M-paper dataset, and describe improvements of up to 5.3% on the CS graphs and 17.5% on DBP15K relative to ComplEx.
• Baselines degrade sharply on the largest and most heterogeneous data. SEPA scores 0.140 on CS-10M and 0.059 on DBP15K; UltraE scores 0.172 on CS-10M; RotatE drops to 0.133 on DBP15K while scoring 0.664 on FB15K.
• Near-linear training scaling. A power-law fit T(|E|, P) = α·|E|^β1·P^β2 over the five dataset scales yields β1 = 1.06 ± 0.02 and β2 = 0.41 ± 0.03, i.e. training time grows as roughly T ∝ |E|^1.06.
• Sublinear inference growth. Inference latency is modeled as L(|E|) = γ·|E|^λ with λ ≈ 0.42, attributed to caching and sharded lookup structures.
• Competitive but not class-leading efficiency. On CS-10M, HyperComplEx trains in 47.85 hours versus 45.92 for ComplEx and 42.97 for TransE, and reaches 85.23 ms per triple of inference latency versus 81.36 for TransE and 91.22 for ComplEx — described as within 10% of the most efficient baselines despite a 1.5–2× larger parameter count. On CS-1M the reported latency is 69.47 ms, and the paper describes an average latency of 69 ms per triple on 1M-entity graphs.
• Learned attention matches relational semantics. Averaged over 1,000 sampled CS-1M triples: BELONGS_TO puts 0.68 weight on hyperbolic space; CITES puts 0.71 on complex; AUTHORED puts 0.61 on complex; COLLABORATES_WITH puts 0.70 on Euclidean; PUBLISHED_IN puts 0.44 on complex (its largest weight, though all three are moderate).
• Ablations show the complex subspace matters most. On CS-1M / DBP15K MRR: full model 0.608 / 0.411; without adaptive attention 0.569 / 0.361; without multi-space consistency 0.582 / 0.372; without the hyperbolic subspace 0.573 / 0.367; without the complex subspace 0.547 / 0.329; without the Euclidean subspace 0.563 / 0.343.
• The paper explicitly flags evaluation limits. The authors state that classical baselines were not exhaustively retuned, that all models were compared under similar hyperparameter budgets rather than mixing best-case results, and that recent multi-geometry or GNN-based methods such as AttH variants and MuRP extensions were excluded due to the lack of unified public implementations. The comparison is described as representative but not exhaustive.
• Hits@K is defined as a metric but not tabulated. The evaluation section states that MRR and Hits@K (K ∈ {1,3,10}) are used, but the results tables report MRR only; per-K figures do not appear in the paper content provided.

Methodology in Plain English

Every entity (paper, author, venue, concept) and every relation is stored as three separate vectors instead of one — one living in a hyperbolic space, one in a complex space, and one in a Euclidean space. Each space scores a candidate triple in its own natural way: the hyperbolic space uses distance inside the Poincaré ball (good for trees and hierarchies), the complex space uses a Hermitian bilinear product with conjugation (good for direction and asymmetry), and the Euclidean space uses the classic translation idea that head + relation should land near tail (good for simple symmetric patterns).

Rather than committing to one of these or splitting dimensions equally, the model learns three numbers per relation type. A softmax turns them into weights that sum to one, and the final score for a triple is just the weighted average of the three space-specific scores. Because those weights are trained by gradient descent, the model discovers for itself that, say, "belongs to" should lean hyperbolic.

Training combines three loss terms: a self-adversarial ranking loss (the RotatE-style objective that upweights hard negative samples using a temperature β), a consistency term that penalizes squared differences between the spaces' scores so they do not drift apart, and standard ℓ2 regularization on embeddings. A scaling trick keeps the total dimension budget fixed while rebalancing it across the three subspaces as the graph grows, and for graphs above a million entities the embeddings are sharded across CPU memory with only active mini-batches resident on GPU. Experiments ran in PyTorch on an NVIDIA RTX 5060 with CUDA, with hyperparameters selected by grid search over embedding dimensions {128, 256, 512}, batch sizes {256, 512, 1024, 2048}, learning rates {5×10^−4, 10^−3, 5×10^−3}, margin γ ∈ {6.0, 9.0, 12.0, 15.0}, and curvature c ∈ {1.0, 2.0}, for up to 200 epochs with early stopping after 20 non-improving epochs. Data was split 80/10/10 after removing duplicate and inverse triples.

Why This Matters

The paper argues that no single geometry fits the mix of relation types found in real knowledge graphs, and that the right choice can be learned rather than hand-specified. Its main practical claim is that this adaptivity costs little: performance improves on every dataset tested while training scale stays close to linear and inference stays in the same range as much simpler models.

• Scientific literature and research analytics: the CS datasets model Paper, Author, Venue, and Concept entities with AUTHORED, CITES, PUBLISHED_IN, BELONGS_TO, and COLLABORATES_WITH relations — the substrate for citation prediction, expert finding, and research-topic mapping.
• Biomedical knowledge discovery: Hetionet's gene, compound, and disease relations are used to test hierarchical and multi-relational reasoning, relevant to drug repurposing pipelines.
• Search and recommendation: the introduction motivates KGE by web search and recommender systems, where link prediction fills in missing user–item or query–entity relations.
• Cross-lingual entity alignment: DBP15K tests alignment across English, Chinese, and French knowledge graphs, the setting behind multilingual knowledge fusion.

Industry relevance: the scaling analysis targets graphs from 10^3 to 10^8 entities, with the authors extrapolating stable behavior at billion-scale. The sharded-embedding and mixed-precision mechanisms are explicitly designed to train on commodity hardware, and the released implementation and dataset family lower the barrier to reproducing and extending the results.

Future Directions

• Temporal and dynamic knowledge graphs. Extending beyond the static triple setting to reason over evolving relational structures; the CS corpora are already temporally filtered by publication year.
• Integration with large language models. Hybrid symbolic–neural reasoning is proposed as a route to better zero-shot and inductive link prediction, addressing factual consistency and scalability limits the paper attributes to LLM-based KG work.
• Automatic curvature tuning and multi-space sparsification. Both are proposed to push toward web-scale graphs with tens of billions of entities.
• Theory and interpretability. The paper leaves formal analysis of curvature interactions and projection stability, convergence guarantees under mixed-curvature manifolds, and methods for tracing decision pathways across geometric spaces as open problems.
• Closing the baseline gap. AttH variants and MuRP extensions were excluded for lack of unified public implementations; the authors state these gaps will be addressed in future work.

Target Audience

Researchers working on knowledge graph embeddings, link prediction, or geometric deep learning who want a concrete, empirically evaluated design for mixing manifolds with learned per-relation weights. It is also relevant to machine learning engineers building large-scale knowledge graph completion systems, since the paper reports training time, inference latency, and memory strategy alongside accuracy. Readers without a background in hyperbolic geometry or complex-valued embeddings will find the methodology sections demanding, but the ablation table and the learned attention weights offer an accessible entry point to what the model actually learns.

Authors’ abstract

Knowledge graphs have emerged as fundamental structures for representing complex relational data across scientific and enterprise domains. However, existing embedding methods face critical limitations when modeling diverse relationship types at scale: Euclidean models struggle with hierarchies, vector space models cannot capture asymmetry, and hyperbolic models fail on symmetric relations. We propose HyperComplEx, a hybrid embedding framework that adaptively combines hyperbolic, complex, and Euclidean spaces via learned attention mechanisms. A relation-specific space weighting strategy dynamically selects optimal geometries for each relation type, while a multi-space consistency loss ensures coherent predictions across spaces. We evaluate HyperComplEx on computer science research knowledge graphs ranging from 1K papers (~25K triples) to 10M papers (~45M triples), demonstrating consistent improvements over state-of-the-art baselines including TransE, RotatE, DistMult, ComplEx, SEPA, and UltraE. Additional tests on standard benchmarks confirm significantly higher results than all baselines. On the 10M-paper dataset, HyperComplEx achieves 0.612 MRR, a 4.8% relative gain over the best baseline, while maintaining efficient training, achieving 85 ms inference per triple. The model scales near-linearly with graph size through adaptive dimension allocation. We release our implementation and dataset family to facilitate reproducible research in scalable knowledge graph embeddings.