Solar-GECO: Perovskite Solar Cell Property Prediction with Geometric-Aware Co-Attention

AI summary

Overview

Research area: Machine learning for materials science, specifically multimodal geometric deep learning applied to perovskite solar cell device efficiency prediction (arXiv:2511.19263v1 [cs.LG], presented at the AI4Mat workshop).

Technical level: Advanced. The paper assumes familiarity with graph neural networks, E(3)-equivariance, multi-head attention, and probabilistic regression, though the core idea is describable in plain terms.

Scope: This paper proposes Solar-GECO, a hybrid model that fuses a geometric graph neural network encoding of a perovskite crystal's atomic structure with language-model embeddings of the surrounding device layers, joined by a co-attention module and trained with a probabilistic loss to predict power conversion efficiency (PCE).

What This Paper Is About

Perovskite solar cells contain several stacked layers (substrate, electron transport layer, hole transport layer, perovskite absorber, back contact), and their efficiency depends on interactions among all of them, not just on the absorber material alone. Searching this combinatorial space experimentally is slow and expensive, and existing machine learning models either predict isolated material properties or represent the perovskite absorber only as text, discarding its 3D crystal geometry. Solar-GECO's goal is to predict full-device PCE by combining atomic-level crystal structure with textual descriptions of the other device layers in a single model.

Key Contributions

1. A hybrid multimodal architecture (Solar-GECO) that pairs a geometric crystal graph encoder for the perovskite absorber with LLM-based molecular embeddings for the device's context layers (substrate, ETL, HTL, back contact).
2. A co-attention fusion module stacking self-attention within each modality and bidirectional cross-attention across modalities, so graph and text representations mutually refine over multiple layers.
3. Uncertainty-aware prediction, where a regression head outputs both the mean and standard deviation of a Gaussian distribution over PCE, trained with a Gaussian negative log-likelihood (NLL) loss.
4. State-of-the-art PCE prediction results on a curated Perovskite Database plus Materials Project dataset, outperforming semantic GNN and other baselines, with ablation and sensitivity studies supporting the design choices.

Main Findings

• Best overall accuracy among compared models. Solar-GECO reaches an R² of 0.4179 ± 0.0042, an MAE of 2.9361 ± 0.0179, and a Spearman's ρ of 0.6192 ± 0.0034, the best values reported in Table 3. All baselines are marked as statistically significantly different from Solar-GECO at p < 0.05 or p < 0.001.

• Improvement over the previous state of the art (semantic GNN). The abstract frames the gain as a reduction in MAE from 3.066 to 2.936; the detailed table reports 3.0668 ± 0.0471 for semantic GNN versus 2.9361 ± 0.0179 for Solar-GECO, along with an R² improvement from 0.3907 to 0.4179.

• Composition-only models do worst. CrabNet records the largest gap to Solar-GECO, with R² 0.2090 ± 0.0058, MAE 3.3655 ± 0.0072, and Spearman's ρ 0.3807 ± 0.0024.

• Geometry alone is not enough; fusion is. The CGCNN+BERT+MLP baseline (encoders concatenated, no co-attention) achieves R² 0.4009 ± 0.0067 and MAE 3.0111 ± 0.0324, worse than Solar-GECO, which the authors attribute to the co-attention module aligning modalities and capturing inter-layer interactions.

• Co-attention without geometry is close but lower. The LLM+Co-Attention ablation reaches MAE 2.9812 ± 0.0104, R² 0.4048 ± 0.0049, and Spearman's ρ 0.6120 ± 0.0003, slightly below Solar-GECO, which the authors say shows the value of the geometric crystal information.

• Uncertainty is well calibrated. With 95% nominal prediction intervals, the model obtains a PICP of 0.9593, an absolute difference of 0.93% from the nominal level. The calibration plot supports the theoretical linear relationship between expected absolute error and predicted σ with c = √(2/π).

• Errors concentrate on low-efficiency devices. The parity plot shows predictions clustering near the diagonal, with more pronounced deviations for low-PCE devices.

• Ablations favor MaterialsBERT, CGCNN, vanilla attention, and Gaussian NLL. MaterialsBERT gives MAE 2.922 versus MatSciBERT's 2.924 (both R² 0.421); CGCNN gives R² 0.421 / MAE 2.922 versus CHGNet's R² 0.394 / MAE 3.032; vanilla attention gives R² 0.421 / MAE 2.922 versus gated cross-attention's R² 0.372 / MAE 3.108; Gaussian NLL gives R² 0.421 versus MSE's R² 0.415, with both at MAE 2.922.

• Group split is harder for every model. Under a device-layer material group split, Solar-GECO still leads with R² 0.3724 and MAE 3.1271, versus semantic GNN (R² 0.3700, MAE 3.2740) and LLM+Co-Attention (R² 0.3374, MAE 3.2820).

• Data curation substantially narrows the chemical space. Filtering reduced total records by 32.4% (43,398 to 29,344), with unique perovskite formulas falling 92.7% (465 to 34). ETL materials went from 1,468 to 1,159 (21.0%), HTL from 1,978 to 1,416 (28.4%), back contacts from 290 to 244 (15.9%), and substrates from 194 to 159 (18.0%).

Methodology in Plain English

The model takes two kinds of input for each solar cell. First, the perovskite absorber is turned into a graph where atoms are nodes and bonds are edges, then encoded by a pre-trained crystal graph convolutional neural network (CGCNN) that uses atomic numbers and interatomic distances. Second, the chemical descriptions of the four other layers (substrate, ETL, HTL, back contact) are fed as text strings into a pre-trained MaterialsBERT language model, and the [CLS] token embedding for each layer is kept, giving four text vectors.

These two representations then pass through a co-attention module repeated over several layers. Inside each layer, self-attention first lets atoms attend to other atoms and text tokens attend to other text tokens, refining each modality on its own. Then cross-attention lets the atom features query the text features and vice versa, so each atom's representation is updated with device-stack information and each layer's representation is updated with crystal information.

After the final fusion layer, masked average pooling collapses the graph and text features into two fixed-size vectors, which are concatenated and passed to an MLP head. That head outputs two numbers: the predicted mean and standard deviation of a Gaussian distribution over PCE. The model is trained to minimize a Gaussian negative log-likelihood loss, which combines a variance-scaled squared error term (pulling the mean toward the true value) with a log-variance term (penalizing inflated uncertainty).

Training uses the AdamW optimizer with a cosine schedule, a 10-epoch warm-up, and the pre-trained CGCNN encoder frozen. The dataset is split 80%/10%/10% into training, validation, and test sets, and experiments are repeated across three random seeds. Data comes from combining device-level specifications in the Perovskite Database with crystallographic structures from the Materials Project.

Why This Matters

Impact on research: The paper argues that single-material property prediction is insufficient for multi-scale devices such as perovskite solar cells, and demonstrates that explicitly encoding crystal geometry alongside device context improves full-device efficiency prediction. It also shows that probabilistic prediction with calibrated uncertainty is feasible in this setting, providing a template for multimodal, geometry-aware device-level models that could extend to batteries, catalysts, and thermoelectrics.

Real-world applications:

• Virtual screening of candidate perovskite solar cell architectures before committing to fabrication.
• Prioritizing which layer-material combinations to test experimentally, given limited lab resources.
• Flagging underperforming or risky device configurations via predicted PCE and its uncertainty.
• Providing uncertainty-aware rankings of candidate materials for downstream decision-making.

Industry relevance: The authors are affiliated with TotalEnergies OneTech, EDF, IPVF (Institut Photovoltaïque d'Ile-de-France), and Télécom Paris, indicating direct energy-sector interest in photovoltaics. The paper frames the core bottleneck as the gap between the growing diversity of prospective materials and the rate at which optimal full-device architectures can be identified, a problem whose solution has commercial value for solar manufacturing and deployment.

Future Directions

• Broaden the crystal structure data. The paper notes that geometric data comes primarily from the Materials Project, which constrains the diversity of available crystal structures and may limit exposure to less-represented materials.
• Incorporate fabrication parameters. The framework currently omits annealing temperature, deposition methods, humidity, and similar process variables, which the authors identify as important for reliable prediction and future exploration.
• Address the low-PCE bias. The authors report a low PCE bias as an observed challenge and suggest balancing strategies such as adaptive sampling, importance weighting, or contrastive loss.
• Improve generalization to unseen material combinations. The group-split results show that all models degrade when tested on novel device-layer material groups, leaving room for architectures that transfer better to unseen combinations.

Target Audience

Researchers and practitioners in machine learning for materials science, computational photovoltaics, and multimodal deep learning; graduate students and engineers familiar with graph neural networks and attention mechanisms; and industry R&D teams in solar energy who need device-level property prediction tools. Readers seeking a gentle introduction to GNNs or attention may find the methodology dense, but the problem framing and headline results are accessible.

Authors’ abstract

Perovskite solar cells are promising candidates for next-generation photovoltaics. However, their performance as multi-scale devices is determined by complex interactions between their constituent layers. This creates a vast combinatorial space of possible materials and device architectures, making the conventional experimental-based screening process slow and expensive. Machine learning models try to address this problem, but they only focus on individual material properties or neglect the important geometric information of the perovskite crystal. To address this problem, we propose to predict perovskite solar cell power conversion efficiency with a geometric-aware co-attention (Solar-GECO) model. Solar-GECO combines a geometric graph neural network (GNN) - that directly encodes the atomic structure of the perovskite absorber - with language model embeddings that process the textual strings representing the chemical compounds of the transport layers and other device components. Solar-GECO also integrates a co-attention module to capture intra-layer dependencies and inter-layer interactions, while a probabilistic regression head predicts both power conversion efficiency (PCE) and its associated uncertainty. Solar-GECO achieves state-of-the-art performance, significantly outperforming several baselines, reducing the mean absolute error (MAE) for PCE prediction from 3.066 to 2.936 compared to semantic GNN (the previous state-of-the-art model). Solar-GECO demonstrates that integrating geometric and textual information provides a more powerful and accurate framework for PCE prediction.