ECSEL: Explainable Classification via Signomial Equation Learning

AI summary

Overview

Research area: Explainable artificial intelligence, symbolic regression, and interpretable machine learning.

Technical level: Intermediate. The paper is readable with a working knowledge of classification and regression, but the core content includes derivations of derivatives, exponents, and softmax-normalized probabilities.

Scope: The paper introduces ECSEL, a classifier that learns closed-form signomial equations (sums of power-law terms) as class score functions, and demonstrates that this form recovers symbolic regression targets more often than state-of-the-art baselines while remaining competitive on standard classification benchmarks.

What This Paper Is About

Most accurate machine learning models, such as deep neural networks and gradient-boosted ensembles, do not explain how they reach a decision, which is a problem in high-stakes settings like medicine, finance, and human resources. Symbolic regression offers an alternative by producing explicit mathematical formulas, but conventional approaches search a vast space of possible expressions and are computationally expensive. ECSEL targets one specific structural pattern, the signomial function, and learns class-specific signomial equations directly, so that the resulting classifier is itself the explanation.

Key Contributions

1. The authors identify signomial functions as a common structure in symbolic regression benchmarks (45 of the 100 AI Feynman equations are signomials) and propose ECSEL, which combines signomial parameterization with softmax and sparsity-inducing regularization into an inherently interpretable classifier.
2. ECSEL recovers target equations on symbolic regression benchmarks more frequently than the competing state-of-the-art methods DGSR, NGGP, and NeSymRes, while requiring substantially less computation.
3. The paper establishes analytical properties of ECSEL across three levels of explanation: global feature behavior through closed-form elasticities, decision-boundary analysis via exact margin sensitivities, and local feature attributions through log-space decompositions.
4. Two real-world case studies, on e-commerce purchase intent and financial fraud detection, show that the learned equations expose dataset characteristics and yield actionable insights.

Main Findings

• Symbolic recovery rate: ECSEL achieves a global average recovery rate of 95.86% on the signomial-rich benchmark collection (AI Feynman, Livermore, Jin, Korns, and DGSR's synthetic power-law equations), compared to DGSR at 59.10%, NGGP at 58.54%, and NeSymRes at 56%.
• Computation time: ECSEL averages 86.4 seconds per equation, against 612.9 seconds for DGSR, 468.7 for NGGP, and 126.3 for NeSymRes. Several DGSR and NGGP runs fail to recover an expression within the 15-minute limit.
• Structural difficulty: Equations consisting of a single signomial term (K = 1) are solved rapidly, while multi-term equations (K > 1) have a higher-dimensional and more nonconvex optimization landscape.
• Near-misses: Even when exact symbolic recovery fails, ECSEL typically yields near-perfect numerical approximations with R² ≈ 1.
• Classification performance: ECSEL achieves the highest F1-score on 4 of 11 datasets (Seeds, Hearts, ILPD, and Compas) and ranks within one percentage point of the best method on 9 of 11 datasets.
• ILPD result: ECSEL reaches 75.86 accuracy, 74.39 F1, and 42.42 minority recall, versus XGBoost at 72.41 accuracy, 63.03 F1, and 6.06 minority recall, a gain of 11.36 F1 points and a 36-point gain in minority recall.
• Compas result: ECSEL scores 68.47 accuracy, 68.36 F1, and 62.82 minority recall, compared with RF at 67.69, 67.63, and 60.40, and XGBoost at 68.18, 68.08, and 62.54.
• Transfusion result: ECSEL scores 79.33 accuracy, 77.95 F1, and 41.67 minority recall, trailing XGBoost's 80.06 accuracy and 78.72 F1 but exceeding its 38.89 minority recall.
• Extreme imbalance: On the two datasets with extreme class imbalance (Skinnonskin and Mammography), ensemble methods retain an advantage, though ECSEL maintains competitive minority recall.
• E-commerce case study: On the Online Shoppers Intention dataset (12,330 user sessions, 15.5% purchase rate), ECSEL learns a seven-feature signomial with a decision threshold of p = 0.559, identifying PageValue_per_ExitRate as the dominant predictor with an exponent of 1.09. It trains in 5.5 seconds and achieves 75.7% minority recall, close to MLP at 75.4% and SVM at 76.6%, and above LR at 67.7%, RF at 70.3%, and XGBoost at 66.4%.
• Explanation agreement: ECSEL's global feature rankings align closely with tree-based methods (Spearman ρ ≥ 0.80, p < 0.001) and show weaker, non-significant agreement with MLP. ECSEL produces attributions at 0.1 seconds versus up to 28.5 seconds for KernelSHAP.
• Fraud detection case study: On the PaySim dataset (approximately 6.3 million transactions over 30 days, 0.13% fraud rate), ECSEL achieves an F1 score of 79.08%, recall of 68.10%, and precision of 94.27%.

Methodology in Plain English

A signomial is a sum of terms, where each term is a coefficient multiplied by each input feature raised to some exponent. Instead of searching through an enormous space of possible formulas the way genetic programming methods do, ECSEL fixes this structural form and learns the coefficients and exponents directly with gradient-based optimization. Each class in a classification problem gets its own signomial score function, and those scores are converted into probabilities with a softmax (or a sigmoid in the binary case). Features are shifted and rescaled so that all inputs are positive, because real-valued exponents require positive arguments.

Training minimizes cross-entropy loss plus an L1 penalty on the exponents. That penalty drives unimportant exponents toward zero, effectively selecting features automatically and producing shorter, more readable equations. For symbolic regression tasks, the cross-entropy is swapped for mean squared error. To handle the nonconvex optimization landscape, the authors use L-BFGS-B for single-term models and a staged approach for multi-term models: Adam-based structure discovery with stronger sparsity, then refinement with reduced regularization, then a final L-BFGS polishing step.

Because the model is a differentiable closed-form expression, its derivatives with respect to log-features can be written down exactly. From these, the authors derive properties for global feature importance (elasticities), counterfactual effects under proportional feature scaling, decision-boundary sensitivity, probability competition, and local attributions. For single-term models, the local attribution is exact; for multi-term models it is a first-order approximation.

Why This Matters

Impact on research: The paper shows that committing to a restricted function class can improve both speed and recovery accuracy on symbolic regression benchmarks, challenging the assumption that general-purpose search is always the right approach. It also reframes signomials, previously studied in geometric programming as minimization objectives, as a supervised learning model class. The universal approximation theorem presented for signomials places them alongside neural networks as a dense function class, but one whose parameters carry direct interpretable meaning.

Real-world applications:

• Clinical decision support, where a closed-form score over tumor size, biomarker levels, and imaging irregularity can be inspected and audited directly.
• Credit and insurance risk scoring, where regulators require an explanation of why an applicant was rejected or flagged.
• E-commerce conversion modeling, where elasticities identify which session behaviors actually drive purchase intent.
• Fraud detection at transaction scale, where exact counterfactual reasoning supports investigation of suspicious activity.

Industry relevance: The practical argument is a cost shift. ECSEL's training times can be higher than some baselines, but once trained, all feature attributions are obtained in closed form at negligible cost. Black-box models require separate post-hoc procedures such as SHAP or LIME, which in the paper's own comparison took up to 28.5 seconds for KernelSHAP and 32.0 seconds for LIME on the e-commerce test set. The paper also notes that post-hoc explanations can be unreliable or misleading for high-stakes decisions, a critique ECSEL sidesteps by making the model itself the explanation.

Future Directions

• Extending the framework beyond the L1 regularizer applied to exponents, since the authors state explicitly that L1 is not essential and other regularizers can be used.
• Addressing the weaker performance on datasets with extreme class imbalance (Skinnonskin and Mammography), where ensemble methods still hold an advantage.
• Investigating calibration, since the paper notes that the resulting scores are not guaranteed to be calibrated and that post-hoc techniques such as temperature scaling could be applied if needed.
• Comparing ECSEL against matched per-class multi-layer perceptrons more systematically, which the paper begins in an appendix, and extending the analysis to cases where exact symbolic recovery fails.

Target Audience

This paper benefits machine learning researchers working on symbolic regression and interpretable models, practitioners in regulated domains such as healthcare and finance who need auditable classifiers, and readers already familiar with SHAP, LIME, and generalized additive models who want to understand how an inherently interpretable closed-form classifier compares to post-hoc explanation methods. The derivations and case studies make it accessible to graduate students comfortable with calculus and standard classification metrics.

Authors’ abstract

We introduce ECSEL, an explainable classification method that learns formal expressions in the form of signomial equations, motivated by the observation that many symbolic regression benchmarks admit compact signomial structure. ECSEL directly constructs a structural, closed-form expression that serves as both a classifier and an explanation. On standard symbolic regression benchmarks, our method recovers a larger fraction of target equations than competing state-of-the-art approaches while requiring substantially less computation. Leveraging this efficiency, ECSEL achieves classification accuracy competitive with established machine learning models without sacrificing interpretability. Further, we show that ECSEL satisfies some desirable properties regarding global feature behavior, decision-boundary analysis, and local feature attributions. Experiments on benchmark datasets and two real-world case studies i.e., e-commerce and fraud detection, demonstrate that the learned equations expose dataset biases, support counterfactual reasoning, and yield actionable insights.