An Open-Access Benchmark of Statistical and Machine-Learning Anomaly Detection Methods for Battery Applications

AI summary

Overview

Research area: Battery data analytics — specifically unsupervised anomaly (outlier) detection in battery cycling data, spanning classical statistics, distance metrics, and machine learning.

Technical level: Intermediate. The paper assumes familiarity with concepts such as z-scores, interquartile ranges, covariance matrices, and unsupervised ML models, but presents them with equations and comparison tables that make the material approachable for data scientists and battery engineers moving into diagnostics.

Scope: The authors introduce OSBAD (Open-Source Benchmark of Anomaly Detection), an open-source framework integrating 15 anomaly detection algorithms, and evaluate it on two battery datasets with different chemistries to establish a unified, reproducible benchmark.

What This Paper Is About

Battery cycling datasets routinely contain abnormal voltage and capacity readings caused by equipment breakdowns, manufacturing defects, sensor errors, or cell failure, and these anomalies are difficult to spot when scaling from a handful of laboratory cells to hundreds of cells over thousands of cycles. While data-driven battery lifetime prediction has been studied extensively, the authors note that reports on data-driven anomaly detection remain limited, and no prior benchmarking study has compared simple statistical approaches against more advanced machine learning methods. The paper's goal is to build a common, open, and reproducible benchmark for comparing anomaly detection methods on battery data, including labeled datasets and a standardized evaluation workflow.

Key Contributions

1. OSBAD framework with 15 algorithms. The authors integrate 15 distinct anomaly detection algorithms into a single open-source framework, spanning statistical methods (Standard Deviation, Median Absolute Deviation, Interquartile Range, Z-score, modified Z-score), distance-based methods (Euclidean, Manhattan, Minkowski, Mahalanobis), and machine learning models (Isolation Forest, K-Nearest Neighbors, Gaussian Mixture Models, Local Outlier Factor, Principal Component Analysis, and Autoencoder).

2. A statistical feature transformation plus physics-informed feature extraction. The authors introduce a transformation dividing by the ratio of the squared median to the IQR to disentangle overlapping normal and abnormal distributions, and then derive cycle-level features — the maximum scaled voltage difference per cycle, the maximum scaled capacity difference per cycle, and their ratio — to convert continuous anomalous sequences into cycle-level point anomalies.

3. Benchmarking datasets with binary labels across two chemistries. The authors created binary labels where 0 denotes inliers and 1 denotes outliers, and benchmarked across a liquid-electrolyte dataset (LiFePO4 positive electrode, graphite negative electrode) and a solid-state dataset from Tohoku University (LiNi0.5Co0.2Mn0.3O2 positive electrode, In/InLi negative electrode), demonstrating cross-chemistry applicability.

4. A probabilistic scoring approach to remove threshold subjectivity. Because each method's outlier scores are not standardized and each requires its own threshold, the authors convert all model outputs into a [0, 1] range using min-max conversion, with a consistent threshold of 0.7 applied across all ML methods for comparison.

Main Findings

• Statistical methods split into two camps. On the feature log(dV/dQ)_scaled,max,cyc, both the SD and MAD detectors showed nearly identical, strong performance with very high true negative rates (approximately 99.4%) and minimal false positives or false negatives. In contrast, the IQR detector produced the highest false positive rate, up to approximately 5%, and flagged many normal cycles near the distribution tails as outliers. The authors conclude that using a boxplot (IQR whiskers) to identify anomalies may not always be reliable.

• All statistical detectors missed true anomalies. Two major anomalies with scores 12.09 and 10.42 exceeded all three statistical limits and were correctly detected. However, the minor anomalies at cycle 147 (score 2.78) and cycle 148 (score 3.06) fell within the feature's distribution and limits, producing two false negatives. The authors attribute this to statistical methods being univariate — the voltage and capacity difference features are measured along different dimensions, limiting cross-detection.

• Distance metrics produce distinctive geometric decision regions. Euclidean distance produced circular contours due to its isotropic variance assumption; Manhattan distance yielded diamond-shaped contours emphasizing axis-aligned deviations; Minkowski distance is flexible depending on the value of p (p = 0.5 shown in the paper's contour figure), with higher p becoming more sensitive to extreme values and lower p producing more conservative detection; Mahalanobis distance produced elliptical contours because it accounts for covariance.

• Mahalanobis distance can underperform on distribution-aligned anomalies. The authors report that it may underperform for anomalies aligned with the distribution (for example, cycles 0 and 40), particularly with a higher MAD threshold of 3.

• Distance metrics double as preprocessing tools. In the Tohoku dataset, the authors computed normalized Mahalanobis distance from the cycle index and maximum discharge capacity per cycle and used this distance as the input feature to ML-based anomaly detection models.

• ML models have qualitatively different decision boundaries. Isolation Forest isolates anomalies by recursive splitting, producing irregular decision boundaries; Local Outlier Factor, being density-based, produces intricate density-based contours rather than smooth symmetric shapes; Autoencoder's boundary looked very similar to PCA's, but the Autoencoder can also capture nonlinear patterns, making it more flexible for complex cases.

• The commonly used MAD scale factor may mislead. The authors report that the commonly used MAD factor (F_MAD = 1.4826) may lead to many false predictions if the underlying distribution is not Gaussian, and that a more robust approach is to compute F_MAD from the reciprocal of the 75th-percentile of the corresponding standard distribution.

• Anomalies are not chemistry- or protocol-specific. The authors found abnormal observations not only in the Severson dataset (in which a discharge voltage in cell 2017-05-12_5_4C-70per_3C_CH17 reached up to 4.5 V against a specified operating window of 2 V to 3.6 V) but also in an open-circuit voltage (OCV) temperature profile and a dynamic voltage profile from different data sources and institutions.

• Anomalies occur throughout cycling, not only at the start. Anomalous cycles were discovered not only in the first cycle but also across multiple cycles beyond the first 100 cycles.

Methodology in Plain English

The authors start from a practical observation: anomalies in battery cycling data are buried inside otherwise normal measurements, so their marginal distributions overlap heavily and cannot be separated by statistics or ML directly. To fix this, they first apply a statistical transformation that subtracts the ratio of the squared median to the IQR from each data point, which amplifies the gap between normal and abnormal cycles while retaining the original physical units. Second, they treat runs of continuously abnormal voltage and capacity readings as a single affected cycle, and reduce each cycle to three numbers: the largest single-step scaled voltage jump, the largest single-step scaled capacity jump, and the largest ratio between the two. These three features turn a messy multi-point anomaly into a cycle-level point anomaly that detectors can rank.

The authors deliberately keep all three features rather than collapsing them into one, because combining features amplifies anomaly scores and can let a few extreme cycles overshadow smaller but meaningful anomalies — and because missing a true anomaly is far more dangerous for battery safety than flagging a normal cycle.

For detection, they compare three families of methods on both datasets. Statistical detectors (SD, MAD, IQR, Z-score, modified Z-score) look at where a value sits in a single feature's distribution. Distance-based detectors compare a cycle's position to the center of the normal data, and differ in the geometry of the boundary they draw. Unsupervised ML models (Isolation Forest, KNN, GMM, LOF, PCA, Autoencoder) look at multiple features at once. Because every model produces its own unscaled anomaly score requiring its own threshold, the authors min-max convert all scores to a [0, 1] probability-like range and apply a single consistent threshold of 0.7 for the ML comparison. For hyperparameter tuning of the unsupervised models, they use two strategies — training with labeled outliers and transferring the best average hyperparameters to unlabeled test data, or using predicted inliers as features for a downstream supervised task — and they use Bayesian optimization rather than random or grid search to sample promising hyperparameters efficiently.

Why This Matters

Anomalies in battery data are not just a data-quality nuisance. The paper stresses that violating temperature and voltage operating constraints through overcharge, overdischarge, or extreme temperatures can cause internal short circuits, toxic chemical reactions, and self-ignition, which can escalate to thermal runaway. Because reports on battery anomalies are scattered, real-life battery monitoring still often relies on manual inspection, which does not scale to hundreds of cells over many cycles. By releasing labeled benchmarking datasets and a reproducible pipeline, OSBAD gives the field a common yardstick and a starting point for trustworthy, data-driven diagnostics in safety-critical energy systems.

Real-world applications:

• Electric vehicle fleets, where anomalous charge and discharge behavior can indicate pack faults.
• Battery-powered data centers and backup power systems, where flagged deviations from baseline voltage profiles can prevent downtime.
• Consumer electronics, including electric scooters, where faulty battery packs have been identified through clustering-based methods.
• Short-range electric aircraft, which depend on reliable performance monitoring for safe operation.

Industry relevance: The authors explicitly motivate simple centroid-based distance methods for situations where low latency, model interpretability, or limited computational resources matter — such as process-control hardware or embedded monitoring systems — offering a practical alternative to computationally intensive ML algorithms.

Future Directions

• Extending OSBAD beyond batteries. The authors state that cross-chemistry benchmarking lays the foundation for robust anomaly detection beyond battery applications in future work.
• Better evaluation metrics for imbalanced data. The paper argues that ROC and AUROC can misrepresent accuracy when anomalies are rare, and points toward AUPRC or Matthew Correlation Coefficient as more robust alternatives — an invitation to re-benchmark methods under those metrics.
• Addressing the univariate limitation. Since all statistical detectors missed minor anomalies that fell inside the distribution, developing consistent multivariate detection that handles subtle anomalies without amplifying scores remains open.
• Reducing label dependence. The hyperparameter tuning strategies rely on labeled outliers for training or downstream supervision; scaling anomaly detection to truly unlabeled settings without domain knowledge remains a challenge.

Target Audience

This paper is most useful to battery engineers and data scientists building monitoring and diagnostics pipelines, to ML practitioners working on unsupervised anomaly detection in time-series sensor data, and to researchers seeking a reproducible benchmark with open labels. It is also relevant to reliability and safety engineers in industries where failure to detect a battery fault carries consequences beyond data quality, such as automotive, aerospace, and energy storage.

Note on reported numbers: The provided content lists anomaly counts, cell indices, and threshold values as described above, but it does not report final aggregated benchmark scores (such as AUPRC or MCC values) or a complete ranked comparison of all 15 algorithms; the excerpt also truncates the detailed descriptions of LOF, PCA, and Autoencoder hyperparameters, so those specifics are not available from the text summarized here.

Authors’ abstract

Battery safety is critical in applications ranging from consumer electronics to electric vehicles and aircraft, where undetected anomalies could trigger safety hazards or costly downtime. In this study, we present OSBAD as an open-source benchmark for anomaly detection frameworks in battery applications. By benchmarking 15 diverse algorithms encompassing statistical, distance-based, and unsupervised machine-learning methods, OSBAD enables a systematic comparison of anomaly detection methods across heterogeneous datasets. In addition, we demonstrate how a physics- and statistics-informed feature transformation workflow enhances anomaly separability by decomposing collective anomalies into point anomalies. To address a major bottleneck in unsupervised anomaly detection due to incomplete labels, we propose a Bayesian optimization pipeline that facilitates automated hyperparameter tuning based on transfer-learning and regression proxies. Through validation on datasets covering both liquid and solid-state chemistries, we further demonstrate the cross-chemistry generalization capability of OSBAD to identify irregularities across different electrochemical systems. By making benchmarking database with open-source reproducible anomaly detection workflows available to the community, OSBAD establishes a unified foundation for developing safe, scalable, and transferable anomaly detection tools in battery analytics. This research underscores the significance of physics- and statistics-informed feature engineering as well as model selection with probabilistic hyperparameter tuning, in advancing trustworthy, data-driven diagnostics for safety-critical energy systems.