NCAA Bracket Prediction Using Machine Learning and Combinatorial Fusion Analysis

AI summary

Overview

Research area: Machine learning applied to sports outcome prediction, specifically ensemble learning and ranking methods for NCAA Men's Basketball Tournament (March Madness) bracket prediction.

Technical level: Intermediate. The paper uses standard ML classifiers (logistic regression, SVM, Random Forest, XGBoost, CNN) but introduces a less common ensemble paradigm — Combinatorial Fusion Analysis (CFA) — built on rank-score characteristic (RSC) functions and cognitive diversity.

Scope: The paper applies five machine learning base models and a CFA ensemble framework to predict 2024 NCAA men's basketball tournament game and team outcomes, and compares its team-ranking accuracy against ten public ranking systems.

What This Paper Is About

Predicting March Madness brackets is usually framed as a classification problem: given two teams, predict which one wins. This paper reframes part of the problem as a ranking problem, asking whether a team can be ranked above another based on the confidence of its predicted wins. The goal is to build a combination of five machine learning models using Combinatorial Fusion Analysis, choose the ensemble that improved performance most consistently over the previous 10 years, and apply it to 2024 tournament data to produce team rankings and an accuracy figure.

Key Contributions

1. Application of CFA to sports prediction. The paper applies Combinatorial Fusion Analysis — previously used in bioinformatics, virtual screening and drug discovery, portfolio management, and information retrieval — to NCAA bracket prediction, combining models in both score space and rank space.

2. A ranking-oriented view of sports prediction. Instead of only classifying each game, the authors convert game-level rankings into team-level rankings (average rank across a team's games, sorted ascending) and evaluate accuracy from those team rankings.

3. A model-selection procedure based on historical consistency. Rather than picking the ensemble that performed best in a single test year, the authors count how often each combination improved on the best individual model across the previous 10 years, selecting the ensemble that appeared most frequently.

4. A benchmark comparison against ten public ranking systems. Rank-combination team ranking reached 74.60% accuracy, compared to the highest public system accuracy of 73.02%.

Main Findings

• Rank combination outperforms public ranking systems. The CFA team ranking using the rank-combination framework achieved 74.60% accuracy, which the paper states is 1.58% higher than the best of the ten public ranking systems in Table I.

• The best public systems were NET Rankings and Logan, both at 73.02%. The other systems reported in Table I were: INCC Stats 71.43%, Jelly Juke 66.67%, Joby Nitty Gritty 69.84%, Massey 69.84%, Moore 71.43%, ESPN SOR 69.84%, Sports Ratings 68.25%, and Donchess Inference 69.84%.

• Score combination was weaker. The "ABE" ensemble using score combination produced a team-ranking accuracy of 71.43%, which the paper notes "beats half of the ranking systems in Table I."

• The selected ensemble was "ABE." The combination of logistic regression (A), SVM (B), and CNN (E) showed improvement 6 times across the previous 10 years, making it the most frequent improving ensemble, and was therefore used for the 2024 prediction.

• Diversity mattered for ensemble success. The paper reports that models with greater cognitive diversity have a higher chance of a successful ensemble when their individual performance is relatively good. On 2022 test data, combinations such as logistic regression (A) with XGBoost (D), and logistic regression (A), XGBoost (D), and CNN (E), showed superior performance.

• Ensemble space explored. With five base models, the framework yields (2^5 − 1 − 5) × 3 × 2 = 156 possible ensemble models. Because the authors used only diversity strength as the weight, this reduced to (2^5 − 1 − 5) × 2 = 52 models, all evaluated with accuracy. Note: the conclusion section separately refers to "56 ensembles from 5 base models," a figure that differs from the 52 reported in the results section; the paper does not explain the difference.

• Feature reduction. From a starting set of 44 features, Recursive Feature Elimination with Cross-Validation (RFECV) using 5-fold cross-validation and the log loss metric selected an optimal subset of 26 features.

Methodology in Plain English

The authors gathered historical NCAA data — regular season results, tournament outcomes, and team statistics — covering tournaments from 2001 to 2022, excluding 2020 because of COVID-19, primarily from the Kaggle "March Machine Learning Mania" competition (hosted since 2011) plus additional team data from the KenPom website.

For each game they computed difference variables by subtracting Team 2's statistics from Team 1's, keeping only these difference features. Because the collected data always listed the winner as Team 1, every game would have carried the label 1, leaving only a single class. To create a second class, they swapped Team 1 and Team 2 variables while keeping their original designations, producing label 0 cases as well.

They then trained five base models: logistic regression (with L1 or L2 regularization chosen via randomized search with 10-fold cross-validation), a support vector machine (kernel and parameters chosen by randomized search on log loss), Random Forest, XGBoost, and a CNN (ReLU in hidden layers, sigmoid in the output layer, Adam optimizer, cross-entropy loss). Each model was trained with stratified 10-fold cross-validation with three repetitions, and parameters were tuned by randomized search.

The five trained models were fed into the CFA pipeline. CFA works with two things for each model: a score function (e.g., a predicted probability) and a derived rank function (the ordering of those scores). The rank-score characteristic (RSC) function maps a rank back to its score, and the cognitive diversity between two models is the root-mean-square distance between their RSC functions. Average diversity strength measures how different a model is from the others. CFA then combines predictions by score combination or rank combination, each of which can use average weighting, weighting by performance, or weighting by diversity strength — six possible forms per ensemble. In this paper, only diversity strength was used as the weight for both score and rank combination.

To decide which ensemble to trust for 2024, the authors examined each of the previous 10 years, drew the RSC functions and combination performances, and counted which combinations improved on the best individual model. The most frequent winner was "ABE." Team rankings were then derived by ranking games by confidence that Team 1 beats Team 2, taking each of the 64 teams' average rank as its score, sorting those scores ascending, and treating the higher-ranked team as the predicted winner.

Why This Matters

• It shows ranking-based ensembles can beat published rating systems. Even a modest edge (74.60% versus 73.02%) is meaningful in a domain where public systems are widely used and heavily optimized.

• It demonstrates a transfer of CFA to a new domain. CFA had been applied to bioinformatics, virtual screening and drug discovery, portfolio management, and information retrieval; this paper extends it to sports analytics.

• It highlights diversity as a design principle. The finding that cognitively diverse model pairs (like SVM and Random Forest, versus SVM and logistic regression) combine better is a practical guide for building ensembles generally.

Real-world applications:

• Bracket prediction for fans, pools, and media coverage of March Madness.
• Sports analytics and betting-related modeling, a context the paper frames with a May 2023 NCAA survey of 3,527 eighteen-to-twenty-two year olds in which 58% reported engaging in at least one sports betting activity.
• Team and opponent evaluation, since the method produces a full team ranking rather than only game-by-game probabilities.
• General ensemble modeling, since the CFA workflow is domain-independent — the RSC function operates on ranks rather than on the data items themselves.

Industry relevance: Any setting that combines several scoring systems and cares about ordering — recommendation, search, credit scoring, risk ranking — faces the same problem of how to fuse predictions. Weighting by diversity strength, as done here, is an alternative to simple averaging or majority voting.

Future Directions

1. Use all three weighting schemes. The authors plan to extend to all three types of weights (average combination, weighted combination by performance, and weighted combination by diversity strength), since each captures a different aspect of each base model's contribution. This study used only diversity strength because of page and space constraints.

2. Compare against other ensemble models in the literature. The paper states this comparison as a planned extension.

3. Resolve and reconcile the reported ensemble counts. The paper reports 52 models in the results section and 56 ensembles in the conclusion; clarifying the exact ensemble space would make the selection procedure fully reproducible.

4. Validate the pattern over more future tournaments. The 2024 season is a single held-out application. Whether "ABE" or a different combination remains the most consistently improving ensemble in later years is an open question the paper does not answer.

Target Audience

Readers who will benefit most are machine learning practitioners and researchers working on ensemble methods and rank aggregation, sports analytics researchers and data scientists who work with Kaggle-style competition data, and statisticians interested in diversity measures for classifier combination. The paper is also useful for readers new to Combinatorial Fusion Analysis who want a concrete, compact case study, since it defines RSC functions, cognitive diversity, and diversity strength and then applies them end to end. A background in basic classification and evaluation metrics (accuracy, cross-validation) is sufficient to follow the argument; the CFA machinery is introduced from scratch.

Authors’ abstract

Machine learning models have demonstrated remarkable success in sports prediction in the past years, often treating sports prediction as a classification task within the field. This paper introduces new perspectives for analyzing sports data to predict outcomes more accurately. We leverage rankings to generate team rankings for the 2024 dataset using Combinatorial Fusion Analysis (CFA), a new paradigm for combining multiple scoring systems through the rank-score characteristic (RSC) function and cognitive diversity (CD). Our result based on rank combination with respect to team ranking has an accuracy rate of $74.60\%$, which is higher than the best of the ten popular public ranking systems ($73.02\%$). This exhibits the efficacy of CFA in enhancing the precision of sports prediction through different lens.