XLinear: A Lightweight and Accurate MLP-Based Model for Long-Term Time Series Forecasting with Exogenous Inputs

AI summary

Overview

Research area: Deep learning for time series forecasting, specifically long-term forecasting models that combine an endogenous target series with exogenous (external driver) variables.

Technical level: Intermediate. The model is architecturally simple (MLPs plus gating), but understanding it requires familiarity with standard forecasting concepts such as look-back windows, forecast horizons, channel independence, and evaluation metrics like MSE/MAE.

Scope: The paper proposes XLinear, an MLP-based forecasting model that uses a learnable "global token" per endogenous variable to absorb information from exogenous variables, and evaluates it against ten baselines across seven benchmark datasets and five real-world environmental datasets.

What This Paper Is About

Most time series forecasting models assume all variables matter equally and are mutually informative, but real applications often have asymmetric relationships: cheap external data (like local weather) drives a target variable (like lake surface temperature) without being affected by it. XLinear's goal is to exploit these one-directional links efficiently, matching the accuracy of heavy Transformer models while training faster and using less memory. The paper targets the trade-off between the efficiency of MLP-based models and the accuracy of patch-based Transformers.

Key Contributions

1. Bridging the efficiency-accuracy gap with exogenous inputs. The work explicitly incorporates exogenous inputs and reconciles temporal dependencies with cross-variable dependencies, addressing the trade-off between the efficiency of MLP-based models and the accuracy associated with patch-based Transformers.

2. Introducing XLinear, a novel MLP-based model. Its core innovation is a gating module built from an MLP with a sigmoid activation function. The model uses learnable global tokens derived from endogenous sequences as hubs for interacting with exogenous variables, enabling selective feature filtering along both the temporal and variable dimensions.

3. Demonstrating superior performance at scale. Across 12 diverse datasets, XLinear is reported to consistently outperform state-of-the-art models in both accuracy and efficiency, with at least 30% faster training speeds than efficient Transformer baselines.

4. Providing public code and interpretability evidence. The implementation is released at https://github.com/Zaiwen/XLinear.git, and weight visualizations on the DE dataset (a subset of the EPF dataset) are used to show which variables and time steps the model emphasizes.

Main Findings

• Univariate forecasting with exogenous inputs on 7 benchmarks: XLinear achieves the lowest MSE in 89.3% (25 of 28) of experimental configurations and the best MAE in 75.0% of cases. It ranks in the top two models in every configuration except the 336-step lead time on ETTh1.

• Efficiency on the Electricity benchmark: Compared with all baselines except DLinear and RLinear, XLinear achieves at least 39.3% faster training, 13.9% lower memory usage, and 2.8% lower MSE. DLinear's and RLinear's MSEs are about 23.5% and 39.6% higher than XLinear's, respectively. Under a unified batch size of 4, XLinear matches the training speed of DLinear and RLinear, consumes less memory than the state-of-the-art models, and delivers the highest forecast accuracy among all 10 comparative models.

• Five real-world environmental datasets: XLinear achieves the lowest MSE and MAE in over 75% of cases. For Nash-Sutcliffe Efficiency (NSE) it ranks first in 75% of rows and second in 20%, reaching the "Good" performance level (NSE > 0.65) in 65% of cases. For Kling-Gupta Efficiency (KGE) and Mean Absolute Percentage Error (MAPE), it ranks among the top two models in 90% and 70% of cases respectively.

• Long-term multivariate forecasting: On the first six benchmark datasets, XLinear achieves the best performance in over 91.7% of cases for both MSE and MAE and ranks second in the remaining cases. Across all seven benchmarks it consistently outperforms the four GNN-based baselines.

• The Traffic exception: The Traffic dataset, with 862 variables, is the one case where iTransformer outperforms XLinear, which the authors attribute to iTransformer's variable-wise attention being better suited to high-dimensional forecasting. Averaged over the four forecast horizons, XLinear still has the lowest MSE and MAE on the first six datasets and ranks second in MSE and third in MAE on Traffic.

• Multivariate analysis on Weather: With a training batch size of 128 and input and output windows of 96, XLinear achieves at least 39.3% faster training, 43.7% lower memory usage, and 5.1% lower MSE than the other models in the figure, except DLinear and RLinear, whose MSEs are 31.5% and 28.9% higher respectively.

• Fine-grained fluctuations on PEMS: On the highly volatile PEMS03, PEMS04, PEMS07, and PEMS08 datasets, XLinear shows superior accuracy in most scenarios compared with TimeXer and PatchTST. The authors attribute this to direct modeling of individual dimensions after embedding, whereas patching can hinder learning of high-resolution temporal details.

• Interpretability: On the DE dataset, the variable-wise weight distribution gives higher weights to Channel 1 (Wind power) and Channel 3 (Electricity price), consistent with correlations visible in the input sequences, and the temporal weight distribution highlights important time steps.

Methodology in Plain English

The setup: given a history of endogenous variables (the series to be predicted) and a history of exogenous variables (external drivers), predict the next S steps for the endogenous variables.

The approach unfolds in four steps:

1. Joint embedding with a global token. Both endogenous and exogenous sequences are first processed with RevIN and then embedded into the same dimension. For each endogenous variable, a learnable global context token is concatenated onto its sequence along the time dimension.

2. Time-wise Gating Module (TGM). An MLP processes the combined endogenous-plus-global-token tensor, and a sigmoid function turns the output into gating weights that are multiplied element-wise with the input. This emphasizes salient temporal patterns and suppresses noise, while also passing temporal information into the global token.

3. Variate-wise Gating Module (VGM). The global token is concatenated with the exogenous embedding along the channel dimension, then processed by another MLP-plus-sigmoid gate in the same gating style. This lets the model pull relevant cross-variable information from exogenous series into the endogenous global token, modeling the asymmetric causal links the paper emphasizes.

4. Prediction head. The temporally enhanced endogenous sequence is concatenated with the updated global token along the time dimension and passed through a single fully connected layer to produce all S future values in a channel-independent manner. Training uses Mean Squared Error, averaged across endogenous channels, with the ADAM optimizer.

For multivariate forecasting, every variable plays both roles: it is predicted as an endogenous variable while also serving as an exogenous driver for other variables. The TGM is shared across all variables and the VGM across features.

Why This Matters

Impact on research: The paper challenges the assumption that patch-based Transformers are necessary for top accuracy in long-term forecasting with exogenous drivers. It shows that a two-block MLP design with sigmoid gating and a global-token hub can match or beat heavier models, providing a lightweight alternative baseline for future work and pointing out where attention still wins (the 862-variable Traffic dataset).

Real-world applications:

• Agriculture: Crop yield forecasting driven by soil moisture, air temperature, humidity, and related factors, using the Crop dataset.
• Water quality monitoring: Dissolved oxygen content in the Darling River and Murray River systems, driven by water temperature, mean discharge rate, and mean water level.
• Dam and reservoir management: Bottom water temperature at 9 m depth in Grahamstown Dam, affected by shortwave radiation, air temperature, cloud cover, and wind speed and direction.
• Energy markets: Electricity price forecasting on the DE dataset using exogenous signals including wind power and Ampirion zonal load.

Industry relevance: Lower memory use and faster training reduce the cost of retraining and deploying forecasting models, which matters for practitioners running many models on limited GPU hardware. The explicit exogenous-input design matches how operational forecasting is actually done in environmental and energy settings, where cheap external data drives an expensive-to-measure target.

Future Directions

• Scaling to high-dimensional settings. The authors state that future work will focus on reducing the VGM input dimensionality while preserving essential inter-variable dependencies, to improve scalability and performance on datasets like Traffic with 862 variables.
• Long look-back windows. The paper begins a section on long lookback windows (the provided content ends there), indicating this is an area of ongoing investigation.
• Better handling of noise in cross-variable modeling. The Traffic failure case shows that extracting informative signals while suppressing noise in high-dimensional spaces remains a significant challenge.
• Comparison with pre-trained foundation models. The paper notes that foundation time series models are almost all Transformer-based and that their accuracy remains limited on new or unseen datasets compared with full-shot state-of-the-art models, leaving room for lightweight models to be tested further in that setting.

Target Audience

Practitioners and researchers working on long-term time series forecasting, especially those who need to incorporate exogenous variables and care about compute budgets. It is also useful for engineers in environmental monitoring, water resource management, agriculture, and energy who need accurate forecasts on modest hardware, and for researchers benchmarking lightweight MLP alternatives against Transformer baselines.

Authors’ abstract

Despite the prevalent assumption of uniform variable importance in long-term time series forecasting models, real world applications often exhibit asymmetric causal relationships and varying data acquisition costs. Specifically, cost-effective exogenous data (e.g., local weather) can unilaterally influence dynamics of endogenous variables, such as lake surface temperature. Exploiting these links enables more effective forecasts when exogenous inputs are readily available. Transformer-based models capture long-range dependencies but incur high computation and suffer from permutation invariance. Patch-based variants improve efficiency yet can miss local temporal patterns. To efficiently exploit informative signals across both the temporal dimension and relevant exogenous variables, this study proposes XLinear, a lightweight time series forecasting model built upon MultiLayer Perceptrons (MLPs). XLinear uses a global token derived from an endogenous variable as a pivotal hub for interacting with exogenous variables, and employs MLPs with sigmoid activation to extract both temporal patterns and variate-wise dependencies. Its prediction head then integrates these signals to forecast the endogenous series. We evaluate XLinear on seven standard benchmarks and five real-world datasets with exogenous inputs. Compared with state-of-the-art models, XLinear delivers superior accuracy and efficiency for both multivariate forecasts and univariate forecasts influenced by exogenous inputs.