MetaCaster: Meta-Harness-Optimized Agent for End-to-End Few-Shot Learning of Lightweight Time Series Forecasters

AI summary

Overview

• Research area: Time series forecasting (TSF), specifically few-shot learning of lightweight (non-pre-trained, compact) forecasters using LLM-based multi-agent systems.
• Technical level: Intermediate. Readers need basic familiarity with time series forecasting metrics (MSE, MAE), the distinction between foundation models and lightweight models, and the concept of LLM agent "harnesses" (system prompts, skills, tools, memory).
• Scope: The paper proposes MetaCaster, a meta-harness-optimized multi-agent framework in which agents act as intermediary engineers that generate training data and train/select a lightweight forecaster from only a few samples plus a textual domain description.

What This Paper Is About

Lightweight time series forecasters are cheap to run but have no large-scale pre-training, so they need substantial downstream training data — which is often unavailable in domains where data is scarce, slowly accumulated, or privacy-sensitive. MetaCaster addresses this by using LLM agents not to forecast directly, but to generate a domain-compliant synthetic training dataset from a small support set and a text description, then train and select the best lightweight forecaster from that generated data. At inference time only the small trained forecaster is kept, making the deployed system far cheaper than LLM- or TSFM-based forecasting.

Key Contributions

1. Problem formulation: The paper investigates few-shot learning for lightweight forecasters, a setting the authors describe as challenging because these models lack large-scale pre-training and are prone to overfitting.
2. The MetaCaster framework: A meta-harness-optimized multi-agent system with three components — MGagent (Meta-Generator, which builds a TS-Generator program), FTagent (Forecaster Trainer, which trains and selects forecasters) and HPagent (Harness Proposer, which optimizes MGagent's harness) — that prepares a specialized lightweight forecaster for a target task rather than forecasting directly.
3. LT-Lib: A unified-API library compiling 23 state-of-the-art lightweight forecasters (2022–2026), including linear-layer models (e.g., MixLinear) and MLP-based (e.g., TSMixer) and frequency-domain (e.g., FITS) models, released alongside MetaCaster.
4. Comprehensive evaluation: Experiments on 18 datasets against 14 baselines, with 23 lightweight forecasters in LT-Lib, across K ∈ {10, 30, 50}, IND and OOD corpora.

The authors also position MetaCaster as, to their knowledge, the first framework to align data generation directly with forecaster quality, and the first time series agent exposed to automatic harness optimization.

Main Findings

• Superior few-shot forecasting: In Table 1 (MSE), MetaCaster is reported as best in 19 of the 30 dataset/K combinations ("Wins (of 30)"), outperforming time series generation models (TimeDP, VerbalTS, T2S, DiffTS, TimeVAE) and augmentation methods (Repeat, Bootstrap, Jitter, MagWarp) in most cases.
• Performance scales with K: MetaCaster improves as K grows from 10 to 30 to 50. For example, on ETTm1 it records 0.376 at K=10, 0.345 at K=30 and 0.267 at K=50; on Electricity, 0.300, 0.226 and 0.191 respectively.
• Can match or beat full-data training: When K ≥ 30, MetaCaster approaches or even surpasses forecasters trained on the full authentic training set D_tr (the upper reference), which the authors attribute to raw data being noisy and optimized data improving training.
• Competitive in the extreme few-shot case: At K = 10 MetaCaster remains competitive — for example 1.522 on Seattle versus 1.420 for D_tr, and 0.158 on Solar versus 0.157 for D_tr.
• OOD generalization: MetaCaster generalizes to out-of-domain datasets (Saugeen, USbirths, M4), generally outperforming baselines despite the increased difficulty.
• Lower-variance forecaster quality: Fig. 4 shows MetaCaster produces more high-quality forecasters with lower variance, and generalizes better to forecasters held out from LT-Lib.
• Agent pipeline alone is insufficient: Fig. 3 shows TimeScientist, which cannot generate time series, struggles to train generalizable forecasters and does not scale with K.
• Large efficiency gains versus TSFMs: On Solar at K = 30, MetaCaster selects MixLinear (243 parameters) at runtime, and achieves up to 10^3 times lower latency and 10^4 times fewer parameters than TSFMs at comparable performance.
• Forecasting-oriented objectives beat distribution matching: Ablation Table 2 (K = 30, overall normalized MSE): MetaCaster 0.267, versus MMD 0.764 and Wasserstein 0.940 when Eq. (1) is replaced by distribution-alignment objectives.
• Context matters: Removing the textual context C_m raises overall normalized MSE from 0.267 to 0.521.
• The harness matters more than the LLM backbone: Swapping LLMs gives overall normalized MSE of 0.288 (Gemini-3.1-Pro), 0.321 (Claude-Opus-4.7), 0.366 (Qwen3.5-122B-A10B) and 0.677 (GPT-5.3-Codex) versus 0.267 for the default GPT-5.4, which was chosen for consistent performance; GPT-5.3-Codex was unstable on datasets such as ETTm1 and USbirths.
• Fast harness convergence: Fig. 6(a) shows the hinge loss (Eq. 4) over 8 harness optimization epochs, converging quickly with the final harness selected from epoch 5.
• Illustrative harness repair: In the alibaba_cluster_2018 case study, HPagent diagnoses broken inter-variate correlation between CPU and memory at epoch 4 and updates MGagent's skills, producing correlated series at epoch 5 that better comply with the few-shot examples.

Methodology in Plain English

The setup begins with a very small support set of K examples (K ∈ {10, 30, 50}) drawn from a target dataset, plus a text description of the domain such as climatology or healthcare.

Instead of asking an LLM to forecast numbers directly — which the authors argue is hampered by the gap between discrete language tokens and continuous values — MetaCaster asks the LLM to write a TS-Generator program. That program uses domain knowledge, rules and models to synthesize a larger dataset from the few examples. The LLM is used for planning, reasoning and coding rather than numerical prediction.

A second agent, FTagent, splits the generated data into training and validation portions matching the originals, then trains every forecaster in LT-Lib with hyperparameter grid search, queuing (forecaster, hyperparameter, dataset) jobs and distributing them across available GPUs, handling errors and restarts without human intervention.

Quality is steered by a third agent, HPagent, which acts as the optimizer. After each round it compares the error of forecasters trained on generated data against forecasters trained on authentic data, using a hinge-loss penalty (Eq. 4) that only triggers when the generated-data forecaster is worse. HPagent then analyzes evidence (the loss, statistical differences such as MMD, training logs), diagnoses causes from MGagent's reasoning traces and harness, and edits the system prompt and skills. Only the prompt and skills are trainable; the LLM is frozen. A batch of B = 8 datasets is processed per epoch and K is resampled uniformly from [10, 50] each epoch, which helps generalizability and reduces overfitting to one dataset.

After optimization, HPagent is discarded, the LLMs are removed, and the harnesses of MGagent and FTagent (including LT-Lib) are attached to whatever LLM API the user chooses. For a new domain, the agents run forward once to produce a trained forecaster; from then on only that one lightweight forecaster is needed for inference.

Why This Matters

The work reframes the role of LLM agents in time series: rather than acting as forecasters, they act as engineers that prepare an efficient, task-specific model for deployment. This is a distinct third paradigm alongside "LLM-as-Forecaster" and "Agent-as-Forecaster." By showing that a few samples plus context can substitute for large training corpora, it opens a practical route to deploying forecasting where data collection is slow or legally constrained. It also shows, consistent with prior harness research, that agent infrastructure can matter more than the choice of LLM backbone, which has implications for how agentic systems are built and optimized generally.

Real-world applications:

• Edge and resource-constrained deployment: the paper explicitly targets edge devices and small organizations that cannot run GPU-intensive TSFMs; on Solar, the selected model has 243 parameters and achieves up to 10^3 times lower latency than TSFMs.
• Healthcare and finance: named as privacy-sensitive domains where acquiring large training datasets may be infeasible.
• Energy, weather and traffic: these are the domains covered by the 18 GIFT-Eval benchmark datasets used in the evaluation.
• IT infrastructure monitoring: the case study uses the alibaba_cluster_2018 dataset with CPU and memory variates, a typical operational forecasting scenario.

Industry relevance: the work is a collaboration spanning the University of Houston, NEC Labs, the University of Waterloo, the University of Connecticut, the University of Illinois at Urbana-Champaign and Singapore Management University, was partially supported by a research gift from NEC Laboratories America and the NVIDIA Academic Grant Program, and the code is released at https://github.com/D2I-Group/metacaster.

Future Directions

• Zero-shot operation: the authors state the current work does not address the extreme zero-shot setting, where no reference examples exist and the agent lacks statistical grounding; this is where TSFMs retain an advantage.
• Broader domain coverage: experiments cover 18 datasets in a limited set of domains; the authors plan to extend to the full pre-training corpora of modern TSFMs to further improve harness optimization.
• Expanding LT-Lib: the library currently holds the lightweight forecasters collected so far and is not claimed to be exhaustive; the authors intend to keep updating it.
• Open question on harness transfer: the paper reports that the optimized harness is transferable across LLMs, enabling flexible API switching, but the limits of that transfer across backends and task types are not fully mapped out here.

Target Audience

Researchers and practitioners working on time series forecasting, especially those interested in few-shot learning, lightweight or edge-deployable models, and agentic AI. It is also relevant to LLM agent researchers interested in harness engineering and automated prompt/skill optimization, and to industry engineers in energy, healthcare, finance, traffic or IT operations who need deployable forecasting without large labeled datasets or GPU-heavy foundation models. Readers seeking a zero-shot solution or an exhaustive survey of lightweight forecasters will not find those here.

Authors’ abstract

Time series forecasting (TSF) is evolving toward multimodal and agentic settings, yet using foundation models remains uneconomical in resource-constrained scenarios, where compact, specialized forecasters are more desirable. However, lightweight forecasters typically require substantial training data, limiting their use in domains with scarce, slowly accumulated, or privacy-sensitive time series. To address this dilemma, we investigate the challenging problem of few-shot learning for lightweight forecasters. We propose MetaCaster, a meta-harness-optimized multi-agent framework that uses agentic data generation to automatically train specialized lightweight forecasters from only a few examples and textual contexts. Our work highlights a new TSF paradigm in which agents act not as forecasters but as intermediary engineers that prepare efficient, task-specific forecasters for deployment. Experiments on 18 datasets, 23 state-of-the-art lightweight forecasters, and 14 baselines demonstrate that MetaCaster achieves both data efficiency and computational efficiency while maintaining high-quality TSF performance.