Meta-Learning and Fast Adaptation

Ordinary supervised learning assumes one task and many examples. Meta-learning assumes a distribution of tasks and tries to acquire a reusable adaptation strategy. A task might be a new user, robot environment, language, product category, class set, or regression function. The model is trained across many such tasks, then evaluated on new tasks from a related distribution. This framing is essential. If meta-training and deployment tasks do not share useful structure, rapid adaptation cannot be expected. A method can perform well on synthetic episodes and fail when real tasks vary in different ways.

Change which tasks are held out and the score moves ten points, with the model untouched. Omniglot shows this cleanly. The dataset is fixed across both papers involved: 1623 characters from 50 alphabets, 20 examples per character, each drawn by a different person. On the original 20-way within-alphabet one-shot task, with 964 background characters from 30 alphabets, humans err on 4.5% of trials and Bayesian Program Learning on 3.3%. A prototypical network errs on 13.7%. That figure comes from Omniglot's own authors, in a 2019 three-year progress report, retraining the method themselves: “we modified the implementation from Snell et al. (2017) for within-alphabet classification”. The same method on the easier between-alphabet split, with 4,800 augmented background classes, errs on 4.0% — the figure the prototypical networks paper reports as 96.0% accuracy at 20-way 1-shot Omniglot. “The neural network performs with an error rate of 13.7% (Table 1 column 1), which is substantially worse than the 4.0% error for the between-alphabet problem,” they write. Cut the background down to five alphabets and the meta-learner degrades further, to 30.8% and 29.3% error, while BPL stays at 4.3% and 4.0%. One method, one dataset, two ways of deciding what to hold out. The human number sits fixed at 4.5% in between.

Meta-learning also overlaps with transfer learning and self-supervision. Pretraining can yield fast adaptation without an explicit meta-objective. The correct question is not whether an algorithm carries the meta-learning label, but whether task-level training produces a measurable advantage on new-task sample efficiency.

The headline benchmark numbers are closer together than the family names suggest. The MAML paper reported 48.70% on 5-way 1-shot miniImageNet. The nearest-neighbour baseline listed in the same table scored 28.86%, and matching networks 43.56%. A first-order variant that drops the second derivative scored 48.07%. The prototypical networks paper then reported 49.42% on the same split. A metric method and a learned initialization landed within a point of each other.

Optimization-based meta-learning differentiates between learning within a task and learning how to start across tasks. Sample a task: draw a support and a query set from the meta-training task distribution. Run the inner loop: update task-specific parameters using the support set. Evaluate the adapted model: measure query loss after adaptation. Run the outer loop: change the shared initialization or meta-parameters so that future adaptation goes better. Then test on unseen tasks, repeating adaptation on tasks excluded from meta-training.

That is the mechanism as designed. How much of the result the inner loop actually produces is a separate question, and it has been measured. A 2020 paper asked whether MAML works by rapid learning or by feature reuse. Its authors removed the inner-loop update from every layer except the task-specific head — the ANIL algorithm — and matched MAML. Table 2 reports 46.7 ± 0.4% for ANIL against 46.9 ± 0.2% for MAML on 5-way 1-shot miniImageNet, 61.5 ± 0.5% against 63.1 ± 0.4% at 5-shot, and 96.2 ± 0.5% against 93.7 ± 0.7% on 20-way 1-shot Omniglot. ANIL is also cheaper, giving “an average speedup of 1.7x per training iteration over MAML and an average speedup of 4.1x per inference iteration”. Their own summary of what the ablations show: “We investigate this question, via ablation studies and analysis of the latent representations, finding that feature reuse is the dominant factor.”

Another group, publishing later in 2020, reaffirmed it with their own experiments. They write of the ANIL paper that “It concluded that feature reuse is the dominant component in MAML's efficacy, which is reaffirmed by experiments conducted in this paper.” Two numbers a fifth of a point apart are the whole argument. On these benchmarks, gradient adaptation of the network body is close to doing nothing. What the outer loop is mostly buying is a representation.

The families differ in what is learned across tasks.

Metric-based methods learn a representation and a comparison rule for support and query examples. Adaptation is often simple at test time, and they are strong for few-shot classification. They depend on episodic class design. Matching and prototypical networks are the standard examples, at 43.56% and 49.42% on 5-way 1-shot miniImageNet.

Optimization-based methods learn parameters that adapt effectively through a few gradient steps. They have explicit inner and outer loops, apply to several model classes, and carry a second-order computation that can be costly. MAML and its first-order variants are the example. The variant that drops the second backward pass scored 48.07% against MAML's 48.70% on that same split. ANIL, which drops the inner-loop update everywhere except the head, scored 46.7 ± 0.4% against MAML's 46.9 ± 0.2% in the 2020 comparison, at 1.7x per training iteration and 4.1x per inference iteration.

Learned-optimizer and memory methods learn an update rule, a recurrent controller, or a memory mechanism. They can express rich adaptation behavior, are harder to train and interpret, may overfit the task generator, and are most useful when update structure repeats.

The taxonomy is real. But the gap between the first two families on the usual benchmarks is roughly a point, which is smaller than the gap between any of them and a plain pretrained embedding measured under matched conditions. That is the comparison the evaluation section makes.

Task boundaries should reflect how the product will encounter new situations. FS-Mol shows what that looks like when a real domain is cut into tasks and the result is measured. It was built in 2021 out of 5,120 assays drawn from ChEMBL27: 233,786 unique compounds, 489,133 measurements, a mean of 94 compounds per task, and 157 held-out test tasks.

At support-set size 16, the mean improvement over a random classifier (ΔAUPRC) across all 157 test tasks was 0.206 ± 0.008 for a prototypical network, 0.160 ± 0.009 for GNN-MAML, 0.112 ± 0.006 for a multitask-pretrained GNN, 0.093 ± 0.007 for a random forest fitted on the support set alone, and 0.021 ± 0.005 for a single-task GNN. The gain tracked how many related training tasks existed. Kinases, with 125 test tasks and 1,497 training tasks, profited most. Ligases and translocases, with fewer than 40 training tasks each, barely at all. “However, we stress that performance is highly task-dependent, as evidenced by the broad range of improvements over the full set of few-shot testing tasks in Figure 2b,” the authors write. A 2025 re-run of the benchmark set those published baselines beside plain fine-tuning probes, which reach 0.224 ± 0.010 (linear) and 0.227 ± 0.010 (quadratic) at the same support size — above every meta-learner in the table except ADKF-IFT (0.231 ± 0.009). Defining the tasks well is what produced the advantage. It is not what guarantees the advantage survives a strong baseline.

Meta-learning should beat simpler pretraining under the same task and data budget. Define the task boundaries, specifying what counts as a new task in deployment. Hold out complete tasks, keeping meta-test tasks separate from every training decision. Match adaptation budgets, using the same support size, steps, and compute for every method. Compare against ordinary pretraining, including fine-tuning from a shared pretrained model. Plot adaptation curves, measuring quality after zero, one, and several updates or examples. Then test task-distribution shift, evaluating tasks that vary in harder ways than the training generator.

The fourth step is the one that changes conclusions, and two groups have run it. Train an ordinary supervised embedding on the merged meta-training set, then fit a logistic-regression classifier per test task. In 2020 that plain recipe scored 62.02 ± 0.63% at 5-way 1-shot on miniImageNet and 79.64 ± 0.44% at 5-shot, rising to 64.82 ± 0.60% and 82.14 ± 0.43% with self-distillation. In the same table, MetaOptNet scored 62.64 ± 0.61% / 78.63 ± 0.46%, prototypical networks 49.42 ± 0.78% / 68.20 ± 0.66%, and MAML 48.70 ± 1.84% / 63.11 ± 0.92%. The abstract states the result plainly: “In this work, we show that a simple baseline: learning a supervised or self-supervised representation on the meta-training set, followed by training a linear classifier on top of this representation, outperforms state-of-the-art few-shot learning methods.”

The same pattern had been reported in 2019, from a re-implementation with matched training, which removes the objection that the baseline simply had a bigger backbone. A plain pretrained classifier with a cosine head, Baseline++, scored 48.24 ± 0.75% / 66.43 ± 0.63% on miniImageNet with a Conv-4 backbone. MAML scored 46.47 ± 0.82% / 62.71 ± 0.71% and ProtoNet 44.42 ± 0.84% / 64.24 ± 0.72%. Run the baseline under the same budget, and the specialized meta-objective has to earn its place against numbers like these rather than against an unmatched control.