Skip to content
AI.info

AI literacy basics

Training, Inference, and Feedback

Learn the high-level lifecycle of an AI model and avoid common misconceptions about training, inference, personalization, and continuous learning.

By the end you can

Steps

The learning lifecycle in one pass

Later paths examine optimization and evaluation in depth. Here the aim is to understand which activity is happening and what evidence each stage can provide.

FigureProcess · 6 steps
  1. 1

    Frame the task

    Choose the target output, objective, users, constraints, and definition of success.

  2. 2

    Prepare evidence

    Collect, label, clean, split, and document data or experience relevant to the task.

  3. 3

    Fit the model

    Adjust parameters so performance improves on the training objective.

  4. 4

    Evaluate separately

    Test behavior on data, scenarios, or people not used to fit the model.

  5. 5

    Deploy for inference

    Use the fixed version to transform new inputs into outputs inside a workflow.

  6. 6

    Monitor and revise

    Observe failures, changes, feedback, and operational evidence before deciding what to update.

Training changes a parameterized model

During training, an algorithm adjusts model parameters so that outputs better match an objective on examples or experience. Two earlier choices set the limits. The model family decides what can change; the training procedure decides how improvement is measured.

Training does not simply pour facts into a container. It shapes a mapping that may generalize to new inputs, memorize parts of the data, or optimize a proxy in unintended ways.

The memorization part has been measured rather than assumed. Nicholas Carlini and colleagues presented “Extracting Training Data from Large Language Models” in August 2021. They queried GPT-2 XL, a 1.5-billion-parameter model trained on scrapes of the public web. They generated 1,800 candidate outputs. An author then manually checked every one of them. “In total across all strategies, we identify 604 unique memorized training examples from among the 1,800 possible candidates, for an aggregate true positive rate of 33.5%.” The recovered strings included names, phone numbers and email addresses. IRC conversations, code and 128-bit UUIDs came out too. The attack worked “even though each of the above sequences are included in just one document in the training data”. Their other result is the one to carry into any conversation about scale. “Worryingly, we find that larger models are more vulnerable than smaller models.”

Training is an optimization process guided by evidence and an objective, not automatic understanding.

Visual

Rehearsal, examination, and performance

Training, evaluation, and inference can use similar-looking inputs, but they serve different purposes and must remain conceptually separate.

The separation is physical as well as conceptual. Meta’s report on Llama 3, first posted on 31 July 2024, records what pre-training cost. A 405-billion-parameter model was trained on 15.6 trillion text tokens. That took 3.8 × 10^25 floating-point operations, on a cluster of up to 16,000 H100 GPUs. The published model card puts that single run at 30.84 million GPU-hours. The same report notes that “During a 54-day snapshot period of pre-training, we experienced a total of 466 job interruptions”, 419 of them unexpected. Inference on the result is one pass through the parameters that run produced. Training is a one-off industrial process measured in weeks of continuous cluster time. Inference is what happens every time someone presses a key.

FigureComparison · 3 columns

Training

Adjusts parameters using a learning objective and selected evidence.

  • Purpose: fit behavior
  • Uses training data or experience
  • Can overfit or exploit shortcuts
  • Output: a model version

Evaluation

Measures behavior on held-out data, stress cases, human studies, or simulations.

  • Purpose: estimate performance and risk
  • Must avoid contamination from training
  • Should reflect intended use
  • Output: evidence and uncertainty

Inference

Applies a selected model version to new inputs inside a product or process.

  • Purpose: produce operational outputs
  • Usually does not change core parameters
  • Adds latency and cost constraints
  • Output: score, ranking, content, or action

Figure

A months-long industrial process that breaks every few hours, against a single pass through the parameters it produced. Meta, arXiv:2407.21783; the GPU-years and the failure interval are derived from the published figures.

Comparison

Deployed systems can change in several different ways

“The AI learns from you” can describe personalization, periodic retraining, online updates, or merely saving preferences. Those are not the same mechanism.

FigureComparison · 3 columns

Fixed model version

Parameters remain unchanged until a new version is trained and deployed.

  • Predictable release process
  • Easier rollback and audit
  • May age as the world changes
  • Common in regulated workflows

Periodic retraining

New data is collected and a replacement model is trained on a schedule or trigger.

  • Updates happen in controlled batches
  • Requires fresh evaluation
  • Can incorporate delayed outcomes
  • Common in forecasting and risk models

Online adaptation

Some parameters, policies, or rankings update as new interactions arrive.

  • Responds quickly to changing behavior
  • Creates feedback-loop risks
  • Needs tight monitoring and safeguards
  • Common in some recommendation settings

Key idea

A regulator names the same three columns

Regulators separate the same three columns by name. In its discussion paper of 2 April 2019, “Proposed Regulatory Framework for Modifications to Artificial Intelligence/Machine Learning (AI/ML)-Based Software as a Medical Device (SaMD)”, the US Food and Drug Administration defined a “locked” algorithm as “an algorithm that provides the same result each time the same input is applied to it and does not change with use”, offering static look-up tables, decision trees and complex classifiers as examples, and set it against an adaptive algorithm that “changes its behavior using a defined learning process”. The paper also recorded where the market actually was: of the AI/ML-based software as a medical device the agency had cleared or approved, “typically, these have only included algorithms that are ‘locked’ prior to marketing”. The middle column has since been given a legal shape. On 4 December 2024 the FDA published final guidance on Predetermined Change Control Plans (89 FR 96259), under which a manufacturer describes the planned modifications and the methodology to develop and validate them in advance, so that the agency reviews the retraining recipe once “without necessitating additional marketing submissions for implementing each modification described in the PCCP”.

Example

Feedback is evidence only after you understand how it was produced

Feedback looks valuable because it comes from real use. It can still be selective, delayed, strategic, or shaped by the very system it is meant to judge.

The click case in particular has been measured under controlled conditions. Thorsten Joachims, Laura Granka, Bing Pan, Helene Hembrooke and Geri Gay published “Accurately Interpreting Clickthrough Data as Implicit Feedback” in 2005. They put an eye-tracker on search users. Then they manipulated the ranking behind their backs. Users “click substantially more often on the first than on the second link, while they view the corresponding abstract with almost equal frequency”. When the top two results were quietly swapped, “there is still a strong bias to click on link one even if the second abstract is more relevant”. When the whole list was reversed, the average rank of a clicked document moved from 2.66 to 4.03. The average number of clicks per query fell from 0.80 to 0.64. The users reacted to the ranking, not only to the documents. The paper’s one-line conclusion is the sentence to keep: “clicks are informative but biased.”

  • Accepted drafts: users may accept mediocre text because editing is slower, so acceptance does not equal correctness.
  • Clicks: ranking determines what users can see, making future click data partly a consequence of earlier rankings.
  • Fraud outcomes: confirmed labels may arrive weeks later and may miss fraud that was never investigated.
  • Inspector corrections: experts may review only uncertain cases, creating a dataset unlike the full production population.
  • Customer ratings: dissatisfied users may abandon the product without leaving feedback, hiding important failures.

Analogy

Rehearsal and live performance

A musician rehearses difficult passages, auditions with unfamiliar pieces, then performs before an audience. Rehearsal changes the musician. The performance applies what was learned, under time and context constraints.

Training, evaluation, and inference follow a similar separation. The useful resemblance ends at deliberate judgment. A human performer can reinterpret the situation, while many deployed models apply a fixed parameterized mapping.

Never use live performance as if it were neutral training evidence without accounting for the conditions that produced it.

Key idea

“Continuous learning” is not a default property

Many deployed systems never update their core model during ordinary use: they store conversation history, retrieve new documents, remember a preference or switch prompts, while the underlying parameters stay exactly where training left them.

Actual online learning introduces versioning, stability, security, fairness, and rollback challenges. It is a design choice, and an expensive one. It should never be an assumed feature of AI.

The paper that popularized the behaviour people mistake for learning said so in its own definition. Tom B. Brown and colleagues published “Language Models are Few-Shot Learners” in 2020. They describe the few-shot setting as one “where the model is given a few demonstrations of the task at inference time as conditioning”. In that setting, “no weight updates are allowed”. Of their 175-billion-parameter model they state that “for all tasks, GPT-3 is applied without any gradient updates or fine-tuning”. Tasks and few-shot demonstrations are “specified purely via text interaction with the model”. A system can visibly get better at your task over the course of a conversation. It may still be a fixed set of numbers. What changed was the text placed in front of it.

Changing context is not the same as changing the model.

Every update creates a new evidence obligation

A new dataset, label policy, threshold, prompt, retrieval index, model version, or interface can change system behavior. Any of them. Versioning and evaluation therefore have to match the real unit of change, which is rarely the model alone.

The central beginner insight is simple: deployment does not end learning work. It changes the questions from “Can we fit a model?” to “Is this version still useful and acceptable here?”

European law now writes that obligation down. The EU AI Act was published in the Official Journal on 12 July 2024. Its Article 43(4) provides that a high-risk system already assessed once “shall undergo a new conformity assessment procedure in the event of a substantial modification”. That holds “regardless of whether the modified system is intended to be further distributed or continues to be used by the current deployer.” Article 3(23) defines a substantial modification. It is a change made after deployment “which is not foreseen or planned in the initial conformity assessment carried out by the provider”. The carve-out is the instructive part. Some systems “continue to learn after being placed on the market or put into service”. For those systems, some changes are “pre-determined by the provider at the moment of the initial conformity assessment”. Written into the technical documentation, those changes “shall not constitute a substantial modification”. Continuous learning is not forbidden. It is allowed to the extent that the shape of the change was described and assessed in advance.

A system update is not complete when it runs; it is complete when its effects are understood well enough for the context.

Key takeaways