Skip to content
AI.info

How machines learn

Features, Representations, and What the Model Can See

Understand how raw observations become features or learned representations and how encoding choices shape what patterns a model can use.

By the end you can

Example

One situation, three representations

A support conversation can be represented in several ways. Each choice gives the model access to different evidence, and makes other information invisible.

  • Counts: number of messages, response delays, attachments, and reopenings summarize the interaction but discard wording.
  • Keywords: selected terms such as “refund” or “cancel” expose specific cues but miss paraphrases and context.
  • Text embedding: a learned vector can capture broader semantic similarity, although its dimensions are difficult to interpret directly.
  • Sequence: the ordered conversation preserves turn structure and escalation, but requires a model able to use order.
  • Metadata: account tier, language, product, and region add context while creating privacy and proxy concerns.

A feature is an input with a job: asthma, 5.4% against 11.3%

A feature is a value handed to a model when it produces an output. It can come straight from a sensor or a database field. It can be calculated from thousands of events. The same raw datum can be useful, irrelevant, or dangerous, depending on the task and on when the decision has to be made.

Features are not neutral windows onto reality. Naming, units, missing-value conventions, aggregation windows and category definitions all encode assumptions. A model learns from the representation it receives, not from an unfiltered world.

One named column can be the whole argument. A pneumonia-risk study covered 14,199 patients, split 9,847 for training and 4,352 for testing, with 46 features. Of those patients 1,542 died, 10.86%. Caruana and five colleagues reported it in 2015. On the same data, an earlier rule-based model had learned something that reads like a medical discovery. The paper prints it: “On one of the pneumonia datasets, the rule-based system learned the rule “HasAsthama(x) ⇒ LowerRisk(x)”, i.e., that patients with pneumonia who have a history of asthma have lower risk of dying from pneumonia than the general population.” The spelling “HasAsthama” is the original's.

The correlation in the data was real. Pneumonia patients with asthma died at 5.4%, against 11.3% for those without; the figures are Cabitza and colleagues' in JAMA. It was real because asthmatic pneumonia patients were routinely admitted straight to intensive care. So the asthma column never measured a patient's lungs. It measured an unrecorded treatment decision. A model using it to decide who could safely be sent home would have sent home precisely the patients the hospital had learned to escalate.

The model cannot use information that its representation omits, and it may exploit shortcuts the representation accidentally reveals.

Case

Word vectors that tracked the Bureau of Labor Statistics at ρ = 0.90

A representation absorbs more than the meanings of words, and the amount has been measured. The published GloVe vectors were trained on a Common Crawl corpus of 840 billion tokens. Caliskan and two colleagues, writing in Science on 14 April 2017, measured how strongly each of 50 occupation words associated with female gender. Then they compared that association with the share of women actually working in those occupations, using 2015 US Bureau of Labor Statistics data. The two lined up at a Pearson correlation of ρ = 0.90, with a p-value below 10⁻¹⁸. Nobody had supplied a gender field. The vectors carried one because the text did.

Visual

From raw signal to task-ready input

Representations often pass through several layers before training. A failure at any layer can look like a model problem. The asthma rule was a failure at feature construction, not at fitting, and no amount of extra training would have removed it.

FigureProcess · 5 steps
  1. 1

    Raw source

    Images, audio, logs, transactions, text, or sensor measurements arrive in operational formats.

  2. 2

    Cleaning and alignment

    Units, timestamps, identities, missing values, and malformed records are reconciled.

  3. 3

    Feature construction

    Values are selected, aggregated, encoded, normalized, or transformed.

  4. 4

    Model representation

    The model receives a vector, tensor, token sequence, graph, or another structured form.

  5. 5

    Learned internal representation

    During training, some models create intermediate features useful for the objective.

Comparison

Learned from pixels, 15.3%; hand-designed descriptors, 26.2%

Modern systems often combine both approaches rather than choosing one. But there is one day on which the two were scored head to head, on the same data, by the same server, and the margin is worth carrying.

The official ILSVRC-2012 results table, published by the challenge organisers on 13 October 2012, puts the SuperVision entry first: a convolutional network trained on raw RGB pixel values, top-5 classification error 0.15315. The next best entry, ISI from the University of Tokyo, scored 0.26172 with a combination of hand-designed descriptors. The table describes it as a “Weighted sum of scores from each classifier with SIFT+FV, LBP+FV, GIST+FV, and CSIFT+FV”. Nobody had told the network about corners, gradients or texture. It had 60 million parameters, 1.2 million images and 1,000 classes, and it found its own intermediate features.

Its three authors state the result in their abstract: “We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.” Read the table honestly and one caveat belongs with the number. The 0.15315 row is annotated “Using extra training data from ImageNet Fall 2011 release”. SuperVision's best entry using only the supplied ILSVRC-2012 training data scored 0.16422 — still a full ten points below ISI.

That gap is what "may miss complex interactions" and "can capture complex structure" mean once somebody measures them. It is not a verdict against hand-designed features, which remain easier to inspect and cheaper to serve. It is the size of what a learned representation found in pixels that a decade of engineered descriptors had not.

FigureComparison · 3 columns

Hand-designed features

People specify measurements believed to matter.

  • Can encode domain knowledge
  • Often easier to inspect
  • May miss complex interactions
  • Example: 30-day purchase count

Learned representations

A model discovers useful intermediate encodings from data.

  • Can capture complex structure
  • Often needs more data and compute
  • May be harder to interpret
  • Example: an image embedding learned from pixels

Hybrid representation

Engineered context is combined with learned embeddings or raw modalities.

  • Balances structure and flexibility
  • Requires careful scale and timing alignment
  • Can inherit risks from both sources
  • Example: text embedding plus account history

Key idea

Google Flu Trends ran high in 100 of 108 weeks

Feature design must include availability, freshness, latency and cost. A warehouse table updated overnight cannot support a prediction that must react within five seconds. A third-party signal may disappear, become expensive, or change definition.

That last risk has a dated record. Google Flu Trends predicted the CDC's influenza-like-illness rate from one kind of feature only: the relative volumes of search terms. It “has missed high for 100 out of 108 weeks starting with August 2011”, Lazer and three colleagues reported in Science on 14 March 2014. Part of the drift they attribute neither to flu nor to the model, but to the owner of the feature source. Google had modified search results in June 2011 to suggest additional search terms, and again in February 2012 to return potential diagnoses for symptom searches. The model had not changed. Its inputs had been re-engineered underneath it.

On what the search terms had been measuring in the first place, the same authors are blunt: “In short, the initial version of GFT was part flu detector, part winter detector.” Another team reached the failure from the other end. Writing in PLOS Computational Biology, Olson and colleagues found GFT “completely missing the first wave of the 2009 influenza A/H1N1 pandemic” and “greatly overestimating the intensity of the A/H3N2 epidemic during the 2012/2013 season”.

Record four things about each feature. When it is produced, how late it can arrive, what happens when it is missing, and whether training data reconstructs the same timing. Add a fifth for any signal you do not own: who is allowed to redefine it, and would you find out when they did.

Feature quality includes operational availability, not only predictive correlation.

Numbers carry units and distributions

A column of annual income in euros and a column of age in years have different scales and different shapes. Some algorithms are sensitive to that difference. Others split or rank values in ways that make scaling matter much less.

Fit transformations — standardization, logarithms, clipping, category encoding — on training data only. Applying them before the split leaks information from validation and test examples into training.

Analogy

The hospital label was 99.95% recoverable from the image

Location, device type, document template and hospital identifier are all available as shortcuts. Real proxy features are rarely wholly irrelevant. They mix legitimate context with sensitive or unstable correlations. How large can that shortcut be? It has been measured.

A cross-sectional study published in PLOS Medicine on 6 November 2018 screened chest radiographs for pneumonia. It first asked a smaller question: could a convolutional network tell where a radiograph had been taken? It could. 22,050 of 22,062 NIH test radiographs (99.95%), 8,386 of 8,388 at Mount Sinai (99.98%) and 737 of 771 at Indiana (95.59%) were assigned to the right hospital system. Nobody had supplied a hospital field. It was in the pixels.

That mattered because pneumonia prevalence was 34.2% at Mount Sinai against 1.2% at NIH and 1.0% at Indiana. On the joint MSH-NIH dataset, sorting images by hospital system and doing nothing else scored AUC 0.861 (95% CI 0.855–0.866) at detecting pneumonia. A model that had learned only the site label would have looked like a diagnostic instrument. And when a jointly trained MSH-NIH model was taken somewhere it had never seen, it fell from an internal test AUC of 0.931 (95% CI 0.927–0.936) to an external AUC of 0.815 at Indiana (95% CI 0.745–0.885, P = 0.001).

The authors' own conclusion is the general lesson: “CNNs robustly identified hospital system and department within a hospital, which can have large differences in disease burden and may confound predictions.” An independent group, surveying shortcut learning in deep neural networks, names the mechanism: “The model had unexpectedly learned to identify particular hospital systems with near-perfect accuracy (e.g. by detecting a hospital-specific metal token on the scan, see Figure 1).” The shortcut was not a column somebody chose to include. It was a piece of metal in the frame.

Steps

Article 10 turns these five steps into law

Reviewing a small set of influential features deeply is often more useful than listing thousands of columns. In the European Union these steps stopped being merely good practice on 12 July 2024, when the AI Act — Regulation (EU) 2024/1689 — was published in the Official Journal.

Article 10, on data and data governance, is close to this review table written as an obligation for high-risk systems. Step 1, define it, is Article 10(2)(d): data governance must cover “the formulation of assumptions, in particular with respect to the information that the data are supposed to measure and represent”. Step 4, question the shortcut, is Article 10(2)(f): “examination in view of possible biases”. The quality bar behind steps 1 to 3 is Article 10(3): “Training, validation and testing data sets shall be relevant, sufficiently representative, and to the best extent possible, free of errors and complete in view of the intended purpose.” Step 5, match production, is Article 10(4). The data must take account of “the characteristics or elements that are particular to the specific geographical, contextual, behavioural or functional setting within which the high-risk AI system is intended to be used”. That is the Indiana result stated as a legal duty.

There is a date attached. The European Commission's regulatory-framework page states that “Starting on 2 December 2027, high-risk AI systems will be subject to strict obligations”, among them “high-quality of the datasets feeding the system to minimise risks of discriminatory outcomes”.

FigureProcess · 5 steps

  1. 1. Define it

    Write the source, unit, calculation, window, and missing-value meaning.

  2. 2. Time-stamp it

    Confirm when the value becomes available relative to prediction.

  3. 3. Stress it

    Inspect range, outliers, rare categories, and changes across time or groups.

  4. 4. Question the shortcut

    Ask what the feature might proxy and whether that relationship should be used.

  5. 5. Match production

    Verify that live and training pipelines compute the same representation.

One column, EUR 2.75 million

A richer representation can improve performance. It also raises cost, privacy exposure, and the number of spurious patterns available. More inputs are not automatically better.

The right representation contains enough evidence for the task. It excludes information that is unavailable, unstable, legally restricted, or unrelated to the intended decision. A regulator has put a price on “legally restricted”. On 7 December 2021 the Dutch Data Protection Authority fined the Minister of Finance EUR 2.75 million. The Tax Administration had processed childcare-benefit applicants' (dual) nationality unlawfully and in a discriminatory manner. Its news release lists the uses, ending with the modeling one: “Lastly, it used applicants’ nationality (Dutch/not Dutch) as an indicator in a system that automatically designated certain applications as risky.” The regulator's position on that column is not a recommendation: “It is unlawful, and therefore prohibited, to use nationality data to assess applications, combat fraud and determine risk.”

The scale was not small and the correction was not fast. Some 1.4 million people were still registered as dual nationals in the Tax Administration's systems in May 2018. Nationality was dropped from risk determination only in October 2018. Amnesty International, documenting the same system, describes it the same way: “Nationality was one of the risk factors used by the tax authorities to assess the risk of inaccuracy and/or fraud in the applications submitted.”

So the boundary of a representation is a product and governance choice as much as a modeling choice, and it is enforceable. Ask of any feature not only whether it predicts, but whether you are permitted to know it, and whether the reason it predicts is a reason you would defend in writing.

Key takeaways