AI, Machine Learning, Deep Learning, and Generative AI

Product names change quickly. The relationships among fields, methods, model families, and applications are more durable, which is what makes them worth learning. The map below is deliberately approximate, because some of these categories overlap rather than nest.

The vocabulary itself has dates. “Foundation model” was proposed in August 2021, by the Stanford report On the Opportunities and Risks of Foundation Models. Its authors wrote: “We call these models foundation models to underscore their critically central yet incomplete character.”

In traditional programming, people specify the procedure and the computer applies it to data; in machine learning, people specify a model family, data, an objective, and a training process, and the fitted parameters then capture a useful mapping.

Machine learning is therefore a major part of modern AI. It is not a synonym for the whole field: search algorithms, logical systems, planners, and hand-authored knowledge are all AI, and none of them learns from a dataset.

Tom Mitchell’s 1997 textbook Machine Learning states the test this shift implies. “A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E.” Ask a vendor to name E, T and P. A system that learns has all three.

The non-learning traditions are equally concrete. A* is a graph-search method published by Peter Hart, Nils Nilsson and Bertram Raphael in 1968. It appeared in IEEE Transactions on Systems Science and Cybernetics. It finds a least-cost route. It always expands the option with the lowest sum of cost so far plus estimated cost remaining. It plans, and it is studied as AI. It has no training data, no fitted parameters, and no experience E to improve with.

Deep learning became influential because it can learn internal representations directly from high-dimensional inputs. That flexibility comes with substantial costs.

The classical side of the comparison below is not a historical footnote. Léo Grinsztajn, Edouard Oyallon and Gaël Varoquaux assembled 45 tabular datasets for a benchmark presented in 2022. They pitted tree ensembles such as XGBoost and random forests against several deep-learning architectures. Each learner got roughly 20,000 compute hours of hyperparameter search. On medium-sized data — around 10,000 samples — the tree-based models still finished ahead. That is before their speed advantage is even counted.

A discriminative system separates or scores possibilities — spam or not spam, likely delay, best-ranked item — while a generative system models enough structure to produce a new candidate: text, an image, audio, code, a molecular design.

Generative AI is not identical to large language models. Language models are one important family, while diffusion models and other generative approaches operate on images, audio, video, and additional data types.

Diffusion is the clearest example of a generative family that is not linguistic at all. Jonathan Ho, Ajay Jain and Pieter Abbeel presented “Denoising Diffusion Probabilistic Models” in 2020. They produced images by learning to reverse a gradual noising process one step at a time. They report a Fréchet Inception Distance of 3.17 on unconditional CIFAR-10. The method says nothing about language, and nothing about truth. A model of this kind will render a convincing photograph of an event that never took place.

The same underlying model can support very different applications depending on data, instructions, tools, permissions, interfaces, and workflow design.

Instead of arguing over names, unpack the label into observable properties.

The labels in this lesson tell you where to investigate next: machine learning points toward training and generalization, deep learning toward neural representations, generative AI toward decoding, grounding, and output validation.

None of these labels proves that a product is advanced, useful, or trustworthy, and a simple model fitted to a well-defined problem can outperform an impressive foundation model embedded in a confused workflow.

A tribunal decision from February 2024 shows how little the label settles. In Moffatt v. Air Canada, British Columbia’s Civil Resolution Tribunal held the airline liable. A support chatbot on the airline’s own website had advised a passenger that a bereavement fare could be claimed after travel. The airline’s own bereavement-travel page did not allow that. Air Canada argued that the chatbot was “a separate legal entity that is responsible for its own actions”. Tribunal member Christopher Rivers called that “a remarkable submission”. He wrote: “While a chatbot has an interactive component, it is still just a part of Air Canada’s website.” The model family was never the question. The deployment was.