Foundations of Large Language Models

AI summary

Overview

Research area: Natural Language Processing (NLP) and large language models — this is a book-length introductory text, not an experimental paper.

Technical level: Beginner-Friendly. The authors state that readers with a machine-learning, NLP, or Transformer background will find the book easy, but that even readers without that background "will not be burdened with too much reading difficulty" because each chapter is written to be as self-contained as possible.

Scope: A six-chapter book that outlines the basic concepts of large language models and the associated techniques, focused on foundational material rather than comprehensive coverage of all cutting-edge methods.

Note on the supplied content: the excerpt contains the copyright page, preface, notation table, and portions of Chapter 1 (Pre-training), ending mid-discussion of permuted language modeling. No experimental results, benchmarks, or dataset sizes appear in this excerpt; where such items would normally be cited, they are marked as not reported below.

What This Paper Is About

Large language models grew out of NLP but have become one of the most consequential technological developments in artificial intelligence. The central insight the book builds on is that knowledge of the world and of languages can be acquired through large-scale language modeling, yielding a universal model that handles diverse problems rather than one system per task. The book's goal is to lay out the foundational concepts and techniques behind this shift — describing the move from training specialized systems from scratch on large amounts of labeled data to a paradigm of large-scale pre-training followed by fine-tuning, alignment, and prompting.

Key Contributions

1. A consolidated introductory treatment of large language model foundations. The book is organized into six chapters covering pre-training (Chapter 1), generative models including scaling and long-text handling (Chapter 2), prompting including chain-of-thought reasoning and automatic prompt design (Chapter 3), alignment including instruction fine-tuning and human-feedback alignment (Chapter 4), inference including decoding algorithms, acceleration, and inference-time scaling (Chapter 5), and methods for improving reasoning ability (Chapter 6).

2. A clear taxonomy of pre-training paradigms. Chapter 1 distinguishes unsupervised, supervised, and self-supervised pre-training, and explains the shift to a new paradigm where pre-trained foundation models are adapted by fine-tuning, alignment, and prompting — in many cases removing the need for large-scale supervised learning for specific tasks.

3. A self-supervised, architecture-oriented account of pre-training objectives. The text derives decoder-only pre-training (next-token prediction with cross-entropy loss, shown to be equivalent to maximum likelihood estimation) and encoder-only pre-training (masked language modeling as used in BERT), and introduces permuted language modeling as a way around the limitations of masking.

4. A companion, openly available resource. The book is described as a selection of chapters from an introductory NLP resource hosted at the NiuTrans NLPBook GitHub repository and project site, licensed for reuse under the terms stated on its copyright page.

Main Findings

• Pre-training predates the current era. The concept goes back decades, including unsupervised learning for RNNs, deep feedforward networks, and autoencoders; in the modern deep-learning era it resurged with large-scale unsupervised training of word embedding models and with pre-training in computer vision on relatively large labeled datasets such as ImageNet.

• Self-supervised pre-training now dominates. The book reports that most current state-of-the-art NLP models are based on self-supervised pre-training, and that pre-training tasks built from unlabeled data (such as predicting masked or missing parts of the input) provide supervision signals directly from text without human annotation.

• Unsupervised pre-training has specific advantages. Among the listed benefits are aiding in the discovery of better local minima and acting as a regularizer, which makes subsequent supervised learning easier and more stable. Its stated drawback is that considerable effort is still required to further train the model with labeled data after pre-training.

• Supervised pre-training has a scaling weakness. Its advantage is straightforward alignment with standard supervised learning paradigms, but as model complexity increases, the demand for labeled data grows accordingly, making the approach challenging when large-scale labeled datasets are unavailable.

• Self-supervised learning differs from classical self-training. In classical self-training, a model needs a small amount of seed data to build an initial classifier, which generates pseudo-labels used to iteratively refine the model; the book cites prior applications to word sense disambiguation and document classification. In self-supervised pre-training in NLP, by contrast, supervision signals come directly from the text and the entire model is trained from scratch without an initial model assigning discrete pseudo-labels.

• Adaptation splits into fine-tuning and prompting. For sequence encoding models, fine-tuning is the common route: the pre-trained encoder is combined with a classifier, and either both are further optimized on labeled data or the encoder is frozen and only the classifier is optimized. For sequence generation models, prompting allows tasks to be framed as text generation, including zero-shot prompts and few-shot in-context learning with demonstrations.

• Masked language modeling has two identified shortcomings. Using the special [MASK] token creates a discrepancy between training and inference, since the token is used only during training; and the auto-encoding process overlooks dependencies between masked tokens, so predictions for different masked positions are made independently even when the tokens are interdependent in context.

• Permuted language modeling is offered as a fix. Like causal language modeling it makes sequential predictions, but it allows predictions in any order — for example x0 → x4 → x2 → x1 → x3 instead of the natural left-to-right order — while the actual order of tokens in the text stays unchanged.

• Prompting is not a full substitute for tuning. The book states that while prompting is a powerful adaptation method, some tuning effort is still needed to ensure models follow instructions accurately, and that fine-tuning is crucial for aligning model values with human values.

• Reported quantitative results, benchmarks, and dataset sizes: not reported in the supplied excerpt. The text is conceptual and mathematical, and the empirical comparisons that would normally appear in an experimental paper are absent from this content.

Methodology in Plain English

The book proceeds by exposition rather than by experiment. It sets up a unified notation for pre-training — an input token sequence, a special beginning symbol, a neural network with parameters theta, and an output that is either a distribution over a vocabulary or a representation of the input — and then uses that notation to walk through each paradigm in turn.

For pre-training, it first contrasts three ways of obtaining a starting point for a model: unsupervised, supervised, and self-supervised. It illustrates the differences with a schematic figure showing what data each method consumes and how the resulting model is adapted (further labeled-data training, transfer to a new task, or fine-tuning and prompting). It then explains the two families of models in use — sequence encoding models and sequence generation models — and how each is adapted after pre-training, working through a text-classification example (polarity of a travel-website comment) to show fine-tuning, a completion-style prompt, an instruction prompt, and an in-context learning prompt with demonstrations.

For the self-supervised objectives, the text derives each one mathematically. Decoder-only pre-training is written as next-token prediction with a cross-entropy loss summed across positions, then shown to be equivalent to maximizing the log-likelihood of the data. Encoder-only pre-training is described as attaching a Softmax layer on top of the encoder during pre-training and dropping it afterward, with masked language modeling defined as maximizing the reconstruction probability of the original sequence given a corrupted version, restricted to the masked positions. The book works a concrete example with the sentence "the early bird catches the worm," masking "early" and "worm" and writing out the resulting loss. Permuted language modeling is introduced by re-deriving the same sequence probability under a different prediction order.

Why This Matters

Impact on research. The book documents a methodological shift: instead of training specialized systems from scratch on large amounts of labeled data, the field now uses large-scale pre-training to obtain foundation models that are then fine-tuned, aligned, and prompted. The authors describe this as having profoundly impacted research methodologies in NLP and many related disciplines, with a stated prospect for more general artificial intelligence.

Real-world applications cited or implied by the text:

• Text classification, such as identifying whether a piece of text is positive, negative, or neutral (the running example is a restaurant or travel review).
• Machine translation, including the use of pre-trained encoder-decoder multilingual models fine-tuned on bilingual data.
• Question answering, listed as a language generation problem addressed by sequence generation models.
• General language understanding and generation tasks, which the book says pre-trained systems handle well enough to outperform previously state-of-the-art supervised systems in a large variety of NLP problems.

Industry relevance. The described paradigm reduces reliance on task-specific human annotations and lets practitioners collect a small amount of labeled data and slightly adjust a pre-trained model instead of building a deep, complex network from scratch. The book notes that the amount of labeled data used in fine-tuning is small compared to pre-training data, so fine-tuning is less computationally expensive — which is what makes adapting pre-trained models efficient in practice. Prompting further lowers the barrier, since instruction-based prompts let large language models solve problems without additional training.

Future Directions

• Prompting as a replacement for training. The book leaves open how much tuning is still required to make large language models follow instructions accurately, and how that interacts with zero-shot and in-context learning; it defers detailed prompting strategies, chain-of-thought reasoning, and automatic prompt design to Chapter 3.
• Alignment with human values. The text flags human-feedback-based alignment and instruction fine-tuning as necessary but treats them as a separate topic for Chapter 4, leaving the question of how alignment relates to prompting and fine-tuning unresolved in this excerpt.
• Scaling and long-context handling. How to scale up model training and how to handle long texts are posed as open topics for Chapter 2.
• Inference cost and inference-time scaling. Decoding algorithms, acceleration methods, and the question of spending additional computation at inference time are reserved for Chapter 5.
• Strengthening reasoning. How training can improve reasoning ability and how extra computation during inference can help models solve complex problems are deferred to Chapter 6.
• Training/inference mismatch in masking. The [MASK] discrepancy and the independence assumption across masked positions are identified as problems; permuted language modeling is presented as one response, leaving the broader question of how best to remove these issues open.
• Data dependence of supervised pre-training. The book notes the growing labeled-data requirement as model complexity increases, without offering a resolution in the excerpt.

Target Audience

The book is written for readers who want an introduction to the foundations of large language models. Readers with a background in machine learning and NLP and some familiarity with neural networks such as Transformers will find it easy going, but the authors state that readers without that prior knowledge can also read it, since each chapter is designed to be self-contained. It is best suited to students and practitioners who want the conceptual and mathematical groundwork — pre-training paradigms, model adaptation, self-supervised objectives — rather than experimental results or a survey of the latest methods. Readers seeking background topics such as sequence modeling and attention mechanisms are directed to the companion online NLP resource.

Authors’ abstract

This is a book about large language models. As indicated by the title, it primarily focuses on foundational concepts rather than comprehensive coverage of all cutting-edge technologies. The book is structured into six main chapters, each exploring a key area: pre-training, generative models, prompting, alignment, inference, and reasoning. It is intended for college students, professionals, and practitioners in natural language processing and related fields, and can serve as a reference for anyone interested in large language models.