AI.info
Neural networks
Differentiable programs, traced by hand from a single neuron to an architecture you can defend in a review.
- Neural Networks as Differentiable Programs
- Inside an Artificial Neuron
- Perceptrons and the Geometry of Linear Separation
- Dense Layers: Many Neurons, One Matrix
- Tensors, Axes, Shapes, and Batches
- Activation Functions: Where Nonlinearity Enters
- Depth, Composition, and Representation Hierarchies
- Parameter Sharing and Inductive Bias
- The Forward Pass and Computational Graphs
- Output Heads: Connecting a Network to a Task
- Loss Functions as Learning Contracts
- Gradients: Local Sensitivity in Parameter Space
- The Chain Rule and Credit Assignment
- Backpropagation, Worked From Output to Input
- Automatic Differentiation in Practice
- The Training Loop: From Mini-Batch to Parameter Update
- Initialization and the Scale of Signals
- Vanishing, Exploding, Saturated, and Dead Gradients
- Normalization and Train–Evaluation Behavior
- Residual Connections and Stable Depth
- Capacity, Regularization, Dropout, and Weight Decay
- Embeddings and Learned Representation Spaces
- Convolutions: Locality, Reuse, and Equivariance
- Recurrence: State, Memory, and Sequence Order
- Attention: Content-Based Interaction
- Architecture Motifs: Encoders, Decoders, Bottlenecks, and Towers
- Reading Hidden Representations Without Overclaiming
- Debugging and Testing Neural Networks
- Capstone: Design, Trace, and Defend a Neural Network