Skip to content
AI.info

Neural networks

Convolutions: Locality, Reuse, and Equivariance

Build intuition for receptive fields, channels, stride, padding, parameter sharing, translation equivariance, and the limits of convolutional assumptions.

By the end you can

Visual

One kernel reused across an image

A convolutional layer slides a small set of weights across positions. The result is a feature map.

Everything after this diagram is a measurement of what that reuse buys and of what it quietly costs at the edges of the grid.

FigureProcess · 5 steps
  1. 1

    Local window

    Select nearby input positions across all input channels.

  2. 2

    Shared kernel

    Apply the same learned coefficients at each valid location.

  3. 3

    Local response

    Compute one output value for one filter and position.

  4. 4

    Spatial sweep

    Repeat across the grid using the chosen stride.

  5. 5

    Feature maps

    Stack responses from several filters as output channels.

Case

ILSVRC-2012 was won at 0.15315 against a runner-up at 0.26172

In 2012 a network with five convolutional layers entered the ImageNet ILSVRC-2012 competition and won it. Krizhevsky and two colleagues report that it “achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry”. The competition’s own results table records the same two submissions: 0.15315 for the SuperVision entry and 0.26172 for the runner-up from ISI.

Everything else in that field was a hand-engineered feature pipeline. The date convolution stopped being one option among several is public.

Figure

One margin read two ways: “won by more than ten percentage points” and “cut the error by two fifths” describe the same two bars. Krizhevsky, Sutskever and Hinton, NIPS 2012, with the competition’s own results table; the absolute and relative margins are derived from both pairs of figures.

Channels are feature coordinates, not separate images

An input image may begin with red, green, and blue channels. Deeper channels are learned feature maps, and what those maps mean depends on the entire network that produced them.

Each convolutional filter spans the input-channel dimension and emits one output channel. A layer with many filters learns several local transformations in parallel.

The cleanest demonstration that the channel axis is a separate, compressible dimension is a kernel with no spatial extent at all. The GoogLeNet paper records that “1×1 convolutions are used to compute reductions before the expensive 3×3 and 5×5 convolutions”. That filter touches exactly one position. It does nothing but mix and shrink channels. GoogLeNet won the ILSVRC-2014 classification task with 6.67% top-5 error, and the challenge organisers’ own results table records 0.06656 for it against 0.07325 for the runner-up VGG and 0.0806 for MSRA Visual Computing.

The saving is the part worth carrying. Szegedy and his co-authors write in their introduction that “Our GoogLeNet submission to ILSVRC 2014 actually uses 12 times fewer parameters than the winning architecture of Krizhevsky et al [9] from two years ago, while being significantly more accurate.” Kernel extent and channel count are two separate budgets. That architecture won by spending only one of them.

Kernel height and width describe locality; channel dimensions describe feature mixing.

Comparison

Stride, padding, and dilation reshape the grid

These settings change coverage and output resolution without changing the basic sharing principle. Two of the three carry documented side effects that their names do not suggest.

Dilation is the one with a published promise attached. Yu and Koltun introduced the dilated-convolution context module in 2016 on exactly this basis: “The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage.” A separate group described the same mechanism two years later. Wang and colleagues call dilation a way of maintaining resolution and receptive field “by inserting 'holes' in the convolution kernels, thus eliminating the need for downsampling”, and report 80.1% mIoU on the Cityscapes test set.

The holes are also the cost, and the cost is arithmetic rather than atmosphere. Wang and colleagues count the positions: “If k = 3, r = 2, only 9 out of 25 pixels in the region are used for the computation”. A “pixel p can only view information in a checkerboard fashion, and lose a large portion (at least 75% when r = 2) of information”. The technique’s own authors documented the artifact. Yu and Koltun write in a 2017 paper on dilated residual networks that “The use of dilated convolutions can cause gridding artifacts.”, and they name the condition that produces them: “Gridding artifacts occur when a feature map has higher-frequency content than the sampling rate of the dilated convolution.” They then build a degridding scheme to remove it. The setting that costs no extra parameters is not free.

Padding is the entry that understates itself most. Islam and colleagues concluded in 2020 that “We believe that the padding near the border delivers position information to learn.” “Controls output size” is true. It is measurably not all that padding does. The equivariance section below carries the numbers.

FigureComparison · 3 columns

Stride

Moves the kernel by more than one position.

  • Reduces output resolution
  • Lowers computation
  • May skip fine detail
  • Acts as learned downsampling with the kernel

Padding

Adds boundary values around the input.

  • Controls output size
  • Lets border positions participate more often
  • Introduces a boundary assumption
  • Zero, reflect, and circular choices differ

Dilation

Spreads kernel taps across a wider area.

  • Expands receptive field
  • Keeps parameter count fixed
  • Can create gridding effects
  • Useful for multiscale context

Receptive field grows through composition

One small kernel sees a local neighborhood. Stacking layers lets later units depend on progressively larger regions, and the trade that makes stacking preferable to widening is arithmetic anyone can redo.

Three stacked 3×3 convolutional layers “have a 7 × 7 effective receptive field”. Simonyan and Zisserman wrote that in 2015, then priced the two ways of buying that reach. The stack “is parametrised by 3(3²C²) = 27C² weights; at the same time, a single 7 × 7 conv. layer would require 7²C² = 49C² parameters, i.e. 81% more”. Szegedy and four colleagues ran the same comparison one size down in 2016: “For example, a 5 × 5 convolution with n filters over a grid with m filters is 25/9 = 2.78 times more computationally expensive than a 3 × 3 convolution with the same number of filters.” Replacing that 5×5 with two 3×3 layers, they write, means “we end up with a net 9+9/25 × reduction with a relative gain of 28% by this factorization. The exact same saving holds for the parameter count”. Two groups, two kernel sizes, one conclusion. Depth buys reach more cheaply than width.

The theoretical receptive field states which inputs can influence an output. The influence a network actually learns may concentrate in a smaller region.

That gap has a name and a measurement. Luo and colleagues introduced the effective receptive field in 2016 and found that it “both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field”. They also trace how nonlinear activations, dropout, sub-sampling and skip connections change it. Counting kernel sizes tells you which inputs could reach an output. It does not tell you which ones do.

A large theoretical receptive field does not guarantee that distant evidence contributes strongly.

Key idea

Translation equivariance is useful but imperfect

Away from boundaries, translating an input can translate a convolutional feature map in a corresponding way. Pooling or global aggregation can later create partial invariance.

Stride, padding, resizing, finite boundaries, and preprocessing can break exact equivariance. A convolutional network is not automatically invariant to rotation, scale, or viewpoint.

Two independent groups measured how far ordinary networks fall short of this. Azulay and Weiss wrote in 2019 about the assumption itself. CNNs are “commonly assumed to be invariant to small image transformations: either because of the convolutional architecture or because they were trained using data augmentation”. They examine antialiasing and additional data augmentation as remedies. Their conclusion is that “the problem of insuring invariance to small image transformations in neural networks while preserving high accuracy remains unsolved”. Richard Zhang opens from the same observation in 2019, that “modern convolutional networks are not shift-invariant, as small input shifts or translations can cause drastic changes in the output”. He attributes it to “commonly used downsampling methods, such as max-pooling, strided-convolution, and average-pooling” that “ignore the sampling theorem”.

The boundary is not merely where the property frays. It is where the network learns to read absolute position, and in 2020 two groups showed this from opposite directions. Islam and colleagues took the padding away: a VGG16 trained without zero padding scored 12.3% mIoU on PASCAL VOC 2012 validation, against 23.1% with it. Their reading of that is plain — “We believe that the padding near the border delivers position information to learn.” Kayhan and van Gemert reached the same conclusion from the filters’ side: “We show that CNNs can and will exploit the absolute spatial location by learning filters that respond exclusively to particular absolute locations by exploiting image boundary effects.” A filter that fires only near one corner is not a shared local detector. The boundary assumption listed as a size setting is a channel through which position enters a network built to ignore it.

The other half of the shortfall is the symmetries convolution never encoded. Worrall and colleagues put the boundary of the guarantee in one sentence in 2017: “Convolutional neural networks (CNNs) are already translation equivariant: input image translations produce proportionate feature map translations. This is not the case for rotations.” The gap has a price. Cohen and Welling report an ordinary planar CNN at 5.03% test error on rotated MNIST in 2016. Replacing each planar convolution with a p4 group convolution takes the same network to 2.28%, with 22k against 25k parameters. Their abstract says why that is not a capacity effect: “G-convolutions increase the expressive capacity of the network without increasing the number of parameters.” Worrall and colleagues reproduce both figures in their own comparison table, at 22k and 25k parameters, and reach 1.69% with rotation equivariance built in at 33k. The rotational robustness a plain CNN lacks is worth roughly half its error rate on that task. It is not bought with capacity. It is bought by encoding the symmetry.

Convolution encodes one structured preference, not every visual symmetry.

Position

An architecture diagram states an intention, not a property the model has

Architectures are sold on their properties. Convolution is translation-equivariant; the receptive field after n layers is this wide; one kernel applies everywhere, so position stops mattering. Those sentences describe an operation. They get read as descriptions of the trained network, and this lesson has now supplied four measurements of the distance between the two.

Take the famous property first. Azulay and Weiss record the assumption in the words practitioners actually use — that convolutional networks are invariant to small image transformations, either because of the architecture or because of data augmentation. They then examine antialiasing and additional augmentation as remedies, and conclude that ensuring such invariance while preserving high accuracy remains unsolved. Richard Zhang puts the finding plainly: modern convolutional networks are not shift-invariant, and small shifts can cause drastic changes in the output. He traces it to max-pooling, strided convolution and average-pooling, which ignore the sampling theorem. Kayhan and van Gemert go further than qualifying the claim. They invert it, showing that convolutional layers learn filters responding only at particular absolute locations by exploiting image boundary effects. Islam and colleagues measure the same channel from the other side, 23.1% mIoU falling to 12.3% once the padding that carries the position information is removed. The property survives the diagram. It does not survive the implementation, and the boundary the diagram does not draw is where it fails.

The remaining two gaps are quieter and work the same way. Counting kernel sizes gives the theoretical receptive field. Luo and colleagues measured the effective one: Gaussian, and occupying only a fraction of the theoretical extent. And “equivariant” on a block diagram names one group and stays silent about the rest. Worrall and colleagues state that translation equivariance is exactly what a CNN has and rotation is what it does not, a silence worth 5.03% against 2.28% error on rotated MNIST at a comparable parameter count. This is not an argument against convolutions, which won ILSVRC-2012 at 0.15315 against 0.26172 for the best hand-engineered pipeline. It is an argument that every property named on a block diagram is a claim about the operation. It has to be measured again on the system that was built.

Translation invariance is what convolution is famous for, and the built network does not have it.

Example

Convolution is broader than image classification

Any data with meaningful local neighborhoods may benefit from shared local processing, and the dense-output case has been scored in public. Ronneberger and colleagues report of their U-Net in 2015 that “Here we achieve an average IOU (“intersection over union”) of 92%, which is significantly better than the second best algorithm with 83%”. Their tables give the four-decimal figures for the ISBI cell tracking challenge 2015: 0.9203 on PhC-U373 and 0.7756 on DIC-HeLa, against 0.83 and 0.46 for the next best entries. Those came from 35 and 20 annotated training images respectively. In the ISBI EM segmentation challenge ranking of 6 March 2015 the same network took first place, with a warping error of 0.000353 against 0.000420 for the previous best sliding-window network.

The challenge organisers confirmed it independently. Their combined report in Nature Methods, in 2017, records that “The algorithms KTH-SE, FR-Ro-GE and HD-Hau-GE ranked first for one or more data sets”, singling out the machine-learning methods on the contrast-enhancement microscopy data. The reach past classification is a leaderboard entry, not a claim.

  • Audio: one-dimensional kernels detect short temporal patterns in waveforms or features.
  • Text: kernels can detect local character or token n-gram patterns.
  • Medical signals: temporal convolution captures repeated waveform motifs.
  • Segmentation: convolution preserves spatial feature maps for dense output — U-Net's 0.9203 average IOU on PhC-U373 against 0.83 for the runner-up was learned from 35 partially annotated training images.
  • Image restoration: local filters predict corrected pixels while deeper layers add wider context.
  • Time–frequency analysis: two-dimensional kernels process spectrogram neighborhoods.

Analogy

A quality inspector using one movable template

An inspector slides the same transparent template across a manufactured sheet. A match can be detected anywhere without designing a new template for each location.

Local inspection is only part of it. A learned convolution can mix channels, combine with nonlinearities, and participate in a hierarchy that no physical stencil captures.

The sheet also has edges, and the analogy is at its most honest there. What the template does when it hangs over the border is a choice someone made. This lesson has already shown that the choice is where absolute position gets into a network designed not to know it.

Parameter sharing makes location reuse cheap; depth turns local responses into broader representations.

What Path 6 will add

This lesson explains the convolutional primitive. The architecture path will examine modern blocks, multiscale designs, dense prediction networks, transfer learning, and vision transformers.

Keep the boundary clear: knowing a kernel operation does not yet explain why a complete CNN architecture succeeds on a particular task. Carry the habit forward instead. Every property printed beside a block in that path is a claim about an operation, and the question to ask of it is which paper measured it, on what task, and by how much.

Key takeaways