Designing Convolutional Stages

A convolutional architecture is easier to evaluate when every stage declares height, width, channels, stride, and receptive-field growth. Attractive diagrams often hide destructive compression or an unplanned activation-memory spike.

The ledger also clarifies interfaces. A detection head, skip connection, or decoder needs exact spatial and channel contracts, not the vague claim that it receives “deep features.”

One column of that ledger is more treacherous than the rest. Receptive field, as normally computed, is a geometric upper bound. It is the set of input pixels that could in principle reach a unit. What actually reaches it is much smaller. Luo and colleagues measured the real contribution in 2016 and found a Gaussian falloff. On the CamVid segmentation network they examined, a 505×505 theoretical window carried an effective diameter of about 100 pixels at initialisation. A stage plan that reads its receptive-field column as a promise of usable context is reading the wrong number. Keep the ledger anyway. It is what makes the discrepancy visible at all.

The pattern is common. A stem, then a high-resolution stage learning edges and short motifs, then middle stages trading positions for channels and context, then a low-resolution stage holding semantic combinations, then a task head that pools, decodes or fuses according to the output geometry. The five ResNet grids of 112, 56, 28, 14 and 7 are one instance of it.

But each transition creates a tradeoff. The two that follow are the ones with published price tags: what depth actually buys in usable context, and what stride costs in sampling stability.

Stride or pooling reduces the number of spatial positions. It expands the receptive field quickly and cuts later activation cost, at the risk of aliasing and lost detail. Dilation spaces the kernel taps without shrinking the grid. More local layers grow context through repeated mixing, keeping regular neighbourhoods and adding nonlinear transformations. All three enlarge access. But they alter computation and information differently, and two of the three have measured exchange rates.

Depth is the one most often overestimated. The window a unit nominally sees and the window it actually uses are not the same size. Luo and colleagues named the gap in 2016: “We introduce the notion of an effective receptive field, and show that it both has a Gaussian distribution and only occupies a fraction of the full theoretical receptive field.” Because the impact of input pixels inside the window is Gaussian-distributed, the effective receptive field grows only as O(√n) with the number of stacked layers. The theoretical one grows linearly, so the effective share shrinks at O(1/√n). Their measured log-domain slopes were 0.56 for the ERF size and −0.43 for the ERF/theoretical ratio. On a CamVid segmentation ResNet with 16 residual blocks and 4 stride-2 subsamplings, a theoretical receptive field of 505×505 pixels delivered an ERF diameter of only about 100 pixels at initialisation, reaching roughly 150 by the end of training. Four times the nominal window buys twice the usable one. Budget depth in √n.

Dilation has its own discount, and it has a name. Panqu Wang and colleagues called it “gridding” in 2018, and described it plainly: “as zeros are padded between two pixels in a convolutional kernel, the receptive field of this kernel only covers an area with checkerboard patterns - only locations with non-zero values are sampled, losing some neighboring information.” With a 3×3 kernel at rate r=2, only 9 of the 25 pixels in the covered region are actually read. At least 75% of the information in the region is discarded. Repeated equal dilation r leaves the contributing locations at most (w′×h′)/r² of the layer below.

The cost is measurable on one backbone. Their Cityscapes validation accuracy without CRF runs 72.9% mIoU with no dilation, 75.0% for Dilation-conv, 75.4% for Dilation-RF and 76.4% for Dilation-bigger. Zhengyang Wang and Shuiwang Ji confirmed the artifact independently in 2018. Degridding alone lifted a DeepLabv2 baseline on PASCAL VOC 2012 val from 75.1% to 75.4% mIoU, and from 71.7% to 72.3% without COCO pre-training. Spreading the taps is not free. You pay for the reach in samples you never read.

The output contract determines where resolution should be retained, and dense prediction is where that contract bites hardest. Yu and Koltun built a backbone around exactly that in 2016: “The architecture is based on the fact that dilated convolutions support exponential expansion of the receptive field without loss of resolution or coverage.” Their 10-layer Cityscapes variant, Dilation10, scored 68.7% mean class IoU on the validation set and 67.1% on the test set. The Cityscapes paper lists the same 67.1% for the method independently, where it is the best-performing baseline. A stage plan scored by the benchmark's own authors is a different kind of claim from a stage plan defended by its diagram.

Backward design prevents the backbone from discarding evidence needed by the head. State the smallest relevant structure the model must preserve. Set the final resolution the classification, localization or dense output requires. Allocate downsampling events only after sufficient local evidence has been encoded. Budget channels and memory where abstraction demands capacity, while monitoring activations. Then verify every intermediate tensor and interface with executable shape assertions.

Two of those steps should be costed with published numbers, not intuition. When step one sets a context target, size it against the effective receptive field and not the theoretical one. Depth is a slow way to buy reach. Growth goes as O(√n), and the CamVid network reached only about 150 pixels of effective diameter after training inside a 505×505 nominal window. When step three places a dilation schedule instead of a stride, cost the gridding: 9 of 25 taps at r=2, and a schedule change on one fixed backbone running from 72.9% mIoU to 75.0%, then 75.4%, then 76.4%.

A one-dimensional causal convolution can model audio, telemetry, or token sequences without reading future positions. Dilation expands temporal history while preserving parallel training across known sequences. The same arithmetic that gives a 67×67 window at unchanged grid size in two dimensions gives long history at unchanged sample rate in one. It carries the same gridding liability, since a rate-r schedule reads only its non-zero taps along the time axis too.

Three-dimensional convolution couples two spatial axes with time or depth. It captures local motion or volumetric structure. But activation cost grows rapidly, and temporal sampling becomes an architectural assumption.

Channel count is not a universal proxy for information. Too few channels can create a bottleneck. Excessive width at high resolution can exhaust memory without improving the relevant representation.

Ablate stage depth, width, and transition location separately. Compound changes make it impossible to know whether gains came from context, capacity, regularization, or compute. The published ablations show how much rides on the isolated variable. Hold the backbone fixed and change only the dilation schedule: Cityscapes validation mIoU moves from 72.9% to 76.4%. Hold the architecture fixed and change only the downsampling filter: ResNet50's shift consistency moves by up to +2.1%, against +0.6% for doubling the depth. Neither number would have been legible from a run that changed two things at once.