VGG, reproduced
Six configurations rebuilt from Table 1, four of the paper's prose claims measured, and Table 5's protocols reproduced on all 50,000 ILSVRC-2012 validation images.
PAPER
Very Deep Convolutional Networks for Large-Scale Image Recognition
Karen Simonyan, Andrew Zisserman
International Conference on Learning Representations (ICLR) · 2015
PAPER NOTES
Select a section to read the notes one part at a time.
The problem the paper was solving
One variable, held under a microscope.
By 2014 the strong ImageNet entries differed from each other in several ways at once — receptive field size, stride, the number of layers, the training recipe. AlexNet opened with an 11 × 11 kernel at stride 4; others used 7 × 7 at stride 2. Each paper reported a better number, and it was not possible to say which of their differences the number came from.
What this paper argued
- Depth is the variable worth isolating. Fix the kernel at 3 × 3, fix the stride, fix the pooling, and the only thing left to change is how many layers there are.
- A stack of small kernels substitutes for one large one. Three 3 × 3 layers see as far as one 7 × 7 while holding 27C² weights instead of 49C², and they pass through three rectifiers instead of one.
- How you evaluate is part of the result. Dense evaluation and multi-crop are different procedures on the same weights, and the paper reports them separately and combined rather than picking the best.
This paper's design is the response: six configurations, A through E, which differ only in depth — 11 weight layers up to 19 — with every other architectural choice held fixed. That is a controlled experiment rather than a competitive entry, and it is the reason the paper is still read.
The constraint that makes it work is the kernel. 3 × 3 is the smallest size that still has a notion of direction: a 1 × 1 kernel sees one pixel and cannot tell left from right, and even sizes have no centre pixel to sit on. Fixing the kernel at the smallest useful size means depth is the only lever left, which is exactly what the authors wanted to measure.
Two of the paper's conclusions have since been superseded, and the write-up below says where and why — the initialisation procedure of §3.1 above all, which was written before anyone had shown why deep rectified networks need a different variance.