Network in Network - 1x1 Convolutions
Network in Network or more commonly known 1x1 Convolution are used to manipulate the depth of input channels
Convolution and Pooling layers reduce the height and width leaving the dept unchanged (or equal to number of filters).
1x1 convolution can be used to reduce only the depth as it slices through the input volume with just 1x1 unit
Input: 6x6x1 (1 depth)
For Multiple Channels:
1x1Conv: 1x1x32, #filters = n
It generates an element wise product across the depth, slices through the entire volume
It can also be thought of as a fully connected network applied to each of the 36 positions (for 6x6xn) matrix
Where is it used?
To shrink height & width, pool or conv is used
TO shrink depth, 1x1Conv is used
It saves on computation task, also addes non linearity and allows to learn more complex functions.
Widely used in the inception network.