Network in Network - 1x1 Convolutions

Network in Network or more commonly known 1x1 Convolution are used to manipulate the depth of input channels

Convolution and Pooling layers reduce the height and width leaving the dept unchanged (or equal to number of filters).

1x1 convolution can be used to reduce only the depth as it slices through the input volume with just 1x1 unit

Input: 6x6x1 (1 depth)

1x1Conv: 1x1

Output: 6x6

For Multiple Channels:

Input: 6x6x32

1x1Conv: 1x1x32, #filters = n

Output: 6x6x#filters

It generates an element wise product across the depth, slices through the entire volume

Test Image

It can also be thought of as a fully connected network applied to each of the 36 positions (for 6x6xn) matrix

Where is it used?

To shrink height & width, pool or conv is used
TO shrink depth, 1x1Conv is used

It saves on computation task, also addes non linearity and allows to learn more complex functions.

Widely used in the inception network.

Written on December 20, 2017