[edit]

Week 4: Convolutional Neural Networks

[jupyter][google colab][reveal]

Nic Lane

Abstract:

This lecture will introduce convolutional neural networks.

from conv_imports import *
%matplotlib notebook

Plan for the Day

Inductive bias

What is inductive bias?

Why do we need it?

Inductive bias

Inductive bias in NNs

A simple universal approximator that makes no assumptions about its input structure.

Inductive bias in NNs

Inductive bias in NNs

Plan for the day

Translation invariance in CNNs

Convolutional Neural Networks

\[ \large Out_{i,j}= b + \sum_{x=1}^3 \sum_{y=1}^3 In_{(i+x-1),(j+y-1)}W_{x,y}\]

Image Kernel Output

Convolutional Neural Networks

\[ \large D_{out} = \huge \frac{(D_{in} - K + 2P)}{2} \large + 1 \]

Kernel size (3x3) Stride (2) Padding (1)

Weight sharing in CNNs

Numerical example of a CNN

Numerical example of a CNN - code

image = show_car(CIFAR10_trainset)

Numerical example of a CNN - code

conv = show_random_kernel()

Numerical example of a CNN - code

fig, num_steps, im, final_fm = show_conv_anim(image, conv)
anim = animation.FuncAnimation(fig, update_image, num_steps,
                                   fargs=(fig, im, final_fm), interval=200, blit=True)

CNN as an edge detector

image = show_cute_fox()

CNN as an edge detector

kernel = np.array([[0,-1,0],
                  [-1,4,-1],
                  [0,-1,0]])
conv = show_edge_detector(kernel.T)

CNN as an edge detector

fig, num_steps, im, final_fm = show_edge_detection(image, conv)
anim = animation.FuncAnimation(fig, update_image_no_annotation, range(0, num_steps, 500), fargs=(im, final_fm),
                                   interval=100, blit=True)

Learned CNN features - setup

convlayer, bnlayer, relulayer = get_mobilenet_convbnrelu()
blue_fox = show_224px_fox()
Downloading: "https://download.pytorch.org/models/mobilenet_v2-b0353104.pth" to /root/.cache/torch/hub/checkpoints/mobilenet_v2-b0353104.pth
{"model_id":"e55d4ca4ad03451db55cbc25eb7d423a","version_major":2,"version_minor":0}
<IPython.core.display.Javascript object>

Learned CNN features - weights at layer 3

fig, num_steps, ims = show_conv_weights(convlayer)
anim = animation.FuncAnimation(fig, update_row, num_steps,
                               fargs=(ims, convlayer.weight), interval=1000, blit=True)

Learned CNN features - convolution pre-activation at layer 3

fig, num_steps, ims, convoutput = show_layer_output(blue_fox, convlayer)

anim = animation.FuncAnimation(fig, update_row_output, num_steps,
                               fargs=(ims, convoutput), interval=1000, blit=True)