Coursework II:
Learning with Images
COMP0169 Team
Niloy Mitra Tobias Ritschel
Eric-Tuan Le Luca Morreale Pradyumnna Reddy Sanjeev Muralikrishnan David Walton Meng Zhang
November 26, 2020
The total points for this exercise is 100.
Each exercise must be implemented from scratch. Unless differently specified, only numpy is allowed. We
encourage to test the correctness of results using libraries.
Coursework submissions must comprise a single jupyter notebook, containing code and report. We will
provide a skeleton solution notebook. You should complete the tasks below by modifying this notebook as
appropriate, following the instructions given. Code must be provided to reproduce any referred image and
table.
1 Image, audio; all the same 15 points
a Implement a filter to remove low temporal frequencies from the audio file audio.npy, provided under
Datasets/Filtering/. Plot the input and output of your filter as line graphs. (5 points)
b Implement a filter to find vertical edges in an image. We provide a sample file image.png under
Datasets/Filtering/ for testing. Apply your filter to this image and display the input and output
of your filter. (10 points)
2 Denoising 25 points
a Implement denoiseGauss(image) to denoise the image noisy image.png under Filtering/ using a
5×5 Gaussian filter. (5 points)
1
b Implement a pyramidal encoder-decoder to denoise an image. We provide you pairs of noisy and
noise-free image patches of size 128×128 for training and testing, under Datasets/Denoising/. You
may use all the infrastructure of Pytorch. The encoder should have 5 levels and double (half) the
number of internal activations when reducing (increasing) the spatial resolution. Please use ReLU
non-linearities after each layer (15 points).
c Write a markdown block in your notebook, comparing the Gaussian and encoder-decoder denoising
methods. Include example output images from each method and discuss which method performs better
and where. You should write no more than half a page. (5 points)
3 Auto-encoder 20 points
a Implement a convolutional auto-encoder (AE). The loss between in and output should be L2. You can
make use of all the convolution, pooling, back-propagation and build-in functionality of Pytorch here.
(15 points)
||I − dec(enc(I))||22.
b Test it on the CelebA (Datasets/CelebA/) and MNIST (Datasets/MNIST/ dataset by encoding
and decoding their first 5 samples. Compile a document showing for each of the dataset and for each
sample the input and the AE reconstruction, side-by-side. (5 points)
4 Variational auto-encoder 40 points
a Add the reparametrization trick to turn the AE into a variational auto-encoder (VAE). (10 points)
b Test it on the CelebA and an MNIST dataset by generating 5 samples of each to be shown in no
more than half a page. (5 points)
c Compare the samples from the VAE to the samples of the AE, by generating 10 samples and explain in
one sentence what is the difference. (3 points)
d If it is difficult to make VAE optimization converge, please try Stochastic Gradient Descent, and ADAM.
Write no more than three sentences and one plot to argue why one optimizer is better than the other
while nothing else was changed. (2 points)
e In the previous questions, you used a reconstruction loss that measured the difference between input
and output pixels. In this and the next question, you will use a VGG-Loss, that measures the difference
in VGG activations created in layer i out of n for the input and output image:
n∑
i=1
αi||vggi(I)− vggi(dec(enc(I)))||22.
Modify your VAE training to use this VGG Perceptual loss, instead of reconstruction loss. For this loss,
you can use the activation from any of the n layers of VGG - you can choose your list of VGG layers to
use, to generate the best results. (10 points)
f Document a choice of weighting, (α1, . . . , αn), used above that you think works best for generating new
samples. This should include image examples, arguing for your design. (10 points)
2




























































学霸联盟