Assigned: 17 November 2021
Homework #5 – Convolutional Neural Networks (CNN)
EE 541: Fall 2021
Due: Saturday, 27 November 2021 at 23:59. Late penalty: 10% per 24-hours before 22 November at
23:59. Submission instructions will follow separately on canvas.
In this assignment you will apply deep-learning techniques to computer vision. You will devise a
solution to a fashion compatibility problem based on the Polyvore dataset. Your task is to create a
category and fashion compatibility classifier.
Find (minimal) starter code for this project at https://github.com/franzke-usc-21fa/ee541-hw5-starter.
We expect you to explore and tune hyper-parameters and model architecture beyond the starter code.
Follow the instruction in the README file to setup the codebase.
Dataset
The Polyvore Outfits [1] dataset is real-world data created from users' outfit preferences at
polyvore.com. Consider paired items from outfits that receive high-ratings as compatible. The Polyvore
dataset contains 68,306 outfits created from 365,054 items. There are a maximum 19 items per outfit.
Figure 1 shows an example outfit.

Figure 1: A visualization of a partial outfit in the dataset. The number at the bottom of each image is the ID of this item.
Category classification
 The starter repository includes several files. Some are almost blank but give structure to follow.
First: make sure that you set the dataset location in utils.py – Config['root_path'].
1. train_category.py: training script
2. model.py: CNN classification models
Homework #5 EE541: Fall 2021
3. data.py: dataset handlers
4. utils.py: utility functions and configuration
 The train_model function in train_*.py handles model training. The driver applies a batch
of data to the model from the dataloader (data.py) during each iteration. You should modify the
code to record your training accuracy so you can review learning curves.
 Fine-tune your model and train “from scratch'”:
1. Finetune a pretrained ImageNet model (e.g., ResNet50). The PyTorch model-zoo allows ready
access to many standard CNN architectures.
2. Construct your own model and train from scratch.
3. Compare the two models (pretrained vs. custom) and comment on the results. What is the
advantage of fine-tuning a pretrained model vs. creating a custom network? Did you require
learning rates for each model?
Note: the images folder contains all the data images coded by its id. You can find more information
about each image in polyvore_item_metadata.json.
 Modify data.py. Create supervised training pairs (image, category label). The
get_data_transforms function applies input normalization.
 Split no less than 10% data for testing your final model. Generate a category.txt that lists item ids
from your test set and the predicted category vs ground-truth category.
Tips
1. Expect over-fitting. Adjust and tune the model structure, hyper-parameters, and regularization to
reduce over-fitting. You can design any custom model structures you like.
2. Set to(device) flag (device=cuda:0) and increase the batch_size defined in utils.py
to speed up training.
3. To debug try reducing the>number of epochs and set a random seed to aid debugging.
4. It will take many epochs to reach a performance plateau. But this will depend on the network
structure and the learning rate(s). Explore whether training further increases generalization or
tends toward over-fitting.
References
Mariya I Vasileva, Bryan A Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth.
Learning type-aware embeddings for fashion compatibility. Proceedings of the European Conference on
Computer Vision (ECCV), pages 390-405, 2018.
























































学霸联盟