UMKC SSE
CS5567 Deep Learning Project 3 -- FS 24


IMPORTANT: (please read and follow all steps!)
Instructions:
1. Take this step-by-step.
2. Don’t get overwhelmed.
3. Remember that we’re here to support you, so please ASK for assistance!

❖ FILES:
➢ Download “DL Project 3.zip” – it contains (.ipynb) files.
➢ Where appropriate load the scripts given for each experiment in your Kaggle (preferred) / Colab
Or local python workspace. If you need help setting up a local workspace, we’ll be happy to assist!
➢ Most of the data files you’ll need will be loaded as part of the scripts, but some will require a bit
of integration (many examples and links are provided).

❖ Loading the dataset into the notebook:
➢ Copy the URL of the dataset provided
➢ Access the notebook, on the right panel → Add Input → Paste the URL → Click the ‘+’ button at
the bottom → Cool! It’s There!

❖ Instructions:
➢ There are 2 Parts A & B
➢ The Experiments table (below) includes guidelines/instructions.

❖ Results:
➢ Save a record of the values you achieved along with EACH experimental configuration you tested.
➢ Both the TRAIN and TEST metrics are important to the assessment of each task. There are a variety
of ways these data may be displayed and reported, but always remember that a VISUAL
representation {plot} will be superior to a simple numerical representation {table}.
➢ You will provide 1-2 pages of your formal REPORT on your findings from the experiments in PARTS
A and B using appropriate terms, and references* (where applicable).
*Reminder: All generated material must provide reference to the source model.

❖ Project 3 Report: Generate an explanation of your efforts and findings as well as the responses to
the questions requested under “Discussion” at the end of the document.
This will be in the form of 2 documents:
➢ ONE report document (.docx or .pdf)
▪ Notes, observations, Results, and Responses to Discussion – your observations when
implementing the requested processes.
▪ Accessible Links to all 5 Notebooks and Accessible Link to the Panopto Video, Please Re-Check
the accessibility of the links before submitting to get your efforts graded.
▪ This video highlights all your RESULTS, Efforts and understanding on all the topics. Do not
exceed 7 minutes (5 points penalty for every 10 sec under 6:30 min and over 7:30 min)
➢ ONE report file (.xlsx)
▪ Save the output of your training & testing processes as part of this file.
Experiments



Part A – Object Detection and Segmentation
Experiment 1 Localization [ROI]
Data Set Dataset used in the notebook - Dataset (similar format for new datasets)
For new datasets use - https://public.roboflow.com/object-detection
Look for the file format → "YoloV8"
Modifications Modify the hyperparameters
➢ Epochs
➢ Batch size
➢ Optimizer
➢ Learning rate
➢ Regularization techniques
Process Utilize the supplied (Object_detection_using_yolov8.ipynb)
❖ Modify the existing code from the notebook and replicate the process and
results on the dataset. Target validation mAP50-95 value is 0.75.
Reporting Add the results of your training &/or testing (output) to your .xlsx file
➢ Save the images and video generated from your experiments
➢ Save the metrics provided for final performance (IoU, etc.)

Experiment 2 UNet Segmentation
Data Set ❖ Dataset used in the notebook - Dataset
Process Utilize the supplied (UNet Segmentation.ipynb)
❖ Modify the existing code from the notebook and replicate the process and
results on the dataset. Target Test IOU - 0.67 and Dice - 0.75
Modifications Modify the hyperparameters
➢ Batch size
➢ Optimizer
➢ Learning rate
➢ Regularization techniques
Reporting Add the results of your training &/or testing (output) to your .xlsx file
➢ Save the images generated from your experiments
➢ Save the metrics provided for final performance (IoU, Dice_coef, etc.)

Experiment 3 SegFormer Segmentation
Data Set Included
❖ Feel like testing SegFormer on custom dataset? Feel free to import any
image dataset and generate results.
Process Utilize the (Fine_tune_SegFormer.ipynb)
Follow the steps indicated in the script to load and refine the model.
Modifications Select 3 new sample images from the dataset (at random) and assess the
performance of the task optimized network on those images.
Reporting Add the results of your training &/or testing (output) to your .xlsx file
➢ Save the images generated from your experiments
➢ Save the metrics provided for final performance (IoU, etc.)
Experiments



Part B: Transfer Learning

Experiment 1 & 2 Transfer Learning
Data Set CIFAR-100  colab import example
Pre-Trained
Models
❖ Included (vgg19_transfer_learningipynb) Reference → VGG_19
❖ Included (resnet50_transfer_learning.ipynb) Reference → RESNET
Parameters Utilize the Keras API TRANSFER LEARNING documentation:
https://keras.io/guides/transfer_learning/
➢ Look first at the section under the heading →
“The typical transfer-learning workflow”
➢ Load the model and weights for each pre-trained model above.
➢ Add the classification portion for the dataset referenced above.
➢ Train your classification layers using your preferred topology and
optimization functions.
Modifications You should “freeze” the model after importing e.g. →
base_model.trainable = False
Get familiar with Resizing → Resizing (important for model training)

Make whatever hyperparameter changes you need to arrive at a reasonable level
of performance [10-15 % < industry standard] in classification for the listed Data
Set. Make sure the model performance (testing accuracy) is not depleted beyond
60% and 65% for vgg19 and resnet50 respectively.

❖ Utilize knowledge gained thus far to determine which topology and
hyperparameters are likely to provide reliable performance for this task and
don’t forget about regularization methods.
PROCESS For Each Pre-Trained Model → “Train the top layer” (search term in reference)
[add the DENSE elements and build the model appropriate for each Data Set]
(2 Models x 1 Dataset) = 2 total outcomes/models
Reporting Add the results of your training &/or testing (output) to your .xlsx file
➢ Track each of your model’s performance during training and testing.
➢ Plot Training and Testing performance (accuracy)
➢ Plot Metrics (ROC, mAP, etc.)

Discussion:
1. Compare the results of your experiments for Object Detection.
a. Display and discuss the results of object detection on images and video using YOLOv8.
b. Which setting (images or videos) provides the best object localization and classification
performance?
c. What challenges did you encounter in detecting partially visible objects, and how did the
model perform in these cases?
d. In which scenarios did you observe false positives or false negatives, and what might be
the reason behind these misclassifications?
Experiments



2. Compare the results of your experiments for Segmentation.
a. Display and discuss the results of the image segmentation using the UNet model.
b. Display and discuss the results of the image segmentation using SegFormer.
c. Which model do you feel provides the best structural understanding of geometric
structures?
d. Did you observe any issues with overlapping objects?

3. Compare the results of your experiments for Transfer Learning.
a. Display and discuss the selected classification layer training performance.
b. Display and discuss the results of the test performance for each experiment.
c. Which model do you think provided the better set of features for training between the
VGG and RESNET? How did you come to this conclusion?

4. Discuss in a few sentences your observations with each of the experiments. How do the things
you observed relate to the things we’ve covered in this course? Do not simply include the
definition of these topics – instead speak about how you feel your efforts relate to the outcomes.



学霸联盟