1
CMPT 459 Assignment 4

Due: 11:59 pm, August 6, 2021
100 points in total

Please submit your assignment in Coursys.

Every student has to complete the assignment independently. While you are encouraged to learn
through discussion with the instructor, the TAs and the peer students, any plagiarisms are serious
violation of the university’s academic integrity policy. We have absolutely zero tolerance of such
behavior. Using any tutoring platforms for this course, such as Course Hero or 51zuoyejun is
strictly NOT ALLOWED. If detected, a student using this kind of paid services in this course or
uploading materials of this course to such platforms will be regarded as dishonest and will be
reported to the university as a plagiarism case.

This assignment covers the materials on outlier detection and advanced topics.

Question 1 (20 points)
When an outlier detection algorithm presents a detected outlier to a user, the user may want
interpretation or explanation, that is, why the object is an outlier. Can you please suggest at least
2 ideas that you may interpret or explain why an object is (detected) as an outlier? Please use
concrete examples to deliberate your ideas. In your answer, please specify (1) what you want to
interpret (e.g., the detection process or the semantic meaning in applications) and (2) what a
user may get from the interpretation/explanation (e.g., what actions can be taken). Are your
interpretation/explanations specific to some outlier detection methods?

Question 2 (20 points)
Propose a definition of outliers in transaction databases. Please use a small transaction database
as an example to elaborate your definition and justify the rationale briefly. Please also describe
an application scenario where your definition is useful, and explain how the outlier transactions
may be useful to users.

Question 3 (20 points)
Consider a neural network with a single output unit. In a given epoch, we use a single negative
training tuple to update the model parameters. Let = 0 be the true target value of the
negative training tuple and be the output of the output unit, that is, the predicted output for
the given negative tuple.
1. If we use mean-squared error as the loss function, under what condition does the output
unit have the highest error δ in magnitude? What is in this case? Hint: is the error
term in the backpropagation algorithm and you should maximize |δ|.
2. If we use cross-entropy as the loss function, under what condition does the output unit
have the highest error in magnitude? You do not need to report the value of here.
2


Question 4 (40 points)
Implement the basic neural network model which consists of a hidden layer sandwiched by an
input and an output layer. You can use Numpy, Scipy and Pandas, or any other packages you
choose. Your implementation should meet the following requirements.

1. Data set: the famous MNIST dataset, which includes many handwritten digits. MNIST is
very popular in the machine learning and data mining community for verifying learning
models. It is accessible at http://yann.lecun.com/exdb/mnist/. By default, it is split into
the training set and testing set, where the training has 60k images of size 28 × 28 pixels,
and the testing set has 10k images of the same image sizes. The total number of categories
are 10, i.e., the digits from 0 to 9. Please note that these images are stored in grayscale,
so you have to treat them as images with only one color channel.
2. Given weight matrices ! ∈ ℛ"×$ and ! ∈ ℛ!%× ", and bias vectors ! = ℛ" and ' =
ℛ!%, learn a function () = 'σ(! + !) + ', where is the activation function,
either ReLU or sigmoid. Please use = 28 × 28 = 784 input units, one for each of the
pixels in the image, and ℎ = 32.
3. Train the neural network using cross entropy as the loss function.
4. Set the batch size to 256 for training and 1000 for testing. Use Adam as the optimizer and
set the learning rate to 0.001.

Please submit the following.

1. plot the training and testing loss curves with respect to the training epochs (at least 40
epochs);
2. plot the training and testing accuracy with respect to the training epochs (again, at least
40 epochs). You should have at least 3 runs with different random network initialization
and report their mean and standard deviation for the curves.

Hint: Dive into Deep Learning provides a step-by-step hands-on guidance about
how to build such neural networks.