COMP4660/8420 Semester 2 2023
Page 1 of 4
NN Assignment 1 Rubric
The rubric below adds up to a total of 100 points, which are then converted to the 20 marks Assignment 1 is worth. Note that where no specific
criterion is provided, this simply indicates a point mark between other explicated mark levels.
Rubric relating to elements that must be adhered to:
The submitted files are not named as required TRUE -3 points FALSE 0 points
The abstract or title gives away that this is a course assignment TRUE -3 points FALSE 0 points
The paper does not use the provided Springer template with appropriate margins TRUE -3 points FALSE 0 points
Rubric assessment and values
Abstract
5/100 pts
The abstract leaves out most key
information and does not
summarize the work.
0 points
The abstract provides most
of the relevant information;
aims, methods, results.
2 points
…
3
points
The abstract provides all relevant
information; aims, methods, results.
4 points
… and the abstract is clear and concise.
5 points
Task
Selection
5/100 pts
The paper does not explain
what problem or task it is
attempting to solve.
0 points
The paper states the problem
to be solved, but some
aspects are unclear.
2 points
…
3
points
The paper explains what the goal of
the work is. The paper partially
justifies why this task is worth solving.
4 points
The paper explains clearly what the goal of
the work is. The paper justifies why this
task is worth solving.
5 points
Task
Complexity
5/100 pts
The proposed task is not
challenging: a very simple model
easily achieves near perfect
performance.
0 points
The proposed task is somewhat
challenging: a very simple model is
not enough to achieve satisfactory
performance.
2 points
…
3
points
The proposed task is of sufficient
complexity: good pre-processing,
model design, and hyper- parameter
tuning will be needed for this task.
4 points
The proposed task is very
complex: creative thinking and
skillful programming will be
needed for this task.
5 points
COMP4660/8420 Semester 2 2023
Page 2 of 4
Methodol-
ogy
10/100 pts
There is little to
no description
of the
methodology
used.
0 points
The paper proposes a
methodology to solve the
task, however there is no
explanation of how or why
the methodology works.
3 points
…
4
points
…
5
points
…
6
points
The paper proposes a
methodology to solve the
task and explains how the
method works with
sufficient detail that a fellow
class-mate could
understand.
7 points
…
8
point
s
…
9
points
… And the proposed methodology is
well suited for the given dataset and
is justified with reference to lectures,
other papers or experiments. The
proposed methodology, if implemented
correctly, could reasonably be expected
to solve the problem.
10 points
Data
Inspection
5/100 pts
No effort has been made to
inspect or understand the
provided dataset.
0 points
The paper describes the
data at a superficial level.
2 points
The paper provides the size of
the dataset and the types of
each variable .
3 points
…
4 points
And the paper provides summary
statistics or plots of the data and explains
how this information informed the pre-
processing decisions.
5 points
Data
Preparation
5/100 pts
The raw data has been fed directly
into the model, with no justification
of why this is acceptable.
0 points
…
2 points
Basic pre-processing steps have been
applied where appropriate, such as
normalizing numeric variables.
3 points
…
4 points
And pre-processing steps specifically
tailored to the dataset have been
applied, where appropriate.
5 points
Findings
5/100 pts
The raw data has been fed
directly into the model,
with no justification of why
this is acceptable.
0 points
…
2
points
The paper accurately re-
implements the methods of the
chosen paper on a new dataset.
3 points
The paper proposes sensible modifications or
extensions to the methods in the chosen paper and
demonstrates their effectiveness or lack thereof.
4 points
The findings represent new
knowledge that is worthy of
publication.
5 points
Evaluation
10/100 pts
Little to no
attention is paid
to critically
evaluating the
work done.
0 points
The evaluation
methodology has
significant flaws
that invalidate
the comparison.
3 points
…
4
points
…
5
points
The evaluation is fair and
valid. Results are reported
against fresh data that has
not been used for training or
tuning models.
6 points
…
7
points
… And great care has been
taken to make the most of
the available data for
training and evaluation …
8 points
…
9
points
And the compared
methods are rigorously
evaluated against the
paper’s stated goals.
10 points
COMP4660/8420 Semester 2 2023
Page 3 of 4
Discussion
10/100 pts
Little to no
discussion of the
obtained results is
provided.
0 points
The discussion merely
repeats the results
without further
elaboration.
3 points
…
4
points
…
5
points
The discussion shows
good understanding
of the outcomes.
6 points
…
7
points
And limitations
are clearly
detailed.
8 points
…
9
points
And appropriate future work is
detailed, demonstrating deep
understanding of the problem.
10 points
Reproducibility
2/100 pts
Little to no settings or
hyperparameters are provided.
0 points
Most of the setting and hyper parameters are
provided, although some key ones are missing
1 point
All relevant algorithms, hyper-parameters and settings are provided
in the paper Another person would be able to reimplement the code
just by reading the paper
2 points
Writing
Clarity
5/100 pts
Writing is almost
impossible to
understand.
0 points
Writing is often
unclear, making it
difficult to understand
1 point
…
2 points
Writing is generally clear,
however there are some
grammatical mistakes.
3 points
…
4 points
…And writing is concise and to the point,
while still being clear and unambiguous.
5 points
Writing Style
3/100 pts
The paper is written in an inconsistent or
informal style.
0 points
The paper is written in a formal style
throughout.
1 point
…
2
points
The writing is engaging and interesting, presenting
the work as a cohesive story.
3 points
Presentation
5/100 pts
The paper is presented
as a wall of text.
0 points
…
1
point
Paragraphs are used effectively to
separate the text into manageable
chunks.
2 points
…
3
points
And tables and plots are used effectively to
convey results Images are used when
appropriate to explain concepts.
4 points
And the paper is well structured,
with one section following logically
from the next.
5 points
Citations
5/100 pts
Citations are not provided or are
inconsistent.
0 points
…
2 points
Most use of other’s ideas is cited
appropriately and consistently.
3 points
…
4 points
All use of other’s ideas is cited
appropriately and consistently.
5 points
Implementation
10/100 pts
The code crashes
when run.
0 points
…
3
points
The code does not implement
the algorithm described in the
paper correctly.
4 points
…
5
points
…
6
points
…
7
points
The code runs without error and
produces the results presented in
the paper.
8 points
…
9
points
The code implements the
methodology described in the
paper exactly.
10 points
COMP4660/8420 Semester 2 2023
Page 4 of 4
Code
Style
10/100
pts
Code is very difficult
to understand.
Variable names and
comments do little to
clarify the code.
0 points
…
3
points
Common functionality is often
copy-pasted into multiple
places.
Variables are given
appropriately meaningful
names described in the paper
correctly.
4 points
…
5
points
Code is generally
easy to understand,
but some places
could use more
comments or clearer
variable names
6 points
…
7
points
…And code uses
appropriate modular
design (common
functionality
factored out into
functions
8 points
…
9
points
…And comments are used
effectively to help explain
complicated sections of
code.
10 points