Cardiff School of Computer Science and Informatics
Coursework Assessment Pro-forma

Module Code: CMT206
Module Title: Human Centric Computing
Lecturer: Dr Alia Abdelmoty, Dr Daniel J. Finnegan, Dr Hantao Liu
Assessment Title: Evaluation And Analysis Of Interface Design (External Students)
Assessment Number: 1
Date Set: 22 March 2022
Submission Date and Time: 20th May 2022 at 9:30am

This assignment is worth 100% (Q1: 33%, Q2: 33%, Q3:34%) of the total marks available for
this module. If coursework is submitted late (and where there are no extenuating
circumstances):

1 If the assessment is submitted no later than 24 hours after the deadline,
the mark for the assessment will be capped at the minimum pass mark;
2 If the assessment is submitted more than 24 hours after the deadline, a
mark of 0 will be given for the assessment.

Your submission must include the official Coursework Submission Cover sheet, which can be
found here:

https://docs.cs.cf.ac.uk/downloads/coursework/Coversheet.pdf

Submission Instructions

Description Type Name
Cover sheet Compulsory One PDF (.pdf) file per assignment Q1_[student number].pdf
Q2_[student number].pdf
Q3_[student number].pdf
Q1 Compulsory One PDF (.pdf) file (Part 1)
+
Spreadsheet for Q1(Part 2)
Q1_[student number].pdf
Q1_[student number]_HE.xlsx
Q2 Compulsory One PDF (.pdf) file Q2_[student number].pdf
Q3 Compulsory One PDF (.pdf) file Q3_[student number].pdf

Submission is via Email:
Q1 to Alia Abdelmoty: AbdelmotyAI@cardiff.ac.uk ;
Q2 to Daniel Finnegan: FinneganD@cardiff.ac.uk ;
Q3 to Dr Hantao Liu: LiuH35@cardiff.ac.uk .

Any deviation from the submission instructions above (including the number and types of
files submitted) will result in a mark of zero for the assessment or question part.
Staff reserve the right to invite students to a meeting to discuss coursework submissions.

Q1

Virtual meeting applications (such as Zoom, Teams, Webex, Blackboard collaborate, etc.)
have become essential working and social communication tools. There are many
applications on the market, and all compete for users.
In this assignment you will consider the design of one of these tools and write a heuristic
evaluation report on the usability of its interface.
Q1. Part 1: User and Task Analysis (10 marks out of 20)
a. Choose one example of these applications. You can choose to consider a web
interface for the application or a mobile version of it.
Identify and describe One primary Persona for your application. Include all
components of the persona description, as discussed in the lectures.

Note: Applications can serve a wide variety of users and as such, may have multiple
primary personas. The persona you choose will guide your consideration for the
subsequent steps in this exercise. ((3 marks out of 10))

b. Identify and describe one primary and one secondary tasks/use cases in the system
(primary tasks are important and frequently used functions by your primary persona,
secondary tasks are desirable but are of less importance to your primary persona
than primary ones). For every use case, provide a brief (text) description and the
outline of the steps in its basic flow scenario and Two of its alternative flow
scenarios. ( (3 marks out of 10))

Note: Tasks such as Register/Login are not primary use cases. These are
supplementary functions to allow a user to access the primary function of the
system.

c. Produce Two State Transition Networks (STNs) to describe the interaction flow in
your use case scenarios (for the basic and alternative flows in each use case).
Use an abstract depiction of the screens with no detail on design elements of the
interface. Your STNs need only depict states and user actions between states in the
use case scenarios. ((4 marks out of 10))

Note: Generally speaking, an interface state changes when an event occurs. For
example, an entry of text in a text field, a click of a button, etc. Your STNs need to
depict all possible states of interaction in the use case scenarios.

Q1. Part 2: Heuristic Evaluation (10 marks out of 20)
For every use case, do a heuristic evaluation of the individual states of the interface,
identifying all usability issues.
Report on the usability issues in an objective manner and present the issues in a
systematic and standard format using the templates (Excel spreadsheet) provided and
discussed in the lectures. Your report should include,
a. the usability problems you identified with some justification of the severity score
assigned to the problems and your suggestions for addressing the issues you
identified in subsequent revisions of the interface, (5 marks out of 10)
b. the good design features that should be maintained in any subsequent revision of
the application. (5 marks out of 10)

Learning Outcomes Assessed
1. Understand the complex nature of users and apply heuristics to create and evaluate
inclusive and multimodal user experiences.
2. Evaluate competing proposals for interface design and implementation.
3. Apply human centric design methodologies in the context of current and emerging
interaction technologies such as virtual and augmented reality.
Criteria for assessment

Credit will be awarded against the following criteria.
Component &
Contribution
Fail < 50% Pass (50-59%) Merit(60-69%) Distinction >=
70%
Q1 Part 1(a) (3
marks)
Personas are
missing or not
clearly justified
for the
application.
Suitable choice of
Personas to
represent user
segments
Sketchy
description of
Personas that will
not support its
utility for design.
Persona
described in
general terms,
but is not specific
enough to clarify
its specific needs
from the
application.


Persona
explained clearly:
Which functions
of the application
are important to
this person?
How (context,
platform) and
When (how
often) will this
person use the
application?

Q1 Part 1(b) (3
marks)
Use cases are not
described.


Good choice of
primary use
cases, but may
not be optimal
primary functions
for the chosen
persona.

Brief descriptions
provided of one
or both use cases,
but basic and
alternative
scenarios are not
clear or complete
.

Good choice of
primary use cases
that are relevant
to the chosen
primary persona

Good description
of both use cases
with clear and
complete outline
of basic
scenarios.
Use cases are
relevant to the
persona and their
primary status
justified to the
needs to the
persona.

Excellent account
of both use cases
and complete
account of basic
and alternative
scenarios.
Q1 Part 1( c) (4
marks)
STNs are missing
or do not relate
to identified use
case scenarios.
STN includes
most of possible
states of the
interface, but
some states may
be missing.
Some user
actions are
annotated.
Some steps of the
use case scenario
are not included
in the STN.
STN includes all
possible states of
the interface.
User actions are
mostly included,
but some may be
missing
STN accounts for
all steps of the
use case scenario
from start to end.
STN includes all
possible states of
the interface.
User actions are
clear and
completely
describe the
transition needed
between states.
STN accounts for
all steps of the
use case scenario
from start to end.
Q1 Part 2 (10
marks)
HE misses most
of the usability
issues, is not
consistently or
exhaustively
applied.

Mention of good
usability features
is patchy across
the interface.
.
HE considers the
usability of some
aspects of the
considered
states, but there
are obvious
omissions of
major usability
issues.

Usability
problems are
recorded in an
adequate but
sometimes
inconsistent
format.

Mention of good
usability features
is patchy across
the interface.

HE considers the
usability of most
aspects of the
considered
states.

Usability
problems are
recorded in a
consistent
format.

Good usability
features are
explored, but not
exhaustively
reported.

HE considers the
usability of all
aspects (selection
of interface
components,
layout, error
handling,
feedback, etc.) of
the considered
states.

Usability
problems are
recorded in an
consistent format
and is complete,
including,
severity rating
and possible
solutions.

Good usability
features are
explored fully and
documented
appropriately.




Feedback and suggestion for future learning

Feedback on your coursework will address the above criteria. Individual feedback and marks
will be returned to you via Email.

Feedback from this assignment will be useful for your dissertation.

Q2

This question is divided into two parts. Parts I and II are completed individually. Part I
requires you to plan an experiment choosing from one of 3 problems. Part II involves
reporting and reflecting on your experience.

Part I: Experiment Design and Conduct
You are tasked with designing an experiment. Pick ONE of the following problems:

• Design an experiment to assess the impact of keyboard and touch screen-based text
entry on text input speed.
• Design a study to compare the usability of TWO different ecommerce user interfaces
for browsing and choosing fashion products.
• Design a study to compare player experience between any TWO games of your
choice.

It is your responsibility to design the experiment. You must carefully plan out your
experiment and choose appropriate and applicable research methods. Key considerations
you must make are: 1) what variables to manipulate and measure, 2) which instrument
(e.g., questionnaire) is appropriate, 3) if a between or within groups design is appropriate,
and 4) what confounding factors might design decisions introduce and how to mitigate
them. NB: You must NOT conduct your experiment with any participants and/or collect data
in any way. Doing so without prior ethical approval will result in a breach of the university’s
ethics policy for human research: https://intranet.cardiff.ac.uk/staff/supporting-your-
work/research-support/research-integrity-and-governance/research-ethics

Part II: The Report
Write a reflective report. You should detail your experiment design with appropriate critical
reflection. At a minimum, you MUST include the following: a clear description of the IVs
and DVs in your experiment and discussion about and a copy of the questionnaire you
would use in your experiment. The questionnaire should be included as an appendix and
does not count towards the word limit. The rest is up to you. The key here is your
contribution: make sure it is clear to the reader what you learned from the process of
designing human-based experiments.

You should use the criteria for assessment (see below) as a set of guidelines for your writing.

Submission Instructions

• Individual Report: You must submit an individual report via Email by the submission
date and time (max 800 words). You MUST use the template document provided.
You may include figures and graphs in your report (keep these limited and relevant),
but the total report length must not exceed 3 pages of A4 (excluding the title page).
When submitting, export this report in PDF format. This is important: do not submit
your report in any format (e.g., docx, odt) other than PDF. You must also submit a
copy of the school cover sheet as stated above. Please follow the naming
conventions shown in Table 1 when submitting your files.
Learning Outcomes Assessed

• Recognize the importance of identifying and involving users in the design and
evaluation of interactive systems.
• Practical skills for evaluating interactive software systems.
Criteria for assessment

The marking scheme is as follows (percentage weights in brackets parentheses):
• Report (80%):
o A clear description of the experiment, including the IVs, DVs, and
questionnaire: 4 marks.
o Reflective reporting on the experiment design, justifying all decisions made
by reasoned argument and/or evidence: 6 marks.
• Report (20%):
o Reflective reporting demonstrating appreciation of the experience: 10 marks.

Credit will be awarded against the criteria shown in Table 5.

Table 5: Assessment criteria
Fail 3rd (40%-49%) 2.2 (50%-59%) 2.1 (60%-69%) 1st (70%+)
Report is
incomplete,
incoherent, and
lacking minimum
requirements

The report
contains some
detail, but not
enough to
reproduce the
experiment or it
is unclear what
was learned. It
illustrates a lack
of critical
reflection.
The report
contains enough
detail to
reproduce the
experiment
procedure yet is
lacking in depth
to convey
understanding
and appreciation
of the work
done. It
illustrates some
critical reflection.
The report is of a
medium quality,
containing
enough detail to
reproduce the
experiment
procedure
approximately,
with discussion
showing
understanding
and appreciation
of the work
done. It
illustrates good
critical reflection.
The report is of a
high quality,
containing detail
to reproduce the
experiment
procedure
exactly, with
discussion
showing
understanding
and appreciation
of the work
done. It
illustrates
excellent critical
reflection
including
discussion of the
limitations of the
experiment
including its
design.

Feedback and potential impact on future learning

Feedback on your coursework will address the above criteria. Individual feedback and marks
will be returned to you via Email.

Feedback from this assignment will be useful for your dissertation.


Q3

You are asked to conduct a statistical analysis of a dataset resulted from a user study. You
are asked to present your study and findings in the form of a report.

Assessment
type
Max. mark Max. length Format Deadline
Report 20 2 Pages .pdf 20th May 2022
at 9:30am

Your report will be assessed according to how well you are able to
• describe the dataset and research objectives, (4 marks)
• analyse the data and interpret the results of data analysis, and (8 marks)
• visualise the results of data analysis. (8 marks)

You should carry out a statistical analysis of the dataset to determine what the data tells
you about the user study and should visualise your results of data analysis in a way that
allows a reader to understand the findings of your study. You are free to use any software
package to conduct you study and generate visualisations.

1. The dataset
A user study was conducted to investigate the difference in display quality between three
different smartphones (i.e., referred to as S1, S2 and S3). Also, the impact of Age (i.e.,
Youth, Adults, and Seniors) on the perceived image quality should be investigated. A
number of participants were recruited to assess the smartphones. Display quality was
measured with questionnaires, using the following method: each participant was asked to
rate some images displayed on S1, S2 and S3, using a scale ranged from 0 (poor quality) to
100 (excellent quality). The dataset is available on Email as a Microsoft Excel file. You should
base your study entirely on the dataset provided.

2. The report
The report is limited to a maximum of TWO A4 pages including figures and tables. Arial font
size 11, with justification, should be used with Single line spacing throughout.

IMPORTANT! Any deviation from the instructions above will result in a deduction of 10% of
the maximum mark obtainable for the assessment. If you submit more than two pages as
instructed above, then ONLY THE FIRST TWO PAGES of the submission will be marked.
Extra submissions will be ignored.

IMPORTANT! Submission is via Email. Submit your report as a single .pdf file – NO other
attachments or files should be included.

You should write a clear and concise report. Your writing should be brief and to the point. A
suggested structure for your report is shown below.
• Introduction: should provide a brief description of the dataset. You should clearly
state the research objectives.
• Data analysis: should consist of one or more sub-sections on a statistical analysis of
the data. You should formulate your research questions, describe the methods and
procedure of statistical tests and discuss the results of these tests. The titles of
these sub-sections should reflect what the sections contain.
• Data visualisation: should consist of ONE or TWO visualisations on the results of the
statistical data analysis. You should include accurate and informative axis labels and
titles. The captions should provide sufficient information to the readers without
referring to the related text in the report.
• [References are not required for the report.]

Learning Outcomes Assessed

2. Evaluate competing proposals for interface design and implementation.
4. Demonstrate competency in the method of scientific analysis, the control of variables,
analysis, and the presentation of outcomes.
6. Select and apply suitable methodologies for the conduct and analysis of a subjective
experiment.

Criteria for assessment

Credit will be awarded against the following criteria.

Level Fail (0-49%) Pass (50-59%) Merit (60-69%) Distinction (70-
100%)
Description of
dataset and
research aims (4
marks)
No/poor
description of
context of
dataset.
No/poor
description of
research aims.
Context of
dataset and
research aims
described well.
Narrative is
sometimes
unclear.
Context of
dataset and
research aims
documented
well.
Narrative is clear
and coherent.

Context of
dataset and
research aims
thoroughly (with
cogent
justification)
documented.
Narrative is clear,
coherent, and
refined. A high
standard of
writing is
maintained
throughout.
Data analysis and
interpretation of
results (8 marks)
No/basic analysis
of data.
Data analysis
does not produce
meaningful
solutions for the
research aims.
Statistical
methods applied
correctly for the
most part.
Results
interpreted
correctly for the
most part.
Statistical
methods applied
correctly.
Results
interpreted
correctly.
Statistical
methods applied
carefully and
precisely (with
cogent
justification).
Results
interpreted
carefully and
precisely.
Data visualisation
and presentation
(8 marks)
No/poor
visualisation of
results.
No/poor
presentation of
story/findings.
Visualisation(s) of
results mostly
appropriate and
correct.
Presentation of
story/findings
mostly clear to
readers.
Visualisation(s) of
results
appropriate and
correct.
Presentation of
story/findings
clear to readers.
Visualisation(s) of
results carefully
and precisely
(with cogent
justification)
produced.
Presentation of
story/findings
clear and
accessible to
readers.

Feedback and suggestion for future learning

Feedback on your coursework will address the above criteria. Individual feedback and marks
will be returned to you via Email.

Feedback from this assignment will be useful for your dissertation.