INFO20004 Usability Evaluation Methods The University of Melbourne
Copyright © The University of Melbourne Page 1 of 5



INFO20004 Exam (Individual) - 2022, semester 2
ACADEMIC HONESTY
• This is an individual exam. Your submission must be entirely your own work. • You must NOT consult with or work with another student or any other person to any extent. • Your answers must be your OWN writing, so take care not to copy or paraphrase text from a source. • You may use any reference materials to assist you in answering the questions. However, all submitted work must comply with university policy on Academic Honest and Plagiarism (http://academichonesty.unimelb.edu.au/)
CONTRIBUTION OF THIS EXAM TO YOUR SUBJECT GRADE
• The exam requires approximately 30 hours of work and is worth 45% of the subject grade.
• The exam is a hurdle, meaning that you must pass this exam to pass the subject overall.
HOW TO WRITE YOUR ANSWERS
• ANSWER ALL THREE QUESTIONS: Question 1 (10 marks); Question 2 (15 marks); Question 3 (20 marks)
• Write your answers in a Word document. Write your name, Student Id, Subject code and date at the top of the first page. Start your answer to each question on a new page. Clearly indicate which question you are answering at the start of the page.
• Answer each question in full sentences. You may use diagrams or other images to help clarify your points. If appropriate for your answer, you may use bullet point lists to present elements of your answer; but be sure to frame any list with a clear statement of what it contains.
• Referenced material must be clearly cited in the body of the text. List references at the end of any answer where they are cited.
WORD COUNT
• Follow the word limits carefully for each question. The following are NOT counted in your word count: any list of references; the contents of figures, diagrams and images; and the subheadings used in Question 3.
• INCLUDE the WORD COUNT for each question at the end of your answer.
HOW TO SUBMIT YOUR ANSWERS
• Submit your answers in a single Word or PDF document.
• The submission must include a completed coversheet available on Canvas.
• Name your submitted file: SURNAME_StudentID (e.g., " SMITH_123465.pdf")
• Submit the file to Turnitin via Canvas by: Wednesday 2nd November at 11:59pm. (Melbourne time)
INFO20004 Usability Evaluation Methods The University of Melbourne
Copyright © The University of Melbourne Page 2 of 5



Question 1: Usability Heuristics (10 marks) (600 words)
1.1 Evaluating the usability of a microwave oven (6 marks) You are asked to apply your knowledge of Nielsen’s usability heuristics to a specific device: the Breville
One Touch Microwave Oven (See Figure 1). Watch the following video that presents a 5-minute demonstration of the Microwave in action. You will see that the demonstrator is very positive about the microwave oven and describes it as ‘easy to use’. https://www.youtube.com/watch?v=whEnwvSLyMw&ab_channel=WilliamsSonoma Your task is to systematically identify and explain in what ways the oven is usable by reference to Nielsen’s usability heuritics. Comment on whether and how the microwave oven follows the FIRST SEVEN of Nielsen's usability heuristics (shown below). In your answer, address each heuristic in turn. For each heuristic, identify the aspects of the interface that are relevant to the heuristic and explain your interpretation. In your answer, refer ONLY to elements that are shown in the video demonstration provided above.
The first seven of Nielsen’s Usability Heuristics #1: Visibility of system status #2: Match between system and the real world #3: User control and freedom #4: Consistency and standards #5: Error prevention #6: Recognition rather than recall #7: Flexibility and efficiency of use
Figure 1. Breville One Touch Microwave Oven
INFO20004 Usability Evaluation Methods The University of Melbourne
Copyright © The University of Melbourne Page 3 of 5




1.2 Designing out errors (4 marks) Identify an example of a PHYSICAL device that you have used or seen others using that violates Nielsen’s 5th usability heuristic, shown below. In your answer, give a brief description of your chosen device, and give an explanation for how it violates the heuristic. As part of your account, include between 1 to 3 images of the device (either photographs or screenshots) that illustrate the device and your account of its usability.
#5: Error prevention Even better than good error messages is a careful design which prevents a problem from occurring in the first place. Either eliminate error-prone conditions or check for them and present users with a confirmation option before they commit to the action.
ASSESSMENT CRITERIA FOR Q1 • Correct understanding of the concepts of heuristic evaluation. • Correct application of those concepts to real-world phenomena. • Breadth and variety of relevant phenomena identified. • Clarity of description and explanation.
INFO20004 Usability Evaluation Methods The University of Melbourne
Copyright © The University of Melbourne Page 4 of 5



Question 2: Planning an Evaluation (15 marks) (800 words)
A company claims that it has developed an app called SafeMeal designed to allow restaurants and home cooks to prepare meals for people with a range of special dietary needs. SafeMeal provides information about several special diets including: vegetarian, vegan, celiac, gluten-free, nut-free, and keto. Users of
SafeMeal enter a list of all the ingredients of a particular meal, and the app provides feedback on which diet(s) it complies with, if any, and signals which ingredients violate which diets. If the meal involves pre-made products, like flavouring and sauces, the user can enter a product code and the app presents a breakdown of its ingredients based on an automated internet search; and again the app will advise which of these constituent ingredients violates which of the special diets. SafeMeal has another key function that generates suggested meals (with recipes) for each special diet based on an internet search that takes into account the ingredients involved. The user can enter a combination of special diets, and suitable meal options will be generated that comply with all the diets specified. Overall, the company believes that the app will allow home cooks and restaurants to cater more confidently and successfully for their guests’ dietary needs. Imagine that an international health regulator has contracted you to evaluate SafeMeal for use in homes and restaurants. Write a plan for your evaluation of SafeMeal. Your plan should aim to evaluate the various functions of
SafeMeal and it should follow the principles of an experimental design to test the claims made by the company. Address the following: usability, usefulness, and user experience. Describe all aspects of your evaluation such that it could be carried out by another evaluator simply by reading your plan. Include the following aspects:
• The overall aims of the evaluation, including what specific aspects of SafeMeal will be evaluated.
• The participants in the evaluation, and how you will select and recruit them.
• The full procedure that participants in the evaluation will be asked to follow.
• The collection of evaluation data.
• Assume that your evaluation will be carried out in Melbourne. Provide an account of what you MIGHT find out in your evaluation as follows:
• Present ONE data plot that shows how your quantitative data might look if they indicate that
SafeMeal would be useful for restaurants in serving customers with a range of dietary needs. Be sure to reflect the relevant elements of your evaluation plan in your choice of data plot and the data that you present.
• Explain in one or two sentences how this chosen data plot provides support for the usefulness of
SafeMeal. Assume that you have whatever financial and other resources are needed to carry out your evaluation.
ASSESSMENT CRITERIA FOR Q2 • Correct interpretation of the evaluation scenario. • Valid application of evaluation techniques, including those of experimental design. • Completeness of addressing the evaluation questions. • Understanding of the relationship between evaluation aims and evaluation data. • Clarity of description and explanation.
INFO20004 Usability Evaluation Methods The University of Melbourne
Copyright © The University of Melbourne Page 5 of 5



Question 3: Critical Review (20 marks) (1000 words) Your task is to write a critical review (1000 words) of the following article: Zhou, G., Nagle, A., Takahashi, G., Hornbeck, T., Loomis, A., Smith, B., ... & Yu, D. (2022, April). ‘Bringing Patient Mannequins to Life: 3D Projection Enhances Nursing Simulation’. In CHI Conference on Human
Factors in Computing Systems (pp. 1-15). You can find this article by going to Google Scholar and entering the title, or through the electronic journals available at the University Library. A copy will be posted on Canvas with this exam paper. A critical review is very different from a book review. For more information about how to write a critical review, see: • https://www.monash.edu/rlo/quick-study-guides/writing-a-critical-review • https://www.ucl.ac.uk/ioe-writing-centre/critical-reading-and-writing/critical-review Your critical review should cover the following which should serve as subheadings to structure your answer:
• The aim(s) of the study reported in the article
• The research method used
• Summary of the main findings
• Strengths and weaknesses of the method used
• Whether the article’s conclusions or recommendations are justified by the evidence presented
• An alternative method to address the questions investigated * * For the 'alternative method', you are asked to invent another qualitative or quantitative approach that might be taken to pursue the aims of this project: to learn more about some aspect of usability, usefulness and/or user experience in this setting. Your review must engage critically with the ideas presented in the article, and not just summarise its key points. Your review must demonstrate some reading and analysis of scholarly literature beyond the article itself. You should make reference to at least three other HCI/UX publications from scholarly sources. Note especially that “critical” here does not imply that your report should be negative or focused only on errors and shortcomings. Rather, “critical” means that you should examine carefully whether the material presented is consistent, makes sense, is well argued, and reaches valid conclusions. This is unlikely to be an all-or-none appraisal, and you will probably find both strengths and some weaknesses. Suggested sources of references are available in the subject LMS. You can also consult the references that are cited in your article or that were set as readings for this subject. Please use the American Psychological Association (APA 6th edition) referencing style, described here: https://library.unimelb.edu.au/recite

ASSESSMENT CRITERIA FOR Q3 • Understanding of the evaluation study’s aims. • Understanding of the methods used by the researchers. • Critical appraisal of strengths and weaknesses of methods. • Identification and description of an appropriate alternative method. • Appropriate comparison with other relevant concepts of evaluation methods. • Clarity and structure of writing. • Appropriate citation of references.