1
ATHK1001 ANALYTIC THINKING: ASSIGNMENT 1, 2023

Due date: 11:59pm Friday, March 31st (Week 6). Late penalty of 5% per calendar day applies.

Online submission: All submissions are to be made online on the ATHK1001 Canvas website.
Submissions will be checked for plagiarism. Artificial Intelligence tools such as ChatGPT that assist
writing are not permitted.

Incorrect submissions: If you discover before the closing date that the file you submitted on
Turnitin was incorrect, and let us know, you may be given the option to resubmit a corrected
version with a 50% penalty or the relevant lateness penalty, whichever is greater.

Word length: 750 words across all questions (excluding references in Question 13). A penalty of
10% will apply to papers that exceed this limit by more than 10%, a 20% penalty if you exceed
20% of the limit, and 30% if you exceed the limit by 30%.

Total marks: 60 (15% of total grade for class)

Background and Aims
A useful skill when dealing with data is being able to estimate plausible answers to questions you
don’t know the answer to. A strategy for doing this is embodied by what are known as “Fermi problems.”
These are numerical estimation problems that break down a difficult estimation problem into steps. A
common example is “How many piano tuners are there in Chicago?” Most people have no idea of what
the right answer may be, but it can be broken down into a set of estimations about which people have
more confidence: How many people are there in Chicago? How many families? What proportion of
families have a piano? How often does a piano have to be tuned? How long does it take to tune a piano?
How many days a year would a piano tuner work? People can come up with reasonable estimates for the
sub-questions and then combine them to make a reasonable estimate for the main questions. They are
called Fermi problems after the Nobel-prize winning physicist Enrico Fermi who reportedly estimated the
force of the first atomic explosion from how far a dropped piece of paper travelled. He was famous for
being able to use such meagre pieces of information to derive surprisingly good answers to questions.
Most famously Fermi problems are the basis for the Drake equation for estimating the number of extra-
terrestrial civilizations. In both science and engineering education Fermi problems are used to show
students the power of deductive thinking, introduce mathematical modelling, and prepare them for
experimental laboratory work. As such they have been used at educational levels ranging from primary to
tertiary (see Ärlebäck & Albarracín, 2019).
Ärlebäck and Albarracín (2019) conducted a systematic literature review and identified 91 articles
that addressed Fermi problems. Forty-three of these articles are described as empirical studies focused on
teaching or learning using Fermi problems. These have demonstrated that Fermi methods can be taught,
but Ärlebäck and Albarracín report that although many of the articles advocate for the benefits of
teaching Fermi problems to students none of the reviewed research focused on explicitly establishing
evidence supporting the argument that Fermi problems improve students’ estimation skills. So there
appears to be no published research that addresses the basic question: Does treating a problem as a Fermi
problem lead to better estimates?
The experiment conducted for this assignment addressed the question of whether participants
make more accurate estimates for Fermi problems rather than non-Fermi problems. It does by treating the
same question as a Fermi problem when it is broken down into sub-questions, or a non-Fermi problems
the target question is presented alone. To evaluate this issue, we tested a set of hypotheses using the data
we collected in tutorials in Week 2.

Method
Participants
A total of 197 students from analytic thinking course (ATHK1001) participated as part of a class
experiment were analysed. A larger number participated but were eliminated from the analysis due to
2
their data not being correctly recorded, they completed too little of the experiment, or they did not give
consent to having their data analysed. Of the analysed participants 92 were female, 100 were male, and
they had a mean age 19.3 years.

Materials

All question used were given as either Non-Fermi problems or Fermi problems. Fermi problems
asked the participant to answer a series of four or five sub-questions before answering the main question.
For example, the Fermi problem version of the question “How many piano tuners do you think there are
in Chicago?” first asked:
What do you think is the population of Chicago?
How many pianos do you think there are in Chicago?
How many hours do you think it takes to tune a piano?
How often do you think pianos are tuned over a ten-year period?
How many hours a year do you think the average piano tuner works?
The Non-Fermi problem version just asked “How many piano tuners do you think there are in Chicago?”
Two sets of nine questions each were used. Most Fermi problems presented in lists of Fermi
problems seems to encourage estimation by multiplication sub-question answers, such as appears to be
the case for the Chicago piano tuners question. However, some questions seem to encourage addition of
sub-question answers. For example, the question “What is the total military budgets of the members of
the UN Security council (USA, Russia, China, UK, France), in millions of $US?” was broken down into
the sub-questions:
What is the military budget of the USA, in millions of $US?
What is the military budget of Russia, in millions of $US?
What is the military budget of China, in millions of $US?
What is the military budget of the UK, in millions of $US?
What is the military budget of France, in millions of $US?
So, we constructed a set of nine multiplicative questions and a set of nine additive questions. A list of the
nine questions for each set together with their sub-questions can be found in the Appendix. Every
participant received both sets of questions: one set as Fermi problems (i.e., with sub-questions) and the
other set as Non-Fermi problems. The experiment had two conditions: in the “Multiplicative condition”
participants answered the nine multiplicative questions as Fermi problems and the nine additive questions
as Non-Fermi problems, where as in the “Additive condition” participants answered the nine additive
questions as Fermi problems and the nine multiplicative questions as Non-Fermi problems.

Procedure

During tutorials for the class Analytic Thinking at the University of Sydney participants
completed the experiment individually on computers in the classroom or online. All participants were
randomly assigned by the computer to either “Multiplicative condition” or the “Additive condition” and
then they answered either their nine Fermi problems then their nine Non-Fermi problems, their nine Non-
Fermi problems then their nine Fermi problems. The order in which they answered the problems was
varied randomly. Responses to each question or sub-questions were entered into textboxes on a webpage,
and participants had to enter numbers in response to every question before they cold advance to the next
page.
After completing the experiment participants indicated whether or not they consented to having
their data included in the data set.

Hypotheses

We proposed four hypotheses to investigate how participants performed on Fermi and Non-Fermi
problems.

3
First, we will test whether Fermi and Non-Fermi problems differed in terms of accuracy, and based on
the wide endorsement of Fermi problems we predicted that participants would do better on Fermi
problems.
Hypothesis 1: Mean accuracy for Fermi problems will be different than for Non-Fermi problems.
We predict accuracy will be greater for Fermi problems.

Hypotheses 2 and 3 tested if which set of questions were answered as Fermi problems would make a
difference. We had no reason to predict a difference.
Hypothesis 2: Mean accuracy for Fermi problems will not be differ between “Multiplicative
condition” and the “Additive condition”.

Hypothesis 3: Mean accuracy for Non-Fermi problems will not be differ between “Multiplicative
condition” and the “Additive condition”.

Hypothesis tested whether how accurate participants were on Fermi problems was associated with their
accuracy on Non-Fermi problems.
Hypothesis 4: Accuracy for Fermi problems will correlate with accuracy for Non-Fermi problems.


Results

The data set for our class can be found on the Canvas site for ATHK1001 under “Assignment 1”.
This assignment description can be found there as well as an Excel file called “Assignment1_dataset.xls”.
This Excel file contains all the data for the assignment and has 197 data lines, one for each participant.
Each participant has values for four variables, and the values of each variable are in a single column of
the file.
The first variable is an arbitrary id number generated by the computer. The “condition” column
gives each participant’s condition: “1” means participants were in the “Multiplicative condition” and “2”
means they were in the “Additive condition”. The third variable is the participants accuracy on their
Fermi problems. The third variable is the participants accuracy on their Non-Fermi problems.
Accuracy was calculated based on whether participants’ answers were the same order of
magnitude as the correct answer. The order of magnitude of an answer is an approximation of the
logarithm (base 10) of a value, and can be understood as the numbers of digits in a integer answer minus
1. For, example, any answer between 100 and 999 would have and order of magnitude of 2, because 100
is 102. In estimation which is expected to be inaccurate often the aim is to estimate to the correct order of
magnitude, and sometimes this is explicitly seen as the goal when setting a question up as a Fermi
problems. The Appendix presents the correct order of magnitude for each question.
For a participant’s Fermi problems how many of their answers had the correct order of magnitude
was countered up and then divided by the number of questions they answered, so the Fermi accuracy was
a proportion between 0.0 and 1.0. Similarly, participants’ Non-Fermi accuracy was the proportion of their
Non-Fermi problems that their answers had the correct order of magnitude.


WHAT YOU WILL WRITE

Your task is to analyse the data in order to test the four hypotheses proposed above. You will do this by
addressing each of the following 13 questions. Answer all questions with complete sentences, not with
just numbers, notes or tables. You will be penalized if you do not use complete sentences. Do not include
the text of the questions in your assignment (this will trigger a plagiarism warning), but you should
include the number of the question being addressed.

A note on how many digits to report in your answers. Excel will give answers with a huge string of
digits, many more than are necessary to understand the results, so the question arises of how many digits
to report in your answers? The convention is to use the same number of significant digits as the data used
4
in the calculation. Significant digits are the digits excluding the leading and trailing zeros. Therefore
1.23, 0.0000123 and 12300000000, all have three significant digits.

1) For Fermi accuracy and Non-Fermi accuracy report the means and standard deviations. (4 marks)

2) Based on the means you calculated in Question 1 use a t-test to test Hypothesis 1, that mean accuracy
for Fermi problems will be different than for Non-Fermi problems. Report the p-value for the t-test and
state clearly whether or not Hypothesis 1 was supported, and state why. (Note that we will be discussing
hypothesis testing in lectures in Week 4 and practicing using Excel to test hypotheses in tutorials in Week
5. So, you may need to wait to answer this question until we have covered the relevant material in class.)
(3 marks)

3) For Fermi accuracy report the means and standard deviations for both the “Multiplicative condition”
and the “Additive condition”. (4 marks)

4) Based on the means you calculated in Question 3 use a t-test to test Hypothesis 2, that mean accuracy
for Fermi problems will not be differ between “Multiplicative condition” and the “Additive condition”.
Report the p-value for the t-test and state clearly whether or not Hypothesis 2 was supported, and state
why. (3 marks)

5) For Non-Fermi accuracy report the means and standard deviations for both the “Multiplicative
condition” and the “Additive condition”. (4 marks)

6) Based on the means you calculated in Question 5 use a t-test to test Hypothesis 3, that mean accuracy
for Non-Fermi problems will not be differ between “Multiplicative condition” and the “Additive
condition”. Report the p-value for the t-test and state clearly whether or not Hypothesis 3 was supported,
and state why. (3 marks)

7) Present one graph that shows the means for absolute accuracy for Fermi accuracy and Non- Fermi
accuracy for the “Multiplicative condition” and the “Additive condition”. To do this you will need to
present a total of four means, and for each of these means include error bars (error bars visually indicate
standard errors above and below the mean). Either a bar or line graph could be used for this, but a bar
graph is preferable. (6 marks)

8) Consider your answers to Questions 4 and 6. What do they suggest regarding whether the type of
Fermi matters and if it matters how it matters? Explain why you drew this conclusion by referring to our
hypotheses and how strongly you draw it. (4 marks)

9) Calculate the correlation between a participant’s accuracy Fermi accuracy and Non-Fermi accuracy.
Test Hypothesis 4 and state whether Hypothesis 4 was supported, and why. What are the implications of
your tests of this hypothesis? (5 marks)

10) Identify three different issues with the way we collected data which could limit our ability to draw
conclusions from it. These issues could relate to one or more of the hypotheses. Clearly differentiate the
three issues as “Issue 1”, “Issue 2” and “Issue 3” and explain how each of these issues relates to a data
collection consideration raised in ATHK1001 lectures, readings or tutorials. For each issue suggest a way
it might be resolved in future research on this topic or if it cannot be resolved then explain why. (12
marks)

11) Based on your analyses presented for previous questions, state whether we have found evidence that
presenting questions as Fermi problems improves participants’ accuracy. Explain why you drew this
conclusion by referring to our hypotheses and how strongly you draw it. (4 marks)

5
12) Summarize what you think the data analysis you have carried out for this assignment tells us about
Fermi problems. Do they add to Ärlebäck and Albarracín (2019) review of the state of research into
Fermi problems, and if so state specifically how they do so. Explain your responses with explicit
reference to the results of your testing of the hypotheses and possibly the issues you raised in Question
10. (6 marks)

13) Include a reference section which lists the full reference for any paper you have cited when
addressing these questions. You must cite Ärlebäck and Albarracín (2019) in Question 12, and include
citations where ever appropriate. You should use APA style for citations and references, but we will
accept other standard journal article referencing formats. (2 marks)


Reference

The use and potential of Fermi problems in the STEM disciplines to support the development of
twenty‑first century competencies.
Authors: J. B. Ärlebäck and L. Albarracín
Source: ZDM, volume 51, pages 979–990.

THIS IS NOT IN A STANDARD REFERENCING FORMAT; YOU WILL NEED TO REFORMAT
THIS FOR QUESTION 13 USING A STANDARD FORMAT (E,G., APA STYLE)

Note that this paper provides background so you do not have to understand every aspect of it. Also, you
do not have to use other sources for answering these questions, but if you do then you must correctly cite
and reference these sources.

Formatting Recommendations

Our preferences
 Use the font “Times New Roman”, 12-point size, and double-space all the lines.
 Indent the beginning of each paragraph using one tab space.
 Use APA referencing style
















6
Appendix

The following table shows the question and sub-questions in the set of nine multiplicative questions. The
last column “Correct” is the order of magnitude of the correct answer.

Target question Sub-question 1 Sub-question 2 Sub-question 3 Sub-question 4 Sub-question 5 Correct
How many piano
tuners do you think
there are in Chicago?
What do you
think is the
population of
Chicago?
How many
pianos do you
think there are
in Chicago?
How many
hours do you
think it takes
to tune a
piano?
How often do
you think
pianos are
tuned over a
ten year
period?
How many
hours a year do
you think the
average piano
tuner works?
102
How many bald people
are there in the world?
What is the
population of
the world?
How many
men are in the
world?
How many
men over 50
are there in the
world
What
percentage of
men over 50
are bald?

108
How many words are
in the Encyclopedia
Britannica?
How many
volumes does
the
Encyclopedia
Britannica
have?
How many
pages does
each volume
of the
Encyclopedia
Britannica
have?
How many
lines are there
on each page?
How many
words per
line?

107
What is the total annual
revenue for the Sydney
Swans Football team,
in Australian $?
What is the
capacity of the
Sydney Swans
home ground
(the SCG)?
What
percentage of
the SCG is full
for a typical
Sydney Swans
game?
How much
does the
average
Sydney Swans
ticket cost?
How many
home games
do the Sydney
Swans play at
the SCG?
What
percentage of
the Sydney
Swans revenue
comes from
home games?
107
How much does a
typical Australian
household spend on
petrol in a year, in
dollars?
How many
days a year
would a
typical
Australian
household
drive?
When they
drive, how far
does an
Australian
household
drive on a
typical day?
What is the
typical fuel
consumption
of Australian
cars, in liters
per 100
kilometres?
What is the
average cost of
petrol per litre
in Australia (in
$)?

103
How much hair dye is
used in Sydney every
year, in litres?
What is the
population of
Sydney?
What
percentage of
people in
Sydney dye
their hair
regularly?
How many
times a year
does someone
typically dye
their hair?
How much
hair dye is
typically used
each time
someone dyes
their hair, in
millilitres?

105
How many English
teachers do you think
there are in high
schools in Beijing?
What is the
population of
Beijing?
What
percentage of
people in
Beijing go to
high school?
What
percentage of
high school
students in
Beijing learn
English?
How many
English classes
can one
English
teacher teach
per week?
How many
English classes
does the
typical student
have per
week?
105
If the land area of the
earth were divided up
equally for each person
on the planet, about
how much would you
get, in square
kilometres?
What is the
radius of the
earth, in
kilometres?
What is the
total area of
the earth in
kilometers
square?
What
percentage of
the area of the
earth is
covered by
land?
What is the
population of
the earth?

108
How much do all
University of Sydney
student spend on pizza
each week, in dollars?
How many
students does
the University
of Sydney
have?
What
percentage of
University of
Sydney
students eat
pizza each
week?
If a student
eats pizza in a
week, how
many pizzas
do they
normally eat?
How much
does the
typical pizza in
Sydney cost?

104
7



The following table shows the question and sub-questions in the set of nine additive questions. The last
column “Correct” is the order of magnitude of the correct answer.

Target question Sub-question 1 Sub-question 2 Sub-question 3 Sub-question 4 Sub-question 5 Correct
How many litres of
water does each
household in Sydney
use per day?
How many
litres of water
does each
household in
Sydney use per
day for
washing
themselves?
How many
litres of water
does each
household in
Sydney use per
day for
cooking?
How many
litres of water
does each
household in
Sydney use per
day for
drinking?
How many
litres of water
does each
household in
Sydney use per
day for
washing
dishes?
How many
litres of water
does each
household in
Sydney use per
day for other
uses?
102
How much would be
the total cost of fully
equipping a computer
lab with 100
computers, in
Australian $?
How much
would the
computers cost
to equip a
computer lab
with 100
computers, in
Australian $?
How much
would the
tables cost to
equip a
computer lab
with 100
computers, in
Australian $?
How much
would the
chairs cost to
equip a
computer lab
with 100
computers,
Australian $?
How much
would security
(e.g., locks,
security
cameras) cost
for a computer
lab with 100
computers, in
Australian $?

105
What is the total
military budgets of the
members of the UN
Security council (USA,
Russia, China, UK,
France), in millions of
$US?
What is the
military budget
of the USA, in
millions of
$US?
What is the
military budget
of Russia, in
millions of
$US?
What is the
military budget
of China, in
millions of
$US?
What is the
military budget
of the UK, in
millions of
$US?
What is the
military budget
of France, in
millions of
$US?
106
How much total
stainless steel would it
take to equip a typical
restaurant, in
kilograms?
How much
stainless steel
would a
typical
restaurant need
for cutlery, in
kilograms?
How much
stainless steel
would a
typical
restaurant need
for pots, in
kilograms?
How much
stainless steel
would a
typical
restaurant need
for kitchen
utensils, in
kilograms?
How much
stainless steel
would a
typical
restaurant need
for shelfing, in
kilograms?

103
Across Australia's state
capitals (excluding
Hobart), how many
total people are
watching TV at some
point each night?
In Sydney,
typically how
many people
are watching
TV at some
point each
night?
In Melbourne,
typically how
many people
are watching
TV at some
point each
night?
In Brisbane,
typically how
many people
are watching
TV at some
point each
night?
In Perth,
typically how
many people
are watching
TV at some
point each
night?
In Adelaide,
typically how
many people
are watching
TV at some
point each
night?
107
The largest theatres for
musicals or Opera in
Sydney are The Joan
Sutherland Theatre in
the Opera House, the
Capital Theatre, the
Lyric Theatre, the State
Theatre and the Theatre
Royal. If all were full,
how many people
would these theatres
collectively hold?
How many
people are in
The Joan
Sutherland
Theatre in the
Opera House
when it is full?
How many
people are in
the Capital
Theatre when
it is full?
How many
people are in
the Lyric
Theatre when
it is full?
How many
people are in
the State
Theatre when
it is full?
How many
people are in
the Theatre
Royal when it
is full?
103
If the longest rivers in
the world (Nile,
Amazon, Yangtze,
Mississippi) were
connected, what would
How long is
the Nile river,
in kilometres?
How long is
the Amazon
river, in
kilometres?
How long is
the Yangtze
river, in
kilometres?
How long is
the Mississippi
river, in
kilometres?

104
8
be their total length be,
in kilometres?
What is the total area of
the Great Lakes in
North America, in
kilometres square?
What is the
total area of
Lake Superior
in North
America, in
kilometres
square?
What is the
total area of
Lake Michigan
in North
America, in
kilometres
square?
What is the
total area of
Lake Erie in
North
America, in
kilometres
square?
What is the
total area of
Lake Ontario
in North
America, in
kilometres
square?
What is the
total area of
Lake Huron in
North
America, in
kilometres
square?
105
What volume of air is
contained in a typical
two bedroom Sydney
apartment, in cubic
metres?
What volume
of air is
contained in
the larger
bedroom of a
typical two
bedroom
Sydney
apartment, in
cubic metres?
What volume
of air is
contained in
the smaller
bedroom of a
typical two
bedroom
Sydney
apartment, in
cubic metres?
What volume
of air is
contained in
the bathroom
of a typical
one bedroom
Sydney
apartment, in
cubic metres?
What volume
of air is
contained in
the kitchen of
a typical two
bedroom
Sydney
apartment, in
cubic metres?
What volume
of air is
contained in
the corridors
of a typical
two bedroom
Sydney
apartment, in
cubic metres?
102



学霸联盟