COMM1110 - Evidence-Based Problem Solving
Week 3: Discovering Possibilities and
Recognising Patterns
Tutorial Guide to Answers
To
pi
cs
Breaking
Down
Problems
and Gaining
Insights
Discovering
Possibilities
and
Recognising
Patterns
Grounding
Problem
Solving with
Ethical
Frameworks
Revealing
Insights from
Limited
Evidence
Developing
and Testing
Hypotheses
with
Confidence
Identifying
Meaningful
Links to
Connect
Solution
Paths
Evaluating
Evidence
and Making
Decision
Communicating
Evidence-based
Solutions
Getting
Started with
Evidence-
based
Problem
Solving
A
ss
es
sm
en
ts
3
1 2 3 4 5 6 7 8 9 10
Flexibility
Week
11
COMM1110 Evidence-based Problem Solving
Course Map
2
1
Analytical Toolbox
Statistical Toolbox
Ethics Toolbox
Topic
Coverage
Assessments
Assessment 1 (20%):
Excel Training Program
1
W
ee
ks
Assessment 2 (65%):
2A Case Report Part 1 (25%)
2B Case Report Part 2 (40%)
2 3 Assessment 3 (15%):
3A Weekly Online Discussion (10%)
3B Weekly Tutorial Participation (5%)
2A 2B
2
COMM1110 Evidence-based Problem Solving
Course Summary
Problem-solving steps
Problem-solving Toolbox
Analytical Toolbox Statistical Toolbox Ethics Toolbox
1. Scope:
- Define the Problem
- Decompose the Problem
• 7-Step Problem-Solving Framework
[Week 1]
• 5Ws [Week 1]
• Logic Tree - Factor Tree [Week 2]
• Descriptive Statistics [Week 2]
• Graphical Representations [Week 2]
2. Analysis:
- Set Priorities
- Develop Workplan
- Conduct Analysis
• Pattern Recognition: Inductive,
Deductive, Abductive [Week 3]
• Collaborative Decision-making
Techniques: Brainwriting and
Brainstorming [Week 4]
• Logic Tree - Deductive Logic Tree
[Week 7]
• Causality with Quantitative Data:
Research Design (Correlation vs.
Causality, Confoundment and Controlled
Experiment) [Week 8]
• Probability Basics [Week 3]
• Pattern Recognition [Week 3]
• Distributions [Week 3]
• Sample & Population [Week 5]
• Sampling Distributions [Week 5]
• Statistical Inference [Week 5]
• Hypothesis Testing [Week 7]
• Evaluating Relationship between Two
Variables
− Scatter Plot [Week 8]
− Correlation [Week 8]
− Simple Linear Regression [Week 8]
− Multi Linear Regression [Week 9]
• 7-Step Ethical Framework [Week 4]
3. Decision:
- Synthesise Findings
- Communicate
Recommendations
• Evaluating Evidence: Reliability and
Validity [Week 9]
• Synthesis of Findings: Visual Narrative
[Week 10]
• Telling A Compelling Story: Situation-
Observation-Resolution Summary,
Pyramid Story Structure [Week 10]
• Model Building: Application and
Interpretation of Regression [Week 9]
• Statistical Biases [Week 9]
• Ethical Guidelines for Statistical Practice
[Week 10]
3
Case Part 1
4
Part 1: Furniture Store Expansion: Using data to
understand our hiring process
• Step 1: Form groups of 4 – 5 people
• Step 2: Small Group Discussion
• Your tutor will allocate one of the initial branches to your group.
• Your tutor will do the heavy lifting of getting you the required
numbers from the data. You need to interpret these results.
• Consider different perspectives within the group.
• Note down key points or solutions you agree upon.
• Step 3: Whole Group Discussion
• Regroup as a class.
• Going through each question. Each group share their insights on
each question.
• Whole group discussion and feedback
5
a) How likely is it for an applicant to receive and accept a job offer from
us? Does this change based on the position for which they apply?
If there is a difference, what might be causing this and how might it
affect Modern Living’s hiring process?
Based on your findings, can you estimate the number of candidates
that Modern Living will need to interview hire two (2) Floor Manager
and five (5) Retail Assistant positions for your allocated department?
6
a) Pivot Table with required probabilities
7
Here are the relevant proportions/probabilities for part a) based on the data. Whilst it is true that the numbers are facts, they
are also very open to interpretation. One of your roles this week is to make these interpretations.
a) How likely is it for an applicant to receive and accept a job offer from
us? Does this change based on the position/department for which they
apply?
If there is a difference, what might be causing this and how might it
affect Modern Living’s hiring process?
8
We can see the Kitchen department appears to have a higher success rate than
Bathroom and fittings.
There are many reasons that may contribute to this difference. These reasons include:
• The different departments will have different needs.
• Different hiring managers may have different approaches/expectations in their hiring
process.
• Some candidates may not be appropriate for the position.
• There might be a “luck of the draw” element with a relatively small sample size.
Some we want to maintain, others we want to change.
Note that this is just one example of a difference, you could have flagged
a) Based on your findings, can you estimate the number of candidates
that Modern Living will need to interview hire two (2) Floor Manager
and five (5) Retail Assistant positions for your allocated department?
9
To hire one (1) floor manager in Bathroom and Fixtures:
• Only 13.21% !"# applicants are successful.
• On average, this means that we need to interview 53 applicants to get 7 floor managers.
• For one (1) floor manager, we need to interview (on average) "#! ≈ 7.57 applicants.
To hire two (2) floor managers in Bathroom and Fixtures, we need to interview !"# ×2 ≈ 15.1428 applicants on average, i.e. we expect Modern Living to interview
about 15 candidates to fill these two (2) floor manager positions.
b) For the purposes of this activity, continue working on your allocated
department, but jointly analyse across Floor managers and Retail
Assistants.
(i) What proportion of applicants make it past this first stage (i.e. what
proportion of CVs are successful)?
(ii) For applicants that have already made it past the first stage of the
interview process, what proportion make it past the second stage (i.e.
for applicants that receive a phone interview, what proportion are
deemed suitable for the role)?
(iii) What proportion of job offers are accepted?
10
b)/c) Pivot Table with required probabilities
11
Here are the relevant proportions/probabilities for part b)/c) based on the data. Whilst it is true that the numbers are facts,
they are also very open to interpretation. One of your roles this week is to make these interpretations.
c) What do your results in part b) say about the hiring process? Which
stage is the hardest to pass (if we include accepting the job offer as a
“third” stage)? Do you think there are any urgent areas in the hiring
process we need to address immediately?
12
b)/c) Pivot Table with required probabilities
13
Here are the relevant proportions/probabilities for part b)/c) based on the data. Whilst it is true that the numbers are facts,
they are also very open to interpretation. One of your roles this week is to make these interpretations.
c) What do your results in part b) say about the hiring process? Which
stage is the hardest to pass (if we include accepting the job offer as a
“third” stage)? Do you think there are any urgent areas in the hiring
process we need to address immediately?
• By looking at the numbers, passing the first CV stage is the hardest, as it looks
to have the lowest chance of success.
• Without a benchmark for what to expect, it can be hard to identify any areas as
“urgent”.
• However, we can see that there is a noticeably larger discrepancy between the
two departments in stage 2. This signals an area for further investigation.
14
b)/c) Pivot Table split further by role
15
Here are the relevant proportions/probabilities for part b)/c) based on the data. Whilst it is true that the numbers are facts,
they are also very open to interpretation. One of your roles this week is to make these interpretations.
Case Part 2
16
Part 2: Identifying employee turnover
• Step 1: Small Group Discussion
• Return to your previously formed group of 4-5.
• Again, your tutor will do the heavy lifting of getting you the
required numbers from the data. You need to interpret these
results (this may involve a few more clculations).
• Consider different perspectives within the group.
• Note down key points or solutions you collectively agree upon
• Step 2: Whole Group Discussion
• Regroup as a class.
• Each group share their collective solutions.
• Whole Group discussion and feedback.
17
a) Before looking into the data, formulate a set of key questions. It’s
OK if this data set is not able to answer all of these questions, but
these may give you ideas for other places to look for more data.
18
• What are some indicators we can use to determine if there’s a staff turnover
issue?
• Why is staff turnover a problem?
• Assuming we are losing staff, where are we losing them?
• Some employee turnover is to be expected. What would be an appropriate
benchmark for comparison?
This is not intended to be an exhaustive list of questions. You have hopefully come up with even better questions than the
ones here.
b) What comments can you make based on the descriptive stats? If
you had to fit a Normal distribution to this data, what would you do?
19
b) Descriptive stats
20
b) What comments can you make based on the descriptive stats? If
you had to fit a Normal distribution to this data, what would you do?
• The mean tenure is 118 days (approximately 4 months). Based on your own
experiences, do you think this is a long time?
• The standard deviation is about 44.58 days. What does that suggest about a
“typical” tenure?
• The minimum tenure is 10 days. What might have happened here? Should we
investigate this further?
• The maximum tenure is 288 days (a bit over 9 months). Is this a lot? Is it very
little?
21
b) What comments can you make based on the descriptive stats? If
you had to fit a Normal distribution to this data, what would you do?
• We have the mean and standard deviation, so we could fit a normal distribution
if we wanted.
22
c) Draw a histogram of the tenure. Does this affect whether or not
you’d still use a Normal distribution?
23
c) Tenure Histogram
24
[10,
20]
(20,
30]
(30,
40]
(40,
50]
(50,
60]
(60,
70]
(70,
80]
(80,
90]
(90,
100
]
(100
, 11
0]
(110
, 12
0]
(120
, 13
0]
(130
, 14
0]
(140
, 15
0]
(150
, 16
0]
(160
, 17
0]
(170
, 18
0]
(180
, 19
0]
(190
, 20
0]
(200
, 21
0]
(210
, 22
0]
(220
, 23
0]
(230
, 24
0]
(240
, 25
0]
(250
, 26
0]
(260
, 27
0]
(270
, 28
0]
(280
, 29
0]
0
5
10
15
20
25
30
35
40
c) Draw a histogram of the tenure. Does this affect whether or not
you’d still use a Normal distribution?
25
Quite subjective.
Pros:
• Does appear symmetric
• Vaguely bell-shaped
Cons:
• Seems like there’s more than one peak
• Maybe a bit too spread-out or “flat” to be Normal
• Beyond the scope of this course: The “Kurtosis” for a Normal distribution is 3,
this data has kurtosis of about 0.38.
d) What insights can you draw from looking at the descriptive statistics
for the three departments separately?
26
d) (Some) Descriptive Stats
27
d) What insights can you draw from looking at the descriptive statistics
for the three departments separately?
28
• The tenure is very different for the three different departments.
• There are some differences if we compare the stores, but these are relatively
small.
• Electronics is doing noticeably worse than the other 2 departments.
• Whist we can’t be sure what is causing these differences, we should clearly be
analysing the three departments separately!
e) Based on the data, what proportion of employees leave within their
first 60 days ?
29
e) Proportions based on the data
30
e) Assuming it is appropriate to use the Normal distribution for the
individual departments and branches, what proportion of employees
leave within their first 60 days ?
31
There are many ways we can do this. One way is the =NORM.DIST function.
For Kensington Bath and Fixtures:
• The mean is 142.69 and the SD is 49.52.
• The Excel formula is then =NORM.DIST(60,142.69,49.52,TRUE)
• Excel gives us an answer of about 4.75%.
e) Proportions from the fitted Normal distribution
32
f) Based on these findings, what recommendations would you make, or
alternatively, what additional questions would you ask Modern Living?
• We might need some industry data to get a benchmark to compare against
Modern Living’s data.
• What contextual information may affect these values? For example, maybe a lot
of the Kensington employees are UNSW students that are working at Modern
Living as an extra job outside the term.
• The Electronics department seems to be performing poorly at Kensington and
the other stores. This may be worth flagging to Modern Living, especially since
a much larger proportion of employees are leaving before the 2-month mark.
• However, the data only covers employees that have left. We may want to look
at data on existing employees to get a more holistic picture.
33
This is not intended to be an exhaustive list of ideas. You have hopefully come up with even better ideas than the ones here.
34
Reflection and Key Takeaways