University of Toronto Mississauga
STA310 H5S: Bayesian Statistics in Forensic Science -
Winter 2021
Instructor: Dr. Ramya Thinniyam
Practice Test Questions
SOLUTIONS
1. True/False: If the statement is true under all conditions, select T; otherwise select F.
a) Tukey is a more powerful method for pairwise comparisons of group means than
Bonferroni when the design is balanced. T F
b) Consider a One-Way ANOVA with only two levels (Group 1, Group 2). Conducting
the One-Way ANOVA F-test is equivalent to testing H0 : µ1 = µ2 vs Ha : µ1 6= µ2
using the pooled t-test. T F
c) Bayes Theorem says that whatever the odds are in the absence of evidence, the odds
when the evidence is taken into account will be L as large, where L is the likelihood
ratio. T F
d) Least Square means are equal to arithmetic means in an ANOVA model. T F
e) A curved Normal QQ-plot of the residuals from an ANOVA model indicates that the
relationship between the predictor and response is non-linear. T F
f) For the same data set, consider three different models. SSTO will remain the same in
all three models. T F
g) The Bayes Factor is a measure of the relative strength of evidence in favour of one
hypothesis against another. T F
The Effectiveness of a New Migraine Pill: A study is conducted to determine
the effects of a new migraine medication. The initial level of pain was
measured for each of 101 patients who suffer from migraine headaches and
then each was randomly assigned to receive the treatment pill or a placebo
pill. After two hours from consuming the pill, the level of pain is once again
measured. At each time point, pain was measured on a 5-point scale where
lower scores correspond to lower pain levels and higher scores correspond to
higher levels of pain. The difference between the pain levels before and after
taking the pills (i.e. before - after) is calculated. The question of interest is
if the new migraine medication effective? (Refer to the R output attached
at the end of the test.)
The variables are: ‘PainDifference’ (pain score difference before - after), ‘Medicine’ (Treat-
ment=new pill with medication, Placebo=placebo - sugar pill without any medicine).
2. Fill in the Blanks: Refer to The Effectiveness of a New Migraine Pill study.
a) Fill in the following missing numbers from the output:
(A) = 1 , (B) = 99 , (C) = 0.8655 , (D) = 13.4263
.
STA310 - Winter 2021 Practice Test Solutions Page 1 of 6
b) Is the design balanced or unbalanced? Unbalanced [write either ‘balanced’ or ‘unbal-
anced’ ].
c) What was the average pain difference score for patients who took the placebo pill?
0.40 .
d) In ANOVA, we assume that the errors have a common variance, σ2. Give an unbiased
estimate for σ2: 0.8655 .
e) How much more of difference in pain, on average, did patients who took the new
medicine feel over those who took the placebo? 0.6784 .
f) What percent of variability in pain difference is explained by the type of treatment?
11.94 %.
For the following questions, you are given a statement and asked to circle
one of the options, citing numbers from the output as appropriate. If you
cannot answer the question, you should not write down a p-value:
g) We have evidence at the 5% significance level that pain difference scores vary by the
treatment.
Yes / No / Cannot Answer p-value: 0.0004 .
h) We have evidence at the 5% significance level that pain difference scores from all
treatments have the same variance in the population.
Yes / No / Cannot Answer p-value: N/A .
i) We have evidence at the 5% significance level that the new migraine pill is effective.
Yes / No / Cannot Answer p-value: 0.0002 .
without justification will not receive marks. Refer to The Effectiveness of a New Migraine
Pill study.
a) Write out the model that is being fitted. Define any variables you include.
Yi = β0 + β1IT,i + ei ; where
Yi = difference in pain for the ith patient
IT,i =
{
1 , if ith patient receives the treatment pill
0 , if ith patient receives placebo pill.
ei ∼ iid N(0, σ2)
b) Test the question of interest using the above linear regression model. Your test should
be in terms of one or more regression parameters; do not use the ANOVA F-test.
Include all the necessary steps for the hypothesis test (and include a practical conclu-
sion).
Using Model1, µP = β0 and µT = β0 + β1
If the new pill is effective then µT > µP (greater improvement for Treatment) ⇒
β1 > 0
H0 : β1 = 0 vs Ha : β1 > 0
t = 3.664 ∼ t99 under H0
p = P (t99 > 3.664) =
0.0004
2
= 0.0002. Reject H0
There is very strong evidence to conclude that the new pill is effective in reducing pain
from migraines!
STA310 - Winter 2021 Practice Test Solutions Page 2 of 6
c) What is another equivalent way you could test the question of interest (rather than
using a linear regression model)? Name the method and state the hypotheses, defining
any terms you introduce.
Another equivalent test would be the pooled t-test.
H0 : µT − µP = 0 (µT = µP ) vs Ha : µT − µP > 0 (µT > µP ) ;
where µT = mean difference in pain for patients who took the new treatment pill and
µP = mean difference in pain for patients who took the placebo pill.
d) Can you use directly use the ANOVA F-test to answer the question of interest? If
YES, conduct the test and answer the question of interest. If NO, explain why not
and describe how you would answer the question of interest.
We cannot directly use the ANOVA F-test to answer the question of interest because
the question is one-sided (treatment being effective corresponds to µT > µP ), whereas
the ANOVA F-test is always two-sided (testing if there is any differences between the
means). In order to answer the question of interest, we would have to do a post-hoc
analysis and in this case since it is just two means (one pair), we could follow up with
the one-sided t-test or directly compute the p-value for the alternative Ha : µT > µP
using the p-value we got from the test with the two-sided alternative µT 6= µP .
e) Name THREE assumptions that are required to do the inferences in this question.
State your answers in terms of this practical problem (not using statistical sym-
bols/general answers). Beside each, write which diagnostic plot/summary stats you
would use to check if the model assumption is valid.
(1) Error terms / difference in pain scores are Normally distributed - check the Normal
QQ-plot
(2) Difference in pain scores have constant variance - check boxplots of difference in
pain by treatment OR time series plot of residuals OR ratio of treatment sample
standard deviations
(3) For each patient, the difference in pain measurements should be independent from
another patient - check time series plot of residuals
Blood Stain left at a Crime Scene: A crime has been committed and a blood stain
is left behind at the crime scene. A suspect, whose blood type matches that of the stain
found at the crime scene, is arrested. Only 1 in 100, 000 of the population has this rare
blood type found at the crime scene (and in the suspect).
without justification will not receive marks. Refer to Blood Stain left at a Crime Scene
example.
a) Consider the following claim:
“In a city like this with a population of 30,000,000 people who may have committed
the crime, this blood type would be found in approximately 300 people. So the evidence
merely shows that the suspect is one of 300 people in the city who might have committed
this crime. The blood test evidence has provided a probability of guilt of 1 in 300, which
is negligible and cannot prove the suspect is guilty.”
STA310 - Winter 2021 Practice Test Solutions Page 3 of 6
(i) Evaluate the claim by identifying any flaws and the correct parts of the argu-
ment. Define appropriate events/hypotheses to express the quoted numbers as
probabilities using proper notation.
Let Hs be the hypothesis that the blood stain came from the suspect and Ho be
the hypothesis that the blood stain was left by someone other than the suspect.
Let E represent the blood test evidence.
P (Hs|E) = 1/300 and P (Hs) = 1/100000
All parts of this claim are correct until the last line that states “which is neg-
ligible and cannot prove the suspect is guilty.” Before the blood test evidence
was considered, the suspect was one of 30,000,000 people but after the evidence
was presented the suspect is one of 300 people. The evidence has led to a large
decrease in the population of potential suspects and so it cannot be negligible.
(ii) If any, what type of misconception was made? Name it and then explain in
detail.
This misconception is called the “Defender’s Fallacy” because it is often made
by defense attorneys in court trials. The fallacy committed by the defense is
that they argue that the evidence is irrelevant and has no value. If the suspect
was identified solely based on the blood stain the argument is okay, but often
the suspect will be found from other evidence as well.
b) Consider the two statements below:
Statement 1: “The chance of observing this blood type if the blood came from someone
other than the suspect is 1 in a 100,000.”
Statement 2: “The chance that the blood came from someone other than the suspect is
1 in a 100,000.”
(i) Explain the difference between the two statements. Define appropriate events/hypotheses
and state each as a probability using proper notation.
Let Hs be the hypothesis that the blood stain came from the suspect and Ho be
the hypothesis that the blood stain was left by someone other than the suspect.
Let E represent the blood test evidence.
The two statements are conditional inverses of each other.
Statement 1: P (E|Ho) = 0.00001 while
Statement 2: P (Ho|E) = 0.00001
(ii) If one were to think that these two statements are equivalent, what type of
misconception would it be? Name it and then explain in detail.
Thinking that the two statements are equivalent is called the “Prosecutor’s Fal-
lacy”, since this error of transposing the condtionals is often made by prosecutors
in court trials. The Prosecutor’s Fallacy is when the prosecutor wrongfully con-
cludes that the chance of the suspect being innocent based on the evidence is very
small (P (Ho|E) = 0.00001), when in reality it is the probability of the observed
evidence if the suspect is innocent that is very small (P (E|Ho) = 0.00001).
(iii) In order for both statements in b) to always have the same probability (in
this example the probability was quoted as 1/100,000 for both), what must be
mathematically assumed? (Hint: Use Bayes Theorem.) What is the wrong with
making this mathematical assumption in cases like this?
Let G represent the hypothesis that the suspect is guilty.
STA310 - Winter 2021 Practice Test Solutions Page 4 of 6
• If the suspect is guilty, the probability of the evidence (of a match between
the blood stain at the crime scene and from the suspect) is 1: P (E|G) = 1
• If the suspect is innocent, the probability of the evidence (of a match between
the blood stain at the crime scene and from the suspect) is : P (E|Gc)
(Statement 1)
• Given the evidence, the probability that the suspect is innocent is : P (Gc|E)
(Statement 2)
Using Bayes Theorem and letting Statement 1 and 2 have same probabilities
P (E|Gc) = P (Gc|E) = a (recall a is a very small number)
P (G|E)
P (Gc|E) =
P (E|G)
P (E|Gc)
P (G)
P (Gc)
1− a
a
=
1
a
P (G)
P (Gc)
The posterior odds will be approximately 1
a
as well⇒ prior odds is 1⇒ P (G)
P (Gc)
=
1⇒ P (G) = 0.5
The problem with this assumption is that it assumes a 50% chance that the
suspect is guilty (before the evidence is introduced). This goes against the legal
principle of “innocent until proven guilty” that should be followed in our court
system.
(iv) Compute the Likelihood Ratio. Interpret it in plain English.
The Likelihood Ratio is:
LR = P (E|G)
P (E|Gc) =
1
0.00001
= 100, 000
The blood test match is 100,000 times as likely if the crime scene and the suspect
blood samples are from the same person as it is if the samples are from different
persons.
STA310 - Winter 2021 Practice Test Solutions Page 5 of 6
The Effectiveness of a New Migraine Pill: R OUTPUT
> model1 <- lm(PainDifference ~ Medicine)
> summary(model1)
Call:
lm(formula = PainDifference ~ Medicine)
Residuals:
Min 1Q Median 3Q Max
-2.40000 -0.40000 -0.07843 0.60000 2.92157
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 0.4000 0.1316 3.040 0.003025 **
MedicineTreatment 0.6784 0.1852 3.664 0.000401 ***
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
Residual standard error: 0.9303 on 99 degrees of freedom
Multiple R-squared: 0.1194, Adjusted R-squared: 0.1105
F-statistic: 13.43 on 1 and 99 DF, p-value: 0.0004009
> anova(model1)
Analysis of Variance Table
Response: PainDifference
Df Sum Sq Mean Sq F value Pr(>F)
Medicine (A) 11.621 11.6207 (D) 0.0004009 ***
Residuals (B) 85.686 (C)
---
Signif. codes: 0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
> tapply(PainDifference, Medicine, length)
Placebo Treatment
50 51
> tapply(PainDifference, Medicine, mean)
Placebo Treatment
0.400000 1.078431
> tapply(PainDifference, Medicine, sd)
Placebo Treatment
0.9476071 0.9130857
STA310 - Winter 2021 Practice Test Solutions Page 6 of 6 