MAST20005/MAST90058 Assignment 3
1. This problem involves analysis-of-variance decompositions for regression. Consider the regression model
Yi = α+ βxi + i for i = 1, . . . , n,
where x1, . . . , xn are treated as fixed, and 1, . . . , n are independent errors with
E[i] = 0 and var(i) = σ2 for each i.
We will use the notation x¯ = n−1
∑n
i=1 xi.
(a) For K =
∑n
i=1(xi − x¯)2, recall the analysis-of-variance formula
n∑
i=1
(
Yi − µ(xi)
)2
=
n∑
i=1
(Yi − Yˆi)2 + n(αˆ0 − α0)2 +K(βˆ − β)2.
Show that
cov(αˆ0, Yi − Yˆi) = 0.
(Moral: This completes Week 8’s tutorial problem 2(b).) [1]
(b) Consider the alternative analysis-of-variance formula
n∑
i=1
(Yi − Y¯ )2︸ ︷︷ ︸
SS(TO)
=
n∑
i=1
(Yˆi − Y¯ )2︸ ︷︷ ︸
SS(R)
+
n∑
i=1
(Yi − Yˆi)2︸ ︷︷ ︸
SS(E)
. (1)
(i) Prove that SS(R) is independent of SS(E). Hint: You can take for granted the fact that βˆ is
independent of the SS(E), as shown by Week 8’s tutorial problem 2(b). [1]
(ii) Show that under β = 0, SS(R)/σ2 has a χ21 disbribution. Hint: Use (i), the fact that SS(E)/σ
2 ∼
χ2n−2, and an moment-generating-function trick. [2]
(Moral of (i) and (ii): SS(R)/1SS(E)/(n−2) has a F1,n−2 distribution under β = 0.)
(iii) Recall we can use the t-statisitc
Tβ=0 =
βˆ
σˆ/
√∑n
i=1(xi − x¯)2
to test the hypothesis H0 : β = 0, where σˆ
2 = SS(E)/(n− 2). Show the algebraic identity
T 2β=0 =
MS(R)
MS(E)
,
where MS(R) = SS(R)/1 and MSE(E) = SS(E)/(n− 2). [1]
(Moral: a square of a tn−2-distributed random variable is F1,n−2-distributed. )
(c) Let y1, . . . , yn be the observed values for Y1, . . . , Yn respectively. By defining the sample variances and
covariances sxx = (n− 1)−1
∑n
i=1(xi− x¯)2, syy = (n− 1)−1
∑n
i=1(yi− y¯)2, sxy = (n− 1)−1
∑n
i=1(xi−
x¯)(yi − y¯), one can alternatively write βˆ as
βˆ =
∑n
i=1(xi − x¯)(yi − y¯)∑n
i=1(xi − x¯)2
=
sxy
sxx
MAST20005/MAST90058 Assignment 3 Semester 2, 2023
(i) Show that
n∑
i=1
(yˆi − y¯)2 = [
∑n
i=1(xi − x¯)(yi − y¯)]2∑n
i=1(xi − x¯)2
Hint: Recall αˆ0 = y¯ and yˆi = αˆ0 + βˆ(xi − x¯). [1]
(ii) Recall that the sample correlation r =
sxy√
sxxsyy
. Show that r2 =
βˆsxy
syy
. [1]
(Moral: Together with Week 8 Tutorial problem 5(b), from (i) one can also conclude the analysis of
variance formula (1) holds. Further, with (i) and (ii), one can readily see that r2 =
∑n
i=1(yˆi−y¯)2∑n
i=1(yi−y¯)2 , so
r2 represents the proportion of variation in yi’s explained by the xi’s via regression. )
2. Consider the one-way ANOVA model, where for each group i = 1, . . . , k, we observe ni normally distributed
Xij ∼ N (µi, σ2), j = 1, . . . , ni; observations across the groups are also independent. Let n =
∑k
i=1 ni,
X¯i· = 1ni
∑ni
j=1Xij and X¯·· =
1
n
∑k
i=1
∑ni
j=1Xij =
1
n
∑k
i=1 niX¯i·, and define the “sums of squares”
SS(T ) =
k∑
i=1
ni∑
j=1
(X¯i· − X¯··)2 =
k∑
i=1
ni(X¯i· − X¯··)2,
SS(E) =
k∑
i=1
ni∑
j=1
(Xij − X¯i·)2 =
k∑
i=1
(ni − 1)S2i
and SS(TO) =
k∑
i=1
ni∑
j=1
(Xij − X¯··)2,
as well as the decomposition SS(TO) = SS(T ) + SS(E). Show that, if µ1 = · · · = µk, then SS(T )/σ2 is
χ2k−1 distributed.
Hint: You can take for granted that SS(TO)/σ2 ∼ χ2n−1, SS(E)/σ2 ∼ χ2n−k, and the independence
between SS(E) and SS(T ). Use moment generating functions. [1]
3. Consider a two-way anova model with one observation per cell, where we assume Xij ∼ N (µij , σ2),
i = 1, . . . , a, j = 1, . . . , b are ab independent normal random variables with means defined by
µij = µ+ αi + βj ,
where
∑a
i=1 αi = 0 and
∑n
j=1 βj = 0. Let
X¯·· =
1
ab
a∑
i=1
b∑
j=1
Xij , X¯i· =
1
b
b∑
j=1
Xij , X¯·j =
1
a
a∑
i=1
Xij ,
and consider the null hypothesis
H0B : β1 = · · · = βb = 0
for the main effects of the factor B. Recall the standard decomposition:
a∑
i=1
b∑
j=1
(Xij − X¯··)2︸ ︷︷ ︸
SS(TO)
= b
a∑
i=1
(X¯i· − X¯··)2︸ ︷︷ ︸
SS(A)
+ a
b∑
j=1
(X¯·j − X¯··)2︸ ︷︷ ︸
SS(B)
+
a∑
i=1
b∑
j=1
(Xij − X¯i· − X¯·j + X¯··)2︸ ︷︷ ︸
SS(E)
(2)
2
MAST20005/MAST90058 Assignment 3 Semester 2, 2023
(a) Show that the “cross-terms”
C1 =
a∑
i=1
b∑
j=1
(X¯i· − X¯··)(X¯·j − X¯··), (3)
C2 =
a∑
i=1
b∑
j=1
(X¯i· − X¯··)(Xij − X¯i· − X¯·j ,+X¯··) (4)
and
C3 =
a∑
i=1
b∑
j=1
(X¯·j − X¯··)(Xij − X¯i· − X¯·j + X¯··) (5)
all equal zero. It is OK you just show (3) and (4), because one can readily conclude (5) from (4) by
a symmetric argument. [2]
(Moral: Hence, one can conclude the decomposition (2).)
(b) Show that the decomposition below is true:
a∑
i=1
b∑
j=1
(Xij − X¯··)2︸ ︷︷ ︸
SS(TO)
= b
a∑
i=1
(X¯i· − X¯··)2︸ ︷︷ ︸
SS(A)
+
a∑
i=1
b∑
j=1
(Xij − X¯i·)2.
Hint: Show that the “cross terms” in
∑a
i=1
∑b
j=1((Xij − X¯i·) + (X¯i· − X¯··))2 vanish. [1]
(c) Show that the term σ−2
∑a
i=1
∑b
j=1(Xij − X¯i·)2 is χ2a(b−1) distributed if H0B is true. [2]
(d) Show that SS(B)/σ2 is χ2(b−1) distributed if the hypothesis H0B is true. [2]
(e) Using (b)− (d), show that SS(B)/(b−1)SS(E)/((a−1)(b−1)) is Fb−1,(a−1)(b−1) distributed under H0B. [4]
Hint 1 : You can use the following theorem:
Theorem (Hogg and Craig, 1958). Let Q = Q1 +Q2, where Q, Q1 and Q2 are all quadratic forms
in N independent and identically distributed N (0, σ2) random variables, i.e.
Q =
N∑
i=1
N∑
j=1
cijYiYj and Qk =
N∑
i=1
N∑
j=1
cij,kYiYj for k = 1, 2,
where Y1, . . . , YN ∼i.i.d. N (0, σ2), and {cij , cij,1, cij,2}1≤i,j≤N are certain constant coefficients.
Suppose
• Q/σ2 is χ2r distributed,
• Q1/σ2 is χr1 distributed with r1 ≤ r, and
• Q2 is non-negative.
Then the following conclusions are true:
• Q1 and Q2 are independent, and
• Q2/σ2 is χ2r−r1 distributed.
Hint 2 : For constant coefficients c1, . . . , cN and i.i.d. N (0, σ2) variables Y1, . . . , YN , the expression
E = (c1Y1 + · · ·+ cNYN )2 is necessarily a quadratic form in the Yi’s because it is readily seen to be
so upon expansion; you can take this fact for granted. Another fact you can take for granted: If E1
and E2 are two quadratic forms in Y1, . . . , YN , then E1 + E2 is also a quadratic form in Y1, . . . , YN .
3
MAST20005/MAST90058 Assignment 3 Semester 2, 2023
4. The problem considers further aspects of the Wilcoxon tests.
(a) To test the null hypothesis H0 that a given continuous and symmetrically distributed random variable
X has median 0, we could use the Wilcoxon signed rank test statistic W =
∑n
i=1 sgn(Xi)rank(Xi),
where X1, . . . , Xn is a random sample of X. Under H0, we have learn that W has the same distri-
bution as the sum
∑n
i=1Wi, where W1, . . . ,Wn are independent variables with the properties that
Pr(Wi = i) = Pr(Wi = −i) = 1/2 for each i = 1, . . . , n.
(i) Show that, under H0, the moment generating function E[exp(sW )] of W has the form
2−n
n∏
i=1
(e−si + esi)
for any s ∈ (−∞,∞). Hint: The Wi’s are independent. [1]
(ii) For n = 2, expand the mgf in (i) to show that the null distribution of W is given by
k -3 -1 1 3
Pr(W = k) 1/4 1/4 1/4 1/4
Hint: Recall an mgf uniquely defines a random variable. [1]
(b) (R) Consider again the following dataset from the Week 10 tutorial, where yields from a particular
process using raw materials from two different sources, A and B, are tabulated as:
(A) 75.1 80.3 85.7 77.2 83.9 85.1 86.7 80.5 75.2 70.0
(B) 80.4 83.1 85.6 90.7 86.6 95.9 82.8 80.9 78.7 89.4
With the help of the wilcox.test function in R, perform an exact test at level 0.05 on the null
hypothesis that the source B does not render more yield than the source A. Provide your code and
conclusion. Hint: Read the documentation of wilcox.test if needed. [2]
4
MAST20005/MAST90058 Assignment 3 Semester 2, 2023
5. Let X be a multinomial random vector with number of trials n and success probabilities p1, . . . , pk, where∑k
i=1 pk = 1 for the k categories, i.e. if the integers x1, . . . , xk ≥ 0 are counts of the n trials falling into
the k different categories, then
Pr(X = x) =
n!
x1! . . . xk!
px11 · · · pxkk ,
where x = (x1, . . . , xk)
T . This naturally extends the binomial distribution to more than 2 categories.
Consider a generic null hypothesis H0 that specifies the success probabilities as
p1 = p
0
1, p2 = p
0
2 , . . . , pk = p
0
k,
for some non-negative values p01, . . . , p
0
k that sum to 1. Recall that the p-value is defined to be the
probability of the data being at least as “extreme” as what we have observed, under the null being true.
As such, if the observed value for X is x, it is natural to define the p-value for H0 as
p =
∑
x′:
PrH0 (X=x
′)≤PrH0 (X=x)
PrH0(X = x
′); (6)
the summation means we are summing over all the possible realizations x′ of X whose probability
PrH0(X = x
′) is less than or equal to the probability of observing x, PrH0(X = x), under the null.
The p-value defined in (6) is exact since no distributional approximation is involved.
This problem concerns “exact” versions of the goodness-of-fit tests we have introduced.
(a) (R) Let X1, . . . , Xn be n independent flips of a coin, i.e., Xi ∼iid Be(p), where p is the probability
of having the coin turning up head. We are to test the null hypothesis
H0 : p = 0.6.
Suppose n = 20, and we have observed X =
∑n
i=1Xi to be 13.
(i) Use the binom.test function in R to produce an exact p-value for H0. [1]
(ii) Use the function pbinom to compute the exact p-value, with respect to the formulation in (6);
show your code. You are precluded from using binom.test for this part, but your computed
p-value should match the one you have in (i). [2]
(b) (R) A group of rats, one by one, proceed down a ramp of one of three doors. We wish to test the
hypothesis that
H0 : p1 = p2 = p3 = 1/3,
where pi is the probability that a rat will choose door i, for i = 1, 2, 3. Suppose the rats were sent
down the ramp n = 90 times and the observed cell frequencies for the doors are n1 = 23, n2 = 36 and
n3 = 31 for door 1 to 3 respectively. Use the multinomial.test from the package EMT to perform
an exact test for H0 at level 0.05 and provide your conclusion. [1]
Total marks = 27
5

学霸联盟