微信客服：xiaoxionga100
微信客服：ITCS521
R代写-STAT7310
时间：2021-05-15
/////
STAT7310 and STAT8310: Statistical Theory
Confidence Intervals
Semester 1, 2021
/////
Interval Estimation
Interval Estimation
I Let X =
(
X1 X2 · · · Xn
)T
, where X1,X2, . . . ,Xn are i.i.d. with
common pdf or pf fX (x ; θ) and θ ∈ Θ
I Form two statistics
L = l (X) and U = u (X) .
I An interval (L,U) is called a 100 (1− α) % confidence interval for θ if
P (l (X) < θ, u (X) > θ) = 1− α;
or
P (θ ∈ [l (X) ,U (X)]) = 1− α.
I Note that in the above, we are not saying that the probability that
θ < u (X) and θ > l (X) is 1− α. That is, 1− α is not the
probability that θ is in some interval. It is the probability that the
random interval (L,U) contains θ.
I L and U are called the lower and upper confidence limits.
DEPARTMENT OF MATHEMATICS & STATISTICS 3
Symmetric Confidence Intervals
I Suppose we choose functions l and u to satisfy
P (l (X) < θ) = P (u (X) > θ) = α2 .
I Then (L,U) is called a symmetric confidence interval (CI).
I Symmetric C.I.s are not always the shortest. It is generally the
shortest interval that is the one we want.
DEPARTMENT OF MATHEMATICS & STATISTICS 4
Pivotal Quantities
I These are useful in the devising of confidence intervals.
I Let
Q = q (X ; θ)
be a function of the random vector X and the parameter of interest θ
I We say that Q is a pivotal quantity if the distribution of Q does not
depend on θ
I Then, a 100 (1− α) % CI for θ is comprised of the set of θ values
which satisfy
q1 < q (X ; θ) < q2
where
P (Q < q1) + P (Q > q2) = α
DEPARTMENT OF MATHEMATICS & STATISTICS 5
Pivotal Quantities : Examples
1. X ∼ N (θ, σ2); σ2 known
Q =
√n (X − θ)
σ
is pivotal since Q ∼ N (0, 1) for all θ
2. X ∼ N (θ, σ2); σ2 unknown
Q =
√n (X − θ)
S
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
is pivotal since Q ∼ tn−1 , for all θ
DEPARTMENT OF MATHEMATICS & STATISTICS 6
3. X ∼ Γ (α, θ); α known
f (x ; θ) =
{ 1
θkΓ(α)xα−1e−x/θ ; x > 0
0 ; otherwise.
Then Q = Xθ is pivotal since
n∑
i=1
Xi ∼ Γ (nα, θ) .
Thus n∑
i=1
Xi
θ
∼ Γ (nα, 1)
and the distribution of this does not depend on θ
DEPARTMENT OF MATHEMATICS & STATISTICS 7
/////
Confidence Intervals for
Gaussian samples
Core assumption
I Typically for single sample (continuous) numeric data we assume that
X =
(
X1 X2 · · · Xn
)T with,
I Xi are independent of each other.
I Xi follow a normal (Gaussian) distribution.
I Each Xi has EXi = µ and var(Xi) = σ2, fixed.
I This is given the shorthand, Xi i.i.d.∼ N (µ, σ2).
I This is required for exact tests, however, for the results that are
analysing the mean, X can use the CLT,
√
n
(
X − µ
σ
)
d−→ N (0, 1)
I So approximate tests can be derived if the sample size is ‘large’
enough.
DEPARTMENT OF MATHEMATICS & STATISTICS 9
Estimation of the mean, µ, when the variance, σ2, known
Xi i.i.d.∼ N
(
µ, σ2
)
; σ2 known. We know
Z = X − µ
σ/
√n =
√n (X − µ)
σ
∼ N (0, 1) .
Thus
P
(
−z1−α/2 <
√
nX − µ
σ
< z1−α/2
)
= P
(
X − z1−α/2 σ√n < µ,X + z1−α/2
σ√n > µ
)
= 1− α
Hence
L = X − z1−α/2 σ√n
U = X + z1−α/2
σ√n
and the 100(1− α)% CI for µ is (L,U) .
Z ∼ N (0, 1)
0.05
2
0.05
2
DEPARTMENT OF MATHEMATICS & STATISTICS 10
R: Example interval for µ, when σ2 known
set.seed(88888888)
n = 25; sig = 4; mu = 17; mu1 = 16
x = rnorm(n) * sig + mu
x
# [1] 16.318648 13.619067 19.976965 22.040565 18.208809 13.760960 15.103078
# [8] 16.829801 16.054201 16.751592 5.824282 18.767051 11.292923 12.635395
# [15] 17.609836 17.225824 18.500564 26.304074 26.479513 19.521398 14.033471
# [22] 20.222837 15.955225 19.311570 20.268188
alpha = 0.05
mean(x) + c(-1, 1) * qnorm(1 - alpha/2) * sig / sqrt(n)
# [1] 15.73666 18.87260
DEPARTMENT OF MATHEMATICS & STATISTICS 11
Estimation of the mean, µ, when the variance, σ2, unknown
X i.i.d.∼ N (µ, σ2); σ2 unknown. We use the
fact that
T = X − µS/√n =
√n (X − µ)
S ∼ tn−1
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
.
Since
P
(
tn−1,α2 <
X − µ
S/√n < tn−1,1−α2
)
= P
(
X − tn−1,1−α2
S√n < µ,X + tn−1,1−α2
S√n > µ
)
= 1− α,
it follows that(
X − tn−1,1−α2
S√n ,X + tn−1,1−α2
S√n
)
is a 100(1− α) % CI for µ.
tn−1
α
2
α
2
DEPARTMENT OF MATHEMATICS & STATISTICS 12
R: Example interval for µ, when σ2 unknown
t.test(x, mu = 16, alternative = "two.sided")
#
# One Sample t-test
#
# data: x
# t = 1.4871, df = 24, p-value = 0.15
# alternative hypothesis: true mean is not equal to 16
# 95 percent confidence interval:
# 15.49400 19.11527
# sample estimates:
# mean of x
# 17.30463
alpha = 0.05; se = sd(x) / sqrt(n)
mean(x) + c(-1, 1) * qt(1 - alpha/2, df = n - 1) * se
# [1] 15.49400 19.11527
DEPARTMENT OF MATHEMATICS & STATISTICS 13
Interval estimation of the variance, σ2
X i.i.d.∼ N (µ, σ2) We use the fact that
(n − 1)S2
σ2
∼ χ2n−1
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
.
Since
P
(
χ2n−1,α2 <
(n − 1)S2
σ2
< χ2n−1,1−α2
)
= P
(
(n − 1)S2
χ2n−1,1−α2
< σ2,
(n − 1)S2
χ2n−1,α2
> σ2,
)
= 1− α,
it follows that(
(n − 1)S2
χ2n−1,1−α2
,
(n − 1)S2
χ2n−1,α2
)
is a 100(1− α) % CI for σ2.
χn−1,α2 χn−1,1−α2
χ2n−1
1− α
α
2
α
2
DEPARTMENT OF MATHEMATICS & STATISTICS 14
R: Example interval for σ2
alpha = 0.05; n = length(x);
quants = qchisq(c(1-alpha/2, alpha/2), df = n - 1)
quants
# [1] 39.36408 12.40115
(n - 1) * var(x)/quants
# [1] 11.73106 37.23705
DEPARTMENT OF MATHEMATICS & STATISTICS 15
Interval estimation of the standard deviation, σ
X i.i.d.∼ N (µ, σ2) We use the fact that
(n − 1)S2
σ2
∼ χ2n−1
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
.
Since
P
(
χ2n−1,α2 <
(n − 1)S2
σ2
< χ2n−1,1−α2
)
= P
(√
(n − 1)S2
χ2n−1,1−α2
< σ,
√
(n − 1)S2
χ2n−1,α2
> σ
)
= 1− α,
it follows that(√
(n − 1)S2
χ2n−1,1−α2
,
√
(n − 1)S2
χ2n−1,α2
)
is a 100(1− α) % CI for σ.
χn−1,α2 χn−1,1−α2
χ2n−1
1− α
α
2
α
2
DEPARTMENT OF MATHEMATICS & STATISTICS 16
R: Example interval for σ
alpha = 0.05; n = length(x);
quants = qchisq(c(1-alpha/2, alpha/2), df = n - 1)
quants
# [1] 39.36408 12.40115
sqrt((n - 1) * var(x)/quants)
# [1] 3.425063 6.102217
DEPARTMENT OF MATHEMATICS & STATISTICS 17
/////
Approximate Confidence
Intervals for Count data
Approximate interval estimation of the proportion p (Binomial)
I Assume X ∼ Bin (n, p) then using the normal approximation to the
Binomial we have the pivotal result,
X − np√
np(1− p)
d−→ N (0, 1).
I Since p is unknown, use the further approximation,
p̂ − p√
p̂(1−p̂)
n
d−→ N (0, 1) where p̂ = Xn
to give an asymptotic interval,
P
zα
2
<
p̂ − p√
p̂(1−p̂)
n
< z1−α2
n→∞−→ 1− α.
It follows that(
p̂ − z1−α2
√
p̂(1− p̂)
n , p̂ + z1−
α
2
√
p̂(1− p̂)
n
)
is an approximate 100(1− α) % CI for p.
DEPARTMENT OF MATHEMATICS & STATISTICS 19
R: Example approximate interval for p
set.seed(88888888)
n = 50; p = 0.82;
x = rbinom(1, size = 50, prob = p)
x
# [1] 42
alpha = 0.05; phat = x/n
se = sqrt(phat * (1 - phat)/n)
phat + c(-1, 1) * qnorm(1 - alpha/2) * se
# [1] 0.7383839 0.9416161
DEPARTMENT OF MATHEMATICS & STATISTICS 20
Approximate interval estimation of the rate λ (Poisson)
I Assume X ∼ Poisson (λ), with λ ‘large’
I Can rewrite X as a sum of i.i.d. Poisson variables with
X =
n∑
i=1
Xi Xi i.i.d∼ Poisson
(
λ
n
)
I Then using the CLT we have the normal approximation to the
Poisson and the pivotal result,
√
n
X − λn√
λ
n
= X − λ√
λ
d−→ N (0, 1) as n→∞.
I Further approximate the standard error with
√
X to get the
asymptotic result,
P
(
zα
2
<
(
X − λ√
X
)
< z1−α2
)
n→∞−→ 1− α.
It follows that (
X − z1−α2
√
X ,X + z1−α2
√
X
)
is an approximate 100(1− α) % CI for λ.
DEPARTMENT OF MATHEMATICS & STATISTICS 21
R: Example approximate interval for λ
set.seed(88888888)
lambda = 200
x = rpois(1, lambda = lambda)
x
# [1] 197
alpha = 0.05;
x + c(-1, 1) * qnorm(1 - alpha/2) * sqrt(x)
# [1] 169.4906 224.5094
DEPARTMENT OF MATHEMATICS & STATISTICS 22
/////
Exact Confidence Intervals for
Count data
Binomial CDF is monotone as a function in p
I Consider X ∼ Bin(n, p) with cumulative distribution function (cdf),
FX (x |p) = P(X ≤ x)
I For a fixed x , consider the cdf as a function of p for the Binomial
random variable.
I That is, let
cB(p) =
x∑
i=0
(
n
i
)
pi(1− p)n−i = FX (p; x)
I That is, it is a function that compares the accumulated probability
masses for Binomial variables as a function of p (i.e. changing p is
changing the Binomial distribution)
I As a function of p, the Binomial cdf is monotone decreasing
(non-increasing) in p.
I That is, if p1 ≥ p2, then cB(p1) ≤ cB(p2).
I Proof given in the Appendix
DEPARTMENT OF MATHEMATICS & STATISTICS 24
Symmetric CI for p for X ∼ Bin(n, p)
I Denote, pU to be the value of p such that
FX (pU ; x) =
x∑
i=0
(
n
i
)
piU(1− pU)n−i =
α
2
I Due to the monotone decreasing property on the previous slide, we
have, p > pU implies that FX (p; x) < FX (pU ; x) = α2
I Also, denote pL to be the value of p such that
1− FX (pL; x) =
n∑
i=x+1
(
n
i
)
piL(1− pL)n−i =
α
2
I Again due to the monotone decreasing property on the previous slide,
we have, p < pL implies that 1− FX (p; x) < 1− FX (pL; x) = α2 .
I This implies that the interval,
(pL, pU)
is a 100(1− α)% Confidence Interval for p.
DEPARTMENT OF MATHEMATICS & STATISTICS 25
R: Exact interval for p
set.seed(88888888); n = 50; p = 0.82;
x = rbinom(1, size = 50, prob = p)
x
# [1] 42
binom.test(x, n = 50)
#
# Exact binomial test
#
# data: x and 50
# number of successes = 42, number of trials = 50, p-value =
# 1.164e-06
# alternative hypothesis: true probability of success is not equal to 0.5
# 95 percent confidence interval:
# 0.7088737 0.9282992
# sample estimates:
# probability of success
# 0.84
DEPARTMENT OF MATHEMATICS & STATISTICS 26
Poisson CDF is monotone as a function in λ
I Consider X ∼ Poisson(λ) with cumulative distribution function (cdf),
FX (x |p) = P(X ≤ x)
I For a fixed x , consider the cdf as a function of λ for the Poisson
random variable.
I That is, let
cP(λ) = cP(λ; x) =
x∑
i=0
e−λλi
i! = FX (λ, x)
I This is the accumulated probability for observed x as a function of λ.
I That is, it is a function that compares the accumulated probability
masses for Poisson variables where λ changes (i.e. changing λ is
changing the Poisson distribution)
I As a function of λ, the Poisson cdf is monotone decreasing in λ.
I That is, if λ1 ≥ λ2, then cP(λ1) ≤ cP(λ2).
I Proof given in the Appendix
DEPARTMENT OF MATHEMATICS & STATISTICS 27
Symmetric CI for λ for X ∼ Possion(λ)
I Denote, λU to be the value of λ such that
FX (λ; x) =
x∑
i=0
e−λλi
i! =
α
2
I Due to the monotone decreasing property on the previous slide, we
have, λ > λU implies that FX (λ; x) < FX (λU ; x) = α2
I Also, denote λL to be the value of λ such that
1− FX (λL; x) =
n∑
i=x+1
e−λλi
i! =
α
2
I Again due to the monotone decreasing property on the previous slide,
we have, λ < λL implies that 1− FX (λ; x) < 1− FX (λL; x) = α2 .
I This implies that the interval,
(λL, λU)
is a 100(1− α)% Confidence Interval for λ.
DEPARTMENT OF MATHEMATICS & STATISTICS 28
R: Example exact interval for λ
set.seed(88888888); lambda = 200
x = rpois(1, lambda = lambda)
x
# [1] 197
poisson.test(x)
#
# Exact Poisson test
#
# data: x time base: 1
# number of events = 197, time base = 1, p-value < 2.2e-16
# alternative hypothesis: true event rate is not equal to 1
# 95 percent confidence interval:
# 170.4496 226.5137
# sample estimates:
# event rate
# 197
DEPARTMENT OF MATHEMATICS & STATISTICS 29
/////
Two sample confidence
intervals for Gaussian samples
Two sample Gaussian assumption
I For the next batch of tests denote Two sample Gaussian to be the
situation when
X =
(
X1 X2
)
I with X1 independent of X2.
I each of the variables follow the Gaussian distribution such that,
X1 =
(
X11 X12 · · · X1n1
)T and X2 = (X21 X22 · · · X2n2)T
X1i i.i.d∼ N (µ1, σ21); X2i i.i.d∼ N (µ2, σ22)
I depending on the testing situation σ2i is assumed either known or
unknown.
DEPARTMENT OF MATHEMATICS & STATISTICS 31
Estimation of two population means, µ1 and µ2, population
variances, known
I Assume the Two sample Gaussian where σ21 and σ22 are known.
I Then it follows that,
Z = X 1 − X 2 − (µ1 − µ2)√
σ21
n1 +
σ22
n2
∼ N (0, 1) .
and consequently,
P
−z1−α/2 < X 1 − X 2 − (µ1 − µ2)√
σ21
n1 +
σ22
n2
< z1−α/2
= 1− α.
Solving the system in a similar way to previous results yields,X 1 − X 2 − z1−α2
√
σ21
n1
+ σ
2
2
n2
,X 1 − X 2 + z1−α2
√
σ21
n1
+ σ
2
2
n2
is a 100(1− α)% CI for µ1 − µ2.
DEPARTMENT OF MATHEMATICS & STATISTICS 32
R: Example interval for µ1 − µ2, variances known
set.seed(88888888)
x = rnorm(n = 25, sd = 4, mean = 17)
y = rnorm(n = 31, sd = 5, mean = 20)
n = c(length(x), length(y)); sig = c(4, 5)
se = sqrt(sig[1]^2/n[1] + sig[2]^2/n[2])
mean(x) - mean(y) + c(-1, 1) * qnorm(1 - alpha/2) * se
# [1] -5.6331827 -0.9187435
DEPARTMENT OF MATHEMATICS & STATISTICS 33
Estimation of two population means, µ1 and µ2, population
variances, unknown but assumed equal
I Assume the Two sample Gaussian where σ21 and σ22 are unknown but
σ21 = σ22 .
I Then estimate σ2 with the pooled variance estimator,
S2p =
(n1 − 1)S21 + (n2 − 1)S22
n1 + n2 − 2 where S
2
i =
1
ni − 1
ni∑
j=1
(Xij − X i)2
and consequently,
X 1 − X 2 − (µ1 − µ2)
Sp
√
1
n1 +
1
n2
∼ tν where ν = n1 + n2 − 2.
Then solving the system in a similar way to the previous results yields,(
X 1 − X 2 − tνSp
√
1
n1
+ 1n2
,X 1 − X 2 + tνSp
√
1
n1
+ 1n2
)
is a 100(1− α)% CI for µ1 − µ2 (where tν = tn1+n2−2,1−α2 ).
DEPARTMENT OF MATHEMATICS & STATISTICS 34
R: Example interval for µ1 − µ2, when σ21 = σ22
sp = sqrt(((n[1] - 1) * var(x) + (n[2] - 1) * var(y))/(sum(n) - 2))
se = sp * sqrt(1/n[1] + 1/n[2])
mean(x) - mean(y)+ c(-1, 1) * qt(1 - alpha/2, df = sum(n) - 2) * se
# [1] -5.6092031 -0.9427231
t.test(x, y, var.equal = TRUE)
#
# Two Sample t-test
#
# data: x and y
# t = -2.8149, df = 54, p-value = 0.006795
# alternative hypothesis: true difference in means is not equal to 0
# 95 percent confidence interval:
# -5.6092031 -0.9427231
# sample estimates:
# mean of x mean of y
# 17.30463 20.58060
DEPARTMENT OF MATHEMATICS & STATISTICS 35
Interval estimation of the ratio of two population variances
I Assume the Two sample Gaussian where σ21 and σ22 are unknown
I We use the fact that
S21/σ21
S22/σ22
∼ Fn1−1,n2−1 where S2i =
1
ni − 1
ni∑
j=1
(Xij − X i)2
Since
P
(
Fn1−1,n2−1,α2 <
S21σ22
S22σ21
< Fn1−1,n2−1,1−α2
)
= P
(
1
Fn1−1,n2−1,1−α2
S21
S22
<
σ21
σ22
<
1
Fn1−1,n2−1,α2
S21
S22
)
= 1− α,
it follows that(
S21
S22
1
Fn1−1,n2−1,1−α2
,
S21
S22
1
Fn1−1,n2−1,α2
)
is a 100(1− α) % CI for σ21
σ22
.
Fn1−1,n2−1,α2 Fn1−1,n2−1,1−α2
Fn1−1,n2−1
1− αα2 α2
DEPARTMENT OF MATHEMATICS & STATISTICS 36
R: Example interval for σ21
σ22
quants = qf(c(alpha/2, 1-alpha/2), df1 = n[1] - 1, df2 = n[2] - 1)
quants
# [1] 0.4526985 2.1358787
var(x) / var(y)
# [1] 1.048782
var(x) / var(y) * quants
# [1] 0.4747821 2.2400711
DEPARTMENT OF MATHEMATICS & STATISTICS 37
/////
Approximate Confidence
Intervals for two sample count
data
Approximate interval estimation of two proportions (Binomial)
I Assume X1 ∼ Bin (n1, p1) and X2 ∼ Bin (n2, p2) are independent then
the normal approximation to the Binomial gives the pivotal result,
p̂i − pi√
pi (1−pi )
ni
d−→ N (0, 1) where pi = Xin .
I Since p1 and p2 are unknown, use the approximation
p̂1 − p̂2 − (p1 − p2)√
p̂1(1−p̂1)
n1 +
p̂1(1−p̂1)
n1
d−→ N (0, 1),
to give an asymptotic interval,
P
(
zα
2
<
p̂1 − p̂2 − (p1 − p2)
ŝ.e.(p̂1 − p̂2) < z1−
α
2
)
n→∞−→ 1− α,
where ŝ.e.(p̂1 − p̂2) =
√
p̂1(1−p̂1)
n1 +
p̂2(1−p̂2)
n2 . It follows that(
p̂1 − p̂2 − z1−α2 ŝ.e.(p̂1 − p̂2), p̂1 − p̂2 + z1−α2 ŝ.e.(p̂1 − p̂2)
)
is an approximate 100(1− α) % CI for p1 − p2.
DEPARTMENT OF MATHEMATICS & STATISTICS 39
R: Example approximate interval for p1 − p2
set.seed(88888888)
n1 = 50; n2 = 90; p1 = 0.82; p2 = 0.79
x1 = rbinom(1, size = n1, prob = p1)
x2 = rbinom(1, size = n2, prob = p2)
c(x1, x2)
# [1] 42 66
alpha = 0.05; phat1 = x1/n1; phat2 = x2/n2
se = sqrt(phat1 * (1 - phat1)/n1 + phat2 * (1 - phat2)/n2)
phat1 - phat2 + c(-1, 1) * qnorm(1 - alpha/2) * se
# [1] -0.02998148 0.24331481
DEPARTMENT OF MATHEMATICS & STATISTICS 40
Approximate interval estimation of two rates (Poisson)
I Assume Xi ∼ Poisson (λi) are two independent variables with λi
‘large’
I Using the CLT from the single sample case, it follows we have the
normal approximation to the Poisson difference and the pivotal result,
Xi − λi√
λi
d−→ N (0, 1) as n→∞.
I Again, estimate the standard error using Xi to give the asymptotic
result,
P
(
zα
2
<
X1 − X2 − (λ1 − λ2)√
X1 + X2
< z1−α2
)
n→∞−→ 1− α.
It follows that(
X1 − X2 − z1−α2
√
X1 + X2,X1 − X2 + z1−α2
√
X1 + X2
)
is an approximate 100(1− α) % CI for λ1 − λ2.
DEPARTMENT OF MATHEMATICS & STATISTICS 41
R: Example approximate interval for λ1 − λ2
set.seed(88888888)
lambda1 = 200; lambda2 = 300
x1 = rpois(1, lambda = lambda1); x2 = rpois(1, lambda = lambda2)
c(x1, x2)
# [1] 197 277
x1 - x2 + c(-1, 1) * qnorm(1 - alpha/2) * sqrt(x1 + x2)
# [1] -122.67144 -37.32856
DEPARTMENT OF MATHEMATICS & STATISTICS 42
/////
Confidence intervals for
means of paired numeric data
Paired data
I Previously two sample data is assumed independent within and
between samples. That is,
X1 =
(
X11 X12 · · · X1n1
)T and X2 = (X21 X22 · · · X2n2)T
I assumed to be independent between X1 and X2.
I assumed to be independent within each Xij .
I Paired data typically form in a ‘before and after’ situation (or
measuring identical twins).
I E.g. Xij is the information on measurement i from pair j.
X1j = first measurement in pair j
X2j = second measurement in pair j
I Data is dependent in index i , independent across index j.
I The data could be written in the following way
X =
(
X11 X12 · · · X1n
X21 X22 · · · X2n
)T
DEPARTMENT OF MATHEMATICS & STATISTICS 44
Analysing paired data
I Exploit the independence and account for the dependence.
I Take differences between each measurement inside the pair.
I Define Di = X1i − X2i (or X2i − X1i)
I Now the Di variables are a single sample of independent observations
I Previously used methods apply!
I Price paid to do this is some information is lost
I Reduced sample size from 2n to n
I Magnitude of individual Xij is lost, only difference Di retained.
DEPARTMENT OF MATHEMATICS & STATISTICS 45
Confidence interval for the mean of paired data.
I Assumed data is paired with
X =
(
X11 X12 · · · X1n
X21 X22 · · · X2n
)T
I Assume the differences Di = X1i − X2i form a single sample with,
Di i.i.d.∼ N (µD , σ2D) µD = µ1 − µ2
I µi = EXi· and σ2D is unknown.
I This leads to the t-distribution
√
n
(
D − µD
SD
)
∼ tn−1
I Consequently, this leads to the result that(
D − tn−1,1−α2
SD√n ,D + tn−1,1−α2
SD√n
)
is a 100(1− α) % CI for µD .
DEPARTMENT OF MATHEMATICS & STATISTICS 46
R: Example interval for mean of differences.
set.seed(88888888); alpha = 0.05
x = rnorm(n = 25, sd = 4, mean = 17)
y = rnorm(n = 25, sd = 5, mean = 20)
t.test(x, y, paired = TRUE)
#
# Paired t-test
#
# data: x and y
# t = -2.076, df = 24, p-value = 0.04877
# alternative hypothesis: true difference in means is not equal to 0
# 95 percent confidence interval:
# -5.20985713 -0.01525898
# sample estimates:
# mean of the differences
# -2.612558
d = x - y; nd = length(d)
se = sd(d)/sqrt(nd)
mean(d) + c(-1, 1) * qt(1 - alpha/2, df = nd - 1) * se
# [1] -5.20985713 -0.01525898
DEPARTMENT OF MATHEMATICS & STATISTICS 47
/////
Bootstrapping
Bootstrapping interval estimators
I In point estimation, re-sampling is conducted either in a parametric or
nonparametric way and estimates recomputed on each resample.
I This gives a sample of estimates that can be analysed.
I The same process can be applied to interval estimators.
I Still resample the data, using the same approach discussed earlier.
I Compute the estimators for each resampled dataset.
I Use the resampled estimators to compute the interval estimator
I Compute the lower bound using the resampled data.
I Compute the upper bound using the resampled data.
I Analyse the bootstrapped interval estimator.
DEPARTMENT OF MATHEMATICS & STATISTICS 49
Computing quantiles for the interval estimators
I Suppose we have a set of bootstrapped estimators.
θ̂∗ =
(
θ̂∗1 θ̂∗2 · · · θ̂∗m
)T
I These bootstrapped estimators can be used to estimate the
distribution of θ̂. The quantiles of this estimated distribution can be
inspected to construct the interval estimator.
I Example: Compute a 100(1− α)% interval for θ using the
bootstrapped estimators θ̂∗
I 100(1− α)% CI for θ given by(
F̂−1
θ̂∗
(α/2), F̂−1
θ̂∗
(1− α/2)
)
I where F−1
θ̂∗
is the estimated quantiles of the the bootstrapped
estimators.
DEPARTMENT OF MATHEMATICS & STATISTICS 50
Quantile function
I The following R code will achieve this
I Suppose we have a set of resampled data in a matrix (dataframe)
called Boot
# Suppose this is the sample mean estimate from the original dataset
mn = 4.60
# Compute the bootstrapped means from the resampled data
means = colMeans(Boot)
# Compute the sample delta 95% quantiles.
CI = quantile(means, prob = c(0.025, 0.975))
# Construct and report the interval
CI
# 2.5% 97.5%
# 3.436639 5.478028
I The exercise in the next class will give you a chance to compute this.
DEPARTMENT OF MATHEMATICS & STATISTICS 51
/////
Appendix
Proof of monotone decreasing Binomial cdf (p)
I It is sufficient to prove that the partial derivative in p is always
negative,
FX (p; x) = P(X ≤ x) =
x∑
i=0
(
n
x
)
pi(1− p)n−x
∂FX (p; x)
∂p =
x∑
i=0
(
n
x
)
∂
∂p p
i(1− p)n−x
=
x∑
i=0
(
n
x
)(
ipi−1(1− p)n−i − (n − i)pi(1− p)n−i−1)
=
x∑
i=0
(
n
x
)(
ipi−1(1− p)n−i − (n − i)pi(1− p)n−i−1)
=
x∑
i=0
(
n
x
)
pi(1− p)n−i
(
i
p −
(n − i)
1− p
)
= 1p(1− p)
x∑
i=0
P(X = i) (i − np)
DEPARTMENT OF MATHEMATICS & STATISTICS 53
I If x ≤ np, then clearly ∂FX (p;x)∂p < 0.I On the other hand, if x > np, re-express the partial derivative as
follows.
∂FX (p; x)
∂p =
1
p(1− p)
x∑
i=0
P(X = i) (i − np)
= 1p(1− p)
( x∑
i=0
iP(X = i)− npFX (x)
)
= 1p(1− p)
(
np(1− FX (x))−
n∑
i=x+1
iP(X = i)
)
= 1p(1− p)
n∑
i=x+1
P(X = i) (np − i)
I Then using the above representation, it is clear that when x > np,
∂FX (p;x)
∂p < 0.
I Therefore, ∂FX (p;x)∂p < 0 for all fixed x ∈ {1, 2, . . . , n} and the result is
proven.
DEPARTMENT OF MATHEMATICS & STATISTICS 54
Proof of monotone decreasing Poisson cdf (λ)
I Again, prove that the partial derivative in λ is always negative,
FX (λ; x) =
x∑
i=0
e−λλx
x !
∂FX (λ; x)
∂λ
= e−λ ∂
∂λ
x∑
i=0
e−λλi
i! − e
−λ
x∑
i=0
e−λλi
i!
= e−λ
{ x∑
i=1
λi−1
(i − 1)! −
x∑
i=0
λi
i!
}
= e−λ
{x−1∑
j=0
λj
j! −
x∑
i=0
λi
i!
}
= −e
−λλx
x ! = −P(X = x) < 0 for all x ∈ Z
+
DEPARTMENT OF MATHEMATICS & STATISTICS 55
学霸联盟
STAT7310 and STAT8310: Statistical Theory
Confidence Intervals
Semester 1, 2021
/////
Interval Estimation
Interval Estimation
I Let X =
(
X1 X2 · · · Xn
)T
, where X1,X2, . . . ,Xn are i.i.d. with
common pdf or pf fX (x ; θ) and θ ∈ Θ
I Form two statistics
L = l (X) and U = u (X) .
I An interval (L,U) is called a 100 (1− α) % confidence interval for θ if
P (l (X) < θ, u (X) > θ) = 1− α;
or
P (θ ∈ [l (X) ,U (X)]) = 1− α.
I Note that in the above, we are not saying that the probability that
θ < u (X) and θ > l (X) is 1− α. That is, 1− α is not the
probability that θ is in some interval. It is the probability that the
random interval (L,U) contains θ.
I L and U are called the lower and upper confidence limits.
DEPARTMENT OF MATHEMATICS & STATISTICS 3
Symmetric Confidence Intervals
I Suppose we choose functions l and u to satisfy
P (l (X) < θ) = P (u (X) > θ) = α2 .
I Then (L,U) is called a symmetric confidence interval (CI).
I Symmetric C.I.s are not always the shortest. It is generally the
shortest interval that is the one we want.
DEPARTMENT OF MATHEMATICS & STATISTICS 4
Pivotal Quantities
I These are useful in the devising of confidence intervals.
I Let
Q = q (X ; θ)
be a function of the random vector X and the parameter of interest θ
I We say that Q is a pivotal quantity if the distribution of Q does not
depend on θ
I Then, a 100 (1− α) % CI for θ is comprised of the set of θ values
which satisfy
q1 < q (X ; θ) < q2
where
P (Q < q1) + P (Q > q2) = α
DEPARTMENT OF MATHEMATICS & STATISTICS 5
Pivotal Quantities : Examples
1. X ∼ N (θ, σ2); σ2 known
Q =
√n (X − θ)
σ
is pivotal since Q ∼ N (0, 1) for all θ
2. X ∼ N (θ, σ2); σ2 unknown
Q =
√n (X − θ)
S
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
is pivotal since Q ∼ tn−1 , for all θ
DEPARTMENT OF MATHEMATICS & STATISTICS 6
3. X ∼ Γ (α, θ); α known
f (x ; θ) =
{ 1
θkΓ(α)xα−1e−x/θ ; x > 0
0 ; otherwise.
Then Q = Xθ is pivotal since
n∑
i=1
Xi ∼ Γ (nα, θ) .
Thus n∑
i=1
Xi
θ
∼ Γ (nα, 1)
and the distribution of this does not depend on θ
DEPARTMENT OF MATHEMATICS & STATISTICS 7
/////
Confidence Intervals for
Gaussian samples
Core assumption
I Typically for single sample (continuous) numeric data we assume that
X =
(
X1 X2 · · · Xn
)T with,
I Xi are independent of each other.
I Xi follow a normal (Gaussian) distribution.
I Each Xi has EXi = µ and var(Xi) = σ2, fixed.
I This is given the shorthand, Xi i.i.d.∼ N (µ, σ2).
I This is required for exact tests, however, for the results that are
analysing the mean, X can use the CLT,
√
n
(
X − µ
σ
)
d−→ N (0, 1)
I So approximate tests can be derived if the sample size is ‘large’
enough.
DEPARTMENT OF MATHEMATICS & STATISTICS 9
Estimation of the mean, µ, when the variance, σ2, known
Xi i.i.d.∼ N
(
µ, σ2
)
; σ2 known. We know
Z = X − µ
σ/
√n =
√n (X − µ)
σ
∼ N (0, 1) .
Thus
P
(
−z1−α/2 <
√
nX − µ
σ
< z1−α/2
)
= P
(
X − z1−α/2 σ√n < µ,X + z1−α/2
σ√n > µ
)
= 1− α
Hence
L = X − z1−α/2 σ√n
U = X + z1−α/2
σ√n
and the 100(1− α)% CI for µ is (L,U) .
Z ∼ N (0, 1)
0.05
2
0.05
2
DEPARTMENT OF MATHEMATICS & STATISTICS 10
R: Example interval for µ, when σ2 known
set.seed(88888888)
n = 25; sig = 4; mu = 17; mu1 = 16
x = rnorm(n) * sig + mu
x
# [1] 16.318648 13.619067 19.976965 22.040565 18.208809 13.760960 15.103078
# [8] 16.829801 16.054201 16.751592 5.824282 18.767051 11.292923 12.635395
# [15] 17.609836 17.225824 18.500564 26.304074 26.479513 19.521398 14.033471
# [22] 20.222837 15.955225 19.311570 20.268188
alpha = 0.05
mean(x) + c(-1, 1) * qnorm(1 - alpha/2) * sig / sqrt(n)
# [1] 15.73666 18.87260
DEPARTMENT OF MATHEMATICS & STATISTICS 11
Estimation of the mean, µ, when the variance, σ2, unknown
X i.i.d.∼ N (µ, σ2); σ2 unknown. We use the
fact that
T = X − µS/√n =
√n (X − µ)
S ∼ tn−1
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
.
Since
P
(
tn−1,α2 <
X − µ
S/√n < tn−1,1−α2
)
= P
(
X − tn−1,1−α2
S√n < µ,X + tn−1,1−α2
S√n > µ
)
= 1− α,
it follows that(
X − tn−1,1−α2
S√n ,X + tn−1,1−α2
S√n
)
is a 100(1− α) % CI for µ.
tn−1
α
2
α
2
DEPARTMENT OF MATHEMATICS & STATISTICS 12
R: Example interval for µ, when σ2 unknown
t.test(x, mu = 16, alternative = "two.sided")
#
# One Sample t-test
#
# data: x
# t = 1.4871, df = 24, p-value = 0.15
# alternative hypothesis: true mean is not equal to 16
# 95 percent confidence interval:
# 15.49400 19.11527
# sample estimates:
# mean of x
# 17.30463
alpha = 0.05; se = sd(x) / sqrt(n)
mean(x) + c(-1, 1) * qt(1 - alpha/2, df = n - 1) * se
# [1] 15.49400 19.11527
DEPARTMENT OF MATHEMATICS & STATISTICS 13
Interval estimation of the variance, σ2
X i.i.d.∼ N (µ, σ2) We use the fact that
(n − 1)S2
σ2
∼ χ2n−1
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
.
Since
P
(
χ2n−1,α2 <
(n − 1)S2
σ2
< χ2n−1,1−α2
)
= P
(
(n − 1)S2
χ2n−1,1−α2
< σ2,
(n − 1)S2
χ2n−1,α2
> σ2,
)
= 1− α,
it follows that(
(n − 1)S2
χ2n−1,1−α2
,
(n − 1)S2
χ2n−1,α2
)
is a 100(1− α) % CI for σ2.
χn−1,α2 χn−1,1−α2
χ2n−1
1− α
α
2
α
2
DEPARTMENT OF MATHEMATICS & STATISTICS 14
R: Example interval for σ2
alpha = 0.05; n = length(x);
quants = qchisq(c(1-alpha/2, alpha/2), df = n - 1)
quants
# [1] 39.36408 12.40115
(n - 1) * var(x)/quants
# [1] 11.73106 37.23705
DEPARTMENT OF MATHEMATICS & STATISTICS 15
Interval estimation of the standard deviation, σ
X i.i.d.∼ N (µ, σ2) We use the fact that
(n − 1)S2
σ2
∼ χ2n−1
where
S2 = 1n − 1
n∑
i=1
(
Xi − X
)2
.
Since
P
(
χ2n−1,α2 <
(n − 1)S2
σ2
< χ2n−1,1−α2
)
= P
(√
(n − 1)S2
χ2n−1,1−α2
< σ,
√
(n − 1)S2
χ2n−1,α2
> σ
)
= 1− α,
it follows that(√
(n − 1)S2
χ2n−1,1−α2
,
√
(n − 1)S2
χ2n−1,α2
)
is a 100(1− α) % CI for σ.
χn−1,α2 χn−1,1−α2
χ2n−1
1− α
α
2
α
2
DEPARTMENT OF MATHEMATICS & STATISTICS 16
R: Example interval for σ
alpha = 0.05; n = length(x);
quants = qchisq(c(1-alpha/2, alpha/2), df = n - 1)
quants
# [1] 39.36408 12.40115
sqrt((n - 1) * var(x)/quants)
# [1] 3.425063 6.102217
DEPARTMENT OF MATHEMATICS & STATISTICS 17
/////
Approximate Confidence
Intervals for Count data
Approximate interval estimation of the proportion p (Binomial)
I Assume X ∼ Bin (n, p) then using the normal approximation to the
Binomial we have the pivotal result,
X − np√
np(1− p)
d−→ N (0, 1).
I Since p is unknown, use the further approximation,
p̂ − p√
p̂(1−p̂)
n
d−→ N (0, 1) where p̂ = Xn
to give an asymptotic interval,
P
zα
2
<
p̂ − p√
p̂(1−p̂)
n
< z1−α2
n→∞−→ 1− α.
It follows that(
p̂ − z1−α2
√
p̂(1− p̂)
n , p̂ + z1−
α
2
√
p̂(1− p̂)
n
)
is an approximate 100(1− α) % CI for p.
DEPARTMENT OF MATHEMATICS & STATISTICS 19
R: Example approximate interval for p
set.seed(88888888)
n = 50; p = 0.82;
x = rbinom(1, size = 50, prob = p)
x
# [1] 42
alpha = 0.05; phat = x/n
se = sqrt(phat * (1 - phat)/n)
phat + c(-1, 1) * qnorm(1 - alpha/2) * se
# [1] 0.7383839 0.9416161
DEPARTMENT OF MATHEMATICS & STATISTICS 20
Approximate interval estimation of the rate λ (Poisson)
I Assume X ∼ Poisson (λ), with λ ‘large’
I Can rewrite X as a sum of i.i.d. Poisson variables with
X =
n∑
i=1
Xi Xi i.i.d∼ Poisson
(
λ
n
)
I Then using the CLT we have the normal approximation to the
Poisson and the pivotal result,
√
n
X − λn√
λ
n
= X − λ√
λ
d−→ N (0, 1) as n→∞.
I Further approximate the standard error with
√
X to get the
asymptotic result,
P
(
zα
2
<
(
X − λ√
X
)
< z1−α2
)
n→∞−→ 1− α.
It follows that (
X − z1−α2
√
X ,X + z1−α2
√
X
)
is an approximate 100(1− α) % CI for λ.
DEPARTMENT OF MATHEMATICS & STATISTICS 21
R: Example approximate interval for λ
set.seed(88888888)
lambda = 200
x = rpois(1, lambda = lambda)
x
# [1] 197
alpha = 0.05;
x + c(-1, 1) * qnorm(1 - alpha/2) * sqrt(x)
# [1] 169.4906 224.5094
DEPARTMENT OF MATHEMATICS & STATISTICS 22
/////
Exact Confidence Intervals for
Count data
Binomial CDF is monotone as a function in p
I Consider X ∼ Bin(n, p) with cumulative distribution function (cdf),
FX (x |p) = P(X ≤ x)
I For a fixed x , consider the cdf as a function of p for the Binomial
random variable.
I That is, let
cB(p) =
x∑
i=0
(
n
i
)
pi(1− p)n−i = FX (p; x)
I That is, it is a function that compares the accumulated probability
masses for Binomial variables as a function of p (i.e. changing p is
changing the Binomial distribution)
I As a function of p, the Binomial cdf is monotone decreasing
(non-increasing) in p.
I That is, if p1 ≥ p2, then cB(p1) ≤ cB(p2).
I Proof given in the Appendix
DEPARTMENT OF MATHEMATICS & STATISTICS 24
Symmetric CI for p for X ∼ Bin(n, p)
I Denote, pU to be the value of p such that
FX (pU ; x) =
x∑
i=0
(
n
i
)
piU(1− pU)n−i =
α
2
I Due to the monotone decreasing property on the previous slide, we
have, p > pU implies that FX (p; x) < FX (pU ; x) = α2
I Also, denote pL to be the value of p such that
1− FX (pL; x) =
n∑
i=x+1
(
n
i
)
piL(1− pL)n−i =
α
2
I Again due to the monotone decreasing property on the previous slide,
we have, p < pL implies that 1− FX (p; x) < 1− FX (pL; x) = α2 .
I This implies that the interval,
(pL, pU)
is a 100(1− α)% Confidence Interval for p.
DEPARTMENT OF MATHEMATICS & STATISTICS 25
R: Exact interval for p
set.seed(88888888); n = 50; p = 0.82;
x = rbinom(1, size = 50, prob = p)
x
# [1] 42
binom.test(x, n = 50)
#
# Exact binomial test
#
# data: x and 50
# number of successes = 42, number of trials = 50, p-value =
# 1.164e-06
# alternative hypothesis: true probability of success is not equal to 0.5
# 95 percent confidence interval:
# 0.7088737 0.9282992
# sample estimates:
# probability of success
# 0.84
DEPARTMENT OF MATHEMATICS & STATISTICS 26
Poisson CDF is monotone as a function in λ
I Consider X ∼ Poisson(λ) with cumulative distribution function (cdf),
FX (x |p) = P(X ≤ x)
I For a fixed x , consider the cdf as a function of λ for the Poisson
random variable.
I That is, let
cP(λ) = cP(λ; x) =
x∑
i=0
e−λλi
i! = FX (λ, x)
I This is the accumulated probability for observed x as a function of λ.
I That is, it is a function that compares the accumulated probability
masses for Poisson variables where λ changes (i.e. changing λ is
changing the Poisson distribution)
I As a function of λ, the Poisson cdf is monotone decreasing in λ.
I That is, if λ1 ≥ λ2, then cP(λ1) ≤ cP(λ2).
I Proof given in the Appendix
DEPARTMENT OF MATHEMATICS & STATISTICS 27
Symmetric CI for λ for X ∼ Possion(λ)
I Denote, λU to be the value of λ such that
FX (λ; x) =
x∑
i=0
e−λλi
i! =
α
2
I Due to the monotone decreasing property on the previous slide, we
have, λ > λU implies that FX (λ; x) < FX (λU ; x) = α2
I Also, denote λL to be the value of λ such that
1− FX (λL; x) =
n∑
i=x+1
e−λλi
i! =
α
2
I Again due to the monotone decreasing property on the previous slide,
we have, λ < λL implies that 1− FX (λ; x) < 1− FX (λL; x) = α2 .
I This implies that the interval,
(λL, λU)
is a 100(1− α)% Confidence Interval for λ.
DEPARTMENT OF MATHEMATICS & STATISTICS 28
R: Example exact interval for λ
set.seed(88888888); lambda = 200
x = rpois(1, lambda = lambda)
x
# [1] 197
poisson.test(x)
#
# Exact Poisson test
#
# data: x time base: 1
# number of events = 197, time base = 1, p-value < 2.2e-16
# alternative hypothesis: true event rate is not equal to 1
# 95 percent confidence interval:
# 170.4496 226.5137
# sample estimates:
# event rate
# 197
DEPARTMENT OF MATHEMATICS & STATISTICS 29
/////
Two sample confidence
intervals for Gaussian samples
Two sample Gaussian assumption
I For the next batch of tests denote Two sample Gaussian to be the
situation when
X =
(
X1 X2
)
I with X1 independent of X2.
I each of the variables follow the Gaussian distribution such that,
X1 =
(
X11 X12 · · · X1n1
)T and X2 = (X21 X22 · · · X2n2)T
X1i i.i.d∼ N (µ1, σ21); X2i i.i.d∼ N (µ2, σ22)
I depending on the testing situation σ2i is assumed either known or
unknown.
DEPARTMENT OF MATHEMATICS & STATISTICS 31
Estimation of two population means, µ1 and µ2, population
variances, known
I Assume the Two sample Gaussian where σ21 and σ22 are known.
I Then it follows that,
Z = X 1 − X 2 − (µ1 − µ2)√
σ21
n1 +
σ22
n2
∼ N (0, 1) .
and consequently,
P
−z1−α/2 < X 1 − X 2 − (µ1 − µ2)√
σ21
n1 +
σ22
n2
< z1−α/2
= 1− α.
Solving the system in a similar way to previous results yields,X 1 − X 2 − z1−α2
√
σ21
n1
+ σ
2
2
n2
,X 1 − X 2 + z1−α2
√
σ21
n1
+ σ
2
2
n2
is a 100(1− α)% CI for µ1 − µ2.
DEPARTMENT OF MATHEMATICS & STATISTICS 32
R: Example interval for µ1 − µ2, variances known
set.seed(88888888)
x = rnorm(n = 25, sd = 4, mean = 17)
y = rnorm(n = 31, sd = 5, mean = 20)
n = c(length(x), length(y)); sig = c(4, 5)
se = sqrt(sig[1]^2/n[1] + sig[2]^2/n[2])
mean(x) - mean(y) + c(-1, 1) * qnorm(1 - alpha/2) * se
# [1] -5.6331827 -0.9187435
DEPARTMENT OF MATHEMATICS & STATISTICS 33
Estimation of two population means, µ1 and µ2, population
variances, unknown but assumed equal
I Assume the Two sample Gaussian where σ21 and σ22 are unknown but
σ21 = σ22 .
I Then estimate σ2 with the pooled variance estimator,
S2p =
(n1 − 1)S21 + (n2 − 1)S22
n1 + n2 − 2 where S
2
i =
1
ni − 1
ni∑
j=1
(Xij − X i)2
and consequently,
X 1 − X 2 − (µ1 − µ2)
Sp
√
1
n1 +
1
n2
∼ tν where ν = n1 + n2 − 2.
Then solving the system in a similar way to the previous results yields,(
X 1 − X 2 − tνSp
√
1
n1
+ 1n2
,X 1 − X 2 + tνSp
√
1
n1
+ 1n2
)
is a 100(1− α)% CI for µ1 − µ2 (where tν = tn1+n2−2,1−α2 ).
DEPARTMENT OF MATHEMATICS & STATISTICS 34
R: Example interval for µ1 − µ2, when σ21 = σ22
sp = sqrt(((n[1] - 1) * var(x) + (n[2] - 1) * var(y))/(sum(n) - 2))
se = sp * sqrt(1/n[1] + 1/n[2])
mean(x) - mean(y)+ c(-1, 1) * qt(1 - alpha/2, df = sum(n) - 2) * se
# [1] -5.6092031 -0.9427231
t.test(x, y, var.equal = TRUE)
#
# Two Sample t-test
#
# data: x and y
# t = -2.8149, df = 54, p-value = 0.006795
# alternative hypothesis: true difference in means is not equal to 0
# 95 percent confidence interval:
# -5.6092031 -0.9427231
# sample estimates:
# mean of x mean of y
# 17.30463 20.58060
DEPARTMENT OF MATHEMATICS & STATISTICS 35
Interval estimation of the ratio of two population variances
I Assume the Two sample Gaussian where σ21 and σ22 are unknown
I We use the fact that
S21/σ21
S22/σ22
∼ Fn1−1,n2−1 where S2i =
1
ni − 1
ni∑
j=1
(Xij − X i)2
Since
P
(
Fn1−1,n2−1,α2 <
S21σ22
S22σ21
< Fn1−1,n2−1,1−α2
)
= P
(
1
Fn1−1,n2−1,1−α2
S21
S22
<
σ21
σ22
<
1
Fn1−1,n2−1,α2
S21
S22
)
= 1− α,
it follows that(
S21
S22
1
Fn1−1,n2−1,1−α2
,
S21
S22
1
Fn1−1,n2−1,α2
)
is a 100(1− α) % CI for σ21
σ22
.
Fn1−1,n2−1,α2 Fn1−1,n2−1,1−α2
Fn1−1,n2−1
1− αα2 α2
DEPARTMENT OF MATHEMATICS & STATISTICS 36
R: Example interval for σ21
σ22
quants = qf(c(alpha/2, 1-alpha/2), df1 = n[1] - 1, df2 = n[2] - 1)
quants
# [1] 0.4526985 2.1358787
var(x) / var(y)
# [1] 1.048782
var(x) / var(y) * quants
# [1] 0.4747821 2.2400711
DEPARTMENT OF MATHEMATICS & STATISTICS 37
/////
Approximate Confidence
Intervals for two sample count
data
Approximate interval estimation of two proportions (Binomial)
I Assume X1 ∼ Bin (n1, p1) and X2 ∼ Bin (n2, p2) are independent then
the normal approximation to the Binomial gives the pivotal result,
p̂i − pi√
pi (1−pi )
ni
d−→ N (0, 1) where pi = Xin .
I Since p1 and p2 are unknown, use the approximation
p̂1 − p̂2 − (p1 − p2)√
p̂1(1−p̂1)
n1 +
p̂1(1−p̂1)
n1
d−→ N (0, 1),
to give an asymptotic interval,
P
(
zα
2
<
p̂1 − p̂2 − (p1 − p2)
ŝ.e.(p̂1 − p̂2) < z1−
α
2
)
n→∞−→ 1− α,
where ŝ.e.(p̂1 − p̂2) =
√
p̂1(1−p̂1)
n1 +
p̂2(1−p̂2)
n2 . It follows that(
p̂1 − p̂2 − z1−α2 ŝ.e.(p̂1 − p̂2), p̂1 − p̂2 + z1−α2 ŝ.e.(p̂1 − p̂2)
)
is an approximate 100(1− α) % CI for p1 − p2.
DEPARTMENT OF MATHEMATICS & STATISTICS 39
R: Example approximate interval for p1 − p2
set.seed(88888888)
n1 = 50; n2 = 90; p1 = 0.82; p2 = 0.79
x1 = rbinom(1, size = n1, prob = p1)
x2 = rbinom(1, size = n2, prob = p2)
c(x1, x2)
# [1] 42 66
alpha = 0.05; phat1 = x1/n1; phat2 = x2/n2
se = sqrt(phat1 * (1 - phat1)/n1 + phat2 * (1 - phat2)/n2)
phat1 - phat2 + c(-1, 1) * qnorm(1 - alpha/2) * se
# [1] -0.02998148 0.24331481
DEPARTMENT OF MATHEMATICS & STATISTICS 40
Approximate interval estimation of two rates (Poisson)
I Assume Xi ∼ Poisson (λi) are two independent variables with λi
‘large’
I Using the CLT from the single sample case, it follows we have the
normal approximation to the Poisson difference and the pivotal result,
Xi − λi√
λi
d−→ N (0, 1) as n→∞.
I Again, estimate the standard error using Xi to give the asymptotic
result,
P
(
zα
2
<
X1 − X2 − (λ1 − λ2)√
X1 + X2
< z1−α2
)
n→∞−→ 1− α.
It follows that(
X1 − X2 − z1−α2
√
X1 + X2,X1 − X2 + z1−α2
√
X1 + X2
)
is an approximate 100(1− α) % CI for λ1 − λ2.
DEPARTMENT OF MATHEMATICS & STATISTICS 41
R: Example approximate interval for λ1 − λ2
set.seed(88888888)
lambda1 = 200; lambda2 = 300
x1 = rpois(1, lambda = lambda1); x2 = rpois(1, lambda = lambda2)
c(x1, x2)
# [1] 197 277
x1 - x2 + c(-1, 1) * qnorm(1 - alpha/2) * sqrt(x1 + x2)
# [1] -122.67144 -37.32856
DEPARTMENT OF MATHEMATICS & STATISTICS 42
/////
Confidence intervals for
means of paired numeric data
Paired data
I Previously two sample data is assumed independent within and
between samples. That is,
X1 =
(
X11 X12 · · · X1n1
)T and X2 = (X21 X22 · · · X2n2)T
I assumed to be independent between X1 and X2.
I assumed to be independent within each Xij .
I Paired data typically form in a ‘before and after’ situation (or
measuring identical twins).
I E.g. Xij is the information on measurement i from pair j.
X1j = first measurement in pair j
X2j = second measurement in pair j
I Data is dependent in index i , independent across index j.
I The data could be written in the following way
X =
(
X11 X12 · · · X1n
X21 X22 · · · X2n
)T
DEPARTMENT OF MATHEMATICS & STATISTICS 44
Analysing paired data
I Exploit the independence and account for the dependence.
I Take differences between each measurement inside the pair.
I Define Di = X1i − X2i (or X2i − X1i)
I Now the Di variables are a single sample of independent observations
I Previously used methods apply!
I Price paid to do this is some information is lost
I Reduced sample size from 2n to n
I Magnitude of individual Xij is lost, only difference Di retained.
DEPARTMENT OF MATHEMATICS & STATISTICS 45
Confidence interval for the mean of paired data.
I Assumed data is paired with
X =
(
X11 X12 · · · X1n
X21 X22 · · · X2n
)T
I Assume the differences Di = X1i − X2i form a single sample with,
Di i.i.d.∼ N (µD , σ2D) µD = µ1 − µ2
I µi = EXi· and σ2D is unknown.
I This leads to the t-distribution
√
n
(
D − µD
SD
)
∼ tn−1
I Consequently, this leads to the result that(
D − tn−1,1−α2
SD√n ,D + tn−1,1−α2
SD√n
)
is a 100(1− α) % CI for µD .
DEPARTMENT OF MATHEMATICS & STATISTICS 46
R: Example interval for mean of differences.
set.seed(88888888); alpha = 0.05
x = rnorm(n = 25, sd = 4, mean = 17)
y = rnorm(n = 25, sd = 5, mean = 20)
t.test(x, y, paired = TRUE)
#
# Paired t-test
#
# data: x and y
# t = -2.076, df = 24, p-value = 0.04877
# alternative hypothesis: true difference in means is not equal to 0
# 95 percent confidence interval:
# -5.20985713 -0.01525898
# sample estimates:
# mean of the differences
# -2.612558
d = x - y; nd = length(d)
se = sd(d)/sqrt(nd)
mean(d) + c(-1, 1) * qt(1 - alpha/2, df = nd - 1) * se
# [1] -5.20985713 -0.01525898
DEPARTMENT OF MATHEMATICS & STATISTICS 47
/////
Bootstrapping
Bootstrapping interval estimators
I In point estimation, re-sampling is conducted either in a parametric or
nonparametric way and estimates recomputed on each resample.
I This gives a sample of estimates that can be analysed.
I The same process can be applied to interval estimators.
I Still resample the data, using the same approach discussed earlier.
I Compute the estimators for each resampled dataset.
I Use the resampled estimators to compute the interval estimator
I Compute the lower bound using the resampled data.
I Compute the upper bound using the resampled data.
I Analyse the bootstrapped interval estimator.
DEPARTMENT OF MATHEMATICS & STATISTICS 49
Computing quantiles for the interval estimators
I Suppose we have a set of bootstrapped estimators.
θ̂∗ =
(
θ̂∗1 θ̂∗2 · · · θ̂∗m
)T
I These bootstrapped estimators can be used to estimate the
distribution of θ̂. The quantiles of this estimated distribution can be
inspected to construct the interval estimator.
I Example: Compute a 100(1− α)% interval for θ using the
bootstrapped estimators θ̂∗
I 100(1− α)% CI for θ given by(
F̂−1
θ̂∗
(α/2), F̂−1
θ̂∗
(1− α/2)
)
I where F−1
θ̂∗
is the estimated quantiles of the the bootstrapped
estimators.
DEPARTMENT OF MATHEMATICS & STATISTICS 50
Quantile function
I The following R code will achieve this
I Suppose we have a set of resampled data in a matrix (dataframe)
called Boot
# Suppose this is the sample mean estimate from the original dataset
mn = 4.60
# Compute the bootstrapped means from the resampled data
means = colMeans(Boot)
# Compute the sample delta 95% quantiles.
CI = quantile(means, prob = c(0.025, 0.975))
# Construct and report the interval
CI
# 2.5% 97.5%
# 3.436639 5.478028
I The exercise in the next class will give you a chance to compute this.
DEPARTMENT OF MATHEMATICS & STATISTICS 51
/////
Appendix
Proof of monotone decreasing Binomial cdf (p)
I It is sufficient to prove that the partial derivative in p is always
negative,
FX (p; x) = P(X ≤ x) =
x∑
i=0
(
n
x
)
pi(1− p)n−x
∂FX (p; x)
∂p =
x∑
i=0
(
n
x
)
∂
∂p p
i(1− p)n−x
=
x∑
i=0
(
n
x
)(
ipi−1(1− p)n−i − (n − i)pi(1− p)n−i−1)
=
x∑
i=0
(
n
x
)(
ipi−1(1− p)n−i − (n − i)pi(1− p)n−i−1)
=
x∑
i=0
(
n
x
)
pi(1− p)n−i
(
i
p −
(n − i)
1− p
)
= 1p(1− p)
x∑
i=0
P(X = i) (i − np)
DEPARTMENT OF MATHEMATICS & STATISTICS 53
I If x ≤ np, then clearly ∂FX (p;x)∂p < 0.I On the other hand, if x > np, re-express the partial derivative as
follows.
∂FX (p; x)
∂p =
1
p(1− p)
x∑
i=0
P(X = i) (i − np)
= 1p(1− p)
( x∑
i=0
iP(X = i)− npFX (x)
)
= 1p(1− p)
(
np(1− FX (x))−
n∑
i=x+1
iP(X = i)
)
= 1p(1− p)
n∑
i=x+1
P(X = i) (np − i)
I Then using the above representation, it is clear that when x > np,
∂FX (p;x)
∂p < 0.
I Therefore, ∂FX (p;x)∂p < 0 for all fixed x ∈ {1, 2, . . . , n} and the result is
proven.
DEPARTMENT OF MATHEMATICS & STATISTICS 54
Proof of monotone decreasing Poisson cdf (λ)
I Again, prove that the partial derivative in λ is always negative,
FX (λ; x) =
x∑
i=0
e−λλx
x !
∂FX (λ; x)
∂λ
= e−λ ∂
∂λ
x∑
i=0
e−λλi
i! − e
−λ
x∑
i=0
e−λλi
i!
= e−λ
{ x∑
i=1
λi−1
(i − 1)! −
x∑
i=0
λi
i!
}
= e−λ
{x−1∑
j=0
λj
j! −
x∑
i=0
λi
i!
}
= −e
−λλx
x ! = −P(X = x) < 0 for all x ∈ Z
+
DEPARTMENT OF MATHEMATICS & STATISTICS 55
学霸联盟
