Camila de Souza
Winter 2023
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 1 / 7
Use of deviance to measure goodness of fit
We can measure how well our proposed model fits the data by
comparing it to the saturated (or full) model.
A saturated model is achievable by adding sufficiently many parameters,
in most cases as many parameters as number of data points.
This comparison (saturated versus proposed) can be done by
calculating the so-called deviance, that is, the difference between the
log likelihood of the saturated model (Sat) and the log likelihood of
our proposed model (M):
DM = 2(log LSat − log LM)
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 2 / 7
Use of deviance to measure goodness of fit
Gaussian linear regression: DM = RSS.
However, we cannot use RSS as a test statistic for goodness of fit
because of the variance (dispersion) parameter.
Instead, we use can R2 = 1− RSS/TSS
Binary logistic regression: DM = −2 log LM , also cannot be used as a
test statistic for goodness fit.
For logistic regression we can use the Hosmer-Lemeshow (HL) test.
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 3 / 7
Use of deviance to measure goodness of fit
In Poisson or Binomial regression we can use use DM as a test
statistic to assess the goodness of fit of the propose model.
H0 : no lack of fit versus H1: lack of fit
Under H0, DM ∼ approx. χ2n−p, where n is the number of observations
and p the number of parameters in the model.
So, we reject “H0 : no lack of fit” if P(DM > Dobs) is sufficiently
small, say smaller than 0.05.
Note: For Poisson and Binomial regression we can also use the Pearson χ2
statistic instead of DM to conduct the goodness of fit test.
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 4 / 7
Use of deviance to measure goodness of fit
Quasi-Binomial or Quasi-Poisson models:
log L is approximated by a function Q that allows for extra variation
Qsat = 0
We cannot use then the deviance based on Q to assess goodness of fit
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 5 / 7
Use of deviance to compare two nested models
For logistic regression consider the likelihood ratio test statistic (LRT):
LRT = 2 log LLLS
= DS − DL ∼ approx. χ2l−s
H0: smaller model is correct
→ H0 : θ ∈ Θ0 vs. H1 : θ ∈ Θ, where Θ0 ⊂ Θ
We reject the null hypothesis (that the simpler model is consistent with
the data), and therefore selet the larger model over the smaller one, if
the LRTobs exceeds the 95% quantile of the reference (null)
distribution (P(LRT > LRTobs) < 0.05).
If the LRTobs lies below this quantile then the null hypothesis is not
rejected, and the smaller model is selected in favour of the larger one.
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 6 / 7
Use of deviance to compare two nested models
For Poisson and Binomial regression:
→ If there is no under or overdispersion: DS − DL ∼ approx. χ2l−s
→ If there is under or overdispersion:
(DS − DL)
(dfS − dfL)
1
σˆ2
∼ approx. F(dfS−dfL),dfL
where (dfS − dfL) = l − s is the difference in number of parameters between
the large and small models.
Note: for quasi-likelihood methods we must use F -tests to compare models
because there is also the dispersion parameter
Camila de Souza SS 3860B/9055B - Deviance Winter 2023 7 / 7