STA304 Chapter 4: Simple Random Sampling (SRS) Dr. Luai Al Labadi Fall 2021 Dr. Luai Al Labadi STA304 Fall 2021 1 / 63 Outline 1 Definition 2 How to Draw a Simple Random Sample 3 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of V (y) Confidence Interval for µ Bound on the Error B Selecting the Sample Size n for Estimating Population Mean µ 4 Estimation of the Population Total τ Estimation of the Population Total τ Selecting the Sample Size n for Estimating τ 5 Estimation of the Population Proportion p Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion 6 Advantages and Disadvantages of SRS 7 Comparing Estimates Dr. Luai Al Labadi STA304 Fall 2021 2 / 63 Definition Simple Random Sampling (SRS) Definition (Simple Random Sampling (SRS)) If a sample of size n is drawn from a population of size N such that every possible sample of size n has the same chance of being selected, the sampling procedure is called simple random sampling. The sample thus obtained is called a simple random sample. • Simple random sampling is often written as SRS. • SRS is the simplest type of sample survey design. Dr. Luai Al Labadi STA304 Fall 2021 3 / 63 Definition Example A population contains {A,B,C,D}. The goal is to draw a SRS of size 2. 1 List all possible samples. 2 Find the probability of drawing {B,C}. Dr. Luai Al Labadi STA304 Fall 2021 4 / 63 How to Draw a Simple Random Sample How to Draw a Simple Random Sample • Suppose that the population of size N has values u1, u2, . . . , uN • There are (Nn) distinct samples of size n. • Assigning probability 1/(Nn) to each obtained sample yields a SRS. • A SRS of size n is denoted by y1, y2, . . . , yn. • Note: In statistics course, upper-case letters (such as Y1, Y2, . . .) denote random variables and lower-case letters (like y1, y2, . . . ) rep- resent observed (fixed) values. In this course, by convention, lower- case letters (such that y1, y2, . . . ) are used to denote random vari- ables. Dr. Luai Al Labadi STA304 Fall 2021 5 / 63 How to Draw a Simple Random Sample How to Draw a Simple Random Sample • Recall that: the total number of all possible samples is (Nn), which could be very large if N and n are large. • For example, ifN = 100, n = 12, then (Nn) = 1, 050, 421, 051, 000, 000. • Therefore, getting a simple random sample by first listing all possi- ble samples and then drawing one at random would not be practical. • An easier way to get a simple random sample is simply to draw n values at random without replacement from the N population values. • That is, we first draw one value at random from the N population values, and then draw another value at random from the remaining N − 1 population values and so on, until we get a sample of n (different) values. Dr. Luai Al Labadi STA304 Fall 2021 6 / 63 How to Draw a Simple Random Sample How to Draw a Simple Random Sample • This can be done as follows: • The units in the population are numbered 1, . . . , N . • A series of numbers are then drawn either by means of a table of random numbers (Table A.2, page 383) or by means of a computer software (R). [SEE THE NEXT EXAMPLE.] Dr. Luai Al Labadi STA304 Fall 2021 7 / 63 How to Draw a Simple Random Sample Example In the college of science, suppose there are N = 5000 students. Use a random number table to construct a SRS of size n = 20 students to use them in a certain study. Solution: • Prepare a list of all students (from the registrar). • Give numbers 0001-5000 for each name. • In R, we use the code: mysample = sample((1:5000), 20,replace=FALSE). Dr. Luai Al Labadi STA304 Fall 2021 8 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of Population Mean µ and Population Variance σ2 • Let u1, u2, . . . , uN be the values of the population of size N . Then 1 The population mean: µ = u1 + u2 + · · ·+ uN N = 1 N N∑ i=1 ui. 2 The population variance: σ2 = 1 N N∑ i=1 (ui − µ)2 = 1 N [ N∑ i=1 u2i −Nµ2 ] . • The goal is to estimate µ and σ2 and to study the accuracy of their estimators. • Let y1, y2, . . . , yn be a SRS of size n. Then a well-known unbiased estimator for µ is the sample mean y. That is, µˆ = y = 1 n n∑ i=1 yi. Dr. Luai Al Labadi STA304 Fall 2021 9 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of Population Mean µ Theorem (Sampling without Replacement) Let y1, y2, . . . , yn be a SRS from the population u1, u2, . . . , uN . Then 1 y is unbiased estimator for µ (i.e. E (y) = µ). 2 V (y) = σ 2 n N−n N−1 . Proof: Dr. Luai Al Labadi STA304 Fall 2021 10 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 11 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 12 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 13 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 14 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 15 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 16 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 17 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 18 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of Population Mean µ • From the proof of the previous theorem, we have E[yi] = µ V [yi] = σ 2 Dr. Luai Al Labadi STA304 Fall 2021 19 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of Population Mean µ • From the previous theorem, y is unbiased estimator for µ. • From the previous theorem, as n gets large (but n ≤ N), V (y) tends to 0. • It follows that y will be a more accurate estimator for µ as n gets larger. • In particular, when n = N , we have a census and V (y) = 0. Dr. Luai Al Labadi STA304 Fall 2021 20 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Sampling with Replacement • Sample: {y1, y2, . . . , yn}. • Sampling with replacement. • Thus, y1, y2, . . . , yn are independent and identically distributed (i.i.d.). • E(y) = µ (why?) and V (y) = σ2n (why?). Dr. Luai Al Labadi STA304 Fall 2021 21 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Sampling with Replacement • Recall: sampling without replacement V (y) = σ2n N−nN−1 . • It follows that: for sampling without replacement V (y) = σ2 n N − n N − 1 < σ2 n (sampling with replacement). [since N−nN−1 ≤ 1 for n = 1, . . . , N .] • Thus, the variance of y for sampling without replacement is less than or equal of the variance of y for sampling with replacement (i.i.d. case). • That is, sampling without replacement produces a less variable es- timator of µ. Dr. Luai Al Labadi STA304 Fall 2021 22 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of σ2 • The population variance σ2 is usually unknown. • Let s2 = 1 n− 1 n∑ i=1 (yi − y)2 = 1 n− 1 ( n∑ i=1 y2i − ny2 ) . Theorem E(s2) = N N − 1σ 2. That is, s2 as defined above is a biased estimator of σ2. Proof: Dr. Luai Al Labadi STA304 Fall 2021 23 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Proof of the Theorem Dr. Luai Al Labadi STA304 Fall 2021 24 / 63 Estimation of Population Mean µ Estimation of Population Mean µ and Population Variance σ2 Estimation of σ2 Theorem σˆ2 = N−1N s 2 is an unbiased estimator of σ2. That is, E ( N − 1 N s2 ) = σ2. Proof: Dr. Luai Al Labadi STA304 Fall 2021 25 / 63 Estimation of Population Mean µ Estimation of V (y) Estimation of V (y) • Let f = n N : the sample proportion 1− f = N − n N = 1− n N : the finite population correction (fpc) Theorem An unbiased estimator for V (y) is V̂ (y) = s2 n N − n N = s2 n (1− f). Recall: V (y) = σ 2 n N−n N−1 Dr. Luai Al Labadi STA304 Fall 2021 26 / 63 Estimation of Population Mean µ Estimation of V (y) Proof Dr. Luai Al Labadi STA304 Fall 2021 27 / 63 Estimation of Population Mean µ Confidence Interval for µ Confidence Interval (CI) for µ • Central Limit Theorem (CLT): If n → ∞ such that n/N → λ ∈ (0, 1), then y − µ√ V (y) ≈ N(0, 1). • Replace V (y) by its estimator V̂ (y). • Thus, P ∣∣∣∣∣∣ y − µ√V̂ (y) ∣∣∣∣∣∣ < zα/2  ≈ 1− α. • That is, P ( y − zα/2 √ V̂ (y) ≤ µ ≤ y − zα/2 √ V̂ (y) ) ≈ 1− α. • Therefore, an approximate (1− α)100% CI for µ is: y ∓ zα/2 √ V̂ (y) = y ∓ zα/2 s√ n √ 1− f. Dr. Luai Al Labadi STA304 Fall 2021 28 / 63 Estimation of Population Mean µ Bound on the Error B Bound on the Error B • B = zα/2 √ V̂ (y) = zα/2 √ s2 n (1− f) is called bound on the error. • Therefore, an approximate (1− α)100% CI for µ is: y ∓B. • Note that, if α = 0.05, then Zα/2 = 1.96 ≈ 2. • In the book, B = 2 √ V̂ (y) = 2 √ s2 n (1− f). • An approximate 95% CI for µ is: y ∓B = y ∓ 2 √ s2 n (1− f). Dr. Luai Al Labadi STA304 Fall 2021 29 / 63 Estimation of Population Mean µ Bound on the Error B Example: #4.16 p. 104 A simple random sample of 100 water meters within a community is monitored to estimate the average daily water consumption per household over a specified dry spell. The sample mean and sample variance are found to be y = 12.5 and s2 = 1252. If we assume that there are N = 10, 000 households within the community, estimate µ, the true mean daily consumption and place a bound on the error of estimation. Also find a 95% C.I. for µ. Solution: Dr. Luai Al Labadi STA304 Fall 2021 30 / 63 Estimation of Population Mean µ Bound on the Error B Dr. Luai Al Labadi STA304 Fall 2021 31 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Selecting the sample size n for estimating µ • Recall: V (y) = σ2n N−nN−1 . • The bigger the sample size n (n ≤ N), the more accurate our esti- mate y is. • Goal: to find out the minimum n such that our estimate is within a bound on the error of estimation with magnitude B with certain probability 1− α: P (|y − µ| < B) ≈ 1− α . • Thus, P (∣∣∣∣y − µV (y) ∣∣∣∣ < BV (y) ) ≈ 1− α. Dr. Luai Al Labadi STA304 Fall 2021 32 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Selecting the sample size n for estimating µ • By CLT, B√ V (y) = B√ σ2 n N−n N−1 = zα/2. • Solve for n: • Thus, n ≈ Nσ 2 (N − 1)D + σ2 , where D = B2 z2α/2 = B2 4 . Dr. Luai Al Labadi STA304 Fall 2021 33 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Selecting the Sample Size n for Estimating µ • Note: The previous formula requires the knowledge of the popula- tion variance σ2, which is typically unknown in practice. However, we can approximate σ2 by the following methods: 1 from pilot studies 2 from previous surveys/studies 3 σ ≈ R/4, where R is the Range of data= max−min. Dr. Luai Al Labadi STA304 Fall 2021 34 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Example: #4.5 p. 89 The average amount of money µ for a hospital’s accounts receivable must be estimated. Although no prior data are available to estimate the population variance, it is known that most accounts lie within a $100 range. There are N = 1000 open accounts. Find the sample size needed to estimate µ with a bound on the error of estimation B = 3. Solution: Dr. Luai Al Labadi STA304 Fall 2021 35 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Dr. Luai Al Labadi STA304 Fall 2021 36 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Example Suppose that a total of 1500 students are to graduate next year. Determine the sample size n needed to ensure that the sample average in starting salary is within $40 of the population average with probability at least 0.95. From previous studies, we know that the standard deviation of the starting salary is approximately $400. Solution: Dr. Luai Al Labadi STA304 Fall 2021 37 / 63 Estimation of Population Mean µ Selecting the Sample Size n for Estimating Population Mean µ Dr. Luai Al Labadi STA304 Fall 2021 38 / 63 Estimation of the Population Total τ Estimation of the Population Total τ Estimation of Population Total τ • Population: u1, u2, . . . , uN . • Sample: y1, y2, . . . , yn. • Populations total: τ = u1 + u2 + . . .+ uN = Nµ. • An estimator of the population τ is τ̂ = Ny. Theorem 1 E (τ̂) = τ . 2 V (τ̂) = N2 σ 2 n N−n N−1 . Proof: Dr. Luai Al Labadi STA304 Fall 2021 39 / 63 Estimation of the Population Total τ Estimation of the Population Total τ Estimation of the Population Total (τ) • Estimated variance of τˆ is given as V̂ (τˆ) = V̂ (Ny¯) = N2V̂ (y¯) = N2 ( s2 n )( N − n N ) = s2 n N(N − n). • Bound on the error of estimation of population total τ is given as B = 2 √ V̂ (Ny¯) = 2 √ N2 ( s2 n )( N − n N ) = 2Ns√ n √ 1− f. • An approximate (1− α)100% CI for τ is: τˆ ∓ zα/2 √ V̂ (τˆ) = τˆ ∓ zα/2N s√ n √ 1− f = N ( y ∓ zα/2 s√ n √ 1− f ) . Dr. Luai Al Labadi STA304 Fall 2021 40 / 63 Estimation of the Population Total τ Selecting the Sample Size n for Estimating τ Selecting the Sample Size n for Estimating τ • The sample size needed to estimate the population total τ with a bound on the error estimation with magnitude B is given by, n = Nσ2 (N − 1)D + σ2 , where, D = B2 4N2 Dr. Luai Al Labadi STA304 Fall 2021 41 / 63 Estimation of the Population Total τ Selecting the Sample Size n for Estimating τ Example: #4.27 & 4.28 p. 106 An investigator is interested in estimating the total number of “count trees” (trees larger than a specified size) on a plantation of N = 1500 acres. This information is used to determine the total volume of lumber for trees on the plantation. A simple random sample of n = 100 one-acre plots was selected, and each plot was examined for the number of count trees. The sample average for the n = 100 one-acre plots was y = 25.2 with a sample variance of s2 = 136. 1 Estimate the total number of count trees on the plantation. 2 Place a bound on the error of estimation. 3 Determine the sample size required to estimate τ , the total number of trees on the plantation, with a bound on the error of estimation of magnitude B = 1500. Solution: Dr. Luai Al Labadi STA304 Fall 2021 42 / 63 Estimation of the Population Total τ Selecting the Sample Size n for Estimating τ Dr. Luai Al Labadi STA304 Fall 2021 43 / 63 Estimation of the Population Total τ Selecting the Sample Size n for Estimating τ Dr. Luai Al Labadi STA304 Fall 2021 44 / 63 Estimation of the Population Proportion p Estimation of the Population Proportion p Estimation of the Population Proportion (p) Let p be the proportion of units in the population belong to a certain category. • Define yi = { 1 the ith unit belongs to the category of interest 0 otherwise. • Examples: • Proportion of males in a certain population. • Proportion of voters who prefer a particular candidate. • It follows that: • µ = E(yi) = p. • σ2 = V (yi) = p− p2 = p(1− p) = pq. Dr. Luai Al Labadi STA304 Fall 2021 45 / 63 Estimation of the Population Proportion p Estimation of the Population Proportion p Estimation of the Population Proportion (p) • ∑n i=1 yi = the total number of elements in the sample of size n that belongs to the category of interest. • An estimator of the population proportion p is pˆ = ∑n i=1 yi n = y. • An estimator of the population variance σ2 = pq is: s2 = 1 n− 1 n∑ i=1 (yi − y)2 = 1 n− 1 ( n∑ i=1 y2i − ny2 ) = 1 n− 1 ( n∑ i=1 yi − npˆ2 ) (WHY ?) = 1 n− 1 ( npˆ− npˆ2) = n n− 1 ( pˆ− pˆ2) = n n− 1 pˆ(1− pˆ) = n n− 1 pˆqˆ. Dr. Luai Al Labadi STA304 Fall 2021 46 / 63 Estimation of the Population Proportion p Estimation of the Population Proportion p Estimation of a Population Proportion p Theorem 1 E (pˆ) = p. 2 V (pˆ) = σ 2 n N−n N−1 = pq n N−n N−1 . Proof: Dr. Luai Al Labadi STA304 Fall 2021 47 / 63 Estimation of the Population Proportion p Estimation of the Population Proportion p Estimation of a Population Proportion p • Variance of p̂ is: V (pˆ) = σ2 n N − n N − 1 = pq n N − n N − 1 . • Estimated variance of p̂ is: V̂ (p̂) = s2 n (1− f) = pˆqˆ n− 1 (1− f) . • Bound on the error of estimation of p is given as B = 2 √ Vˆ (p̂) = 2 √ pˆqˆ n− 1 (1− f). • An approximate (1− α)100% CI for p is p̂∓ zα/2 √ V̂ (p̂) = p̂∓ zα/2 √ p̂q̂√ n− 1 √ 1− f. Dr. Luai Al Labadi STA304 Fall 2021 48 / 63 Estimation of the Population Proportion p Estimation of the Population Proportion p Example: #4.25 106 A sociological study conducted in a small town called for the estimation of the proportion of households that contain at least one member over 65 years of age. The city has 621 households according to the most recent city directory. A simple random sample of n = 60 households was selected from the directory. At the completion of the fieldwork, out of the 60 households sampled, 11 contained at least one member over 65 years of age. 1 Estimate the true population proportion p. 2 Place a bound on the error of estimation.. Solution: Dr. Luai Al Labadi STA304 Fall 2021 49 / 63 Estimation of the Population Proportion p Estimation of the Population Proportion p Dr. Luai Al Labadi STA304 Fall 2021 50 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Selecting the Sample Size n for Estimating p • The sample size (number of observations) needed to estimate the population proportion p with a bound on the error of estimation with magnitude B is given by, n = Npq (N − 1)D + pq where, D = B2 4 • Note: 1 p is often unknown, so we can replace it by some estimate (from previous study, pilot study, etc.). 2 If we don’t have an estimate of p, we can replace it by p = 12 . Thus, pq = 14 . Dr. Luai Al Labadi STA304 Fall 2021 51 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Example A small town has population of N = 800 people. Let p = the proportion of people with blood type A. 1 What sample size n must be drawn in order to estimate p to be within 0.04 of p with probability 0.95? 2 A simple random sample of size n = 200 is taken and it is found that 7% of the sample has blood type A. Find a 95% confidence interval for p. Solution: Dr. Luai Al Labadi STA304 Fall 2021 52 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Dr. Luai Al Labadi STA304 Fall 2021 53 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Dr. Luai Al Labadi STA304 Fall 2021 54 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Example Quiz marks for a class of size N = 15 students are as follows: Students 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 Marks 3 4 3 2 4 0 2 5 2 4 3 3 4 3 5 1 Find the population mean. 2 Find the population variance. 3 Find the exact variance of the sample mean. Take n = 4. 4 A SRS of size 4 is picked: Students 4 10 11 13 Marks 2 4 3 4 Estimate the population total τ and calculate the exact error of the estimate. Solution: Dr. Luai Al Labadi STA304 Fall 2021 55 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Dr. Luai Al Labadi STA304 Fall 2021 56 / 63 Estimation of the Population Proportion p Selecting the Sample Size n for Estimating a Population Proportion Dr. Luai Al Labadi STA304 Fall 2021 57 / 63 Advantages and Disadvantages of SRS Advantages of SRS 1 Easy to implement if a sampling frame is available or the population is not large. 2 Data analysis is simple and flexible. 3 Easy to supplement and reduce the sample. 4 Easy to correct for errors or other problems in selecting the sample. 5 Easy to combine with more sophisticated estimators (ratio and re- gression estimators). Dr. Luai Al Labadi STA304 Fall 2021 58 / 63 Advantages and Disadvantages of SRS Disadvantages of SRS 1 If the population is widely dispersed, it may be extremely costly to reach them. 2 The current list of the whole population we are interested in (sam- pling frame) may not be readily available. 3 Perhaps, the population itself is not homogeneous and the sub- groups are very different in size. In such a case, precision can be increased through stratified sampling. Dr. Luai Al Labadi STA304 Fall 2021 59 / 63 Comparing Estimates Comparing Estimates: Means • Suppose x1, . . . , xm is a random sample from a population with mean µx and y1, . . . , yn is a random sample from a population with mean µy. • We are interested in the difference of means µy − µx, which can be estimated unbiased by y¯ − x¯, as E(y¯ − x¯) = µy − µx. • Moreover, V (y¯ − x¯) = V (y¯) + V (x¯)− 2Cov(y¯, x¯). • Note: If the two samples are independent, then Cov(y¯, x¯) = 0. In this case, 95% CI for µy − µx is y¯ − x¯±B = y¯ − x¯± 2 √ V (y¯) + V (x¯). Dr. Luai Al Labadi STA304 Fall 2021 60 / 63 Comparing Estimates Comparing Estimates: Means • As before, if you may need to replace V (y¯) and V (x¯) by Vˆ (y¯) and Vˆ (x¯), respectively. • Read Example 4.10 P. 95 • As mentioned in the book, when comparing means, we consider only the independent-sample case because the dependent case becomes too complicated to handle at this level. Dr. Luai Al Labadi STA304 Fall 2021 61 / 63 Comparing Estimates Comparing Estimates: Proportions • Suppose an opinion poll asks n people the question “Do you support the death penalty?” • The opinions given are: Yes, No, No Opinion. • Let the proportions of people who answer ‘Yes’, ‘No’, ‘No opinion’ be p1, p2 and p3, respectively. • In particular, we are interested in comparing p1 and p2 by looking at p1 − p2. • Clearly, p1 and p2 are dependent proportions (Why?). Dr. Luai Al Labadi STA304 Fall 2021 62 / 63 Comparing Estimates Comparing Estimates: Proportions • For pˆ1 and pˆ2, we have E(pˆ1 − pˆ2) = p1 − p2. • Moreover, V (pˆ1 − pˆ2)= V (pˆ1) + V (pˆ2)− 2Cov(pˆ1, pˆ2) = p1(1− p1) n + p2(1− p2) n + 2 p1p2 n • Read Example Problem 4.11 p 98 and Example 4.12 p. 99. Dr. Luai Al Labadi STA304 Fall 2021 63 / 63