STAT 3450-Winter-2023: Assignment 2
Quan Yuan B00923505
2023-02-05
Table of Contents
Introduction .............................................................................................................................................. 2
Evaluation ............................................................................................................................... 2
Topics ....................................................................................................................................... 2
Preparation ............................................................................................................................................... 2
Libraries ............................................................................................................................................ 2
Read data .......................................................................................................................................... 3
Problem 1 [10 pts] .................................................................................................................................. 4
Best Subset Selection ................................................................................................................... 4
Question 1.1 [2 pts] ............................................................................................................. 4
Question 1.2 [2 pts] ............................................................................................................. 5
Forward and Backward Stepwise Selection ....................................................................... 6
Question 1.3 [2 pts] ............................................................................................................. 6
Choosing Among Models ................................................................................................... 8
Question 1.4 [2 pts] ............................................................................................................. 8
Question 1.5 [2 pts] ............................................................................................................. 9
Problem 2 [5 pts] ................................................................................................................................. 25
Ridge regression (RR) .............................................................................................................. 25
Question 2.1 [1 pt] ............................................................................................................ 25
Question 2.1 [2 pts] .......................................................................................................... 26
LASSO .............................................................................................................................................. 27
Question 2.2 [1 pt] ............................................................................................................ 27
Question 2.3 [1 pt] ............................................................................................................ 28
Problem 3 [5 pts] ................................................................................................................................. 28
Question 3.1 [1 pt] ............................................................................................................ 29
Question 3.2 [1 pt] ............................................................................................................ 33
Question 3.3 [1 pt] ............................................................................................................ 34
Question 3.4 [1 pt] ............................................................................................................ 35
Question 3.5 [1 pt] ............................................................................................................ 38

Introduction
Evaluation
(Total: 20 pts)
Please print your name and banner ID in the ‘author’ section of the header of this file
(see above).
Topics
In this assignment, you will apply the methods seen in the lectures of Module 2 to
perform various model selection methods.
Preparation
Problem 1: 10 pts Variable selection methods Problem 2: 5 pts Ridge Regression
and the Lasso. Problem 3: 5 pts Principal Components Regression
Preparation
Libraries
You will have to figure out which libraries you need to install or load for this
assignment.
If necessary use ‘install.packages’ to install the packages when they are not alread
listed in your environment.
Run this cell to clear the variables in your global R environment.
rm(list = ls())
ls()
## character(0)
Read data
Download the file dataA2.csv from Brightspace and put it in the same folder as your
assignment’s Rmd file.
Read this file into a dataframe named df. Print the structure of the dataframe df.
df <- read.csv(file='dataA2.csv')
str(df)
## 'data.frame': 506 obs. of 14 variables:
## $ e: num 4.43 6.48 5.42 1.86 6.5 ...
## $ m: num 6.59 2.87 5.08 14.52 8.79 ...
## $ y: num 25.2 36.4 24.6 10.9 20.9 19.2 22.5 22.9 33.1 19.4 ...
## $ b: int 223 398 287 666 345 437 358 311 254 233 ...
## $ d: num 0.464 0.437 0.493 0.679 0.409 0.624 0.4 0.524 0.447 0.448
...
## $ c: num 0.1621 0.0866 0.1919 37.6619 0.1282 ...
## $ g: num 6.96 3.44 7.38 18.1 6.07 ...
## $ f: int 3 5 5 24 4 4 5 5 4 3 ...
## $ h: num 20 45 0 0 12.5 0 70 12.5 40 0 ...
## $ a: int 0 0 0 0 0 0 0 0 1 0 ...
## $ i: num 6.24 7.18 6.43 6.2 5.88 ...
## $ j: num 16.3 26.3 14.7 78.7 33 98.9 20.1 66.6 27.6 62 ...
## $ n: num 396.9 390.5 393.7 18.8 396.9 ...
## $ l: num 18.6 15.2 19.6 20.2 18.9 21.2 14.8 15.2 17.6 17.9 ...
Here is R code that computes the column number that corresponds to the column
named ‘y’. We save this column number in a variable called iy and print it. We will
use this column as the dependent variable of our models and all other columns as
predictors (or ‘X matrix’). How many predictors do we have? We store this value in a
variable called ‘mynvmax’.
str(df)
## 'data.frame': 506 obs. of 14 variables:
## $ e: num 4.43 6.48 5.42 1.86 6.5 ...
## $ m: num 6.59 2.87 5.08 14.52 8.79 ...
## $ y: num 25.2 36.4 24.6 10.9 20.9 19.2 22.5 22.9 33.1 19.4 ...
## $ b: int 223 398 287 666 345 437 358 311 254 233 ...
## $ d: num 0.464 0.437 0.493 0.679 0.409 0.624 0.4 0.524 0.447 0.448
...
## $ c: num 0.1621 0.0866 0.1919 37.6619 0.1282 ...
## $ g: num 6.96 3.44 7.38 18.1 6.07 ...
## $ f: int 3 5 5 24 4 4 5 5 4 3 ...
## $ h: num 20 45 0 0 12.5 0 70 12.5 40 0 ...
## $ a: int 0 0 0 0 0 0 0 0 1 0 ...
## $ i: num 6.24 7.18 6.43 6.2 5.88 ...
## $ j: num 16.3 26.3 14.7 78.7 33 98.9 20.1 66.6 27.6 62 ...
## $ n: num 396.9 390.5 393.7 18.8 396.9 ...
## $ l: num 18.6 15.2 19.6 20.2 18.9 21.2 14.8 15.2 17.6 17.9 ...
iy=which(names(df)=='y')
iy
## [1] 3
mynvmax=13
We will use the dataframe ‘df’ in this assignment.
Problem 1 [10 pts]
Subset Selection Methods
Best Subset Selection
Question 1.1 [2 pts]
Load the leaps library
library(leaps)
Call ‘the proper function’ (I used it in the lectures of Module 2) to perform best
subset selection among all possible linear regression models of the response for
dataset ‘df’.
Save your results in an object called ‘regfit.bestsub’. Make sure that you pass the
right value to the ‘nvmax’ argument so that all possible subsets of the entire list of
predictors are used.
regfit.bestsub <- regsubsets(y~., data=df, nvmax=mynvmax)
Now save the summary of the ‘regfit.bestsub’ object into an object called
‘regfit.summary’.Print this summary object.
regfit.summary <- summary(regfit.bestsub)
Now extract the names of the summary and print them. Print the ‘adjr2’ member of
the summary.
names(regfit.summary)
## [1] "which" "rsq" "rss" "adjr2" "cp" "bic" "outmat" "
obj"
regfit.summary["adjr2"]
## $adjr2
## [1] 0.5432418 0.6371245 0.6767036 0.6878351 0.7051702 0.7123567 0.7
182560
## [8] 0.7222072 0.7252743 0.7299149 0.7348058 0.7343282 0.7337897
Question 1.2 [2 pts]
Use function which.min to compute the position ‘ib’ where the bic member of the
summary is object reg.summary is minimum. Print the value of ib.
# use which.min
ib <- which.min(regfit.summary$bic)
ib
## [1] 11
Plot the bic member of the summary.
Use the function points to overlay one red point for the best model (for this BIC
scale) of size ‘ib’ predictors.
Hint: You may if you are lost look at the code provided in video-2 of module 2. Or
simply look at the template Rmd codes of the lectures.
# plot the bic
plot(regfit.summary$bic,type = "l")
# overlay a red point where bic is
points(ib, regfit.summary$bic[ib], cex=2, pch=20, col="red")

Make a plot of object ‘regfit.bestsub’ on the Mallows Cp scale:
plot(regfit.bestsub, scale = "Cp")

Forward and Backward Stepwise Selection
In all the questions, make sure to use all predictors in the dataframe.
Question 1.3 [2 pts]
Use the same function as in Question 1.1, but now modify the value of the ‘method’
argument to compute forward selection.
Save the returned resilts to an object named ‘regfit.fwd’.
Print a summary of this object.
(0.75 point)
regfit.fwd <- regsubsets(y~., data=df, nvmax=mynvmax, method="forward")
summary(regfit.fwd)
## Subset selection object
## Call: regsubsets.formula(y ~ ., data = df, nvmax = mynvmax, method =
"forward")
## 13 Variables (and intercept)
## Forced in Forced out
## e FALSE FALSE
## m FALSE FALSE
## b FALSE FALSE
## d FALSE FALSE
## c FALSE FALSE
## g FALSE FALSE
## f FALSE FALSE
## h FALSE FALSE
## a FALSE FALSE
## i FALSE FALSE
## j FALSE FALSE
## n FALSE FALSE
## l FALSE FALSE
## 1 subsets of each size up to 13
## Selection Algorithm: forward
## e m b d c g f h a i j n l
## 1 ( 1 ) " " "*" " " " " " " " " " " " " " " " " " " " " " "
## 2 ( 1 ) " " "*" " " " " " " " " " " " " " " "*" " " " " " "
## 3 ( 1 ) " " "*" " " " " " " " " " " " " " " "*" " " " " "*"
## 4 ( 1 ) "*" "*" " " " " " " " " " " " " " " "*" " " " " "*"
## 5 ( 1 ) "*" "*" " " "*" " " " " " " " " " " "*" " " " " "*"
## 6 ( 1 ) "*" "*" " " "*" " " " " " " " " "*" "*" " " " " "*"
## 7 ( 1 ) "*" "*" " " "*" " " " " " " " " "*" "*" " " "*" "*"
## 8 ( 1 ) "*" "*" " " "*" " " " " " " "*" "*" "*" " " "*" "*"
## 9 ( 1 ) "*" "*" " " "*" "*" " " " " "*" "*" "*" " " "*" "*"
## 10 ( 1 ) "*" "*" " " "*" "*" " " "*" "*" "*" "*" " " "*" "*"
## 11 ( 1 ) "*" "*" "*" "*" "*" " " "*" "*" "*" "*" " " "*" "*"
## 12 ( 1 ) "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" " " "*" "*"
## 13 ( 1 ) "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" "*"
Again use the same function but now compute backward selection.
Save the results to an object named ‘regfit.bwd’.
Print a summary of this object.
(0.75 point)
regfit.bwd <- regsubsets(y~., data=df, nvmax=mynvmax, method="backward")
summary(regfit.bwd)
## Subset selection object
## Call: regsubsets.formula(y ~ ., data = df, nvmax = mynvmax, method =
"backward")
## 13 Variables (and intercept)
## Forced in Forced out
## e FALSE FALSE
## m FALSE FALSE
## b FALSE FALSE
## d FALSE FALSE
## c FALSE FALSE
## g FALSE FALSE
## f FALSE FALSE
## h FALSE FALSE
## a FALSE FALSE
## i FALSE FALSE
## j FALSE FALSE
## n FALSE FALSE
## l FALSE FALSE
## 1 subsets of each size up to 13
## Selection Algorithm: backward
## e m b d c g f h a i j n l
## 1 ( 1 ) " " "*" " " " " " " " " " " " " " " " " " " " " " "
## 2 ( 1 ) " " "*" " " " " " " " " " " " " " " "*" " " " " " "
## 3 ( 1 ) " " "*" " " " " " " " " " " " " " " "*" " " " " "*"
## 4 ( 1 ) "*" "*" " " " " " " " " " " " " " " "*" " " " " "*"
## 5 ( 1 ) "*" "*" " " "*" " " " " " " " " " " "*" " " " " "*"
## 6 ( 1 ) "*" "*" " " "*" " " " " " " " " " " "*" " " "*" "*"
## 7 ( 1 ) "*" "*" " " "*" " " " " "*" " " " " "*" " " "*" "*"
## 8 ( 1 ) "*" "*" " " "*" "*" " " "*" " " " " "*" " " "*" "*"
## 9 ( 1 ) "*" "*" "*" "*" "*" " " "*" " " " " "*" " " "*" "*"
## 10 ( 1 ) "*" "*" "*" "*" "*" " " "*" "*" " " "*" " " "*" "*"
## 11 ( 1 ) "*" "*" "*" "*" "*" " " "*" "*" "*" "*" " " "*" "*"
## 12 ( 1 ) "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" " " "*" "*"
## 13 ( 1 ) "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" "*" "*"
Use the coef function to extract the coefficients of ‘regfit.bestsub’ (calculated in
question Question 1.1) for the best subset of 9 predictors (.05 point)
# extracts the reg coefts of The best size 9 subset of predictors
coef(regfit.bestsub, 9)
## (Intercept) e m d
c
## 35.543360341 -1.177844841 -0.526715556 -20.895746238 -0.09691476
8
## f a i n
l
## 0.142965701 2.922953058 4.116861442 0.009691244 -1.12876977
9
Choosing Among Models
Question 1.4 [2 pts]
Generate a random train-test split. [Here you will use method-2 seen in the the
video coding summary of Module 2].
If we note n the number of rows in the dataframe df, we will randomly sample,
WITH replacement, n values from the set of two elements TRUE, and FALSE, and use
this sample vector to create a logical array named ‘train’ of size n. This will be used
to select the training rows of the dataframe.
Because we use this random scheme, approximatly half of the dataframe records
will be kept for training.
Check that this is the case applying the table function to the ‘train’ array.
Then define the ‘test’ array as the logical negation of the ‘train’ array. Try to
remember how you negate a boolean variable or vector in R.
set.seed(23)
train <- sample(c(TRUE, FALSE), nrow(df), replace=T)
table(train)
## train
## FALSE TRUE
## 242 264
test <- !train
From now on, we will use the ‘train’ and ‘test’ logical arrays to feed training data or
testing data to functions.
For example, if we want to pass training data to the function ‘regsubsets’ of leaps,
we can pass the argument ‘data=df[train,]’
Question 1.5 [2 pts]
In this question, we simply want to write a small R code that calculates and stores
the mean square of the prediction errors made on the testing set by each best model
of size 1, 2, 3, up to mynvmax.
mynvmax is the total number of predictors in the dataset. You have calculated it in
question 1.
We will call val.errors the vector that we use to store these errors.
You can use the template code seen e.g. in video-2 of module 2 to calculate a vector
val.errors which stores the MSE of the prediction error for each best size subset of
predictors.
In order to do this: 1. call the ‘appropriate function’ of ‘leaps’ to perform best subset
selection on the training dataset. 2. use the function ‘model.matrix’ to extract the
design matrix on the testing set. Save the result to an object called ‘test.mat’.
regfit.bestmoel <- regsubsets(y~., data=df[train,], nvmax=mynvmax)
test.mat <- model.matrix(y~., data=df[test,])
Print test.mat:
test.mat
## (Intercept) e m b d c g f h a
i j
## 2 1 6.4798 2.87 398 0.4370 0.08664 3.44 5 45.0 0 7.
178 26.3
## 3 1 5.4159 5.08 287 0.4930 0.19186 7.38 5 0.0 0 6.
431 14.7
## 5 1 6.4980 8.79 345 0.4090 0.12816 6.07 4 12.5 0 5.
885 33.0
## 10 1 6.0877 16.20 233 0.4480 0.21977 6.91 3 0.0 0 5.
602 62.0
## 11 1 2.8561 7.67 384 0.5200 0.11432 8.56 5 0.0 0 6.
781 71.3
## 16 1 1.4395 30.63 666 0.7000 20.08490 18.10 24 0.0 0 4.
368 91.2
## 17 1 4.2330 22.60 307 0.5380 1.13081 8.14 4 0.0 0 5.
713 94.1
## 19 1 3.0334 16.29 666 0.5800 4.34879 18.10 24 0.0 0 6.
167 84.0
## 20 1 3.3317 7.74 666 0.5320 5.70818 18.10 24 0.0 0 6.
750 74.9
## 22 1 5.9604 7.39 289 0.4370 0.05372 13.92 4 0.0 0 6.
549 51.0
## 24 1 5.4509 15.71 311 0.5240 0.09378 7.87 5 12.5 0 5.
889 39.0
## 25 1 4.8122 8.51 224 0.5150 0.03306 5.19 5 0.0 0 6.
059 37.3
## 26 1 4.5026 9.10 398 0.4370 0.08387 12.83 5 0.0 0 5.
874 36.6
## 28 1 3.9917 10.58 666 0.5830 3.67367 18.10 24 0.0 0 6.
312 51.9
## 30 1 1.7883 17.19 437 0.6240 0.25915 21.89 4 0.0 0 5.
693 96.0
## 33 1 3.2628 7.90 304 0.5440 0.26938 9.90 4 0.0 0 6.
266 82.8
## 35 1 1.8773 4.59 403 0.6050 1.22358 19.58 5 0.0 0 6.
943 97.4
## 36 1 7.6534 4.56 402 0.4030 0.03150 1.47 3 95.0 0 6.
975 15.3
## 40 1 6.0622 5.21 222 0.4580 0.02985 2.18 3 0.0 0 6.
430 58.7
## 43 1 7.3967 5.90 330 0.4310 0.14030 5.86 7 22.0 0 6.
487 13.0
## 46 1 1.4394 34.41 437 0.6240 1.62864 21.89 4 0.0 0 5.
019 100.0
## 47 1 2.1980 16.44 666 0.7400 9.92485 18.10 24 0.0 0 6.
251 96.6
## 50 1 3.6519 10.88 307 0.5070 0.33045 6.20 8 0.0 0 6.
086 61.5
## 52 1 2.4216 7.44 264 0.5750 0.57834 3.97 5 20.0 0 8.
297 67.0
## 53 1 1.6232 15.79 403 0.8710 2.14918 19.58 5 0.0 0 5.
709 98.5
## 60 1 2.1224 14.09 384 0.5200 0.13117 8.56 5 0.0 0 6.
127 85.2
## 61 1 4.0123 13.83 307 0.5380 0.85204 8.14 4 0.0 0 5.
965 89.2
## 62 1 1.9047 14.64 666 0.7700 4.22239 18.10 24 0.0 1 5.
803 89.0
## 65 1 3.5875 29.55 277 0.4890 0.28955 10.59 4 0.0 0 5.
412 9.8
## 68 1 7.8148 6.86 284 0.4530 0.15445 5.13 8 25.0 0 6.
145 29.2
## 69 1 6.2669 8.65 422 0.5180 0.02498 1.89 1 0.0 0 6.
540 59.7
## 70 1 2.3889 6.48 273 0.5730 0.10959 11.93 1 0.0 0 6.
794 89.3
## 73 1 2.8470 5.68 193 0.4880 0.10008 2.46 3 0.0 0 6.
563 95.6
## 75 1 2.6775 10.16 432 0.5470 0.10084 10.01 6 0.0 0 6.
715 81.6
## 76 1 2.4961 18.71 666 0.7130 9.51363 18.10 24 0.0 0 6.
728 94.1
## 78 1 4.4290 13.00 223 0.4640 0.29916 6.96 3 20.0 0 5.
856 42.1
## 79 1 1.4896 30.59 666 0.6930 38.35180 18.10 24 0.0 0 5.
453 100.0
## 82 1 2.3682 17.27 666 0.7130 5.09017 18.10 24 0.0 0 6.
297 91.8
## 86 1 1.8681 18.07 711 0.6090 0.10574 27.74 4 0.0 0 5.
983 98.8
## 87 1 2.0048 16.59 666 0.7400 5.66637 18.10 24 0.0 0 6.
219 100.0
## 88 1 1.6132 7.12 666 0.7180 4.55587 18.10 24 0.0 0 3.
561 87.9
## 89 1 4.2579 14.67 307 0.5380 0.78420 8.14 4 0.0 0 5.
990 81.7
## 90 1 2.4210 12.33 384 0.5200 0.13960 8.56 5 0.0 0 6.
167 90.0
## 91 1 6.4798 4.69 398 0.4370 0.06911 3.44 5 45.0 0 6.
739 30.8
## 92 1 1.7984 5.50 403 0.6050 1.27346 19.58 5 0.0 1 6.
250 92.6
## 94 1 2.3817 13.59 391 0.5850 0.27957 9.69 6 0.0 0 5.
926 42.6
## 95 1 4.2392 14.66 277 0.4890 0.13587 10.59 4 0.0 1 6.
064 59.1
## 97 1 3.8473 10.13 279 0.4990 0.17505 5.96 5 0.0 0 5.
966 30.2
## 98 1 1.8746 15.70 666 0.7180 7.02259 18.10 24 0.0 0 6.
006 95.3
## 100 1 2.6403 5.98 304 0.5440 0.79041 9.90 4 0.0 0 6.
122 52.8
## 104 1 2.4527 16.74 666 0.7130 8.24809 18.10 24 0.0 0 7.
393 99.3
## 111 1 7.3090 6.62 329 0.4040 0.03768 1.52 2 80.0 0 7.
274 38.3
## 114 1 2.7831 16.23 666 0.7130 7.75223 18.10 24 0.0 0 6.
301 83.7
## 117 1 2.4259 12.14 403 0.6050 1.80028 19.58 5 0.0 0 5.
877 79.2
## 118 1 2.0407 1.92 403 0.6050 1.83377 19.58 5 0.0 1 7.
802 98.2
## 119 1 3.3633 10.50 276 0.5500 0.11425 13.89 5 0.0 1 6.
373 92.4
## 120 1 9.2203 6.57 315 0.3920 0.04819 3.64 1 80.0 0 6.
108 32.0
## 121 1 4.4547 12.80 307 0.5380 0.77299 8.14 4 0.0 0 6.
495 94.4
## 124 1 2.5091 12.67 666 0.7700 4.26131 18.10 24 0.0 0 6.
112 81.3
## 125 1 2.2222 15.17 666 0.7130 13.91340 18.10 24 0.0 0 6.
208 95.0
## 126 1 1.8946 7.79 264 0.6470 0.66351 3.97 5 20.0 0 7.
333 100.0
## 130 1 2.4220 11.64 403 0.6050 2.24236 19.58 5 0.0 0 5.
854 91.8
## 131 1 5.2447 4.59 216 0.4429 0.03705 3.33 5 20.0 0 6.
968 37.2
## 134 1 5.1180 3.81 224 0.4161 0.03510 2.68 4 95.0 0 7.
853 33.2
## 135 1 1.8662 22.11 666 0.7400 22.05110 18.10 24 0.0 0 5.
818 92.4
## 144 1 1.3459 29.29 403 0.8710 2.77974 19.58 5 0.0 0 4.
903 97.8
## 145 1 3.3779 11.41 279 0.4990 0.09744 5.96 5 0.0 0 5.
841 61.4
## 147 1 4.7211 7.70 287 0.4930 0.35114 7.38 5 0.0 0 6.
041 49.9
## 148 1 2.7474 10.30 432 0.5470 0.15098 10.01 6 0.0 0 6.
021 82.6
## 150 1 6.6407 12.43 304 0.4379 0.05023 6.06 1 35.0 0 5.
706 28.4
## 153 1 6.3361 11.38 300 0.4280 0.11329 4.93 6 30.0 0 6.
897 54.3
## 155 1 2.7592 13.61 432 0.5470 0.14476 10.01 6 0.0 0 5.
731 65.2
## 156 1 3.3175 4.54 304 0.5440 0.49298 9.90 4 0.0 0 6.
635 82.5
## 157 1 2.6463 9.04 296 0.5100 0.09178 4.05 5 0.0 0 6.
416 84.1
## 160 1 5.7209 5.81 233 0.4480 0.14150 6.91 3 0.0 0 6.
169 6.6
## 161 1 1.6334 19.92 666 0.6930 7.67202 18.10 24 0.0 0 5.
747 98.9
## 162 1 1.4608 29.53 403 0.8710 2.36862 19.58 5 0.0 0 4.
926 95.7
## 168 1 8.0555 10.15 330 0.4310 0.19657 5.86 7 22.0 0 6.
226 79.2
## 169 1 7.0355 11.22 300 0.4280 0.10290 4.93 6 30.0 0 6.
358 52.9
## 170 1 2.1974 14.27 188 0.5810 0.07165 25.65 2 0.0 0 6.
004 84.1
## 172 1 2.1000 11.10 403 0.6050 2.30040 19.58 5 0.0 0 6.
319 96.1
## 176 1 2.1620 3.32 403 0.6050 1.51902 19.58 5 0.0 1 8.
375 93.9
## 179 1 3.8750 23.09 277 0.4890 0.43571 10.59 4 0.0 1 5.
344 100.0
## 183 1 6.8147 8.43 243 0.4390 0.04981 5.64 4 21.0 0 5.
998 21.4
## 184 1 10.5857 8.05 334 0.4130 0.04301 1.91 4 80.0 0 5.
663 21.9
## 186 1 6.8147 13.45 243 0.4390 0.08873 5.64 4 21.0 0 5.
963 45.7
## 187 1 3.2157 3.13 307 0.5040 0.38214 6.20 8 0.0 0 8.
040 86.5
## 188 1 6.8147 5.28 243 0.4390 0.05360 5.64 4 21.0 0 6.
511 21.1
## 189 1 2.1705 18.03 666 0.6140 10.23300 18.10 24 0.0 0 6.
185 96.7
## 190 1 2.7301 12.04 432 0.5470 0.13158 10.01 6 0.0 0 6.
176 72.5
## 191 1 6.4980 8.10 345 0.4090 0.05789 6.07 4 12.5 0 5.
878 21.4
## 201 1 3.3751 3.92 307 0.5040 0.29819 6.20 8 0.0 0 7.
686 17.0
## 203 1 4.7794 9.62 247 0.4490 0.05059 4.49 3 0.0 0 6.
389 48.0
## 204 1 2.5480 17.09 432 0.5470 0.22212 10.01 6 0.0 0 6.
092 95.4
## 205 1 2.0635 17.16 666 0.5840 8.79212 18.10 24 0.0 0 5.
565 70.6
## 206 1 2.5975 16.42 666 0.7130 4.81213 18.10 24 0.0 0 6.
701 90.0
## 209 1 2.1069 16.47 384 0.5200 0.13262 8.56 5 0.0 0 5.
851 96.7
## 211 1 1.4672 28.28 666 0.7000 24.39380 18.10 24 0.0 0 4.
652 100.0
## 212 1 1.9784 18.85 666 0.7400 9.96654 18.10 24 0.0 0 6.
485 100.0
## 213 1 2.5403 21.32 666 0.5840 4.42228 18.10 24 0.0 0 6.
003 94.5
## 214 1 4.7211 11.74 287 0.4930 0.28392 7.38 5 0.0 0 5.
708 74.3
## 215 1 2.5671 14.65 666 0.7130 3.69311 18.10 24 0.0 0 6.
376 88.4
## 224 1 2.7410 7.56 193 0.4880 0.06588 2.46 3 0.0 0 7.
765 83.3
## 226 1 1.7573 6.43 403 0.6050 1.34284 19.58 5 0.0 0 6.
066 100.0
## 227 1 1.9265 17.11 666 0.7000 6.96215 18.10 24 0.0 0 5.
713 97.0
## 231 1 3.7240 13.34 666 0.5830 2.37857 18.10 24 0.0 0 5.
871 41.9
## 232 1 2.3534 16.21 432 0.5470 0.12329 10.01 6 0.0 0 5.
913 92.9
## 235 1 2.2834 9.81 403 0.6050 2.92400 19.58 5 0.0 0 6.
101 93.0
## 236 1 4.0905 10.27 398 0.4370 0.08707 12.83 5 0.0 0 6.
140 45.8
## 237 1 4.1403 7.19 254 0.4470 0.07978 6.41 4 40.0 0 6.
482 32.1
## 238 1 9.2203 9.25 315 0.3920 0.03548 3.64 1 80.0 0 5.
876 19.1
## 239 1 3.2157 6.36 307 0.5040 0.41238 6.20 8 0.0 0 7.
163 79.9
## 240 1 7.3197 14.80 469 0.4100 0.01360 4.00 3 75.0 0 5.
888 47.6
## 241 1 7.8278 4.74 358 0.4000 0.05561 2.24 5 70.0 0 7.
041 10.0
## 243 1 6.3200 15.84 289 0.4370 0.14103 13.92 4 0.0 0 5.
790 58.0
## 245 1 1.9444 25.41 188 0.5810 0.15038 25.65 2 0.0 0 5.
856 97.0
## 247 1 6.3361 12.40 300 0.4280 0.10612 4.93 6 30.0 0 6.
095 65.1
## 249 1 1.6687 18.46 437 0.6240 0.54452 21.89 4 0.0 0 6.
151 97.9
## 250 1 1.5184 31.99 666 0.7000 22.59710 18.10 24 0.0 0 5.
000 89.5
## 251 1 5.6484 2.97 255 0.4220 0.01381 0.46 4 80.0 0 7.
875 32.0
## 252 1 5.6894 18.80 233 0.4480 0.22927 6.91 3 0.0 0 6.
030 85.5
## 253 1 2.1222 17.60 666 0.7700 8.98296 18.10 24 0.0 1 6.
212 97.4
## 255 1 4.4546 16.51 307 0.5380 0.84054 8.14 4 0.0 0 5.
599 85.7
## 257 1 2.4091 12.92 391 0.5850 0.23912 9.69 6 0.0 0 6.
019 65.3
## 258 1 7.9549 18.46 330 0.4310 0.19133 5.86 7 22.0 0 5.
605 70.2
## 259 1 5.8700 30.81 233 0.4480 0.25387 6.91 3 0.0 0 5.
399 95.3
## 260 1 2.0720 17.79 666 0.7400 6.28807 18.10 24 0.0 0 6.
341 96.4
## 265 1 3.4242 10.74 666 0.5320 5.82401 18.10 24 0.0 0 6.
242 64.7
## 266 1 2.4982 14.33 391 0.5850 0.22438 9.69 6 0.0 0 6.
027 79.7
## 267 1 2.5052 13.27 666 0.7700 3.84970 18.10 24 0.0 1 6.
395 91.0
## 269 1 1.4655 19.78 666 0.5970 14.43830 18.10 24 0.0 0 6.
852 100.0
## 275 1 1.9356 21.52 666 0.6790 7.36711 18.10 24 0.0 0 6.
193 78.1
## 276 1 1.1296 8.88 666 0.6680 8.26725 18.10 24 0.0 1 5.
875 89.6
## 277 1 3.9342 8.77 279 0.4990 0.08014 5.96 5 0.0 0 5.
850 41.5
## 279 1 8.3440 5.99 280 0.4350 0.01501 2.01 4 80.0 0 6.
635 29.7
## 284 1 2.7778 13.00 384 0.5200 0.10793 8.56 5 0.0 0 6.
195 54.4
## 285 1 2.3158 16.22 666 0.7130 5.58107 18.10 24 0.0 0 6.
436 87.9
## 286 1 5.1167 4.70 245 0.4110 0.03615 4.95 4 80.0 0 6.
630 23.4
## 287 1 2.2004 12.03 666 0.5840 6.44405 18.10 24 0.0 0 6.
425 74.8
## 288 1 1.3216 26.82 403 0.8710 3.32105 19.58 5 0.0 1 5.
403 100.0
## 291 1 5.4917 4.86 329 0.4330 0.10000 6.09 7 34.0 0 6.
982 17.7
## 293 1 2.8927 14.10 391 0.5850 0.26838 9.69 6 0.0 0 5.
794 70.6
## 294 1 1.9096 24.39 666 0.6790 15.86030 18.10 24 0.0 0 5.
896 95.4
## 296 1 5.4007 7.51 281 0.4260 0.03551 4.86 4 25.0 0 6.
167 46.7
## 301 1 4.5667 5.39 398 0.4370 0.08370 3.44 5 45.0 0 7.
185 38.9
## 302 1 3.2797 13.15 193 0.4880 0.06047 2.46 3 0.0 0 6.
153 68.8
## 305 1 1.5804 23.60 666 0.7000 9.18702 18.10 24 0.0 0 5.
536 100.0
## 307 1 2.0527 17.64 666 0.5840 8.49213 18.10 24 0.0 0 6.
348 86.1
## 308 1 1.7257 17.12 666 0.6930 8.71675 18.10 24 0.0 0 6.
471 98.8
## 310 1 1.7659 7.39 403 0.8710 1.42502 19.58 5 0.0 0 6.
510 100.0
## 313 1 2.5050 7.88 273 0.5730 0.04741 11.93 1 0.0 0 6.
030 80.8
## 315 1 1.8010 5.12 264 0.6470 0.61154 3.97 5 20.0 0 8.
704 86.9
## 317 1 4.2390 11.98 307 0.5380 1.00245 8.14 4 0.0 0 6.
674 87.3
## 319 1 4.0900 4.98 296 0.5380 0.00632 2.31 1 18.0 0 6.
575 65.2
## 326 1 5.4159 6.15 287 0.4930 0.30347 7.38 5 0.0 0 6.
312 28.9
## 328 1 7.9809 9.50 284 0.4530 0.12650 5.13 8 25.0 0 6.
762 43.4
## 329 1 2.8944 4.63 307 0.5040 0.52693 6.20 8 0.0 0 8.
725 83.0
## 330 1 6.9320 9.22 284 0.4530 0.10328 5.13 8 25.0 0 5.
927 47.2
## 331 1 5.1167 3.33 245 0.4110 0.03502 4.95 4 80.0 0 6.
861 27.9
## 337 1 6.2196 5.03 265 0.4010 0.02187 2.93 1 60.0 0 6.
800 9.9
## 339 1 3.3603 9.68 279 0.4990 0.06417 5.96 5 0.0 0 5.
933 68.2
## 340 1 3.0480 21.46 307 0.5070 0.40771 6.20 8 0.0 1 6.
164 91.3
## 342 1 3.8771 17.27 277 0.4890 0.17446 10.59 4 0.0 1 5.
960 92.1
## 344 1 1.3163 13.44 666 0.6710 19.60910 18.10 24 0.0 0 7.
313 97.9
## 345 1 4.5404 7.20 287 0.4930 0.16760 7.38 5 0.0 0 6.
426 52.3
## 348 1 3.6519 3.95 307 0.5070 0.33147 6.20 8 0.0 0 8.
247 70.4
## 349 1 3.9450 11.50 304 0.5440 0.25356 9.90 4 0.0 0 5.
705 77.7
## 351 1 1.7494 15.12 403 0.8710 1.41385 19.58 5 0.0 1 6.
129 96.0
## 353 1 3.1025 9.97 304 0.5440 0.34940 9.90 4 0.0 0 5.
972 76.7
## 356 1 1.6475 26.64 666 0.6790 25.94060 18.10 24 0.0 0 5.
304 89.1
## 359 1 3.7598 20.34 307 0.5380 1.61282 8.14 4 0.0 0 6.
096 96.9
## 360 1 2.3460 17.31 437 0.6240 0.97617 21.89 4 0.0 0 5.
757 98.4
## 362 1 2.0459 3.70 403 0.6050 2.01019 19.58 5 0.0 0 7.
929 96.2
## 365 1 2.4298 18.14 666 0.5840 8.05579 18.10 24 0.0 0 5.
427 95.4
## 368 1 6.0821 29.93 311 0.5240 0.21124 7.87 5 12.5 0 5.
631 100.0
## 369 1 2.1675 5.64 273 0.5730 0.06076 11.93 1 0.0 0 6.
976 91.0
## 371 1 5.4917 8.67 329 0.4330 0.09266 6.09 7 34.0 0 6.
495 18.4
## 377 1 3.4952 6.65 276 0.4450 0.12204 2.89 2 0.0 0 6.
625 57.8
## 379 1 6.2700 3.11 348 0.4150 0.02177 2.03 2 82.5 0 7.
610 15.7
## 380 1 1.6768 19.37 666 0.6930 5.87205 18.10 24 0.0 0 6.
405 96.0
## 381 1 3.5325 10.36 304 0.5440 0.40202 9.90 4 0.0 0 6.
382 67.2
## 382 1 3.6659 6.21 270 0.4640 0.02875 15.04 4 28.0 0 6.
211 28.9
## 389 1 1.9301 8.10 264 0.6470 0.55007 3.97 5 20.0 0 7.
206 91.6
## 391 1 3.6715 5.25 307 0.5040 0.46296 6.20 8 0.0 0 7.
412 76.9
## 392 1 1.6102 12.12 403 0.8710 1.12658 19.58 5 0.0 1 5.
012 88.0
## 393 1 1.6180 14.10 403 0.8710 1.65660 19.58 5 0.0 0 6.
122 97.3
## 397 1 7.2255 6.73 284 0.4530 0.11027 5.13 8 25.0 0 6.
456 67.8
## 399 1 2.3053 18.68 666 0.6140 4.87141 18.10 24 0.0 0 6.
484 93.6
## 401 1 3.5459 14.98 666 0.5830 5.69175 18.10 24 0.0 0 6.
114 79.8
## 403 1 3.7886 6.68 398 0.4370 0.07875 3.44 5 45.0 0 6.
782 41.1
## 404 1 1.9669 16.90 437 0.6240 0.32264 21.89 4 0.0 0 5.
942 93.5
## 407 1 4.4272 12.86 247 0.4490 0.05188 4.49 3 0.0 0 6.
015 45.1
## 408 1 2.8617 9.71 307 0.5070 0.35809 6.20 8 0.0 1 6.
951 88.5
## 409 1 6.0622 5.33 222 0.4580 0.06905 2.18 3 0.0 0 7.
147 54.2
## 410 1 2.1036 10.19 666 0.7700 3.67822 18.10 24 0.0 0 5.
362 96.2
## 411 1 2.0026 18.05 666 0.7400 14.42080 18.10 24 0.0 0 6.
461 93.3
## 412 1 5.9853 8.01 224 0.5150 0.03961 5.19 5 0.0 0 6.
037 34.5
## 416 1 1.1370 37.97 666 0.6680 18.49820 18.10 24 0.0 0 4.
138 100.0
## 418 1 2.7147 10.63 384 0.5200 0.22876 8.56 5 0.0 0 6.
405 85.4
## 420 1 4.1480 9.54 307 0.5070 0.52058 6.20 8 0.0 1 6.
631 76.5
## 425 1 2.1121 9.59 264 0.6470 0.54011 3.97 5 20.0 0 7.
203 81.8
## 430 1 2.4775 15.37 432 0.5470 0.13058 10.01 6 0.0 0 5.
872 73.1
## 431 1 1.5166 16.65 403 0.8710 2.15505 19.58 5 0.0 0 5.
628 100.0
## 434 1 10.5857 5.57 334 0.4130 0.10659 1.91 4 80.0 0 5.
936 19.5
## 435 1 2.0788 11.25 264 0.6470 0.82526 3.97 5 20.0 0 7.
327 94.5
## 436 1 4.4619 10.26 307 0.5380 0.63796 8.14 4 0.0 0 6.
096 84.5
## 437 1 4.0952 19.88 307 0.5380 0.98843 8.14 4 0.0 0 5.
813 100.0
## 438 1 1.9799 18.34 437 0.6240 0.88125 21.89 4 0.0 0 5.
637 94.7
## 441 1 4.7075 8.26 307 0.5380 0.62976 8.14 4 0.0 0 5.
949 61.8
## 444 1 3.3175 6.29 296 0.5100 0.05425 4.05 5 0.0 0 6.
315 73.4
## 448 1 2.2565 10.45 432 0.5470 0.14231 10.01 6 0.0 0 6.
254 84.2
## 449 1 1.9512 13.11 666 0.6140 14.33370 18.10 24 0.0 0 6.
229 88.0
## 451 1 2.4699 15.03 437 0.6240 0.32982 21.89 4 0.0 0 5.
822 95.4
## 453 1 10.7103 5.49 411 0.4110 0.07950 1.69 4 60.0 0 6.
579 35.9
## 457 1 3.3751 3.76 307 0.5040 0.44178 6.20 8 0.0 0 6.
552 21.4
## 459 1 2.2955 14.19 666 0.7700 3.83684 18.10 24 0.0 0 6.
251 91.1
## 461 1 4.4377 6.53 247 0.4490 0.05735 4.49 3 0.0 0 6.
630 56.1
## 464 1 6.5921 17.10 311 0.5240 0.17004 7.87 5 12.5 0 6.
004 85.9
## 465 1 3.1523 11.45 666 0.5830 4.83567 18.10 24 0.0 0 5.
905 53.2
## 466 1 4.8122 9.74 224 0.5150 0.05497 5.19 5 0.0 0 5.
985 45.4
## 467 1 3.9454 9.47 277 0.4890 0.19802 10.59 4 0.0 0 6.
182 42.4
## 468 1 1.4191 27.80 403 0.8710 2.37934 19.58 5 0.0 0 6.
130 100.0
## 470 1 4.0522 11.97 398 0.4370 0.10153 12.83 5 0.0 0 6.
279 74.5
## 472 1 2.7175 14.70 666 0.7130 6.80117 18.10 24 0.0 0 6.
081 84.4
## 474 1 3.9175 13.65 223 0.4640 0.09065 6.96 3 20.0 1 5.
920 61.5
## 479 1 2.8715 17.15 666 0.6550 3.77498 18.10 24 0.0 0 5.
952 84.7
## 481 1 3.0923 8.81 270 0.4890 0.04684 3.41 2 0.0 0 6.
417 66.1
## 482 1 6.4584 6.75 224 0.5150 0.03738 5.19 5 0.0 0 6.
310 38.5
## 483 1 8.0555 9.16 330 0.4310 0.33983 5.86 7 22.0 0 6.
108 34.9
## 485 1 1.4261 30.81 666 0.7000 16.81180 18.10 24 0.0 0 5.
277 98.1
## 488 1 3.4952 3.57 276 0.4450 0.08187 2.89 2 0.0 0 7.
820 36.9
## 489 1 1.9142 26.45 666 0.7400 15.17720 18.10 24 0.0 0 6.
152 100.0
## 490 1 10.7103 7.79 411 0.4110 0.07244 1.69 4 60.0 0 5.
884 18.5
## 492 1 1.5192 21.08 666 0.6710 15.87440 18.10 24 0.0 0 6.
545 99.1
## 493 1 3.7965 11.28 307 0.5380 0.72580 8.14 4 0.0 0 5.
727 69.5
## 494 1 3.6150 10.59 270 0.4640 0.04294 15.04 4 28.0 0 6.
249 77.3
## 495 1 6.2196 4.38 265 0.4010 0.01439 2.93 1 60.0 0 6.
604 18.8
## 496 1 2.1099 13.35 711 0.6090 0.11132 27.74 4 0.0 0 5.
983 83.5
## 497 1 1.5004 29.97 666 0.6930 9.91655 18.10 24 0.0 0 5.
852 77.8
## 498 1 1.9512 14.10 666 0.6140 12.04820 18.10 24 0.0 0 5.
648 87.6
## 499 1 5.9604 6.27 289 0.4370 0.12932 13.92 4 0.0 0 6.
678 31.1
## 506 1 2.1678 18.76 666 0.7000 5.29305 18.10 24 0.0 0 6.
051 82.5
## n l
## 2 390.49 15.2
## 3 393.68 19.6
## 5 396.90 18.9
## 10 396.90 17.9
## 11 395.58 20.9
## 16 285.83 20.2
## 17 360.17 21.0
## 19 396.90 20.2
## 20 393.07 20.2
## 22 392.85 16.0
## 24 390.50 15.2
## 25 396.14 20.2
## 26 396.06 18.7
## 28 388.62 20.2
## 30 392.11 21.2
## 33 393.39 18.4
## 35 363.43 14.7
## 36 396.90 17.0
## 40 394.12 18.7
## 43 396.28 19.1
## 46 396.90 21.2
## 47 388.52 20.2
## 50 376.75 17.4
## 52 384.54 13.0
## 53 261.95 14.7
## 60 387.69 20.9
## 61 392.53 21.0
## 62 353.04 20.2
## 65 348.93 18.6
## 68 390.68 19.7
## 69 389.96 15.9
## 70 393.45 21.0
## 73 396.90 17.8
## 75 395.59 17.8
## 76 6.68 20.2
## 78 388.65 18.6
## 79 396.90 20.2
## 82 385.09 20.2
## 86 390.11 20.1
## 87 395.69 20.2
## 88 354.70 20.2
## 89 386.75 21.0
## 90 392.69 20.9
## 91 389.71 15.2
## 92 338.92 14.7
## 94 396.90 19.2
## 95 381.32 18.6
## 97 393.43 19.2
## 98 319.98 20.2
## 100 396.90 18.4
## 104 375.87 20.2
## 111 392.20 12.6
## 114 272.21 20.2
## 117 227.61 14.7
## 118 389.61 14.7
## 119 393.74 16.4
## 120 392.89 16.4
## 121 387.94 21.0
## 124 390.74 20.2
## 125 100.63 20.2
## 126 383.29 13.0
## 130 395.11 14.7
## 131 392.23 14.9
## 134 392.78 14.7
## 135 391.45 20.2
## 144 396.90 14.7
## 145 377.56 19.2
## 147 396.90 19.6
## 148 394.51 17.8
## 150 394.02 16.9
## 153 391.25 16.6
## 155 391.50 17.8
## 156 396.90 18.4
## 157 395.50 16.6
## 160 383.37 17.9
## 161 393.10 20.2
## 162 391.71 14.7
## 168 376.14 19.1
## 169 372.75 16.6
## 170 377.67 19.1
## 172 297.09 14.7
## 176 388.45 14.7
## 179 396.90 18.6
## 183 396.90 16.8
## 184 382.80 22.0
## 186 395.56 16.8
## 187 387.38 17.4
## 188 396.90 16.8
## 189 379.70 20.2
## 190 393.30 17.8
## 191 396.21 18.9
## 201 377.51 17.4
## 203 396.90 18.5
## 204 396.90 17.8
## 205 3.65 20.2
## 206 255.23 20.2
## 209 394.05 20.9
## 211 396.90 20.2
## 212 386.73 20.2
## 213 331.29 20.2
## 214 391.13 19.6
## 215 391.43 20.2
## 224 395.56 17.8
## 226 353.89 14.7
## 227 394.43 20.2
## 231 370.73 20.2
## 232 394.95 17.8
## 235 240.16 14.7
## 236 386.96 18.7
## 237 396.90 17.6
## 238 395.18 16.4
## 239 372.08 17.4
## 240 396.90 21.1
## 241 371.58 14.8
## 243 396.90 16.0
## 245 370.31 19.1
## 247 394.62 16.6
## 249 396.90 21.2
## 250 396.90 20.2
## 251 394.23 14.4
## 252 392.74 17.9
## 253 377.73 20.2
## 255 303.42 21.0
## 257 396.90 19.2
## 258 389.13 19.1
## 259 396.90 17.9
## 260 318.01 20.2
## 265 396.90 20.2
## 266 396.90 19.2
## 267 391.34 20.2
## 269 179.36 20.2
## 275 96.73 20.2
## 276 347.88 20.2
## 277 396.90 19.2
## 279 390.94 17.0
## 284 393.49 20.9
## 285 100.19 20.2
## 286 396.90 19.2
## 287 97.95 20.2
## 288 396.90 14.7
## 291 390.43 16.1
## 293 396.90 19.2
## 294 7.68 20.2
## 296 390.64 19.0
## 301 396.90 15.2
## 302 387.11 17.8
## 305 396.90 20.2
## 307 83.45 20.2
## 308 391.98 20.2
## 310 364.31 14.7
## 313 396.90 21.0
## 315 389.70 13.0
## 317 380.23 21.0
## 319 396.90 15.3
## 326 396.90 19.6
## 328 395.58 19.7
## 329 382.00 17.4
## 330 396.90 19.7
## 331 396.90 19.2
## 337 393.37 15.6
## 339 396.90 19.2
## 340 395.24 17.4
## 342 393.25 18.6
## 344 396.90 20.2
## 345 396.90 19.6
## 348 378.95 17.4
## 349 396.42 18.4
## 351 321.02 14.7
## 353 396.24 18.4
## 356 127.36 20.2
## 359 248.31 21.0
## 360 262.76 21.2
## 362 369.30 14.7
## 365 352.58 20.2
## 368 386.63 15.2
## 369 396.90 21.0
## 371 383.61 16.1
## 377 357.98 18.0
## 379 395.38 14.7
## 380 396.90 20.2
## 381 395.21 18.4
## 382 396.33 18.2
## 389 387.89 13.0
## 391 376.14 17.4
## 392 343.28 14.7
## 393 372.80 14.7
## 397 396.90 19.7
## 399 396.21 20.2
## 401 392.68 20.2
## 403 393.87 15.2
## 404 378.25 21.2
## 407 395.99 18.5
## 408 391.70 17.4
## 409 396.90 18.7
## 410 380.79 20.2
## 411 27.49 20.2
## 412 396.90 20.2
## 416 396.90 20.2
## 418 70.80 20.9
## 420 388.45 17.4
## 425 392.80 13.0
## 430 338.63 17.8
## 431 169.27 14.7
## 434 376.04 22.0
## 435 393.42 13.0
## 436 380.02 21.0
## 437 394.54 21.0
## 438 396.90 21.2
## 441 396.90 21.0
## 444 395.60 16.6
## 448 388.74 17.8
## 449 383.32 20.2
## 451 388.69 21.2
## 453 370.78 18.3
## 457 380.34 17.4
## 459 350.65 20.2
## 461 392.30 18.5
## 464 386.71 15.2
## 465 388.22 20.2
## 466 396.90 20.2
## 467 393.63 18.6
## 468 172.91 14.7
## 470 373.66 18.7
## 472 396.90 20.2
## 474 391.34 18.6
## 479 22.01 20.2
## 481 392.18 17.8
## 482 389.40 20.2
## 483 390.18 19.1
## 485 396.90 20.2
## 488 393.53 18.0
## 489 9.32 20.2
## 490 392.33 18.3
## 492 396.90 20.2
## 493 390.95 21.0
## 494 396.90 18.2
## 495 376.70 15.6
## 496 396.90 20.1
## 497 338.16 20.2
## 498 291.55 20.2
## 499 396.90 16.0
## 506 378.38 20.2
## attr(,"assign")
## [1] 0 1 2 3 4 5 6 7 8 9 10 11 12 13
Now 1. use the function rep (replicate) to define an array of mynvmax values all
equal to NA. 2. use a for loop over all sizes of predictors (use a loop index named ‘i’
that runs from 1 to mynvmax), and for each size ‘i’, compute the MSE and store it in
the i’th position of the val.errors array.
After the loop is finished, val.errors[k] should contain the MSE of predictions on the
testing set for the best subset of k predictors.
Print the array val.errors.
Finally write a line of code to retrieve and print which value of k gives the lowest
MSE value in the array val.errors (use the function which.min and apply it to
val.errors)..
val.errors <- rep(NA, mynvmax)
for(i in 1:mynvmax){
coefi=coef(regfit.bestmoel, id=i)
pred=test.mat[, names(coefi)]%*%coefi
val.errors[i]=mean((df$y[test]-pred)^2)
}
val.errors
## [1] 40.53897 30.02298 25.03273 25.57349 23.34972 22.69760 23.32652
23.00910
## [9] 22.61540 22.03925 21.32035 21.97549 21.98025
which.min(val.errors)
## [1] 11
Problem 2 [5 pts]
TOPICS: Ridge Regression and the Lasso.
Ridge regression (RR)
Question 2.1 [1 pt]
Let’s prepare the data. We need to create a x matrix and a y vector.
First, let us build two objects: x,the design matrix of predictors (this will
automatically include a column of 1 for the intercept) and the response column y.
Here are my instructions.
Use the function model.matrix to define a matrix named ‘x’ of predictors for the data
frame ‘df’. Do not use [,-1]. Instead, provide a formula to the function model.matrix
saying that you want to model the response column y as a function of all other
variables. If you proceed like this, the matrix x returned by the function
model.matrix will NOT contain the y column (it will only be the design matrix of
oredictors of the regression).
Print the head of x.
Assign the column ‘y’ of dataframe ‘df’ to a vector called ‘y’. Print the first few
records of the column y for the testing set (use the logical array ‘test’ prepared in
question Q5)
x <- model.matrix(~., df[, -c(iy)])
head(x)
## (Intercept) e m b d c g f h a i
j n
## 1 1 4.4290 6.59 223 0.464 0.16211 6.96 3 20.0 0 6.240
16.3 396.90
## 2 1 6.4798 2.87 398 0.437 0.08664 3.44 5 45.0 0 7.178
26.3 390.49
## 3 1 5.4159 5.08 287 0.493 0.19186 7.38 5 0.0 0 6.431
14.7 393.68
## 4 1 1.8629 14.52 666 0.679 37.66190 18.10 24 0.0 0 6.202
78.7 18.82
## 5 1 6.4980 8.79 345 0.409 0.12816 6.07 4 12.5 0 5.885
33.0 396.90
## 6 1 2.1185 12.60 437 0.624 0.34006 21.89 4 0.0 0 6.458
98.9 395.04
## l
## 1 18.6
## 2 15.2
## 3 19.6
## 4 20.2
## 5 18.9
## 6 21.2
y <- df$y
we will keep the x and y arrays for all subsequent questions. If you need to feed only
the rows of the training set to a function that uses x, you can use x[train,]. Similarly,
you can use y[train] for feeding training valuesof y. Of course similar comments
apply to the testing set (use x[test,] and y[test]).
Question 2.1 [2 pts]
Load the library that contains the cv.glmnet function:
library(glmnet)
## 载入需要的程辑包：Matrix
## Loaded glmnet 4.1-6
Use the cv.glmnet function to compute cross-validation for the ridge regression
model. For this, use the default arguments of the function ‘cv.glmnet’ and pass the
training data for x and y. Store the results in an object named ‘cv.out’. Plot this object.
cv.out <- cv.glmnet(x[train,], y[train], alpha=0)
plot(cv.out)
Compute the best value of the regularization parameter (lambda) and print it (hint:
this value is stored in the object ‘cv.out’).
cv.out["lambda.min"]
## $lambda.min
## [1] 0.7268458
LASSO
Question 2.2 [1 pt]
Use the ‘cv.glmnet’ function to compute cross-validation for the LASSO model. Use
default arguments and the training data for x and y. store the result in ‘cv.out’.
cv.out <- cv.glmnet(x[train,], y[train], alpha=1)
plot(cv.out)
Question 2.3 [1 pt]
Plot ‘cv.out’. Extract and print the best value of lambda (the optimal regularization
parameter).
cv.out["lambda.min"]
## $lambda.min
## [1] 0.007439492
Problem 3 [5 pts]
TOPICS: PCR regression
Download the cc.csv dataset, place it in the folder of this assignment.
Run the following code. This will prepare the data that will be needed for this
problem.
set.seed(654321)
d<-read.csv('cc.csv',header=FALSE)
#39,23,74118,Springfieldcity,
d=d[,-c(1,2,3,4)]
cr=NULL
nc=ncol(d)
for(j in 1:nc){
if(sum(d[,j]=='?')>0)cr=c(cr,j)
}
d=d[,-cr]
ncol(d)
## [1] 101
y=d[,101]
x=d[,1:100]
names(d)[101]='y'
n=nrow(d)
nt=floor(n*0.8)
id=sample(1:n,nt)
dtrain=d[id,]
dtest=d[-id,]
xtest=dtest[,-101]
ytest=dtest[,101]
Question 3.1 [1 pt]
Call the function pcr to fit a model of y as a function of all other columns in training
set dtrain. Set scale to true and ask for cross-validation. Save the result to pcrfit. Call
the function validationplot on model pcrfit and ask for val.type MSEP.
library(pls)
##
## 载入程辑包：'pls'
## The following object is masked from 'package:stats':
##
## loadings
pcrfit <- pcr(y~., data=dtrain, scale=TRUE, validation="CV")
summary(pcrfit)
## Data: X dimension: 1595 100
## Y dimension: 1595 1
## Fit method: svdpc
## Number of components considered: 100
##
## VALIDATION: RMSEP
## Cross-validated using 10 random segments.
## (Intercept) 1 comps 2 comps 3 comps 4 comps 5 comps 6 c
omps
## CV 0.231 0.1793 0.1681 0.1627 0.1556 0.1552 0.
1547
## adjCV 0.231 0.1793 0.1681 0.1627 0.1556 0.1551 0.
1546
## 7 comps 8 comps 9 comps 10 comps 11 comps 12 comps 13 c
omps
## CV 0.1513 0.1412 0.1412 0.1409 0.1408 0.1399 0.
1399
## adjCV 0.1513 0.1411 0.1412 0.1409 0.1407 0.1398 0.
1398
## 14 comps 15 comps 16 comps 17 comps 18 comps 19 comps 2
0 comps
## CV 0.1399 0.1400 0.1399 0.1399 0.1400 0.1401
0.1401
## adjCV 0.1398 0.1399 0.1398 0.1398 0.1399 0.1400
0.1400
## 21 comps 22 comps 23 comps 24 comps 25 comps 26 comps 2
7 comps
## CV 0.1394 0.1393 0.1394 0.1392 0.1393 0.1394
0.1394
## adjCV 0.1392 0.1391 0.1393 0.1390 0.1392 0.1392
0.1392
## 28 comps 29 comps 30 comps 31 comps 32 comps 33 comps 3
4 comps
## CV 0.1395 0.1396 0.1395 0.1392 0.1389 0.1388
0.1388
## adjCV 0.1393 0.1394 0.1393 0.1391 0.1387 0.1386
0.1386
## 35 comps 36 comps 37 comps 38 comps 39 comps 40 comps 4
1 comps
## CV 0.1387 0.1388 0.1388 0.1387 0.1385 0.1385
0.1384
## adjCV 0.1385 0.1386 0.1386 0.1387 0.1382 0.1384
0.1381
## 42 comps 43 comps 44 comps 45 comps 46 comps 47 comps 4
8 comps
## CV 0.1386 0.1386 0.1386 0.1386 0.1384 0.1387
0.1386
## adjCV 0.1383 0.1384 0.1384 0.1384 0.1381 0.1384
0.1384
## 49 comps 50 comps 51 comps 52 comps 53 comps 54 comps 5
5 comps
## CV 0.1385 0.1388 0.1387 0.1388 0.1389 0.1390
0.1388
## adjCV 0.1383 0.1385 0.1384 0.1385 0.1386 0.1387
0.1385
## 56 comps 57 comps 58 comps 59 comps 60 comps 61 comps 6
2 comps
## CV 0.1390 0.1391 0.1390 0.1387 0.1388 0.1388
0.1386
## adjCV 0.1387 0.1388 0.1387 0.1383 0.1384 0.1385
0.1383
## 63 comps 64 comps 65 comps 66 comps 67 comps 68 comps 6
9 comps
## CV 0.1388 0.1388 0.1386 0.1384 0.1381 0.1380
0.1380
## adjCV 0.1384 0.1384 0.1382 0.1381 0.1378 0.1376
0.1377
## 70 comps 71 comps 72 comps 73 comps 74 comps 75 comps 7
6 comps
## CV 0.1377 0.1377 0.1375 0.1376 0.1376 0.1373
0.1375
## adjCV 0.1373 0.1373 0.1371 0.1372 0.1371 0.1369
0.1371
## 77 comps 78 comps 79 comps 80 comps 81 comps 82 comps 8
3 comps
## CV 0.1376 0.1375 0.1376 0.1377 0.1376 0.1370
0.1371
## adjCV 0.1371 0.1371 0.1371 0.1372 0.1371 0.1366
0.1367
## 84 comps 85 comps 86 comps 87 comps 88 comps 89 comps 9
0 comps
## CV 0.1373 0.1373 0.1374 0.1371 0.1371 0.1370
0.1369
## adjCV 0.1368 0.1369 0.1369 0.1367 0.1366 0.1365
0.1365
## 91 comps 92 comps 93 comps 94 comps 95 comps 96 comps 9
7 comps
## CV 0.1371 0.1369 0.1369 0.1370 0.1370 0.1371
0.1371
## adjCV 0.1366 0.1364 0.1364 0.1365 0.1365 0.1366
0.1366
## 98 comps 99 comps 100 comps
## CV 0.1373 0.1373 0.1373
## adjCV 0.1368 0.1367 0.1367
##
## TRAINING: % variance explained
## 1 comps 2 comps 3 comps 4 comps 5 comps 6 comps 7 comps 8
comps
## X 25.25 42.14 51.39 59.05 64.56 68.76 72.01
74.94
## y 39.70 47.07 50.53 54.83 55.27 55.62 57.54
63.09
## 9 comps 10 comps 11 comps 12 comps 13 comps 14 comps 15 com
ps
## X 77.01 78.59 80.08 81.49 82.84 83.85 84.
83
## y 63.09 63.21 63.47 63.95 63.96 64.07 64.
08
## 16 comps 17 comps 18 comps 19 comps 20 comps 21 comps 22 co
mps
## X 85.75 86.64 87.39 88.09 88.75 89.36 89.
94
## y 64.11 64.25 64.30 64.30 64.32 64.79 64.
96
## 23 comps 24 comps 25 comps 26 comps 27 comps 28 comps 29 co
mps
## X 90.47 90.99 91.51 91.99 92.45 92.89 93.
31
## y 65.00 65.15 65.15 65.18 65.19 65.21 65.
21
## 30 comps 31 comps 32 comps 33 comps 34 comps 35 comps 36 co
mps
## X 93.69 94.05 94.41 94.75 95.05 95.34 95.
61
## y 65.27 65.37 65.62 65.68 65.73 65.77 65.
78
## 37 comps 38 comps 39 comps 40 comps 41 comps 42 comps 43 co
mps
## X 95.86 96.10 96.34 96.55 96.76 96.96 97.
16
## y 65.79 65.79 66.07 66.07 66.20 66.23 66.
25
## 44 comps 45 comps 46 comps 47 comps 48 comps 49 comps 50 co
mps
## X 97.34 97.52 97.68 97.83 97.98 98.11 98.
24
## y 66.26 66.34 66.42 66.50 66.52 66.67 66.
69
## 51 comps 52 comps 53 comps 54 comps 55 comps 56 comps 57 co
mps
## X 98.35 98.46 98.57 98.66 98.75 98.83 98.
91
## y 66.76 66.85 66.85 66.85 66.97 66.97 67.
00
## 58 comps 59 comps 60 comps 61 comps 62 comps 63 comps 64 co
mps
## X 98.98 99.05 99.12 99.18 99.24 99.30 99.
35
## y 67.06 67.24 67.30 67.30 67.42 67.42 67.
48
## 65 comps 66 comps 67 comps 68 comps 69 comps 70 comps 71 co
mps
## X 99.39 99.44 99.48 99.52 99.56 99.60 99.
63
## y 67.56 67.73 67.86 67.98 67.98 68.14 68.
19
## 72 comps 73 comps 74 comps 75 comps 76 comps 77 comps 78 co
mps
## X 99.67 99.70 99.72 99.75 99.78 99.80 99.
82
## y 68.27 68.35 68.41 68.45 68.46 68.54 68.
55
## 79 comps 80 comps 81 comps 82 comps 83 comps 84 comps 85 co
mps
## X 99.84 99.86 99.88 99.90 99.91 99.92 99.
94
## y 68.63 68.64 68.80 68.94 68.94 68.98 68.
99
## 86 comps 87 comps 88 comps 89 comps 90 comps 91 comps 92 co
mps
## X 99.95 99.96 99.96 99.97 99.98 99.98 99.
99
## y 69.01 69.16 69.17 69.27 69.28 69.28 69.
38
## 93 comps 94 comps 95 comps 96 comps 97 comps 98 comps 99 co
mps
## X 99.99 99.99 99.99 100.00 100.00 100.00 100.
00
## y 69.40 69.40 69.45 69.47 69.51 69.51 69.
55
## 100 comps
## X 100.00
## y 69.58
validationplot(pcrfit,val.type = "MSEP")

Question 3.2 [1 pt]
Set nvmax to 50, load the caret library, set the random seed to 123. now call the
train function to train a pcr model. Ask for scaling, use dtrain for data, and ask for
10-fold cross-validation. use expand.grid to create a grid of values of ncomp that
goes from 1 to nvmax. Save the result of train to object pcrmodel.
library(caret)
## 载入需要的程辑包：ggplot2
## 载入需要的程辑包：lattice
##
## 载入程辑包：'caret'
## The following object is masked from 'package:pls':
##
## R2
nvmax=50
pcrmodle <- train(y~., data=dtrain, method="pcr", scale=TRUE, trControl
=trainControl(method="cv", number=10),
tuneGrid=expand.grid(ncomp=1:nvmax))
Question 3.3 [1 pt]
Plot pcrmodel.
# Plot model RMSE vs different values of components
plot(pcrmodle)

Extract the member ncomp of the member bestTune of the pcrmodel object and
copy this to ncompb. Print ncomb:
# Print the best tuning parameter ncomp that
# minimize the cross-validation error, RMSE
# best value of hyperparameter
ncompb <- pcrmodle$bestTune$ncomp
ncompb
## [1] 49
Question 3.4 [1 pt]
IN this question you need to write 4 lines of code. 1. we want to predict the value of
the model pcrfit with ncomp = 8 on testing set xtest. To do so, apply the predict
function to pcrfit, in an appropriate manner, and save the result to object pcrpred8.
2. plot ytest against pcrpred8 and give the title: Test using ncomp=8 3. use abline to
overlay the diagonal line y=x on the scatter plot 4. apply summary to the linear
regression (lm function) of prcrpred8 on ytest.
pcrpred8 <- predict(pcrfit, ncomp=8, newdata=xtest)
plot(ytest, pcrpred8, main="Test using ncomp=8")
abline(0,1)

summary(lm(pcrpred8~ytest))
##
## Call:
## lm(formula = pcrpred8 ~ ytest)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.36639 -0.07268 -0.00715 0.06784 0.42550
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.093734 0.007741 12.11 <2e-16 ***
## ytest 0.579313 0.022249 26.04 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.107 on 397 degrees of freedom
## Multiple R-squared: 0.6307, Adjusted R-squared: 0.6297
## F-statistic: 677.9 on 1 and 397 DF, p-value: < 2.2e-16
What is the value of R2 for the regression of pcrpred8 on ytest?
For the pcr model with 8 components, the R2 of the test prediction vs
ytest is?
0.6307
Repeat (4 lines of code) for ncomp=2 (name the prediction object pcrpred2 now)
pcrpred2 <- predict(pcrfit, ncomp=2, newdata=xtest)
plot(ytest, pcrpred2, main="Test using ncomp=2")
abline(0,1)
summary(lm(pcrpred2~ytest))
##
## Call:
## lm(formula = pcrpred2 ~ ytest)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.29001 -0.08074 -0.00769 0.06893 0.36609
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.129408 0.008102 15.97 <2e-16 ***
## ytest 0.432038 0.023287 18.55 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.112 on 397 degrees of freedom
## Multiple R-squared: 0.4644, Adjusted R-squared: 0.463
## F-statistic: 344.2 on 1 and 397 DF, p-value: < 2.2e-16
The value of R2 for ncomp=2 is?
0.4644
Question 3.5 [1 pt]
same question as 3.4 but now use ncomp=ncompb computed above. call the
prediction object pcrpredl. IN THE PLOT, use the title: ‘Test using the lowest CV
error’
pcrpred1 <- predict(pcrfit, ncomp=ncompb, newdata=xtest)
plot(ytest, pcrpred1, main="Test using the lowest CV error")
abline(0,1)

summary(lm(pcrpred1~ytest))
##
## Call:
## lm(formula = pcrpred1 ~ ytest)
##
## Residuals:
## Min 1Q Median 3Q Max
## -0.39345 -0.06453 -0.00578 0.05603 0.36885
##
## Coefficients:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.087182 0.007528 11.58 <2e-16 ***
## ytest 0.612081 0.021638 28.29 <2e-16 ***
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
##
## Residual standard error: 0.104 on 397 degrees of freedom
## Multiple R-squared: 0.6684, Adjusted R-squared: 0.6676
## F-statistic: 800.2 on 1 and 397 DF, p-value: < 2.2e-16
The value of R2 for ncomb is:?
0.6684
for which value of ncomp (2 or 8 or ncompb) did you find the highest va
lue of R2 of the pcrprediction vs ytest?
ncompb