STAT3450是一门统计学课程，主要涵盖了统计推断、假设检验、回归分析等内容。在这门课程中，学生将学习如何使用统计方法来分析数据和解决实际的统计问题。
Assignment 3 STAT3450: Questions [40 pts]
Quan Yuan B00923505
2023/2/27
Intro
Objectives
You will need to demonstrate that you can:
1. fit random forest and gradient boosted models for a classification problem
2. use caret to validate models
3. produce a ROC curve plot.
Notes
1. If you see in my instructions an expression such as rfmodel, ignore the *
characters (they are just used to highlight the text).
2. Use a dark theme (e.g.:global options>appearance>gob) to better visualize
this assignment in Rstudio. For knitting: use a default theme?.
This assignment contains two problems.
1. Problem-1: Phoneme classification (25 pts)
2. Problem-2: Caret training and ROC curve plot (15 pts)
Context and data
We will be using a dataframe of ‘sound features’ of phonemes.
Phonemes constitute the elementary acoustic component of spoken voice.
Several speakers were recorded and asked to pronounce one of 5 phonemes. The
goal here is to classify the recordings, based on preprocessed sound features
prepared in the dataframe.
This data set comes from the TIMIT database (TIMIT Acoustic-Phonetic Continuous
Speech Corpus, NTIS, US Dept of Commerce). It has been used by Tibshirani et al. in
their book Elements of Statistical Learning.
The raw dataframe has 4509 observations and 258 variables:
The 256 variables named x.1, x.2 etc. will be used as features or predictors (they are
obtained as log-periodograms of the acoustic signal in short time segments).
The speaker column identifies the speaker.
The g column identifies the phoneme class.
The phonemes are transcribed as follows:
“sh” as in “she”, “dcl” as in “dark”, “iy” as the vowel in “she”, “aa” as the vowel in
“dark”, and “ao” as the first vowel in “water”.
The columns g and speaker are two categorical attributes (not audio data).
Phoneme classification [25 pts]
QUESTION 1 [3 points]
Load the data eslphonemes.csv into a dataframe named p. Set the column X of p
(which has no meaning) to NULL in order to remove it. Print the number of rows
and the number of columns of p.
[1 pt]
p <- read.csv("eslphonemes.csv")
p$X=NULL
names(p)
## [1] "x.1" "x.2" "x.3" "x.4" "x.5" "x.6" "x.
7"
## [8] "x.8" "x.9" "x.10" "x.11" "x.12" "x.13" "x.
14"
## [15] "x.15" "x.16" "x.17" "x.18" "x.19" "x.20" "x.
21"
## [22] "x.22" "x.23" "x.24" "x.25" "x.26" "x.27" "x.
28"
## [29] "x.29" "x.30" "x.31" "x.32" "x.33" "x.34" "x.
35"
## [36] "x.36" "x.37" "x.38" "x.39" "x.40" "x.41" "x.
42"
## [43] "x.43" "x.44" "x.45" "x.46" "x.47" "x.48" "x.
49"
## [50] "x.50" "x.51" "x.52" "x.53" "x.54" "x.55" "x.
56"
## [57] "x.57" "x.58" "x.59" "x.60" "x.61" "x.62" "x.
63"
## [64] "x.64" "x.65" "x.66" "x.67" "x.68" "x.69" "x.
70"
## [71] "x.71" "x.72" "x.73" "x.74" "x.75" "x.76" "x.
77"
## [78] "x.78" "x.79" "x.80" "x.81" "x.82" "x.83" "x.
84"
## [85] "x.85" "x.86" "x.87" "x.88" "x.89" "x.90" "x.
91"
## [92] "x.92" "x.93" "x.94" "x.95" "x.96" "x.97" "x.
98"
## [99] "x.99" "x.100" "x.101" "x.102" "x.103" "x.104" "x.
105"
## [106] "x.106" "x.107" "x.108" "x.109" "x.110" "x.111" "x.
112"
## [113] "x.113" "x.114" "x.115" "x.116" "x.117" "x.118" "x.
119"
## [120] "x.120" "x.121" "x.122" "x.123" "x.124" "x.125" "x.
126"
## [127] "x.127" "x.128" "x.129" "x.130" "x.131" "x.132" "x.
133"
## [134] "x.134" "x.135" "x.136" "x.137" "x.138" "x.139" "x.
140"
## [141] "x.141" "x.142" "x.143" "x.144" "x.145" "x.146" "x.
147"
## [148] "x.148" "x.149" "x.150" "x.151" "x.152" "x.153" "x.
154"
## [155] "x.155" "x.156" "x.157" "x.158" "x.159" "x.160" "x.
161"
## [162] "x.162" "x.163" "x.164" "x.165" "x.166" "x.167" "x.
168"
## [169] "x.169" "x.170" "x.171" "x.172" "x.173" "x.174" "x.
175"
## [176] "x.176" "x.177" "x.178" "x.179" "x.180" "x.181" "x.
182"
## [183] "x.183" "x.184" "x.185" "x.186" "x.187" "x.188" "x.
189"
## [190] "x.190" "x.191" "x.192" "x.193" "x.194" "x.195" "x.
196"
## [197] "x.197" "x.198" "x.199" "x.200" "x.201" "x.202" "x.
203"
## [204] "x.204" "x.205" "x.206" "x.207" "x.208" "x.209" "x.
210"
## [211] "x.211" "x.212" "x.213" "x.214" "x.215" "x.216" "x.
217"
## [218] "x.218" "x.219" "x.220" "x.221" "x.222" "x.223" "x.
224"
## [225] "x.225" "x.226" "x.227" "x.228" "x.229" "x.230" "x.
231"
## [232] "x.232" "x.233" "x.234" "x.235" "x.236" "x.237" "x.
238"
## [239] "x.239" "x.240" "x.241" "x.242" "x.243" "x.244" "x.
245"
## [246] "x.246" "x.247" "x.248" "x.249" "x.250" "x.251" "x.
252"
## [253] "x.253" "x.254" "x.255" "x.256" "g" "speaker"
This dataset has a column named g which is the outcome of interest, i.e. the class
(name of the phoneme) of the sound that is pronounced.
Convert g to a factor. Print the levels of g.
Use the table function to print the frequencies of each class (aa ao dcl iy sh) found in
this dataframe.
[1 pt]
p$g <- as.factor(p$g)
levels(p$g)
## [1] "aa" "ao" "dcl" "iy" "sh"
table(p$g)
##
## aa ao dcl iy sh
## 695 1022 757 1163 872
The column speaker of dataframe p contains the name of the person whose voice is
recorded, for each row.
Each speaker has been recorded 10 times - or close to 10 times. One of the speakers
is called rain.dr7.mbom0.sa1.
Write 2 lines of R code to print the value of the phonemes (column g) spoken by this
speaker. To do this, you will first calculate the vector id1 of record numbers which
correspond to this speaker. After that, you simply can simply print the phonemes for
these rows.
Hints:
1. prepare a ‘keep’ array where you put the names of all the speakers (this can
be a single name) that you want to keep. Use the c operator for this.
2. You can calculate id1 by applying the which function to an expression with a
syntax similar to:
dataframe[ID %in% keep]
[1 point]
# compute keep
# use keep to compute id1
# use id1 to display the subset of g values belonging to this speaker
keep <- c("train.dr7.mbom0.sa1")
id1 <- p[p$speaker %in% keep,]
QUESTION 2 [2 pts]
In this question, we want to create a simpler dataframe and keep only two
phonemes so that we can treat the response as binary.
We will build a subset of the data that contains only the ao and aa phonemes, and use
it for binary classification.
Question: build a dataframe named d by selecting those rows of p whose g values fall
into the set of classes {ao,aa} (keep all the columns of p in d). Use the %in% operator
and the c operator. Do not forget to bracket the values of the phoneme class by
double quotes (e.g. “aa”).
Hint: syntax is ‘similar’ to d = p[ column g in the set (ao,aa) , ]
Here is a short example of use of the in operator:
uu<-c('a','b','c','a','d','b')
uu %in% c('a','b')
## [1] TRUE TRUE FALSE TRUE FALSE TRUE
# define d by subsetting the rows of p
d <- p[p$g %in% c("ao","aa"), ]
Now you should have a dataset d of records with only two classes of phonemes aa
and ao.
[Tabulate the g column of d for a check]
table(d$g)
##
## aa ao dcl iy sh
## 695 1022 0 0 0
Now we want to get rid of the speaker column. Write one line to remove this column
from dataframe d
# suppress the column speaker of d
d$speaker = NULL
Finally, we want to recode the g column of d to 1 if g is equal to phoneme ‘ao’ and to
0 otherwise. Use the ifelse function to do this.
IMPORTANT: remember that the ‘positive’ class here (i.e. the class recoded as 1) is
the phoneme ‘ao’ (sound of the vowel a in ‘water’ ). The sound of the a vowel as in
‘dark’ will be coded as 0.
# replace the g column of d
d$g = ifelse(d$g == "ao",1,0)
# convert d$g to a factor
d$g = as.factor(d$g)
# you can tabulate column g to make sure it contains 0's and 1's
table(d$g)
##
## 0 1
## 695 1022
IN the remaining questions problem, WE WILL USE THE DATASET d provided on BS
with the assignment:
d<-read.csv('d.csv')
d$g=as.factor(d$g)
This dataframe (d) will also be used in Problem 2. So make sure that you do not
accidentally remove it or overwrite it from now on.
QUESTION 3 [2 pts]
Use the sample function to build a random train/test split (using 90% of the data for
training and 10% for testing) for model validation.
The number of training example should be floor((number of rows of dataframe d) *
trainingproportion))
You will first calculate a vector trows containing the row numbers of d you want to
keep in the training set.
# keep this line
set.seed(322)
# compute the number n of rows of d
n <- nrow(d)
# compute the number nt of training records. keep a proportion of 90% f
or training
nt <- floor(0.9*n)
# draw nt random rows
trows <- sample(1:n, nt, replace = FALSE)
# use these rows to build your dtrain and dtest dataframes
dtrain <- d[trows, ]
dtest <- d[-trows, ]
QUESTION 4 [2 pts]
Fit a random forest model to the training set data to predict g. You will have the
library required for this model. Do NOT use caret here. Save your result to an object
called rfmodel1
#library(??)
library(randomForest)
# keep this line
set.seed(322)
# fit the random forest model
rfmodel1 = randomForest(g~., data=dtrain)
QUESTION 5 [2 pts]
Use the plot method to graph the random forest stabilization (Out-of-bag error
versus number of trees)
plot(rfmodel1)

QUESTION 6 [4 pts]
Use the predict function to compute the probabilities of each class predicted by the
model rfmodel1 for the records in the testing set. Use type=‘prob’. Save the results
of predict into an object called probability and use the head function to print the top
records of this object.
[1 pt /4 pts]
probability <- predict(rfmodel1, newdata = dtest, type = "prob")
head(probability)
## 0 1
## 35 0.048 0.952
## 37 0.786 0.214
## 43 0.110 0.890
## 45 0.844 0.156
## 51 0.068 0.932
## 66 0.934 0.066
[1 pt /4 pts]
Predict the probabilities of each class and classes on the test set
Same question, but now for predicted classes (not the probabilities of the responses
but the actual values 1 or 0). Use type=‘class’ and save the results to an object called
predicted.
predicted <- predict(rfmodel1, newdata = dtest, type = "class")
head(predicted)
## 35 37 43 45 51 66
## 1 0 1 0 1 0
## Levels: 0 1
Compute and print the confusion table named trf that compares the true outcome
for the test dataset (dtest)ℎℎ(ℎ ∗
∗ g)
[1 pt/ 2 pts]
trf <- table(predicted, dtest$g)
head(trf)
##
## predicted 0 1
## 0 43 11
## 1 22 96
Use trf to compute the prediction accuracy [1 pt / 2 pts]
# formula using trf that computes the accuracy = fraction of correctly
classified test items
(trf[1,1]+trf[2,2])/sum(trf)
## [1] 0.8081395
QUESTION 7 [4 pts]
Plot the importance of predictor variales for the model rfmodel1 [2 pts]
varImpPlot(rfmodel1)
Use the ggplot function (+geom_density, with an alpha value of 0.25) to plot overlay
the histograms of the x.41 predictor for each phoneme (use the arguments x= and
fill= in the aesthetics of ggplot). [2 pts]
library(ggplot2)
##
## 载入程辑包：'ggplot2'
## The following object is masked from 'package:randomForest':
##
## margin
ggplot(d,aes(x=x.41, fill = g))+geom_density(alpha = 0.25)

QUESTION 8 [2 pts]
We will repeat the classification exercise for the entire phoneme dataset (dataframe
p) (i.e. all phonemes are allowed).
Write one line of code tp nullify the speaker column of dataframe p.
d <- p
d$speaker = NULL
You can tabulate the column of g of p to check that you see all phonemes
#table(p$g)
table(p$g)
##
## aa ao dcl iy sh
## 695 1022 757 1163 872
Now you may recycle previous codes to answer the following questions
Prepare a train/test split (90% of data in training set) (see question)
CALL ptrain the training set and ptest the testing set.
These subsets of dataframe p will contain records belonging to all classe of
phonemes.
DO NOT USE the names dtrain and dtest here.
# keep this line
set.seed(322)
# compute the number of rows of d
n <- nrow(d)
# compute the number nt of training records keep 90%
nt <- floor(0.9*n)
# draw nt random rows
trows <- sample(1:n, nt, replace = FALSE)
# use these rows to build your ptrain and ptest dataframes
ptrain <- d[trows,]
ptest <- d[-trows,]
QUESTION 9 [2 pts]
Use the training data to fit a random forest model of the response g based on all
predictors. Save your model to an object called rfpall.
rfpall <- randomForest(g~., data = ptrain)
[0.5pt / 2 pts] Plot the OOB error rates of model rfpall as a function of the number of
trees in the model.
plot(rfpall)
[0.5pt / 2 pts]
Use the predict function to compute the probability predicted by model rfp for each
test observation and each class of response (use type=prob). Save this in an object
called probability. Use the head function to show the first rows probability.
probability <- predict(rfpall, newdata = ptest, type = "prob")
head(probability)
## aa ao dcl iy sh
## 3 0.000 0.000 1.000 0.000 0.000
## 15 0.020 0.024 0.014 0.936 0.006
## 28 0.140 0.860 0.000 0.000 0.000
## 37 0.076 0.922 0.000 0.002 0.000
## 50 0.002 0.002 0.002 0.984 0.010
## 53 0.012 0.008 0.010 0.962 0.008
Use the predict function to compute the class of phoneme predicted by model rfp for
each test observation (use type=class). Save this in an object called predicted. Use
the head function to show the first rows predicted.
[0.5 pt / 2 pts]
predicted <- predict(rfpall, newdata = ptest, type = "class")
head(predicted)
## 3 15 28 37 50 53
## dcl iy ao ao iy iy
## Levels: aa ao dcl iy sh
QUESTION 10 [2 pts]
Use table to compute the confusion table t which compares the predicted outcomes
(vector predicted) and the true outcome (column g) on the test dataset. Print the
confusion table t.
Use t to compute and print the classifier testing accuracy (percentage correct).
[1 pt / 2 pts]
# compute table t
t <- table(predicted, ptest$g)
# print it
print(t)
##
## predicted aa ao dcl iy sh
## aa 46 10 0 0 0
## ao 17 85 0 0 0
## dcl 0 0 88 0 0
## iy 0 0 1 115 0
## sh 0 0 1 0 88
# compute the accuracy
mean(predicted != ptest$g)
## [1] 0.06430155
[1 pt]
what is the pair of phonemes that is the most confused in the classification?
The pair: aa, ao
Caret training and ROC curve plot [15 pts]
In this problem, you will use the caret library to train a random forest and gradient
boosted models.
You will also use the ROCR library to compute the ROC curve of a random forest
classifier.
IN this problem, WE WILL USE THE DATASET d defined in question 2 of problem 1.
For safety use the dataset provided with the assignment:
d<-read.csv('d.csv')
d$g=as.factor(d$g)
QUESTION 1 [1 pt]
Load the caret library
library(caret)
## 载入需要的程辑包：lattice
Use the function createDataPartition of library caret to create a vector of row
numbers called trows that you wish to keep for your training set. You will want to
keep 90% of the full dataset for training. Use trows to define a training set
( dataframe dtrain) and a testing set (dataframe dtest). Keep all the columns of d in
your new dataframes.
# keep this line:
set.seed(322)
# create a testing and training set
trows <- createDataPartition(d$g, p = 0.9, list = FALSE)
dtrain <- d[trows,]
dtest <- d[-trows,]
QUESTION 2 [2 pts]
BASIC CV training
Use the function trainControl to define an object called tcontrol that will specify the
sampling method that you want to use for training. Choose the arguments of
trainControl so that training will perform a 10 fold cross-validation.
# specify that the resampling method is 10-fold CV
tcontrol = trainControl(method = "cv", number = 10, search = "grid")
Use the expand.grid function to specify 4 values of the mtry hyperparameter that
you want to use during training. Call the result of the expand.grid function rfgrid.
choose the following values for mtry
2 5 20 80
# define rfgrid
rfgrid <- expand.grid(mtry = c(2,5,20,80))
Call the function train to train a random forest model on the data dtrain with the
training control set to the object tcontrol. Save your results into an object called
rfmodel2.
Print the object rfmodel2.
7 minutes on my laptop.
# keep this line
set.seed(123)
# train a random forest model rfmodel2 with train
rfmodel2 <- train(g~., data = dtrain, method = "rf", trControl = tcontr
ol, tuneGrid = rfgrid)
# print the model result rfmodel2
print(rfmodel2)
## Random Forest
##
## 1546 samples
## 256 predictor
## 2 classes: '0', '1'
##
## No pre-processing
## Resampling: Cross-Validated (10 fold)
## Summary of sample sizes: 1391, 1391, 1392, 1392, 1391, 1392, ...
## Resampling results across tuning parameters:
##
## mtry Accuracy Kappa
## 2 0.8001257 0.5756646
## 5 0.8117721 0.6030824
## 20 0.8130708 0.6068592
## 80 0.8136950 0.6089072
##
## Accuracy was used to select the optimal model using the largest valu
e.
## The final value used for the model was mtry = 80.
Which value of mtry gives the highest cross-validated accuracy of prediction?
mtry=80
QUESTION 3 [4 pts]
In this question, we will fit gradient boosted trees models with the packages caret
and gbm.
Use the trainControl function of caret to request repeated cross-validation (2
repeats and 5-folds cross-validation). Save the value returned by trainControl to an
object called tcontrol. You will use later tcontrol as a value for the trControl
argument of function train.
[0.5 pt / 4 pts]
# define tcontrol to repeat 2 times a 5-folds cross-validation
tcontrol = trainControl(method = "repeatedcv", repeats = 2, number = 5,
search = "grid")
Then use the train function to fit gbm models (method=‘gbm’) to the training set
dtrain.
Save the results of train in an object called gbm1.
[1 pt / 4 pts]
# keep this line
set.seed(825)
# train the model gbm1
# print the model gbm1
gbm1 <- train(g~., data = dtrain, method = "gbm", trControl = tcontrol)
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2936 nan 0.1000 0.0270
## 2 1.2431 nan 0.1000 0.0224
## 3 1.2028 nan 0.1000 0.0182
## 4 1.1635 nan 0.1000 0.0185
## 5 1.1304 nan 0.1000 0.0137
## 6 1.1020 nan 0.1000 0.0117
## 7 1.0756 nan 0.1000 0.0115
## 8 1.0521 nan 0.1000 0.0099
## 9 1.0308 nan 0.1000 0.0099
## 10 1.0095 nan 0.1000 0.0099
## 20 0.8870 nan 0.1000 0.0025
## 40 0.8015 nan 0.1000 0.0001
## 60 0.7630 nan 0.1000 -0.0005
## 80 0.7261 nan 0.1000 0.0002
## 100 0.6948 nan 0.1000 -0.0001
## 120 0.6694 nan 0.1000 -0.0000
## 140 0.6470 nan 0.1000 -0.0000
## 150 0.6335 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2831 nan 0.1000 0.0280
## 2 1.2185 nan 0.1000 0.0321
## 3 1.1711 nan 0.1000 0.0220
## 4 1.1271 nan 0.1000 0.0211
## 5 1.0825 nan 0.1000 0.0199
## 6 1.0499 nan 0.1000 0.0135
## 7 1.0180 nan 0.1000 0.0128
## 8 0.9863 nan 0.1000 0.0122
## 9 0.9626 nan 0.1000 0.0092
## 10 0.9416 nan 0.1000 0.0084
## 20 0.8259 nan 0.1000 0.0020
## 40 0.7331 nan 0.1000 -0.0008
## 60 0.6636 nan 0.1000 -0.0007
## 80 0.6053 nan 0.1000 -0.0004
## 100 0.5605 nan 0.1000 -0.0011
## 120 0.5196 nan 0.1000 0.0000
## 140 0.4839 nan 0.1000 -0.0004
## 150 0.4703 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2756 nan 0.1000 0.0335
## 2 1.2085 nan 0.1000 0.0281
## 3 1.1553 nan 0.1000 0.0227
## 4 1.1073 nan 0.1000 0.0205
## 5 1.0627 nan 0.1000 0.0213
## 6 1.0283 nan 0.1000 0.0144
## 7 0.9995 nan 0.1000 0.0098
## 8 0.9702 nan 0.1000 0.0112
## 9 0.9462 nan 0.1000 0.0095
## 10 0.9255 nan 0.1000 0.0069
## 20 0.7956 nan 0.1000 0.0004
## 40 0.6731 nan 0.1000 0.0003
## 60 0.5878 nan 0.1000 -0.0002
## 80 0.5231 nan 0.1000 -0.0007
## 100 0.4689 nan 0.1000 -0.0008
## 120 0.4252 nan 0.1000 -0.0007
## 140 0.3829 nan 0.1000 -0.0005
## 150 0.3668 nan 0.1000 0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2915 nan 0.1000 0.0262
## 2 1.2440 nan 0.1000 0.0203
## 3 1.2042 nan 0.1000 0.0168
## 4 1.1696 nan 0.1000 0.0140
## 5 1.1352 nan 0.1000 0.0150
## 6 1.1029 nan 0.1000 0.0155
## 7 1.0739 nan 0.1000 0.0121
## 8 1.0501 nan 0.1000 0.0109
## 9 1.0266 nan 0.1000 0.0090
## 10 1.0058 nan 0.1000 0.0099
## 20 0.8797 nan 0.1000 0.0029
## 40 0.7922 nan 0.1000 -0.0007
## 60 0.7486 nan 0.1000 0.0000
## 80 0.7138 nan 0.1000 -0.0011
## 100 0.6869 nan 0.1000 -0.0010
## 120 0.6603 nan 0.1000 -0.0007
## 140 0.6348 nan 0.1000 -0.0005
## 150 0.6240 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2791 nan 0.1000 0.0332
## 2 1.2140 nan 0.1000 0.0313
## 3 1.1644 nan 0.1000 0.0213
## 4 1.1175 nan 0.1000 0.0216
## 5 1.0800 nan 0.1000 0.0183
## 6 1.0457 nan 0.1000 0.0137
## 7 1.0154 nan 0.1000 0.0130
## 8 0.9881 nan 0.1000 0.0116
## 9 0.9654 nan 0.1000 0.0087
## 10 0.9452 nan 0.1000 0.0082
## 20 0.8285 nan 0.1000 0.0025
## 40 0.7278 nan 0.1000 -0.0008
## 60 0.6552 nan 0.1000 -0.0003
## 80 0.6008 nan 0.1000 -0.0001
## 100 0.5607 nan 0.1000 -0.0013
## 120 0.5165 nan 0.1000 -0.0002
## 140 0.4822 nan 0.1000 -0.0006
## 150 0.4667 nan 0.1000 -0.0004
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2703 nan 0.1000 0.0373
## 2 1.2066 nan 0.1000 0.0271
## 3 1.1535 nan 0.1000 0.0236
## 4 1.1055 nan 0.1000 0.0209
## 5 1.0614 nan 0.1000 0.0193
## 6 1.0271 nan 0.1000 0.0150
## 7 0.9967 nan 0.1000 0.0115
## 8 0.9690 nan 0.1000 0.0111
## 9 0.9466 nan 0.1000 0.0094
## 10 0.9226 nan 0.1000 0.0099
## 20 0.7869 nan 0.1000 0.0004
## 40 0.6505 nan 0.1000 0.0004
## 60 0.5699 nan 0.1000 -0.0006
## 80 0.5000 nan 0.1000 -0.0012
## 100 0.4486 nan 0.1000 -0.0010
## 120 0.4041 nan 0.1000 -0.0007
## 140 0.3657 nan 0.1000 -0.0003
## 150 0.3483 nan 0.1000 -0.0008
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.3014 nan 0.1000 0.0232
## 2 1.2527 nan 0.1000 0.0231
## 3 1.2058 nan 0.1000 0.0206
## 4 1.1655 nan 0.1000 0.0179
## 5 1.1340 nan 0.1000 0.0128
## 6 1.1034 nan 0.1000 0.0136
## 7 1.0784 nan 0.1000 0.0103
## 8 1.0544 nan 0.1000 0.0100
## 9 1.0313 nan 0.1000 0.0108
## 10 1.0080 nan 0.1000 0.0095
## 20 0.8780 nan 0.1000 0.0019
## 40 0.7807 nan 0.1000 0.0013
## 60 0.7335 nan 0.1000 0.0002
## 80 0.6950 nan 0.1000 -0.0005
## 100 0.6654 nan 0.1000 -0.0011
## 120 0.6385 nan 0.1000 -0.0005
## 140 0.6098 nan 0.1000 0.0002
## 150 0.5985 nan 0.1000 -0.0008
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2847 nan 0.1000 0.0325
## 2 1.2267 nan 0.1000 0.0232
## 3 1.1732 nan 0.1000 0.0230
## 4 1.1288 nan 0.1000 0.0202
## 5 1.0880 nan 0.1000 0.0182
## 6 1.0522 nan 0.1000 0.0148
## 7 1.0202 nan 0.1000 0.0137
## 8 0.9901 nan 0.1000 0.0136
## 9 0.9632 nan 0.1000 0.0135
## 10 0.9418 nan 0.1000 0.0080
## 20 0.8132 nan 0.1000 0.0014
## 40 0.6998 nan 0.1000 0.0003
## 60 0.6266 nan 0.1000 -0.0001
## 80 0.5719 nan 0.1000 -0.0010
## 100 0.5266 nan 0.1000 -0.0003
## 120 0.4912 nan 0.1000 -0.0004
## 140 0.4563 nan 0.1000 -0.0004
## 150 0.4421 nan 0.1000 -0.0010
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2705 nan 0.1000 0.0366
## 2 1.2007 nan 0.1000 0.0315
## 3 1.1483 nan 0.1000 0.0216
## 4 1.1016 nan 0.1000 0.0201
## 5 1.0607 nan 0.1000 0.0174
## 6 1.0246 nan 0.1000 0.0166
## 7 0.9904 nan 0.1000 0.0156
## 8 0.9614 nan 0.1000 0.0111
## 9 0.9370 nan 0.1000 0.0087
## 10 0.9138 nan 0.1000 0.0093
## 20 0.7731 nan 0.1000 0.0022
## 40 0.6459 nan 0.1000 0.0002
## 60 0.5565 nan 0.1000 -0.0001
## 80 0.4956 nan 0.1000 -0.0003
## 100 0.4425 nan 0.1000 -0.0003
## 120 0.3996 nan 0.1000 -0.0000
## 140 0.3616 nan 0.1000 -0.0002
## 150 0.3439 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2919 nan 0.1000 0.0263
## 2 1.2420 nan 0.1000 0.0229
## 3 1.1969 nan 0.1000 0.0201
## 4 1.1588 nan 0.1000 0.0172
## 5 1.1231 nan 0.1000 0.0156
## 6 1.0896 nan 0.1000 0.0148
## 7 1.0618 nan 0.1000 0.0119
## 8 1.0353 nan 0.1000 0.0105
## 9 1.0130 nan 0.1000 0.0103
## 10 0.9927 nan 0.1000 0.0088
## 20 0.8640 nan 0.1000 0.0036
## 40 0.7712 nan 0.1000 0.0003
## 60 0.7286 nan 0.1000 -0.0005
## 80 0.6974 nan 0.1000 0.0001
## 100 0.6657 nan 0.1000 -0.0001
## 120 0.6416 nan 0.1000 -0.0006
## 140 0.6180 nan 0.1000 0.0000
## 150 0.6060 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2785 nan 0.1000 0.0347
## 2 1.2163 nan 0.1000 0.0267
## 3 1.1580 nan 0.1000 0.0263
## 4 1.1147 nan 0.1000 0.0190
## 5 1.0756 nan 0.1000 0.0169
## 6 1.0395 nan 0.1000 0.0165
## 7 1.0076 nan 0.1000 0.0143
## 8 0.9782 nan 0.1000 0.0122
## 9 0.9520 nan 0.1000 0.0114
## 10 0.9318 nan 0.1000 0.0089
## 20 0.8057 nan 0.1000 0.0019
## 40 0.7058 nan 0.1000 0.0003
## 60 0.6380 nan 0.1000 0.0001
## 80 0.5871 nan 0.1000 -0.0004
## 100 0.5466 nan 0.1000 -0.0006
## 120 0.5074 nan 0.1000 -0.0013
## 140 0.4730 nan 0.1000 -0.0002
## 150 0.4586 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2693 nan 0.1000 0.0379
## 2 1.2002 nan 0.1000 0.0285
## 3 1.1461 nan 0.1000 0.0230
## 4 1.0985 nan 0.1000 0.0209
## 5 1.0579 nan 0.1000 0.0168
## 6 1.0208 nan 0.1000 0.0155
## 7 0.9877 nan 0.1000 0.0134
## 8 0.9600 nan 0.1000 0.0120
## 9 0.9371 nan 0.1000 0.0082
## 10 0.9178 nan 0.1000 0.0065
## 20 0.7699 nan 0.1000 0.0016
## 40 0.6458 nan 0.1000 0.0003
## 60 0.5673 nan 0.1000 -0.0007
## 80 0.5052 nan 0.1000 -0.0008
## 100 0.4529 nan 0.1000 -0.0011
## 120 0.4104 nan 0.1000 -0.0005
## 140 0.3707 nan 0.1000 -0.0009
## 150 0.3507 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2957 nan 0.1000 0.0235
## 2 1.2491 nan 0.1000 0.0218
## 3 1.2050 nan 0.1000 0.0203
## 4 1.1659 nan 0.1000 0.0175
## 5 1.1324 nan 0.1000 0.0132
## 6 1.1031 nan 0.1000 0.0134
## 7 1.0790 nan 0.1000 0.0097
## 8 1.0558 nan 0.1000 0.0088
## 9 1.0329 nan 0.1000 0.0098
## 10 1.0147 nan 0.1000 0.0082
## 20 0.8912 nan 0.1000 0.0035
## 40 0.8075 nan 0.1000 0.0004
## 60 0.7625 nan 0.1000 -0.0001
## 80 0.7281 nan 0.1000 -0.0008
## 100 0.6993 nan 0.1000 -0.0013
## 120 0.6724 nan 0.1000 0.0001
## 140 0.6497 nan 0.1000 -0.0007
## 150 0.6398 nan 0.1000 -0.0006
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2797 nan 0.1000 0.0310
## 2 1.2211 nan 0.1000 0.0266
## 3 1.1709 nan 0.1000 0.0235
## 4 1.1298 nan 0.1000 0.0192
## 5 1.0933 nan 0.1000 0.0161
## 6 1.0596 nan 0.1000 0.0149
## 7 1.0284 nan 0.1000 0.0132
## 8 1.0040 nan 0.1000 0.0101
## 9 0.9837 nan 0.1000 0.0088
## 10 0.9629 nan 0.1000 0.0084
## 20 0.8409 nan 0.1000 0.0018
## 40 0.7334 nan 0.1000 0.0003
## 60 0.6618 nan 0.1000 -0.0004
## 80 0.6179 nan 0.1000 -0.0016
## 100 0.5718 nan 0.1000 -0.0004
## 120 0.5336 nan 0.1000 -0.0013
## 140 0.4988 nan 0.1000 -0.0006
## 150 0.4826 nan 0.1000 -0.0007
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2744 nan 0.1000 0.0352
## 2 1.2118 nan 0.1000 0.0298
## 3 1.1543 nan 0.1000 0.0245
## 4 1.1065 nan 0.1000 0.0207
## 5 1.0677 nan 0.1000 0.0175
## 6 1.0341 nan 0.1000 0.0158
## 7 0.9999 nan 0.1000 0.0141
## 8 0.9743 nan 0.1000 0.0099
## 9 0.9501 nan 0.1000 0.0091
## 10 0.9320 nan 0.1000 0.0061
## 20 0.7926 nan 0.1000 0.0024
## 40 0.6601 nan 0.1000 -0.0005
## 60 0.5800 nan 0.1000 -0.0014
## 80 0.5156 nan 0.1000 -0.0012
## 100 0.4634 nan 0.1000 -0.0010
## 120 0.4187 nan 0.1000 -0.0006
## 140 0.3822 nan 0.1000 -0.0008
## 150 0.3654 nan 0.1000 -0.0006
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2910 nan 0.1000 0.0280
## 2 1.2427 nan 0.1000 0.0208
## 3 1.1975 nan 0.1000 0.0197
## 4 1.1630 nan 0.1000 0.0137
## 5 1.1274 nan 0.1000 0.0160
## 6 1.0968 nan 0.1000 0.0133
## 7 1.0684 nan 0.1000 0.0125
## 8 1.0466 nan 0.1000 0.0098
## 9 1.0230 nan 0.1000 0.0101
## 10 1.0041 nan 0.1000 0.0089
## 20 0.8781 nan 0.1000 0.0027
## 40 0.7870 nan 0.1000 0.0003
## 60 0.7460 nan 0.1000 -0.0003
## 80 0.7137 nan 0.1000 -0.0000
## 100 0.6845 nan 0.1000 -0.0005
## 120 0.6574 nan 0.1000 -0.0002
## 140 0.6329 nan 0.1000 -0.0004
## 150 0.6212 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2797 nan 0.1000 0.0317
## 2 1.2207 nan 0.1000 0.0287
## 3 1.1741 nan 0.1000 0.0203
## 4 1.1269 nan 0.1000 0.0219
## 5 1.0926 nan 0.1000 0.0123
## 6 1.0561 nan 0.1000 0.0151
## 7 1.0267 nan 0.1000 0.0119
## 8 1.0005 nan 0.1000 0.0113
## 9 0.9758 nan 0.1000 0.0105
## 10 0.9527 nan 0.1000 0.0097
## 20 0.8254 nan 0.1000 0.0029
## 40 0.7181 nan 0.1000 -0.0014
## 60 0.6465 nan 0.1000 0.0003
## 80 0.5877 nan 0.1000 -0.0000
## 100 0.5433 nan 0.1000 -0.0010
## 120 0.5054 nan 0.1000 -0.0009
## 140 0.4719 nan 0.1000 -0.0003
## 150 0.4548 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2737 nan 0.1000 0.0375
## 2 1.2042 nan 0.1000 0.0323
## 3 1.1495 nan 0.1000 0.0255
## 4 1.1022 nan 0.1000 0.0221
## 5 1.0600 nan 0.1000 0.0164
## 6 1.0250 nan 0.1000 0.0149
## 7 0.9929 nan 0.1000 0.0140
## 8 0.9686 nan 0.1000 0.0096
## 9 0.9447 nan 0.1000 0.0078
## 10 0.9236 nan 0.1000 0.0085
## 20 0.7829 nan 0.1000 0.0010
## 40 0.6468 nan 0.1000 -0.0006
## 60 0.5628 nan 0.1000 -0.0015
## 80 0.5020 nan 0.1000 -0.0006
## 100 0.4489 nan 0.1000 -0.0005
## 120 0.4054 nan 0.1000 -0.0009
## 140 0.3701 nan 0.1000 -0.0018
## 150 0.3537 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2972 nan 0.1000 0.0245
## 2 1.2499 nan 0.1000 0.0214
## 3 1.2053 nan 0.1000 0.0190
## 4 1.1659 nan 0.1000 0.0175
## 5 1.1282 nan 0.1000 0.0158
## 6 1.0999 nan 0.1000 0.0117
## 7 1.0742 nan 0.1000 0.0108
## 8 1.0502 nan 0.1000 0.0105
## 9 1.0280 nan 0.1000 0.0098
## 10 1.0072 nan 0.1000 0.0094
## 20 0.8856 nan 0.1000 0.0022
## 40 0.8017 nan 0.1000 0.0009
## 60 0.7532 nan 0.1000 -0.0003
## 80 0.7145 nan 0.1000 0.0000
## 100 0.6846 nan 0.1000 0.0003
## 120 0.6588 nan 0.1000 -0.0010
## 140 0.6367 nan 0.1000 -0.0005
## 150 0.6241 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2832 nan 0.1000 0.0316
## 2 1.2274 nan 0.1000 0.0272
## 3 1.1808 nan 0.1000 0.0214
## 4 1.1383 nan 0.1000 0.0175
## 5 1.1034 nan 0.1000 0.0166
## 6 1.0658 nan 0.1000 0.0171
## 7 1.0333 nan 0.1000 0.0134
## 8 1.0041 nan 0.1000 0.0119
## 9 0.9799 nan 0.1000 0.0108
## 10 0.9572 nan 0.1000 0.0098
## 20 0.8326 nan 0.1000 0.0014
## 40 0.7244 nan 0.1000 0.0021
## 60 0.6551 nan 0.1000 0.0002
## 80 0.6031 nan 0.1000 -0.0019
## 100 0.5529 nan 0.1000 0.0006
## 120 0.5166 nan 0.1000 -0.0015
## 140 0.4794 nan 0.1000 0.0002
## 150 0.4619 nan 0.1000 -0.0006
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2736 nan 0.1000 0.0349
## 2 1.2064 nan 0.1000 0.0288
## 3 1.1513 nan 0.1000 0.0239
## 4 1.1058 nan 0.1000 0.0188
## 5 1.0657 nan 0.1000 0.0159
## 6 1.0301 nan 0.1000 0.0149
## 7 0.9978 nan 0.1000 0.0134
## 8 0.9713 nan 0.1000 0.0102
## 9 0.9482 nan 0.1000 0.0090
## 10 0.9256 nan 0.1000 0.0097
## 20 0.7854 nan 0.1000 0.0023
## 40 0.6518 nan 0.1000 0.0004
## 60 0.5723 nan 0.1000 -0.0015
## 80 0.5100 nan 0.1000 0.0002
## 100 0.4582 nan 0.1000 -0.0007
## 120 0.4095 nan 0.1000 -0.0001
## 140 0.3714 nan 0.1000 -0.0003
## 150 0.3537 nan 0.1000 -0.0006
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2995 nan 0.1000 0.0218
## 2 1.2505 nan 0.1000 0.0220
## 3 1.2077 nan 0.1000 0.0178
## 4 1.1712 nan 0.1000 0.0176
## 5 1.1396 nan 0.1000 0.0144
## 6 1.1058 nan 0.1000 0.0143
## 7 1.0795 nan 0.1000 0.0120
## 8 1.0553 nan 0.1000 0.0111
## 9 1.0330 nan 0.1000 0.0099
## 10 1.0145 nan 0.1000 0.0079
## 20 0.8929 nan 0.1000 0.0022
## 40 0.8130 nan 0.1000 0.0011
## 60 0.7682 nan 0.1000 0.0002
## 80 0.7357 nan 0.1000 -0.0003
## 100 0.7059 nan 0.1000 -0.0002
## 120 0.6787 nan 0.1000 -0.0012
## 140 0.6588 nan 0.1000 -0.0009
## 150 0.6495 nan 0.1000 -0.0007
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2893 nan 0.1000 0.0285
## 2 1.2361 nan 0.1000 0.0234
## 3 1.1855 nan 0.1000 0.0245
## 4 1.1344 nan 0.1000 0.0232
## 5 1.0969 nan 0.1000 0.0155
## 6 1.0641 nan 0.1000 0.0159
## 7 1.0338 nan 0.1000 0.0139
## 8 1.0089 nan 0.1000 0.0103
## 9 0.9818 nan 0.1000 0.0112
## 10 0.9643 nan 0.1000 0.0076
## 20 0.8446 nan 0.1000 0.0027
## 40 0.7453 nan 0.1000 -0.0009
## 60 0.6782 nan 0.1000 -0.0002
## 80 0.6300 nan 0.1000 -0.0006
## 100 0.5787 nan 0.1000 -0.0005
## 120 0.5429 nan 0.1000 0.0000
## 140 0.5054 nan 0.1000 -0.0003
## 150 0.4876 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2752 nan 0.1000 0.0339
## 2 1.2142 nan 0.1000 0.0282
## 3 1.1612 nan 0.1000 0.0230
## 4 1.1108 nan 0.1000 0.0227
## 5 1.0709 nan 0.1000 0.0179
## 6 1.0380 nan 0.1000 0.0140
## 7 1.0059 nan 0.1000 0.0143
## 8 0.9806 nan 0.1000 0.0101
## 9 0.9583 nan 0.1000 0.0081
## 10 0.9367 nan 0.1000 0.0091
## 20 0.8089 nan 0.1000 0.0034
## 40 0.6792 nan 0.1000 -0.0007
## 60 0.6029 nan 0.1000 0.0008
## 80 0.5403 nan 0.1000 -0.0014
## 100 0.4846 nan 0.1000 -0.0008
## 120 0.4363 nan 0.1000 0.0001
## 140 0.3941 nan 0.1000 -0.0008
## 150 0.3751 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2921 nan 0.1000 0.0263
## 2 1.2452 nan 0.1000 0.0222
## 3 1.2020 nan 0.1000 0.0214
## 4 1.1640 nan 0.1000 0.0172
## 5 1.1316 nan 0.1000 0.0149
## 6 1.1004 nan 0.1000 0.0127
## 7 1.0696 nan 0.1000 0.0133
## 8 1.0449 nan 0.1000 0.0116
## 9 1.0170 nan 0.1000 0.0108
## 10 0.9976 nan 0.1000 0.0081
## 20 0.8681 nan 0.1000 0.0015
## 40 0.7709 nan 0.1000 -0.0004
## 60 0.7228 nan 0.1000 -0.0003
## 80 0.6876 nan 0.1000 -0.0006
## 100 0.6587 nan 0.1000 -0.0001
## 120 0.6323 nan 0.1000 -0.0004
## 140 0.6068 nan 0.1000 -0.0010
## 150 0.5964 nan 0.1000 -0.0009
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2749 nan 0.1000 0.0356
## 2 1.2147 nan 0.1000 0.0298
## 3 1.1612 nan 0.1000 0.0255
## 4 1.1134 nan 0.1000 0.0212
## 5 1.0763 nan 0.1000 0.0183
## 6 1.0436 nan 0.1000 0.0140
## 7 1.0137 nan 0.1000 0.0132
## 8 0.9885 nan 0.1000 0.0107
## 9 0.9652 nan 0.1000 0.0095
## 10 0.9438 nan 0.1000 0.0075
## 20 0.8100 nan 0.1000 0.0023
## 40 0.7044 nan 0.1000 0.0010
## 60 0.6327 nan 0.1000 -0.0003
## 80 0.5818 nan 0.1000 -0.0004
## 100 0.5379 nan 0.1000 -0.0006
## 120 0.5009 nan 0.1000 -0.0003
## 140 0.4640 nan 0.1000 -0.0004
## 150 0.4494 nan 0.1000 -0.0008
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2721 nan 0.1000 0.0340
## 2 1.2026 nan 0.1000 0.0321
## 3 1.1392 nan 0.1000 0.0277
## 4 1.0892 nan 0.1000 0.0204
## 5 1.0452 nan 0.1000 0.0193
## 6 1.0106 nan 0.1000 0.0175
## 7 0.9803 nan 0.1000 0.0143
## 8 0.9546 nan 0.1000 0.0120
## 9 0.9283 nan 0.1000 0.0085
## 10 0.9062 nan 0.1000 0.0080
## 20 0.7720 nan 0.1000 0.0023
## 40 0.6366 nan 0.1000 0.0010
## 60 0.5561 nan 0.1000 -0.0007
## 80 0.4925 nan 0.1000 -0.0002
## 100 0.4422 nan 0.1000 0.0002
## 120 0.3963 nan 0.1000 -0.0010
## 140 0.3629 nan 0.1000 -0.0007
## 150 0.3462 nan 0.1000 -0.0007
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2969 nan 0.1000 0.0235
## 2 1.2451 nan 0.1000 0.0232
## 3 1.2022 nan 0.1000 0.0186
## 4 1.1623 nan 0.1000 0.0181
## 5 1.1283 nan 0.1000 0.0151
## 6 1.0965 nan 0.1000 0.0140
## 7 1.0690 nan 0.1000 0.0126
## 8 1.0439 nan 0.1000 0.0119
## 9 1.0188 nan 0.1000 0.0112
## 10 0.9979 nan 0.1000 0.0069
## 20 0.8712 nan 0.1000 0.0029
## 40 0.7810 nan 0.1000 0.0006
## 60 0.7388 nan 0.1000 -0.0002
## 80 0.7063 nan 0.1000 -0.0011
## 100 0.6759 nan 0.1000 -0.0011
## 120 0.6537 nan 0.1000 -0.0004
## 140 0.6318 nan 0.1000 -0.0005
## 150 0.6201 nan 0.1000 -0.0013
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2842 nan 0.1000 0.0304
## 2 1.2228 nan 0.1000 0.0254
## 3 1.1711 nan 0.1000 0.0237
## 4 1.1255 nan 0.1000 0.0213
## 5 1.0837 nan 0.1000 0.0196
## 6 1.0461 nan 0.1000 0.0167
## 7 1.0202 nan 0.1000 0.0103
## 8 0.9908 nan 0.1000 0.0122
## 9 0.9652 nan 0.1000 0.0101
## 10 0.9429 nan 0.1000 0.0093
## 20 0.8147 nan 0.1000 0.0010
## 40 0.7131 nan 0.1000 -0.0009
## 60 0.6470 nan 0.1000 -0.0009
## 80 0.5972 nan 0.1000 -0.0007
## 100 0.5534 nan 0.1000 -0.0003
## 120 0.5141 nan 0.1000 -0.0006
## 140 0.4815 nan 0.1000 -0.0001
## 150 0.4635 nan 0.1000 -0.0004
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2685 nan 0.1000 0.0372
## 2 1.1984 nan 0.1000 0.0302
## 3 1.1428 nan 0.1000 0.0256
## 4 1.0984 nan 0.1000 0.0183
## 5 1.0585 nan 0.1000 0.0174
## 6 1.0211 nan 0.1000 0.0160
## 7 0.9916 nan 0.1000 0.0112
## 8 0.9684 nan 0.1000 0.0091
## 9 0.9410 nan 0.1000 0.0117
## 10 0.9170 nan 0.1000 0.0099
## 20 0.7800 nan 0.1000 -0.0004
## 40 0.6563 nan 0.1000 -0.0006
## 60 0.5742 nan 0.1000 -0.0004
## 80 0.5103 nan 0.1000 -0.0004
## 100 0.4563 nan 0.1000 -0.0013
## 120 0.4053 nan 0.1000 -0.0001
## 140 0.3647 nan 0.1000 -0.0001
## 150 0.3458 nan 0.1000 -0.0011
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2799 nan 0.1000 0.0323
## 2 1.2211 nan 0.1000 0.0294
## 3 1.1706 nan 0.1000 0.0237
## 4 1.1253 nan 0.1000 0.0196
## 5 1.0892 nan 0.1000 0.0160
## 6 1.0528 nan 0.1000 0.0165
## 7 1.0241 nan 0.1000 0.0120
## 8 0.9983 nan 0.1000 0.0116
## 9 0.9730 nan 0.1000 0.0111
## 10 0.9532 nan 0.1000 0.0081
## 20 0.8346 nan 0.1000 0.0015
## 40 0.7388 nan 0.1000 -0.0004
## 60 0.6755 nan 0.1000 -0.0004
## 80 0.6253 nan 0.1000 -0.0003
## 100 0.5817 nan 0.1000 -0.0003
print(gbm1)
## Stochastic Gradient Boosting
##
## 1546 samples
## 256 predictor
## 2 classes: '0', '1'
##
## No pre-processing
## Resampling: Cross-Validated (5 fold, repeated 2 times)
## Summary of sample sizes: 1237, 1237, 1236, 1237, 1237, 1237, ...
## Resampling results across tuning parameters:
##
## interaction.depth n.trees Accuracy Kappa
## 1 50 0.8056342 0.5900303
## 1 100 0.8078964 0.5953983
## 1 150 0.8133960 0.6077585
## 2 50 0.8137123 0.6076946
## 2 100 0.8188892 0.6191838
## 2 150 0.8150120 0.6103993
## 3 50 0.8117768 0.6043601
## 3 100 0.8146873 0.6106935
## 3 150 0.8163086 0.6138186
##
## Tuning parameter 'shrinkage' was held constant at a value of 0.1
##
## Tuning parameter 'n.minobsinnode' was held constant at a value of 10
## Accuracy was used to select the optimal model using the largest valu
e.
## The final values used for the model were n.trees = 100, interaction.
depth =
## 2, shrinkage = 0.1 and n.minobsinnode = 10.
[0.5 pt / 4 pts] Write the values of the best (highest accuracy) hyperparameters
below each corresponding hyperparameter name of gbm: :
interaction.depth n.trees Accuracy

2 100 0.8188892
Use expand.grid to define a grid of hyperparameters, named gbmgrid, that will
request the values 1,5,9 for the interaction.depth parameter, the values 50,150,300
for the n.trees hyperparameter, the value 0.1 for the shrinkage hyperparameter and
the value 10 for the n.minobsinnode hyperparameter.
[0.5 pt / 4 pts]
# define gbmgrid
gbmgrid <- expand.grid(interaction.depth = c(1,5,9), n.trees = c(50,150,
300), shrinkage = 0.1, n.minobsinnode = 10)
Use the train function of caret (and the tcontrol and gbmgrid objects) to train a gbm
model. Save the results to an object called gbm2.
[1. pt / 4 pts]
# keep this line
set.seed(825)
# gbm2 <- ...
# print gbm2
gbm2 <- train(g~., data = dtrain, method = "gbm", trControl = tcontrol,
tuneGrid = gbmgrid)
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2936 nan 0.1000 0.0270
## 2 1.2431 nan 0.1000 0.0224
## 3 1.2028 nan 0.1000 0.0182
## 4 1.1635 nan 0.1000 0.0185
## 5 1.1304 nan 0.1000 0.0137
## 6 1.1020 nan 0.1000 0.0117
## 7 1.0756 nan 0.1000 0.0115
## 8 1.0521 nan 0.1000 0.0099
## 9 1.0308 nan 0.1000 0.0099
## 10 1.0095 nan 0.1000 0.0099
## 20 0.8870 nan 0.1000 0.0025
## 40 0.8015 nan 0.1000 0.0001
## 60 0.7630 nan 0.1000 -0.0005
## 80 0.7261 nan 0.1000 0.0002
## 100 0.6948 nan 0.1000 -0.0001
## 120 0.6694 nan 0.1000 -0.0000
## 140 0.6470 nan 0.1000 -0.0000
## 160 0.6240 nan 0.1000 -0.0007
## 180 0.6047 nan 0.1000 -0.0004
## 200 0.5852 nan 0.1000 -0.0006
## 220 0.5663 nan 0.1000 -0.0004
## 240 0.5514 nan 0.1000 0.0001
## 260 0.5360 nan 0.1000 -0.0001
## 280 0.5196 nan 0.1000 -0.0005
## 300 0.5055 nan 0.1000 -0.0007
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2672 nan 0.1000 0.0348
## 2 1.1982 nan 0.1000 0.0316
## 3 1.1421 nan 0.1000 0.0232
## 4 1.0888 nan 0.1000 0.0247
## 5 1.0408 nan 0.1000 0.0198
## 6 1.0027 nan 0.1000 0.0150
## 7 0.9691 nan 0.1000 0.0118
## 8 0.9365 nan 0.1000 0.0108
## 9 0.9077 nan 0.1000 0.0101
## 10 0.8823 nan 0.1000 0.0084
## 20 0.7308 nan 0.1000 0.0019
## 40 0.5752 nan 0.1000 -0.0003
## 60 0.4689 nan 0.1000 -0.0001
## 80 0.3900 nan 0.1000 -0.0009
## 100 0.3299 nan 0.1000 -0.0008
## 120 0.2811 nan 0.1000 -0.0002
## 140 0.2400 nan 0.1000 -0.0001
## 160 0.2070 nan 0.1000 -0.0006
## 180 0.1786 nan 0.1000 -0.0004
## 200 0.1567 nan 0.1000 -0.0003
## 220 0.1366 nan 0.1000 -0.0003
## 240 0.1187 nan 0.1000 -0.0004
## 260 0.1036 nan 0.1000 -0.0003
## 280 0.0918 nan 0.1000 -0.0002
## 300 0.0812 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2601 nan 0.1000 0.0356
## 2 1.1798 nan 0.1000 0.0299
## 3 1.1118 nan 0.1000 0.0273
## 4 1.0550 nan 0.1000 0.0212
## 5 1.0038 nan 0.1000 0.0191
## 6 0.9578 nan 0.1000 0.0143
## 7 0.9186 nan 0.1000 0.0117
## 8 0.8833 nan 0.1000 0.0107
## 9 0.8504 nan 0.1000 0.0102
## 10 0.8217 nan 0.1000 0.0077
## 20 0.6314 nan 0.1000 0.0008
## 40 0.4341 nan 0.1000 -0.0011
## 60 0.3159 nan 0.1000 -0.0004
## 80 0.2344 nan 0.1000 -0.0001
## 100 0.1795 nan 0.1000 -0.0004
## 120 0.1387 nan 0.1000 -0.0003
## 140 0.1096 nan 0.1000 -0.0003
## 160 0.0871 nan 0.1000 -0.0001
## 180 0.0686 nan 0.1000 -0.0002
## 200 0.0553 nan 0.1000 -0.0001
## 220 0.0439 nan 0.1000 -0.0001
## 240 0.0349 nan 0.1000 -0.0001
## 260 0.0280 nan 0.1000 -0.0001
## 280 0.0224 nan 0.1000 -0.0000
## 300 0.0181 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2915 nan 0.1000 0.0262
## 2 1.2440 nan 0.1000 0.0203
## 3 1.2042 nan 0.1000 0.0168
## 4 1.1696 nan 0.1000 0.0140
## 5 1.1352 nan 0.1000 0.0150
## 6 1.1029 nan 0.1000 0.0155
## 7 1.0739 nan 0.1000 0.0121
## 8 1.0501 nan 0.1000 0.0109
## 9 1.0266 nan 0.1000 0.0090
## 10 1.0058 nan 0.1000 0.0099
## 20 0.8797 nan 0.1000 0.0029
## 40 0.7922 nan 0.1000 -0.0007
## 60 0.7486 nan 0.1000 0.0000
## 80 0.7138 nan 0.1000 -0.0011
## 100 0.6869 nan 0.1000 -0.0010
## 120 0.6603 nan 0.1000 -0.0007
## 140 0.6348 nan 0.1000 -0.0005
## 160 0.6117 nan 0.1000 -0.0004
## 180 0.5930 nan 0.1000 -0.0004
## 200 0.5725 nan 0.1000 -0.0000
## 220 0.5561 nan 0.1000 -0.0005
## 240 0.5381 nan 0.1000 -0.0006
## 260 0.5201 nan 0.1000 -0.0002
## 280 0.5033 nan 0.1000 -0.0000
## 300 0.4894 nan 0.1000 -0.0004
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2654 nan 0.1000 0.0384
## 2 1.1957 nan 0.1000 0.0305
## 3 1.1349 nan 0.1000 0.0250
## 4 1.0827 nan 0.1000 0.0213
## 5 1.0400 nan 0.1000 0.0175
## 6 0.9976 nan 0.1000 0.0166
## 7 0.9626 nan 0.1000 0.0148
## 8 0.9283 nan 0.1000 0.0137
## 9 0.8991 nan 0.1000 0.0097
## 10 0.8755 nan 0.1000 0.0064
## 20 0.7183 nan 0.1000 0.0031
## 40 0.5624 nan 0.1000 0.0005
## 60 0.4583 nan 0.1000 -0.0013
## 80 0.3850 nan 0.1000 -0.0007
## 100 0.3274 nan 0.1000 -0.0005
## 120 0.2772 nan 0.1000 -0.0008
## 140 0.2394 nan 0.1000 -0.0001
## 160 0.2081 nan 0.1000 -0.0005
## 180 0.1820 nan 0.1000 -0.0002
## 200 0.1574 nan 0.1000 -0.0004
## 220 0.1366 nan 0.1000 -0.0001
## 240 0.1199 nan 0.1000 -0.0002
## 260 0.1041 nan 0.1000 -0.0001
## 280 0.0914 nan 0.1000 -0.0002
## 300 0.0806 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2494 nan 0.1000 0.0428
## 2 1.1698 nan 0.1000 0.0306
## 3 1.1051 nan 0.1000 0.0259
## 4 1.0498 nan 0.1000 0.0207
## 5 0.9985 nan 0.1000 0.0188
## 6 0.9569 nan 0.1000 0.0140
## 7 0.9160 nan 0.1000 0.0137
## 8 0.8820 nan 0.1000 0.0110
## 9 0.8511 nan 0.1000 0.0099
## 10 0.8201 nan 0.1000 0.0087
## 20 0.6248 nan 0.1000 0.0007
## 40 0.4254 nan 0.1000 -0.0002
## 60 0.3078 nan 0.1000 0.0001
## 80 0.2332 nan 0.1000 -0.0006
## 100 0.1800 nan 0.1000 -0.0003
## 120 0.1393 nan 0.1000 -0.0006
## 140 0.1109 nan 0.1000 -0.0002
## 160 0.0863 nan 0.1000 0.0000
## 180 0.0682 nan 0.1000 -0.0000
## 200 0.0542 nan 0.1000 0.0000
## 220 0.0428 nan 0.1000 -0.0001
## 240 0.0346 nan 0.1000 -0.0001
## 260 0.0279 nan 0.1000 -0.0001
## 280 0.0224 nan 0.1000 -0.0001
## 300 0.0178 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.3014 nan 0.1000 0.0232
## 2 1.2527 nan 0.1000 0.0231
## 3 1.2058 nan 0.1000 0.0206
## 4 1.1655 nan 0.1000 0.0179
## 5 1.1340 nan 0.1000 0.0128
## 6 1.1034 nan 0.1000 0.0136
## 7 1.0784 nan 0.1000 0.0103
## 8 1.0544 nan 0.1000 0.0100
## 9 1.0313 nan 0.1000 0.0108
## 10 1.0080 nan 0.1000 0.0095
## 20 0.8780 nan 0.1000 0.0019
## 40 0.7807 nan 0.1000 0.0013
## 60 0.7335 nan 0.1000 0.0002
## 80 0.6950 nan 0.1000 -0.0005
## 100 0.6654 nan 0.1000 -0.0011
## 120 0.6385 nan 0.1000 -0.0005
## 140 0.6098 nan 0.1000 0.0002
## 160 0.5881 nan 0.1000 -0.0011
## 180 0.5681 nan 0.1000 -0.0008
## 200 0.5483 nan 0.1000 -0.0006
## 220 0.5312 nan 0.1000 -0.0003
## 240 0.5143 nan 0.1000 -0.0007
## 260 0.4978 nan 0.1000 -0.0006
## 280 0.4829 nan 0.1000 -0.0003
## 300 0.4690 nan 0.1000 -0.0003
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2705 nan 0.1000 0.0367
## 2 1.1959 nan 0.1000 0.0305
## 3 1.1302 nan 0.1000 0.0260
## 4 1.0805 nan 0.1000 0.0213
## 5 1.0342 nan 0.1000 0.0186
## 6 0.9920 nan 0.1000 0.0158
## 7 0.9537 nan 0.1000 0.0148
## 8 0.9222 nan 0.1000 0.0127
## 9 0.8950 nan 0.1000 0.0107
## 10 0.8684 nan 0.1000 0.0087
## 20 0.7021 nan 0.1000 -0.0004
## 40 0.5388 nan 0.1000 -0.0007
## 60 0.4380 nan 0.1000 -0.0007
## 80 0.3639 nan 0.1000 0.0001
## 100 0.3063 nan 0.1000 -0.0002
## 120 0.2609 nan 0.1000 -0.0006
## 140 0.2243 nan 0.1000 -0.0006
## 160 0.1926 nan 0.1000 -0.0004
## 180 0.1675 nan 0.1000 -0.0001
## 200 0.1478 nan 0.1000 -0.0003
## 220 0.1292 nan 0.1000 -0.0002
## 240 0.1124 nan 0.1000 -0.0001
## 260 0.0988 nan 0.1000 -0.0001
## 280 0.0868 nan 0.1000 -0.0002
## 300 0.0763 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2506 nan 0.1000 0.0428
## 2 1.1706 nan 0.1000 0.0310
## 3 1.1033 nan 0.1000 0.0251
## 4 1.0466 nan 0.1000 0.0185
## 5 0.9953 nan 0.1000 0.0168
## 6 0.9499 nan 0.1000 0.0173
## 7 0.9061 nan 0.1000 0.0159
## 8 0.8693 nan 0.1000 0.0114
## 9 0.8332 nan 0.1000 0.0109
## 10 0.7992 nan 0.1000 0.0101
## 20 0.6006 nan 0.1000 0.0014
## 40 0.4097 nan 0.1000 -0.0007
## 60 0.2965 nan 0.1000 0.0000
## 80 0.2249 nan 0.1000 -0.0013
## 100 0.1725 nan 0.1000 -0.0003
## 120 0.1343 nan 0.1000 -0.0002
## 140 0.1039 nan 0.1000 -0.0001
## 160 0.0814 nan 0.1000 -0.0004
## 180 0.0649 nan 0.1000 -0.0002
## 200 0.0519 nan 0.1000 -0.0002
## 220 0.0416 nan 0.1000 -0.0000
## 240 0.0330 nan 0.1000 -0.0000
## 260 0.0263 nan 0.1000 -0.0000
## 280 0.0210 nan 0.1000 -0.0000
## 300 0.0169 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2919 nan 0.1000 0.0263
## 2 1.2420 nan 0.1000 0.0229
## 3 1.1969 nan 0.1000 0.0201
## 4 1.1588 nan 0.1000 0.0172
## 5 1.1231 nan 0.1000 0.0156
## 6 1.0896 nan 0.1000 0.0148
## 7 1.0618 nan 0.1000 0.0119
## 8 1.0353 nan 0.1000 0.0105
## 9 1.0130 nan 0.1000 0.0103
## 10 0.9927 nan 0.1000 0.0088
## 20 0.8640 nan 0.1000 0.0036
## 40 0.7712 nan 0.1000 0.0003
## 60 0.7286 nan 0.1000 -0.0005
## 80 0.6974 nan 0.1000 0.0001
## 100 0.6657 nan 0.1000 -0.0001
## 120 0.6416 nan 0.1000 -0.0006
## 140 0.6180 nan 0.1000 0.0000
## 160 0.5957 nan 0.1000 0.0001
## 180 0.5762 nan 0.1000 -0.0006
## 200 0.5573 nan 0.1000 -0.0003
## 220 0.5392 nan 0.1000 -0.0002
## 240 0.5228 nan 0.1000 -0.0006
## 260 0.5087 nan 0.1000 -0.0002
## 280 0.4930 nan 0.1000 -0.0001
## 300 0.4777 nan 0.1000 -0.0003
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2612 nan 0.1000 0.0421
## 2 1.1862 nan 0.1000 0.0314
## 3 1.1232 nan 0.1000 0.0274
## 4 1.0713 nan 0.1000 0.0213
## 5 1.0293 nan 0.1000 0.0155
## 6 0.9893 nan 0.1000 0.0164
## 7 0.9554 nan 0.1000 0.0128
## 8 0.9242 nan 0.1000 0.0104
## 9 0.8933 nan 0.1000 0.0106
## 10 0.8688 nan 0.1000 0.0090
## 20 0.7028 nan 0.1000 0.0019
## 40 0.5487 nan 0.1000 -0.0017
## 60 0.4503 nan 0.1000 -0.0005
## 80 0.3779 nan 0.1000 -0.0008
## 100 0.3209 nan 0.1000 0.0001
## 120 0.2732 nan 0.1000 -0.0001
## 140 0.2339 nan 0.1000 -0.0006
## 160 0.2029 nan 0.1000 -0.0003
## 180 0.1746 nan 0.1000 -0.0001
## 200 0.1534 nan 0.1000 -0.0003
## 220 0.1332 nan 0.1000 -0.0002
## 240 0.1163 nan 0.1000 -0.0002
## 260 0.1031 nan 0.1000 -0.0002
## 280 0.0905 nan 0.1000 -0.0001
## 300 0.0789 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2530 nan 0.1000 0.0414
## 2 1.1705 nan 0.1000 0.0309
## 3 1.1012 nan 0.1000 0.0253
## 4 1.0427 nan 0.1000 0.0211
## 5 0.9905 nan 0.1000 0.0195
## 6 0.9452 nan 0.1000 0.0153
## 7 0.9066 nan 0.1000 0.0124
## 8 0.8699 nan 0.1000 0.0130
## 9 0.8398 nan 0.1000 0.0090
## 10 0.8141 nan 0.1000 0.0072
## 20 0.6129 nan 0.1000 -0.0001
## 40 0.4211 nan 0.1000 0.0011
## 60 0.3053 nan 0.1000 -0.0010
## 80 0.2309 nan 0.1000 -0.0005
## 100 0.1763 nan 0.1000 -0.0005
## 120 0.1388 nan 0.1000 -0.0003
## 140 0.1096 nan 0.1000 0.0000
## 160 0.0870 nan 0.1000 -0.0001
## 180 0.0689 nan 0.1000 -0.0001
## 200 0.0547 nan 0.1000 -0.0001
## 220 0.0437 nan 0.1000 -0.0001
## 240 0.0354 nan 0.1000 -0.0001
## 260 0.0284 nan 0.1000 -0.0001
## 280 0.0227 nan 0.1000 -0.0000
## 300 0.0182 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2957 nan 0.1000 0.0235
## 2 1.2491 nan 0.1000 0.0218
## 3 1.2050 nan 0.1000 0.0203
## 4 1.1659 nan 0.1000 0.0175
## 5 1.1324 nan 0.1000 0.0132
## 6 1.1031 nan 0.1000 0.0134
## 7 1.0790 nan 0.1000 0.0097
## 8 1.0558 nan 0.1000 0.0088
## 9 1.0329 nan 0.1000 0.0098
## 10 1.0147 nan 0.1000 0.0082
## 20 0.8912 nan 0.1000 0.0035
## 40 0.8075 nan 0.1000 0.0004
## 60 0.7625 nan 0.1000 -0.0001
## 80 0.7281 nan 0.1000 -0.0008
## 100 0.6993 nan 0.1000 -0.0013
## 120 0.6724 nan 0.1000 0.0001
## 140 0.6497 nan 0.1000 -0.0007
## 160 0.6277 nan 0.1000 -0.0007
## 180 0.6084 nan 0.1000 -0.0002
## 200 0.5913 nan 0.1000 -0.0014
## 220 0.5731 nan 0.1000 -0.0010
## 240 0.5553 nan 0.1000 -0.0000
## 260 0.5387 nan 0.1000 -0.0012
## 280 0.5254 nan 0.1000 -0.0000
## 300 0.5109 nan 0.1000 -0.0011
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2640 nan 0.1000 0.0382
## 2 1.1890 nan 0.1000 0.0337
## 3 1.1315 nan 0.1000 0.0249
## 4 1.0819 nan 0.1000 0.0220
## 5 1.0368 nan 0.1000 0.0187
## 6 1.0014 nan 0.1000 0.0130
## 7 0.9686 nan 0.1000 0.0119
## 8 0.9389 nan 0.1000 0.0101
## 9 0.9133 nan 0.1000 0.0096
## 10 0.8872 nan 0.1000 0.0088
## 20 0.7251 nan 0.1000 0.0013
## 40 0.5683 nan 0.1000 -0.0006
## 60 0.4650 nan 0.1000 -0.0009
## 80 0.3952 nan 0.1000 -0.0021
## 100 0.3360 nan 0.1000 -0.0009
## 120 0.2886 nan 0.1000 0.0004
## 140 0.2470 nan 0.1000 -0.0003
## 160 0.2110 nan 0.1000 -0.0003
## 180 0.1823 nan 0.1000 -0.0001
## 200 0.1604 nan 0.1000 -0.0002
## 220 0.1401 nan 0.1000 -0.0002
## 240 0.1237 nan 0.1000 -0.0001
## 260 0.1087 nan 0.1000 -0.0003
## 280 0.0972 nan 0.1000 -0.0002
## 300 0.0860 nan 0.1000 -0.0004
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2575 nan 0.1000 0.0401
## 2 1.1837 nan 0.1000 0.0271
## 3 1.1133 nan 0.1000 0.0272
## 4 1.0554 nan 0.1000 0.0212
## 5 1.0060 nan 0.1000 0.0184
## 6 0.9636 nan 0.1000 0.0142
## 7 0.9235 nan 0.1000 0.0122
## 8 0.8887 nan 0.1000 0.0115
## 9 0.8609 nan 0.1000 0.0074
## 10 0.8343 nan 0.1000 0.0071
## 20 0.6358 nan 0.1000 0.0023
## 40 0.4304 nan 0.1000 -0.0014
## 60 0.3074 nan 0.1000 -0.0002
## 80 0.2316 nan 0.1000 -0.0002
## 100 0.1782 nan 0.1000 -0.0009
## 120 0.1375 nan 0.1000 -0.0004
## 140 0.1099 nan 0.1000 -0.0001
## 160 0.0865 nan 0.1000 -0.0003
## 180 0.0688 nan 0.1000 -0.0002
## 200 0.0554 nan 0.1000 -0.0001
## 220 0.0441 nan 0.1000 -0.0001
## 240 0.0354 nan 0.1000 -0.0001
## 260 0.0286 nan 0.1000 -0.0001
## 280 0.0231 nan 0.1000 -0.0000
## 300 0.0187 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2910 nan 0.1000 0.0280
## 2 1.2427 nan 0.1000 0.0208
## 3 1.1975 nan 0.1000 0.0197
## 4 1.1630 nan 0.1000 0.0137
## 5 1.1274 nan 0.1000 0.0160
## 6 1.0968 nan 0.1000 0.0133
## 7 1.0684 nan 0.1000 0.0125
## 8 1.0466 nan 0.1000 0.0098
## 9 1.0230 nan 0.1000 0.0101
## 10 1.0041 nan 0.1000 0.0089
## 20 0.8781 nan 0.1000 0.0027
## 40 0.7870 nan 0.1000 0.0003
## 60 0.7460 nan 0.1000 -0.0003
## 80 0.7137 nan 0.1000 -0.0000
## 100 0.6845 nan 0.1000 -0.0005
## 120 0.6574 nan 0.1000 -0.0002
## 140 0.6329 nan 0.1000 -0.0004
## 160 0.6130 nan 0.1000 -0.0004
## 180 0.5938 nan 0.1000 -0.0003
## 200 0.5739 nan 0.1000 -0.0002
## 220 0.5564 nan 0.1000 -0.0006
## 240 0.5402 nan 0.1000 -0.0005
## 260 0.5253 nan 0.1000 -0.0007
## 280 0.5103 nan 0.1000 -0.0007
## 300 0.4961 nan 0.1000 -0.0005
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2664 nan 0.1000 0.0360
## 2 1.1948 nan 0.1000 0.0329
## 3 1.1388 nan 0.1000 0.0234
## 4 1.0871 nan 0.1000 0.0207
## 5 1.0437 nan 0.1000 0.0160
## 6 1.0031 nan 0.1000 0.0156
## 7 0.9683 nan 0.1000 0.0133
## 8 0.9366 nan 0.1000 0.0124
## 9 0.9097 nan 0.1000 0.0096
## 10 0.8828 nan 0.1000 0.0091
## 20 0.7273 nan 0.1000 0.0034
## 40 0.5648 nan 0.1000 -0.0016
## 60 0.4578 nan 0.1000 -0.0005
## 80 0.3861 nan 0.1000 -0.0003
## 100 0.3239 nan 0.1000 -0.0005
## 120 0.2784 nan 0.1000 -0.0005
## 140 0.2403 nan 0.1000 -0.0003
## 160 0.2080 nan 0.1000 -0.0004
## 180 0.1804 nan 0.1000 -0.0003
## 200 0.1555 nan 0.1000 -0.0002
## 220 0.1362 nan 0.1000 -0.0001
## 240 0.1186 nan 0.1000 -0.0003
## 260 0.1041 nan 0.1000 -0.0003
## 280 0.0915 nan 0.1000 -0.0003
## 300 0.0803 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2537 nan 0.1000 0.0428
## 2 1.1733 nan 0.1000 0.0328
## 3 1.1030 nan 0.1000 0.0300
## 4 1.0434 nan 0.1000 0.0250
## 5 0.9920 nan 0.1000 0.0177
## 6 0.9484 nan 0.1000 0.0154
## 7 0.9065 nan 0.1000 0.0140
## 8 0.8735 nan 0.1000 0.0101
## 9 0.8427 nan 0.1000 0.0075
## 10 0.8148 nan 0.1000 0.0085
## 20 0.6205 nan 0.1000 0.0012
## 40 0.4177 nan 0.1000 -0.0010
## 60 0.3002 nan 0.1000 -0.0012
## 80 0.2239 nan 0.1000 -0.0009
## 100 0.1718 nan 0.1000 -0.0003
## 120 0.1323 nan 0.1000 -0.0001
## 140 0.1039 nan 0.1000 -0.0000
## 160 0.0820 nan 0.1000 -0.0002
## 180 0.0648 nan 0.1000 -0.0002
## 200 0.0521 nan 0.1000 -0.0002
## 220 0.0416 nan 0.1000 -0.0001
## 240 0.0334 nan 0.1000 -0.0001
## 260 0.0267 nan 0.1000 -0.0000
## 280 0.0216 nan 0.1000 -0.0001
## 300 0.0175 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2972 nan 0.1000 0.0245
## 2 1.2499 nan 0.1000 0.0214
## 3 1.2053 nan 0.1000 0.0190
## 4 1.1659 nan 0.1000 0.0175
## 5 1.1282 nan 0.1000 0.0158
## 6 1.0999 nan 0.1000 0.0117
## 7 1.0742 nan 0.1000 0.0108
## 8 1.0502 nan 0.1000 0.0105
## 9 1.0280 nan 0.1000 0.0098
## 10 1.0072 nan 0.1000 0.0094
## 20 0.8856 nan 0.1000 0.0022
## 40 0.8017 nan 0.1000 0.0009
## 60 0.7532 nan 0.1000 -0.0003
## 80 0.7145 nan 0.1000 0.0000
## 100 0.6846 nan 0.1000 0.0003
## 120 0.6588 nan 0.1000 -0.0010
## 140 0.6367 nan 0.1000 -0.0005
## 160 0.6155 nan 0.1000 -0.0009
## 180 0.5919 nan 0.1000 -0.0004
## 200 0.5712 nan 0.1000 -0.0002
## 220 0.5527 nan 0.1000 -0.0000
## 240 0.5350 nan 0.1000 -0.0004
## 260 0.5198 nan 0.1000 -0.0005
## 280 0.5039 nan 0.1000 -0.0003
## 300 0.4900 nan 0.1000 -0.0007
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2646 nan 0.1000 0.0379
## 2 1.1954 nan 0.1000 0.0320
## 3 1.1364 nan 0.1000 0.0264
## 4 1.0854 nan 0.1000 0.0206
## 5 1.0411 nan 0.1000 0.0193
## 6 1.0048 nan 0.1000 0.0133
## 7 0.9685 nan 0.1000 0.0132
## 8 0.9362 nan 0.1000 0.0121
## 9 0.9127 nan 0.1000 0.0072
## 10 0.8904 nan 0.1000 0.0067
## 20 0.7248 nan 0.1000 0.0007
## 40 0.5604 nan 0.1000 -0.0005
## 60 0.4570 nan 0.1000 0.0007
## 80 0.3875 nan 0.1000 -0.0007
## 100 0.3247 nan 0.1000 -0.0003
## 120 0.2784 nan 0.1000 -0.0006
## 140 0.2370 nan 0.1000 -0.0001
## 160 0.2040 nan 0.1000 -0.0001
## 180 0.1765 nan 0.1000 -0.0004
## 200 0.1536 nan 0.1000 -0.0002
## 220 0.1344 nan 0.1000 -0.0002
## 240 0.1177 nan 0.1000 -0.0004
## 260 0.1024 nan 0.1000 -0.0002
## 280 0.0900 nan 0.1000 -0.0000
## 300 0.0789 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2604 nan 0.1000 0.0336
## 2 1.1811 nan 0.1000 0.0289
## 3 1.1131 nan 0.1000 0.0260
## 4 1.0567 nan 0.1000 0.0196
## 5 1.0105 nan 0.1000 0.0139
## 6 0.9664 nan 0.1000 0.0150
## 7 0.9263 nan 0.1000 0.0115
## 8 0.8890 nan 0.1000 0.0125
## 9 0.8571 nan 0.1000 0.0094
## 10 0.8249 nan 0.1000 0.0102
## 20 0.6273 nan 0.1000 0.0027
## 40 0.4260 nan 0.1000 -0.0008
## 60 0.3114 nan 0.1000 0.0000
## 80 0.2313 nan 0.1000 -0.0006
## 100 0.1797 nan 0.1000 -0.0003
## 120 0.1377 nan 0.1000 -0.0002
## 140 0.1065 nan 0.1000 -0.0000
## 160 0.0842 nan 0.1000 -0.0002
## 180 0.0668 nan 0.1000 -0.0001
## 200 0.0528 nan 0.1000 0.0000
## 220 0.0423 nan 0.1000 -0.0001
## 240 0.0338 nan 0.1000 -0.0001
## 260 0.0274 nan 0.1000 -0.0001
## 280 0.0220 nan 0.1000 -0.0001
## 300 0.0177 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2995 nan 0.1000 0.0218
## 2 1.2505 nan 0.1000 0.0220
## 3 1.2077 nan 0.1000 0.0178
## 4 1.1712 nan 0.1000 0.0176
## 5 1.1396 nan 0.1000 0.0144
## 6 1.1058 nan 0.1000 0.0143
## 7 1.0795 nan 0.1000 0.0120
## 8 1.0553 nan 0.1000 0.0111
## 9 1.0330 nan 0.1000 0.0099
## 10 1.0145 nan 0.1000 0.0079
## 20 0.8929 nan 0.1000 0.0022
## 40 0.8130 nan 0.1000 0.0011
## 60 0.7682 nan 0.1000 0.0002
## 80 0.7357 nan 0.1000 -0.0003
## 100 0.7059 nan 0.1000 -0.0002
## 120 0.6787 nan 0.1000 -0.0012
## 140 0.6588 nan 0.1000 -0.0009
## 160 0.6373 nan 0.1000 -0.0007
## 180 0.6150 nan 0.1000 -0.0008
## 200 0.5953 nan 0.1000 -0.0008
## 220 0.5762 nan 0.1000 -0.0002
## 240 0.5589 nan 0.1000 -0.0007
## 260 0.5437 nan 0.1000 -0.0009
## 280 0.5303 nan 0.1000 -0.0007
## 300 0.5140 nan 0.1000 -0.0004
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2724 nan 0.1000 0.0345
## 2 1.2058 nan 0.1000 0.0284
## 3 1.1478 nan 0.1000 0.0253
## 4 1.0890 nan 0.1000 0.0253
## 5 1.0445 nan 0.1000 0.0169
## 6 1.0066 nan 0.1000 0.0160
## 7 0.9709 nan 0.1000 0.0134
## 8 0.9427 nan 0.1000 0.0098
## 9 0.9150 nan 0.1000 0.0096
## 10 0.8907 nan 0.1000 0.0084
## 20 0.7372 nan 0.1000 0.0030
## 40 0.5793 nan 0.1000 -0.0015
## 60 0.4738 nan 0.1000 0.0005
## 80 0.4002 nan 0.1000 -0.0011
## 100 0.3384 nan 0.1000 -0.0011
## 120 0.2883 nan 0.1000 -0.0002
## 140 0.2513 nan 0.1000 -0.0007
## 160 0.2154 nan 0.1000 -0.0002
## 180 0.1895 nan 0.1000 -0.0004
## 200 0.1654 nan 0.1000 -0.0005
## 220 0.1451 nan 0.1000 -0.0004
## 240 0.1253 nan 0.1000 -0.0003
## 260 0.1095 nan 0.1000 -0.0000
## 280 0.0966 nan 0.1000 -0.0002
## 300 0.0856 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2563 nan 0.1000 0.0385
## 2 1.1822 nan 0.1000 0.0306
## 3 1.1145 nan 0.1000 0.0252
## 4 1.0588 nan 0.1000 0.0206
## 5 1.0114 nan 0.1000 0.0175
## 6 0.9716 nan 0.1000 0.0124
## 7 0.9320 nan 0.1000 0.0144
## 8 0.8961 nan 0.1000 0.0119
## 9 0.8653 nan 0.1000 0.0091
## 10 0.8372 nan 0.1000 0.0088
## 20 0.6430 nan 0.1000 0.0015
## 40 0.4407 nan 0.1000 -0.0004
## 60 0.3266 nan 0.1000 0.0000
## 80 0.2466 nan 0.1000 -0.0006
## 100 0.1872 nan 0.1000 -0.0005
## 120 0.1454 nan 0.1000 -0.0003
## 140 0.1158 nan 0.1000 -0.0002
## 160 0.0923 nan 0.1000 -0.0005
## 180 0.0740 nan 0.1000 -0.0002
## 200 0.0595 nan 0.1000 -0.0001
## 220 0.0484 nan 0.1000 -0.0001
## 240 0.0391 nan 0.1000 -0.0002
## 260 0.0311 nan 0.1000 -0.0000
## 280 0.0251 nan 0.1000 -0.0001
## 300 0.0204 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2921 nan 0.1000 0.0263
## 2 1.2452 nan 0.1000 0.0222
## 3 1.2020 nan 0.1000 0.0214
## 4 1.1640 nan 0.1000 0.0172
## 5 1.1316 nan 0.1000 0.0149
## 6 1.1004 nan 0.1000 0.0127
## 7 1.0696 nan 0.1000 0.0133
## 8 1.0449 nan 0.1000 0.0116
## 9 1.0170 nan 0.1000 0.0108
## 10 0.9976 nan 0.1000 0.0081
## 20 0.8681 nan 0.1000 0.0015
## 40 0.7709 nan 0.1000 -0.0004
## 60 0.7228 nan 0.1000 -0.0003
## 80 0.6876 nan 0.1000 -0.0006
## 100 0.6587 nan 0.1000 -0.0001
## 120 0.6323 nan 0.1000 -0.0004
## 140 0.6068 nan 0.1000 -0.0010
## 160 0.5871 nan 0.1000 -0.0005
## 180 0.5662 nan 0.1000 -0.0003
## 200 0.5483 nan 0.1000 -0.0003
## 220 0.5294 nan 0.1000 -0.0004
## 240 0.5143 nan 0.1000 -0.0004
## 260 0.4989 nan 0.1000 -0.0001
## 280 0.4852 nan 0.1000 -0.0005
## 300 0.4710 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2660 nan 0.1000 0.0372
## 2 1.1929 nan 0.1000 0.0344
## 3 1.1293 nan 0.1000 0.0297
## 4 1.0758 nan 0.1000 0.0221
## 5 1.0324 nan 0.1000 0.0190
## 6 0.9901 nan 0.1000 0.0185
## 7 0.9543 nan 0.1000 0.0144
## 8 0.9243 nan 0.1000 0.0115
## 9 0.8954 nan 0.1000 0.0098
## 10 0.8698 nan 0.1000 0.0082
## 20 0.7081 nan 0.1000 0.0012
## 40 0.5578 nan 0.1000 -0.0009
## 60 0.4574 nan 0.1000 -0.0007
## 80 0.3811 nan 0.1000 -0.0012
## 100 0.3269 nan 0.1000 -0.0003
## 120 0.2804 nan 0.1000 -0.0008
## 140 0.2422 nan 0.1000 -0.0003
## 160 0.2087 nan 0.1000 -0.0004
## 180 0.1796 nan 0.1000 -0.0001
## 200 0.1557 nan 0.1000 -0.0005
## 220 0.1353 nan 0.1000 -0.0000
## 240 0.1174 nan 0.1000 -0.0001
## 260 0.1028 nan 0.1000 -0.0001
## 280 0.0903 nan 0.1000 -0.0002
## 300 0.0793 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2543 nan 0.1000 0.0391
## 2 1.1772 nan 0.1000 0.0306
## 3 1.1035 nan 0.1000 0.0290
## 4 1.0461 nan 0.1000 0.0182
## 5 0.9876 nan 0.1000 0.0226
## 6 0.9445 nan 0.1000 0.0175
## 7 0.9062 nan 0.1000 0.0130
## 8 0.8685 nan 0.1000 0.0135
## 9 0.8359 nan 0.1000 0.0079
## 10 0.8050 nan 0.1000 0.0107
## 20 0.5997 nan 0.1000 0.0019
## 40 0.4055 nan 0.1000 0.0006
## 60 0.2908 nan 0.1000 -0.0007
## 80 0.2196 nan 0.1000 -0.0007
## 100 0.1676 nan 0.1000 -0.0002
## 120 0.1292 nan 0.1000 -0.0005
## 140 0.1001 nan 0.1000 -0.0004
## 160 0.0792 nan 0.1000 -0.0002
## 180 0.0627 nan 0.1000 -0.0003
## 200 0.0506 nan 0.1000 -0.0002
## 220 0.0404 nan 0.1000 -0.0001
## 240 0.0322 nan 0.1000 -0.0000
## 260 0.0258 nan 0.1000 -0.0001
## 280 0.0207 nan 0.1000 -0.0001
## 300 0.0167 nan 0.1000 -0.0001
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2969 nan 0.1000 0.0235
## 2 1.2451 nan 0.1000 0.0232
## 3 1.2022 nan 0.1000 0.0186
## 4 1.1623 nan 0.1000 0.0181
## 5 1.1283 nan 0.1000 0.0151
## 6 1.0965 nan 0.1000 0.0140
## 7 1.0690 nan 0.1000 0.0126
## 8 1.0439 nan 0.1000 0.0119
## 9 1.0188 nan 0.1000 0.0112
## 10 0.9979 nan 0.1000 0.0069
## 20 0.8712 nan 0.1000 0.0029
## 40 0.7810 nan 0.1000 0.0006
## 60 0.7388 nan 0.1000 -0.0002
## 80 0.7063 nan 0.1000 -0.0011
## 100 0.6759 nan 0.1000 -0.0011
## 120 0.6537 nan 0.1000 -0.0004
## 140 0.6318 nan 0.1000 -0.0005
## 160 0.6079 nan 0.1000 -0.0002
## 180 0.5907 nan 0.1000 -0.0003
## 200 0.5731 nan 0.1000 -0.0004
## 220 0.5564 nan 0.1000 -0.0005
## 240 0.5391 nan 0.1000 0.0002
## 260 0.5211 nan 0.1000 -0.0003
## 280 0.5033 nan 0.1000 -0.0003
## 300 0.4888 nan 0.1000 -0.0007
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2662 nan 0.1000 0.0388
## 2 1.1927 nan 0.1000 0.0315
## 3 1.1343 nan 0.1000 0.0257
## 4 1.0814 nan 0.1000 0.0230
## 5 1.0352 nan 0.1000 0.0201
## 6 0.9943 nan 0.1000 0.0157
## 7 0.9618 nan 0.1000 0.0116
## 8 0.9299 nan 0.1000 0.0113
## 9 0.9011 nan 0.1000 0.0092
## 10 0.8763 nan 0.1000 0.0088
## 20 0.7109 nan 0.1000 0.0012
## 40 0.5531 nan 0.1000 -0.0008
## 60 0.4587 nan 0.1000 -0.0012
## 80 0.3866 nan 0.1000 -0.0002
## 100 0.3294 nan 0.1000 -0.0004
## 120 0.2767 nan 0.1000 -0.0005
## 140 0.2384 nan 0.1000 -0.0002
## 160 0.2071 nan 0.1000 -0.0002
## 180 0.1771 nan 0.1000 -0.0002
## 200 0.1524 nan 0.1000 -0.0003
## 220 0.1325 nan 0.1000 -0.0002
## 240 0.1161 nan 0.1000 -0.0003
## 260 0.1036 nan 0.1000 -0.0001
## 280 0.0908 nan 0.1000 0.0000
## 300 0.0795 nan 0.1000 -0.0002
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2513 nan 0.1000 0.0395
## 2 1.1684 nan 0.1000 0.0320
## 3 1.1022 nan 0.1000 0.0267
## 4 1.0476 nan 0.1000 0.0197
## 5 0.9977 nan 0.1000 0.0173
## 6 0.9543 nan 0.1000 0.0136
## 7 0.9121 nan 0.1000 0.0144
## 8 0.8762 nan 0.1000 0.0121
## 9 0.8450 nan 0.1000 0.0098
## 10 0.8137 nan 0.1000 0.0106
## 20 0.6183 nan 0.1000 0.0021
## 40 0.4222 nan 0.1000 -0.0005
## 60 0.3113 nan 0.1000 -0.0007
## 80 0.2363 nan 0.1000 -0.0006
## 100 0.1826 nan 0.1000 -0.0005
## 120 0.1396 nan 0.1000 0.0000
## 140 0.1084 nan 0.1000 -0.0003
## 160 0.0850 nan 0.1000 -0.0000
## 180 0.0665 nan 0.1000 -0.0000
## 200 0.0534 nan 0.1000 -0.0001
## 220 0.0431 nan 0.1000 -0.0000
## 240 0.0340 nan 0.1000 -0.0001
## 260 0.0272 nan 0.1000 -0.0001
## 280 0.0216 nan 0.1000 -0.0000
## 300 0.0172 nan 0.1000 -0.0000
##
## Iter TrainDeviance ValidDeviance StepSize Improve
## 1 1.2584 nan 0.1000 0.0392
## 2 1.1795 nan 0.1000 0.0357
## 3 1.1121 nan 0.1000 0.0279
## 4 1.0555 nan 0.1000 0.0207
## 5 1.0065 nan 0.1000 0.0190
## 6 0.9637 nan 0.1000 0.0157
## 7 0.9298 nan 0.1000 0.0113
## 8 0.8966 nan 0.1000 0.0103
## 9 0.8646 nan 0.1000 0.0112
## 10 0.8379 nan 0.1000 0.0075
## 20 0.6637 nan 0.1000 0.0015
## 40 0.4743 nan 0.1000 -0.0013
## 60 0.3646 nan 0.1000 -0.0001
## 80 0.2905 nan 0.1000 -0.0003
## 100 0.2316 nan 0.1000 -0.0004
## 120 0.1867 nan 0.1000 -0.0002
## 140 0.1544 nan 0.1000 -0.0000
## 160 0.1271 nan 0.1000 -0.0002
## 180 0.1029 nan 0.1000 -0.0001
## 200 0.0853 nan 0.1000 -0.0002
## 220 0.0713 nan 0.1000 -0.0001
## 240 0.0589 nan 0.1000 -0.0000
## 260 0.0493 nan 0.1000 -0.0002
## 280 0.0415 nan 0.1000 -0.0001
## 300 0.0343 nan 0.1000 -0.0000
print(gbm2)
## Stochastic Gradient Boosting
##
## 1546 samples
## 256 predictor
## 2 classes: '0', '1'
##
## No pre-processing
## Resampling: Cross-Validated (5 fold, repeated 2 times)
## Summary of sample sizes: 1237, 1237, 1236, 1237, 1237, 1237, ...
## Resampling results across tuning parameters:
##
## interaction.depth n.trees Accuracy Kappa
## 1 50 0.8056342 0.5900303
## 1 150 0.8133960 0.6077585
## 1 300 0.8137133 0.6084665
## 5 50 0.8159850 0.6132240
## 5 150 0.8150130 0.6106199
## 5 300 0.8137238 0.6076744
## 9 50 0.8159839 0.6126004
## 9 150 0.8163044 0.6132187
## 9 300 0.8163075 0.6125792
##
## Tuning parameter 'shrinkage' was held constant at a value of 0.1
##
## Tuning parameter 'n.minobsinnode' was held constant at a value of 10
## Accuracy was used to select the optimal model using the largest valu
e.
## The final values used for the model were n.trees = 300, interaction.
depth =
## 9, shrinkage = 0.1 and n.minobsinnode = 10.
List the values of the hyperparameters for the final model (best accuracy) gbm2:
[0.5 pt / 4 pts]
The final values used for the model were

n.trees = 300
interaction.depth = 9
shrinkage = 0.1
n.minobsinnode = 10
IMPORTANT: for the remaining questions, we will use the final (optimally tuned)
classification model gbm2.
QUESTION 4 [1 pt]
# do not remove this
rm(ypred,ypredp,yscore,ytruth)
## Warning in rm(ypred, ypredp, yscore, ytruth): 找不到对象'ypred'
## Warning in rm(ypred, ypredp, yscore, ytruth): 找不到对象'ypredp'
## Warning in rm(ypred, ypredp, yscore, ytruth): 找不到对象'yscore'
## Warning in rm(ypred, ypredp, yscore, ytruth): 找不到对象'ytruth'
Call the predict function on the testing set to calculate the predicted value of the
phoneme class g (0 or 1). Save the result to an object called ypred. Be careful to pass
the correct value to the type parameter. Print the top section of the ypred vector.
# ypred=...
# show top of ypred
ypred=predict(gbm2, newdata = dtest, type = "raw")
head(ypred)
## [1] 0 0 0 1 1 0
## Levels: 0 1
QUESTION 5 [1 pt]
Call the predict function on the testing set to calculate the predicted probabilities of
the two outcomes (g=0 or g=1). Save the result to an object called ypredp. Be careful
to pass the correct value to the type parameter. Print the top section of the ypredp
vector.
# ypredp
# print the top of ypredp
ypred=predict(gbm2, newdata = dtest, type = "prob")
print(ypred)
## 0 1
## 1 0.6817518261 0.3182481739
## 2 0.9976748769 0.0023251231
## 3 0.8791884571 0.1208115429
## 4 0.0013125415 0.9986874585
## 5 0.0016926687 0.9983073313
## 6 0.5155624394 0.4844375606
## 7 0.6172948863 0.3827051137
## 8 0.0074846705 0.9925153295
## 9 0.8142319641 0.1857680359
## 10 0.9936121459 0.0063878541
## 11 0.0455444013 0.9544555987
## 12 0.8839110041 0.1160889959
## 13 0.9490800142 0.0509199858
## 14 0.9880335211 0.0119664789
## 15 0.3962840920 0.6037159080
## 16 0.0451512226 0.9548487774
## 17 0.0478878973 0.9521121027
## 18 0.2182914095 0.7817085905
## 19 0.6298141280 0.3701858720
## 20 0.0821687666 0.9178312334
## 21 0.0054030364 0.9945969636
## 22 0.9874268352 0.0125731648
## 23 0.0028122464 0.9971877536
## 24 0.0020772039 0.9979227961
## 25 0.0442061694 0.9557938306
## 26 0.9890595414 0.0109404586
## 27 0.0021173590 0.9978826410
## 28 0.9952997408 0.0047002592
## 29 0.9866077718 0.0133922282
## 30 0.9016050068 0.0983949932
## 31 0.2245258044 0.7754741956
## 32 0.0016648524 0.9983351476
## 33 0.9801640557 0.0198359443
## 34 0.9903917864 0.0096082136
## 35 0.9994806963 0.0005193037
## 36 0.4027756921 0.5972243079
## 37 0.4990610700 0.5009389300
## 38 0.8833804227 0.1166195773
## 39 0.9172989440 0.0827010560
## 40 0.5996840852 0.4003159148
## 41 0.0026219045 0.9973780955
## 42 0.0475351885 0.9524648115
## 43 0.0008113484 0.9991886516
## 44 0.9936730175 0.0063269825
## 45 0.0699291915 0.9300708085
## 46 0.9945918357 0.0054081643
## 47 0.0047483929 0.9952516071
## 48 0.0043756984 0.9956243016
## 49 0.0037687235 0.9962312765
## 50 0.8287974329 0.1712025671
## 51 0.0166378490 0.9833621510
## 52 0.0017685988 0.9982314012
## 53 0.0022892131 0.9977107869
## 54 0.0844414229 0.9155585771
## 55 0.0046325210 0.9953674790
## 56 0.0235565618 0.9764434382
## 57 0.0045348852 0.9954651148
## 58 0.5505952576 0.4494047424
## 59 0.9919802807 0.0080197193
## 60 0.2694848966 0.7305151034
## 61 0.9947197216 0.0052802784
## 62 0.0031642161 0.9968357839
## 63 0.1748655722 0.8251344278
## 64 0.7780355502 0.2219644498
## 65 0.0229930890 0.9770069110
## 66 0.8627493264 0.1372506736
## 67 0.0189454403 0.9810545597
## 68 0.9575969287 0.0424030713
## 69 0.6032250428 0.3967749572
## 70 0.0033547165 0.9966452835
## 71 0.0641365534 0.9358634466
## 72 0.1180229192 0.8819770808
## 73 0.0065784173 0.9934215827
## 74 0.0109495167 0.9890504833
## 75 0.6375309110 0.3624690890
## 76 0.0066029098 0.9933970902
## 77 0.9363608859 0.0636391141
## 78 0.0070507630 0.9929492370
## 79 0.0003725416 0.9996274584
## 80 0.1141623225 0.8858376775
## 81 0.0030256023 0.9969743977
## 82 0.0060008334 0.9939991666
## 83 0.0026058524 0.9973941476
## 84 0.9079492066 0.0920507934
## 85 0.0026266408 0.9973733592
## 86 0.0056954111 0.9943045889
## 87 0.0024944538 0.9975055462
## 88 0.3733965214 0.6266034786
## 89 0.0124665498 0.9875334502
## 90 0.1835182765 0.8164817235
## 91 0.0012620525 0.9987379475
## 92 0.9733711525 0.0266288475
## 93 0.0052817821 0.9947182179
## 94 0.9895100400 0.0104899600
## 95 0.9993222240 0.0006777760
## 96 0.9017712748 0.0982287252
## 97 0.0103330716 0.9896669284
## 98 0.0311039741 0.9688960259
## 99 0.2907944001 0.7092055999
## 100 0.8805498277 0.1194501723
## 101 0.4052412497 0.5947587503
## 102 0.4636301143 0.5363698857
## 103 0.3101457426 0.6898542574
## 104 0.0088958291 0.9911041709
## 105 0.0367353635 0.9632646365
## 106 0.0018207629 0.9981792371
## 107 0.9827781079 0.0172218921
## 108 0.0981745389 0.9018254611
## 109 0.9952380027 0.0047619973
## 110 0.1159038820 0.8840961180
## 111 0.3911516938 0.6088483062
## 112 0.9977473333 0.0022526667
## 113 0.0456077417 0.9543922583
## 114 0.0920902574 0.9079097426
## 115 0.8654244857 0.1345755143
## 116 0.5627780932 0.4372219068
## 117 0.9863199164 0.0136800836
## 118 0.9940510163 0.0059489837
## 119 0.9892325662 0.0107674338
## 120 0.0037011113 0.9962988887
## 121 0.3280786698 0.6719213302
## 122 0.5938611738 0.4061388262
## 123 0.0012883546 0.9987116454
## 124 0.9590258850 0.0409741150
## 125 0.8523898673 0.1476101327
## 126 0.9817168418 0.0182831582
## 127 0.0111034018 0.9888965982
## 128 0.0158789742 0.9841210258
## 129 0.0229824421 0.9770175579
## 130 0.0063395958 0.9936604042
## 131 0.2139683703 0.7860316297
## 132 0.0120698523 0.9879301477
## 133 0.8616656049 0.1383343951
## 134 0.2634594396 0.7365405604
## 135 0.9936538335 0.0063461665
## 136 0.9998763105 0.0001236895
## 137 0.9013030131 0.0986969869
## 138 0.9461697830 0.0538302170
## 139 0.8814515580 0.1185484420
## 140 0.9378060473 0.0621939527
## 141 0.0075536993 0.9924463007
## 142 0.4927222246 0.5072777754
## 143 0.2617343451 0.7382656549
## 144 0.7648600391 0.2351399609
## 145 0.9994006161 0.0005993839
## 146 0.0099538930 0.9900461070
## 147 0.0034840725 0.9965159275
## 148 0.0526213540 0.9473786460
## 149 0.0032488359 0.9967511641
## 150 0.0119321728 0.9880678272
## 151 0.3712291458 0.6287708542
## 152 0.0835902627 0.9164097373
## 153 0.9843694885 0.0156305115
## 154 0.3473137627 0.6526862373
## 155 0.0025870137 0.9974129863
## 156 0.0213775712 0.9786224288
## 157 0.8820694326 0.1179305674
## 158 0.1008162232 0.8991837768
## 159 0.9945495707 0.0054504293
## 160 0.8638994528 0.1361005472
## 161 0.0185369557 0.9814630443
## 162 0.1015667624 0.8984332376
## 163 0.9027156266 0.0972843734
## 164 0.0053523708 0.9946476292
## 165 0.9966004116 0.0033995884
## 166 0.0199719367 0.9800280633
## 167 0.0026859061 0.9973140939
## 168 0.0014374697 0.9985625303
## 169 0.0144366911 0.9855633089
## 170 0.8989226985 0.1010773015
## 171 0.0548401451 0.9451598549
QUESTION 6 [2 pts]
Define the yscore array as the second column of the ypredp object. This score is
simply the estimated probability that g=1. We naturally wish to classify a record as
belonging to class 0 if this score is less than 0.5 and 1 otherwise.
Extract the g column of the testing set to a vector called ytruth. Use the table
function to display a confusion table that compares the decision yscore more than
0.5 to ytruth.
# compute yscore
yscore <- ypred[,2]
# set ytruth value
ytruth <- dtest$g
# compute and show the confusion table of prediction
table(yscore > 0.5, ytruth)
## ytruth
## 0 1
## FALSE 51 17
## TRUE 18 85
QUESTION 7 [2 pts]
You will use the library ROCR to plot the ROC curve of the classifier. Load the
appropriate library to plot the ROC curve. Apply the prediction function of ROCR to
yscore and ytruth, and save the result into an object called pred.
# load the library
# compute the pred object
library(ROCR)
pred <- prediction(yscore, ytruth)
QUESTION 8 [2 pts]
Plot the ROC curve of the classifier in blue, and overlay in red the ROC curve of a
random classifier. Your R code must have three lines of codes.
I will provide an example of ROC curve plotting in class.
In the first line of code, define an object called roc, obtained by applying the
performance function to pred, and requesting tpr as measure and fpr as x.measure.
In the plot, use the title: ROC of GBM classifier of vowel ao (vs aa)
Use lwd=2 in the call to plot and lines (this will set line width to 2).
# apply the function performance to obtain the roc object
roc <- performance(pred, measure = "tpr", x.measure = "fpr")
# apply the function plot to the roc object
plot(roc,lwd = 2)
# overlay a red line going from 0,0 to 1,1
lines(x=c(0,1), y=c(0,1), lwd = 2)