Project 4, APS1070 Fall 2022
Linear Regression - 14 points
Deadline: Dec 1st, 21:00
Academic Integrity
This project is individual - it is to be completed on your own. If you have questions, please post your query in the APS1070 Piazza Q&A forums (the
answer might be useful to others!).
Do not share your code with others, or post your work online. Do not submit code that you have not written yourself. Students suspected of plagiarism on a
project, midterm or exam will be referred to the department for formal discipline for breaches of the Student Code of Conduct.
Please fill out the following:
Your name:
Your student number:
How to submit (HTML [This notebook] + IPYNB + GitHub [Part 7])
1. Download your notebook: File -> Download .ipynb
2. Click on the Files icon on the far left menu of Colab
3. Select & upload your .ipynb file you just downloaded, and then obtain its path (right click) (you might need to hit the Refresh button before your file
shows up)
1. execute the following in a Colab cell:
%%shell
jupyter nbconvert --to html /PATH/TO/YOUR/NOTEBOOKFILE.ipynb
2. An HTML version of your notebook will appear in the files, so you can download it.
3. Submit both `HTML` and `IPYNB` files for this notebook on Quercus for grading.
4. Make sure you compeleted the last part on **GitHub** and added your link to the last cell here. The teaching team will view and grade your part 7 on
GitHub.
Ref: https://stackoverflow.com/a/64487858 (https://stackoverflow.com/a/64487858)
In [ ]:
%%shell
jupyter nbconvert --to html /PATH/TO/YOUR/NOTEBOOKFILE.ipynb
Part 1 - Getting Started [1 marks]
Ailerons are small hinged sections on the outboard portion of a wing used to control the roll of an airplane. In this project, we are going to design a
controller to manage the ailerons of an aircraft based on supervised learning.
The following dataset contains 13750 instances, where each instance is a set of 40 features describing the airplane's status. Our goal is to use these
features to predict the Goal column, which is a command that our controller should issue. We will make our predictions by implementing linear
regression.
In [ ]:
import pandas as pd
import numpy as np
df=pd.read_csv("https://raw.githubusercontent.com/aps1070-2019/datasets/master/F16L.csv" , skipinitialspace=True)
In [ ]:
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from scipy import linalg
from sklearn.metrics import mean_squared_error
import matplotlib.pyplot as plt
Here are the steps to complete this portion:
1. Print the dataframe.
2. Prepare your dataset as follows: [0.5]
Using train_test_split from Sklearn, split the dataset into training, validation, and test sets ( training, validation, and
test). When splitting, set random_state=1 .
Standardize the data using StandardScaler from sklearn.
Insert the first column of all s in the training, validation, and test set.
3. Explain the difference between epoch and iteration in the Gradient descent algorithm (SGD/mini-batch)? [0.5]
70% 15% 15%
1
In [ ]:
# 1:
# Print the dataframe.
df
In [ ]:
# 2:
# Prepare your dataset as follows.
# 70:15:15 = 7:3
X1 = df.drop('Goal', axis = 1).values
y1 = df['Goal'].values
X1_train, X1_val_test, y_train, y1_val_test = train_test_split(X1, y1, test_size=0.3, random_state=1)
X1_val, X1_test, y_val, y_test = train_test_split(X1_val_test, y1_val_test, test_size=0.5, random_state=1)
X1_train.shape, X1_val.shape, X1_test.shape, y_train.shape, y_val.shape, y_test.shape
Out[ ]:
inputs
ClimbRate Sgz P Q CurPitch CurRoll AbsRoll DiffClb DiffRollRate DiffDiffClb ... DiffSeTime8 DiffSeTime9 DiffSeTime10 DiffSeTime11 DiffSeTime12
0 -17 -6 -0.03 0.14 0.59 -0.5 -11 -9 0.015 -0.2 ... 0.0 0.000 0.0 0.000
1 -791 15 -0.28 -0.34 1.11 0.9 -10 27 -0.002 4.7 ... 0.0 0.000 0.0 0.000
2 -46 48 -0.42 0.07 0.57 0.6 -5 -8 -0.008 0.0 ... 0.0 0.000 0.0 0.000
3 496 -5 -0.21 -0.04 0.03 -0.5 -7 6 -0.002 -0.1 ... 0.0 0.000 0.0 0.001
4 -124 16 -0.02 0.05 0.61 -0.2 -9 -3 0.000 0.0 ... 0.0 0.000 0.0 0.000
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
13745 -53 -13 -0.07 0.02 0.58 -2.3 -15 12 0.022 -0.8 ... 0.0 0.000 0.0 0.000
13746 1 4 -0.11 -0.07 0.47 0.8 -5 8 -0.003 -0.2 ... 0.0 0.000 0.0 0.000
13747 425 -14 0.40 0.07 0.41 -2.1 -16 12 0.031 -0.7 ... 0.0 0.001 0.0 0.000
13748 102 -10 -0.19 0.08 1.30 0.5 -10 -3 -0.003 -0.2 ... 0.0 0.000 0.0 0.000
13749 -165 52 0.08 0.42 1.07 -1.6 -22 -15 0.007 0.0 ... 0.0 0.000 0.0 0.000
13750 rows × 41 columns
Out[ ]:
((9625, 40), (2062, 40), (2063, 40), (9625,), (2062,), (2063,))
In [ ]:
# Standardization
scl = StandardScaler()
Xtr_std = scl.fit_transform(X1_train)
Xtr_std = pd.DataFrame(Xtr_std)
Xtr_std.columns = df.iloc[:, 0:40].columns
Xvl_std = scl.fit_transform(X1_val)
Xvl_std = pd.DataFrame(Xvl_std)
Xvl_std.columns = df.iloc[:, 0:40].columns
Xts_std = scl.fit_transform(X1_test)
Xts_std = pd.DataFrame(Xts_std)
Xts_std.columns = df.iloc[:, 0:40].columns
In [ ]:
# Insert the first column of all 1s
Xtr_std['All_1'] = 1
Xtr_std.insert(0, 'All_1', Xtr_std.pop('All_1'))
Xvl_std['All_1'] = 1
Xvl_std.insert(0, 'All_1', Xvl_std.pop('All_1'))
Xts_std['All_1'] = 1
Xts_std.insert(0, 'All_1', Xts_std.pop('All_1'))
Part 2 - Linear Regression Using Direct Solution [1 marks]
Implement the direct solution of the linear regression problem on the training set. [0.5]
Note: You should use scipy.linalg.inv to perform the matrix inversion, as numpy.linalg.inv may cause numerical issues.
Report the root-mean-square error (RMSE) for both the training and validation sets. [0.5]
You may use mean_squared_error from Sklearn for computing the RMSE.
In [ ]:
# 1:
# Implement the direct solution of the linear regression problem on the training set.
W = np.dot(linalg.inv(np.dot(Xtr_std.T, Xtr_std)), np.dot(Xtr_std.T, y_train))
In [ ]:
y_train_pred = np.dot(Xtr_std, W)
y_val_pred = np.dot(Xvl_std, W)
In [ ]:
# 1:
# Report the root-mean-square error (RMSE) for both the training and validation sets.
print('The RMSE of standardized training set is: ', np.sqrt(mean_squared_error(y_train_pred, y_train)))
print('The RMSE of standardized validation set is: ', np.sqrt(mean_squared_error(y_val_pred, y_val)))
Part 3 - Full Batch Gradient Descent [2 marks]
The RMSE of standardized training set is: 0.000181615454715037
The RMSE of standardized validation set is: 0.00018403000289419626
We will now implement a "full batch" gradient descent algorithm and record the training time for our model. Recall that the full batch gradient descent is,
where is the learning rate and is your gradient, computed on the entire data.
Here are the steps for this part:
Implement gradient descent for linear regression using a fixed learning rate of , and iterate until your model's validation RMSE converges.
We consider the gradient descent as having converged when RMSE on the validation set using gradient descent satisfies:
where is the RMSE on the validation set using the direct solution that you have calculated in the previous part.
We refer to the quantity as the convergence threshold (CT).
Record the training time (from the first iteration until convergence) using the time.time() function. Be sure to compute the gradients yourself!
Take a look at the code provided in the tutorial. [0.5]
Plot the training RMSE and the validation RMSE vs. epoch on the same figure. [0.5]
Comment on overfitting/underfitting by observing the training and validation RMSE [1]
Hint: Initialize your weights with small random numbers (< )
= − α wt wt−1 gt
α gt
α = 0.01
RMS ≤ 1.001 ×RMSEGD EDirect Solution
RMSEDirect Solution
RMS × 1.001EDirect Solution
0.001
In [ ]:
(np.random.random(41))
In [ ]:
import time
start_time = time.time() ## Records current time
CT = 1.001 * np.sqrt(mean_squared_error(y_val_pred, y_val))
lr = 0.01
w = np.random.random(41)/1500
rmse_train = []
rmse_val = []
for epoch in range (0, 1500):
y_train_p = np.dot(Xtr_std, w.T)
rmse_train.append(np.sqrt(mean_squared_error(y_train_p, y_train)))
gradient = (1 / len(y_train_p)) * (np.dot(Xtr_std.T, y_train_p - y_train))
w = w - lr * gradient
y_val_p = np.dot(Xvl_std, w.T)
rmse_val_value = np.sqrt(mean_squared_error(y_val_p, y_val))
rmse_val.append(rmse_val_value)
print('Epoch {} done with rmse val = {}, CT = {}'.format(epoch, rmse_val_value, CT))
if np.sqrt(mean_squared_error(y_val_p, y_val)) <= CT:
print('Loop breaks at epoch = ', epoch)
break
print("--- Total Training Time: %s (s) ---" % (time.time() - start_time))
Out[ ]:
array([0.99005339, 0.40187839, 0.74079032, 0.70957531, 0.94749885,
0.32595822, 0.40486802, 0.50385905, 0.08968792, 0.78185192,
0.65739725, 0.70435831, 0.76225397, 0.28109569, 0.56646975,
0.27255346, 0.68183414, 0.5858784 , 0.78941335, 0.64134047,
0.33142455, 0.77132565, 0.76388711, 0.41956906, 0.15063067,
0.29680645, 0.89510859, 0.66710086, 0.65975106, 0.08489766,
0.99586304, 0.34496044, 0.40701317, 0.50469564, 0.49941891,
0.10419883, 0.91130101, 0.34872421, 0.96049002, 0.58452437,
0.13296187])
Epoch 0 done with rmse val = 0.005044755128855716, CT = 0.00018421403289709044
Epoch 1 done with rmse val = 0.004355113825555625, CT = 0.00018421403289709044
Epoch 2 done with rmse val = 0.0037975512614875236, CT = 0.00018421403289709044
Epoch 3 done with rmse val = 0.0033502706362310546, CT = 0.00018421403289709044
Epoch 4 done with rmse val = 0.002994397518787471, CT = 0.00018421403289709044
Epoch 5 done with rmse val = 0.002713467625344955, CT = 0.00018421403289709044
Epoch 6 done with rmse val = 0.002493118494109315, CT = 0.00018421403289709044
Epoch 7 done with rmse val = 0.0023209390184699467, CT = 0.00018421403289709044
Epoch 8 done with rmse val = 0.002186399103093229, CT = 0.00018421403289709044
Epoch 9 done with rmse val = 0.0020807795394066054, CT = 0.00018421403289709044
Epoch 10 done with rmse val = 0.0019970511913146153, CT = 0.00018421403289709044
Epoch 11 done with rmse val = 0.0019296930029653082, CT = 0.00018421403289709044
Epoch 12 done with rmse val = 0.0018744681501961652, CT = 0.00018421403289709044
Epoch 13 done with rmse val = 0.00182818819638647, CT = 0.00018421403289709044
Epoch 14 done with rmse val = 0.0017884908733822693, CT = 0.00018421403289709044
Epoch 15 done with rmse val = 0.0017536468993614815, CT = 0.00018421403289709044
Epoch 16 done with rmse val = 0.001722401536032142, CT = 0.00018421403289709044
Epoch 17 done with rmse val = 0.0016938499987242772, CT = 0.00018421403289709044
Epoch 18 done with rmse val = 0.0016673424277287312, CT = 0.00018421403289709044
Epoch 19 done with rmse val = 0.001642412987692755, CT = 0.00018421403289709044
Epoch 20 done with rmse val = 0.0016187278061809988, CT = 0.00018421403289709044
Epoch 21 done with rmse val = 0.0015960471850331296, CT = 0.00018421403289709044
Epoch 22 done with rmse val = 0.001574198401198882, CT = 0.00018421403289709044
Epoch 23 done with rmse val = 0.0015530562495890024, CT = 0.00018421403289709044
Epoch 24 done with rmse val = 0.001532529187319462, CT = 0.00018421403289709044
Epoch 25 done with rmse val = 0.0015125495003230065, CT = 0.00018421403289709044
Epoch 26 done with rmse val = 0.0014930663427481722, CT = 0.00018421403289709044
Epoch 27 done with rmse val = 0.0014740408198975815, CT = 0.00018421403289709044
Epoch 28 done with rmse val = 0.0014554425204410914, CT = 0.00018421403289709044
Epoch 29 done with rmse val = 0.0014372470740473353, CT = 0.00018421403289709044
Epoch 30 done with rmse val = 0.0014194344331678172, CT = 0.00018421403289709044
Epoch 31 done with rmse val = 0.0014019876653987846, CT = 0.00018421403289709044
Epoch 32 done with rmse val = 0.0013848921053171805, CT = 0.00018421403289709044
Epoch 33 done with rmse val = 0.0013681347590587247, CT = 0.00018421403289709044
Epoch 34 done with rmse val = 0.0013517038863522306, CT = 0.00018421403289709044
Epoch 35 done with rmse val = 0.0013355887069718313, CT = 0.00018421403289709044
Epoch 36 done with rmse val = 0.0013197791942869352, CT = 0.00018421403289709044
Epoch 37 done with rmse val = 0.0013042659296819924, CT = 0.00018421403289709044
Epoch 38 done with rmse val = 0.0012890399994377967, CT = 0.00018421403289709044
Epoch 39 done with rmse val = 0.0012740929211731802, CT = 0.00018421403289709044
Epoch 40 done with rmse val = 0.0012594165908205925, CT = 0.00018421403289709044
Epoch 41 done with rmse val = 0.0012450032438321977, CT = 0.00018421403289709044
Epoch 42 done with rmse val = 0.0012308454262247288, CT = 0.00018421403289709044
Epoch 43 done with rmse val = 0.0012169359724114605, CT = 0.00018421403289709044
Epoch 44 done with rmse val = 0.0012032679877076577, CT = 0.00018421403289709044
Epoch 45 done with rmse val = 0.0011898348340511908, CT = 0.00018421403289709044
Epoch 46 done with rmse val = 0.0011766301179368332, CT = 0.00018421403289709044
Epoch 47 done with rmse val = 0.0011636476798803667, CT = 0.00018421403289709044
Epoch 48 done with rmse val = 0.0011508815849487024, CT = 0.00018421403289709044
Epoch 49 done with rmse val = 0.0011383261140441388, CT = 0.00018421403289709044
Epoch 50 done with rmse val = 0.0011259757557352144, CT = 0.00018421403289709044
Epoch 51 done with rmse val = 0.001113825198497815, CT = 0.00018421403289709044
Epoch 52 done with rmse val = 0.0011018693232784395, CT = 0.00018421403289709044
Epoch 53 done with rmse val = 0.0010901031963238595, CT = 0.00018421403289709044
Epoch 54 done with rmse val = 0.0010785220622428016, CT = 0.00018421403289709044
Epoch 55 done with rmse val = 0.0010671213372792075, CT = 0.00018421403289709044
Epoch 56 done with rmse val = 0.0010558966027854398, CT = 0.00018421403289709044
Epoch 57 done with rmse val = 0.0010448435988892123, CT = 0.00018421403289709044
Epoch 58 done with rmse val = 0.0010339582183511468, CT = 0.00018421403289709044
Epoch 59 done with rmse val = 0.0010232365006114985, CT = 0.00018421403289709044
Epoch 60 done with rmse val = 0.00101267462602528, CT = 0.00018421403289709044
Epoch 61 done with rmse val = 0.0010022689102851142, CT = 0.00018421403289709044
Epoch 62 done with rmse val = 0.0009920157990309041, CT = 0.00018421403289709044
Epoch 63 done with rmse val = 0.0009819118626449835, CT = 0.00018421403289709044
Epoch 64 done with rmse val = 0.0009719537912309016, CT = 0.00018421403289709044
Epoch 65 done with rmse val = 0.0009621383897734893, CT = 0.00018421403289709044
Epoch 66 done with rmse val = 0.0009524625734773443, CT = 0.00018421403289709044
Epoch 67 done with rmse val = 0.0009429233632804441, CT = 0.00018421403289709044
Epoch 68 done with rmse val = 0.0009335178815391992, CT = 0.00018421403289709044
Epoch 69 done with rmse val = 0.000924243347880943, CT = 0.00018421403289709044
Epoch 70 done with rmse val = 0.0009150970752195836, CT = 0.00018421403289709044
Epoch 71 done with rmse val = 0.0009060764659299486, CT = 0.00018421403289709044
Epoch 72 done with rmse val = 0.0008971790081761923, CT = 0.00018421403289709044
Epoch 73 done with rmse val = 0.0008884022723895394, CT = 0.00018421403289709044
Epoch 74 done with rmse val = 0.0008797439078905677, CT = 0.00018421403289709044
Epoch 75 done with rmse val = 0.0008712016396512129, CT = 0.00018421403289709044
Epoch 76 done with rmse val = 0.0008627732651916752, CT = 0.00018421403289709044
Epoch 77 done with rmse val = 0.0008544566516074422, CT = 0.00018421403289709044
Epoch 78 done with rmse val = 0.0008462497327216923, CT = 0.00018421403289709044
Epoch 79 done with rmse val = 0.0008381505063584098, CT = 0.00018421403289709044
Epoch 80 done with rmse val = 0.000830157031731632, CT = 0.00018421403289709044
Epoch 81 done with rmse val = 0.0008222674269463382, CT = 0.00018421403289709044
Epoch 82 done with rmse val = 0.000814479866606603, CT = 0.00018421403289709044
Epoch 83 done with rmse val = 0.0008067925795267419, CT = 0.00018421403289709044
Epoch 84 done with rmse val = 0.0007992038465412999, CT = 0.00018421403289709044
Epoch 85 done with rmse val = 0.0007917119984098547, CT = 0.00018421403289709044
Epoch 86 done with rmse val = 0.000784315413812729, CT = 0.00018421403289709044
Epoch 87 done with rmse val = 0.0007770125174338332, CT = 0.00018421403289709044
Epoch 88 done with rmse val = 0.0007698017781269873, CT = 0.00018421403289709044
Epoch 89 done with rmse val = 0.0007626817071621954, CT = 0.00018421403289709044
Epoch 90 done with rmse val = 0.0007556508565484713, CT = 0.00018421403289709044
Epoch 91 done with rmse val = 0.0007487078174299386, CT = 0.00018421403289709044
Epoch 92 done with rmse val = 0.00074185121855205, CT = 0.00018421403289709044
Epoch 93 done with rmse val = 0.0007350797247948895, CT = 0.00018421403289709044
Epoch 94 done with rmse val = 0.0007283920357706389, CT = 0.00018421403289709044
Epoch 95 done with rmse val = 0.0007217868844824049, CT = 0.00018421403289709044
Epoch 96 done with rmse val = 0.0007152630360417125, CT = 0.00018421403289709044
Epoch 97 done with rmse val = 0.0007088192864420806, CT = 0.00018421403289709044
Epoch 98 done with rmse val = 0.0007024544613862001, CT = 0.00018421403289709044
Epoch 99 done with rmse val = 0.0006961674151643371, CT = 0.00018421403289709044
Epoch 100 done with rmse val = 0.000689957029581681, CT = 0.00018421403289709044
Epoch 101 done with rmse val = 0.0006838222129324564, CT = 0.00018421403289709044
Epoch 102 done with rmse val = 0.000677761899018704, CT = 0.00018421403289709044
Epoch 103 done with rmse val = 0.000671775046211732, CT = 0.00018421403289709044
Epoch 104 done with rmse val = 0.0006658606365543175, CT = 0.00018421403289709044
Epoch 105 done with rmse val = 0.0006600176749018283, CT = 0.00018421403289709044
Epoch 106 done with rmse val = 0.0006542451881005062, CT = 0.00018421403289709044
Epoch 107 done with rmse val = 0.000648542224201239, CT = 0.00018421403289709044
Epoch 108 done with rmse val = 0.0006429078517072108, CT = 0.00018421403289709044
Epoch 109 done with rmse val = 0.000637341158853905, CT = 0.00018421403289709044
Epoch 110 done with rmse val = 0.0006318412529199869, CT = 0.00018421403289709044
Epoch 111 done with rmse val = 0.00062640725956767, CT = 0.00018421403289709044
Epoch 112 done with rmse val = 0.0006210383222112286, CT = 0.00018421403289709044
Epoch 113 done with rmse val = 0.0006157336014123751, CT = 0.00018421403289709044
Epoch 114 done with rmse val = 0.0006104922743012865, CT = 0.00018421403289709044
Epoch 115 done with rmse val = 0.0006053135340221113, CT = 0.00018421403289709044
Epoch 116 done with rmse val = 0.0006001965892018467, CT = 0.00018421403289709044
Epoch 117 done with rmse val = 0.0005951406634415233, CT = 0.00018421403289709044
Epoch 118 done with rmse val = 0.0005901449948286846, CT = 0.00018421403289709044
Epoch 119 done with rmse val = 0.0005852088354701939, CT = 0.00018421403289709044
Epoch 120 done with rmse val = 0.0005803314510444456, CT = 0.00018421403289709044
Epoch 121 done with rmse val = 0.0005755121203721023, CT = 0.00018421403289709044
Epoch 122 done with rmse val = 0.0005707501350045153, CT = 0.00018421403289709044
Epoch 123 done with rmse val = 0.0005660447988290308, CT = 0.00018421403289709044
Epoch 124 done with rmse val = 0.000561395427690415, CT = 0.00018421403289709044
Epoch 125 done with rmse val = 0.0005568013490276716, CT = 0.00018421403289709044
Epoch 126 done with rmse val = 0.0005522619015255559, CT = 0.00018421403289709044
Epoch 127 done with rmse val = 0.0005477764347801248, CT = 0.00018421403289709044
Epoch 128 done with rmse val = 0.0005433443089776896, CT = 0.00018421403289709044
Epoch 129 done with rmse val = 0.0005389648945865705, CT = 0.00018421403289709044
Epoch 130 done with rmse val = 0.0005346375720610785, CT = 0.00018421403289709044
Epoch 131 done with rmse val = 0.0005303617315571802, CT = 0.00018421403289709044
Epoch 132 done with rmse val = 0.0005261367726593201, CT = 0.00018421403289709044
Epoch 133 done with rmse val = 0.0005219621041179073, CT = 0.00018421403289709044
Epoch 134 done with rmse val = 0.0005178371435969919, CT = 0.00018421403289709044
Epoch 135 done with rmse val = 0.0005137613174316795, CT = 0.00018421403289709044
Epoch 136 done with rmse val = 0.0005097340603948549, CT = 0.00018421403289709044
Epoch 137 done with rmse val = 0.0005057548154728044, CT = 0.00018421403289709044
Epoch 138 done with rmse val = 0.0005018230336493479, CT = 0.00018421403289709044
Epoch 139 done with rmse val = 0.0004979381736981074, CT = 0.00018421403289709044
Epoch 140 done with rmse val = 0.0004940997019825586, CT = 0.00018421403289709044
Epoch 141 done with rmse val = 0.0004903070922635279, CT = 0.00018421403289709044
Epoch 142 done with rmse val = 0.0004865598255138132, CT = 0.00018421403289709044
Epoch 143 done with rmse val = 0.0004828573897396225, CT = 0.00018421403289709044
Epoch 144 done with rmse val = 0.00047919927980853786, CT = 0.00018421403289709044
Epoch 145 done with rmse val = 0.0004755849972837264, CT = 0.00018421403289709044
Epoch 146 done with rmse val = 0.00047201405026413545, CT = 0.00018421403289709044
Epoch 147 done with rmse val = 0.000468485953230417, CT = 0.00018421403289709044
Epoch 148 done with rmse val = 0.000465000226896343, CT = 0.00018421403289709044
Epoch 149 done with rmse val = 0.0004615563980654802, CT = 0.00018421403289709044
Epoch 150 done with rmse val = 0.0004581539994929094, CT = 0.00018421403289709044
Epoch 151 done with rmse val = 0.00045479256975177807, CT = 0.00018421403289709044
Epoch 152 done with rmse val = 0.00045147165310449047, CT = 0.00018421403289709044
Epoch 153 done with rmse val = 0.00044819079937834477, CT = 0.00018421403289709044
Epoch 154 done with rmse val = 0.00044494956384543815, CT = 0.00018421403289709044
Epoch 155 done with rmse val = 0.0004417475071066682, CT = 0.00018421403289709044
Epoch 156 done with rmse val = 0.00043858419497966734, CT = 0.00018421403289709044
Epoch 157 done with rmse val = 0.00043545919839051335, CT = 0.00018421403289709044
Epoch 158 done with rmse val = 0.00043237209326906845, CT = 0.00018421403289709044
Epoch 159 done with rmse val = 0.00042932246044780526, CT = 0.00018421403289709044
Epoch 160 done with rmse val = 0.0004263098855639836, CT = 0.00018421403289709044
Epoch 161 done with rmse val = 0.0004233339589650496, CT = 0.00018421403289709044
Epoch 162 done with rmse val = 0.00042039427561713506, CT = 0.00018421403289709044
Epoch 163 done with rmse val = 0.00041749043501653794, CT = 0.00018421403289709044
Epoch 164 done with rmse val = 0.00041462204110407334, CT = 0.00018421403289709044
Epoch 165 done with rmse val = 0.00041178870218218724, CT = 0.00018421403289709044
Epoch 166 done with rmse val = 0.00040899003083473177, CT = 0.00018421403289709044
Epoch 167 done with rmse val = 0.00040622564384930256, CT = 0.00018421403289709044
Epoch 168 done with rmse val = 0.0004034951621420482, CT = 0.00018421403289709044
Epoch 169 done with rmse val = 0.00040079821068485835, CT = 0.00018421403289709044
Epoch 170 done with rmse val = 0.0003981344184348499, CT = 0.00018421403289709044
Epoch 171 done with rmse val = 0.0003955034182660659, CT = 0.00018421403289709044
Epoch 172 done with rmse val = 0.0003929048469033117, CT = 0.00018421403289709044
Epoch 173 done with rmse val = 0.0003903383448580523, CT = 0.00018421403289709044
Epoch 174 done with rmse val = 0.0003878035563663006, CT = 0.00018421403289709044
Epoch 175 done with rmse val = 0.0003853001293284268, CT = 0.00018421403289709044
Epoch 176 done with rmse val = 0.00038282771525082403, CT = 0.00018421403289709044
Epoch 177 done with rmse val = 0.00038038596918936744, CT = 0.00018421403289709044
Epoch 178 done with rmse val = 0.0003779745496946061, CT = 0.00018421403289709044
Epoch 179 done with rmse val = 0.0003755931187586288, CT = 0.00018421403289709044
Epoch 180 done with rmse val = 0.00037324134176355014, CT = 0.00018421403289709044
Epoch 181 done with rmse val = 0.0003709188874315612, CT = 0.00018421403289709044
Epoch 182 done with rmse val = 0.0003686254277764942, CT = 0.00018421403289709044
Epoch 183 done with rmse val = 0.0003663606380568526, CT = 0.00018421403289709044
Epoch 184 done with rmse val = 0.0003641241967302564, CT = 0.00018421403289709044
Epoch 185 done with rmse val = 0.00036191578540925915, CT = 0.00018421403289709044
Epoch 186 done with rmse val = 0.0003597350888184911, CT = 0.00018421403289709044
Epoch 187 done with rmse val = 0.0003575817947530859, CT = 0.00018421403289709044
Epoch 188 done with rmse val = 0.00035545559403834913, CT = 0.00018421403289709044
Epoch 189 done with rmse val = 0.00035335618049062953, CT = 0.00018421403289709044
Epoch 190 done with rmse val = 0.0003512832508793543, CT = 0.00018421403289709044
Epoch 191 done with rmse val = 0.0003492365048901902, CT = 0.00018421403289709044
Epoch 192 done with rmse val = 0.00034721564508929443, CT = 0.00018421403289709044
Epoch 193 done with rmse val = 0.0003452203768886219, CT = 0.00018421403289709044
Epoch 194 done with rmse val = 0.00034325040851225286, CT = 0.00018421403289709044
Epoch 195 done with rmse val = 0.00034130545096370873, CT = 0.00018421403289709044
Epoch 196 done with rmse val = 0.00033938521799422466, CT = 0.00018421403289709044
Epoch 197 done with rmse val = 0.0003374894260719464, CT = 0.00018421403289709044
Epoch 198 done with rmse val = 0.00033561779435202325, CT = 0.00018421403289709044
Epoch 199 done with rmse val = 0.00033377004464756554, CT = 0.00018421403289709044
Epoch 200 done with rmse val = 0.0003319459014014391, CT = 0.00018421403289709044
Epoch 201 done with rmse val = 0.0003301450916588696, CT = 0.00018421403289709044
Epoch 202 done with rmse val = 0.00032836734504082814, CT = 0.00018421403289709044
Epoch 203 done with rmse val = 0.000326612393718172, CT = 0.00018421403289709044
Epoch 204 done with rmse val = 0.0003248799723865158, CT = 0.00018421403289709044
Epoch 205 done with rmse val = 0.00032316981824180693, CT = 0.00018421403289709044
Epoch 206 done with rmse val = 0.0003214816709565804, CT = 0.00018421403289709044
Epoch 207 done with rmse val = 0.0003198152726568702, CT = 0.00018421403289709044
Epoch 208 done with rmse val = 0.0003181703678997528, CT = 0.00018421403289709044
Epoch 209 done with rmse val = 0.0003165467036515004, CT = 0.00018421403289709044
Epoch 210 done with rmse val = 0.0003149440292663212, CT = 0.00018421403289709044
Epoch 211 done with rmse val = 0.00031336209646566534, CT = 0.00018421403289709044
Epoch 212 done with rmse val = 0.0003118006593180744, CT = 0.00018421403289709044
Epoch 213 done with rmse val = 0.0003102594742195544, CT = 0.00018421403289709044
Epoch 214 done with rmse val = 0.0003087382998744511, CT = 0.00018421403289709044
Epoch 215 done with rmse val = 0.0003072368972768079, CT = 0.00018421403289709044
Epoch 216 done with rmse val = 0.00030575502969218757, CT = 0.00018421403289709044
Epoch 217 done with rmse val = 0.0003042924626399368, CT = 0.00018421403289709044
Epoch 218 done with rmse val = 0.00030284896387587705, CT = 0.00018421403289709044
Epoch 219 done with rmse val = 0.0003014243033754022, CT = 0.00018421403289709044
Epoch 220 done with rmse val = 0.0003000182533169654, CT = 0.00018421403289709044
Epoch 221 done with rmse val = 0.00029863058806593783, CT = 0.00018421403289709044
Epoch 222 done with rmse val = 0.0002972610841588217, CT = 0.00018421403289709044
Epoch 223 done with rmse val = 0.0002959095202878023, CT = 0.00018421403289709044
Epoch 224 done with rmse val = 0.00029457567728562157, CT = 0.00018421403289709044
Epoch 225 done with rmse val = 0.00029325933811075725, CT = 0.00018421403289709044
Epoch 226 done with rmse val = 0.0002919602878328935, CT = 0.00018421403289709044
Epoch 227 done with rmse val = 0.0002906783136186667, CT = 0.00018421403289709044
Epoch 228 done with rmse val = 0.00028941320471767145, CT = 0.00018421403289709044
Epoch 229 done with rmse val = 0.00028816475244871476, CT = 0.00018421403289709044
Epoch 230 done with rmse val = 0.00028693275018630137, CT = 0.00018421403289709044
Epoch 231 done with rmse val = 0.0002857169933473386, CT = 0.00018421403289709044
Epoch 232 done with rmse val = 0.0002845172793780481, CT = 0.00018421403289709044
Epoch 233 done with rmse val = 0.0002833334077410694, CT = 0.00018421403289709044
Epoch 234 done with rmse val = 0.00028216517990274496, CT = 0.00018421403289709044
Epoch 235 done with rmse val = 0.00028101239932057446, CT = 0.00018421403289709044
Epoch 236 done with rmse val = 0.0002798748714308252, CT = 0.00018421403289709044
Epoch 237 done with rmse val = 0.00027875240363628873, CT = 0.00018421403289709044
Epoch 238 done with rmse val = 0.00027764480529417326, CT = 0.00018421403289709044
Epoch 239 done with rmse val = 0.00027655188770411864, CT = 0.00018421403289709044
Epoch 240 done with rmse val = 0.0002754734640963274, CT = 0.00018421403289709044
Epoch 241 done with rmse val = 0.0002744093496197991, CT = 0.00018421403289709044
Epoch 242 done with rmse val = 0.00027335936133065966, CT = 0.00018421403289709044
Epoch 243 done with rmse val = 0.0002723233181805774, CT = 0.00018421403289709044
Epoch 244 done with rmse val = 0.00027130104100525523, CT = 0.00018421403289709044
Epoch 245 done with rmse val = 0.0002702923525129923, CT = 0.00018421403289709044
Epoch 246 done with rmse val = 0.0002692970772733062, CT = 0.00018421403289709044
Epoch 247 done with rmse val = 0.0002683150417056081, CT = 0.00018421403289709044
Epoch 248 done with rmse val = 0.0002673460740679242, CT = 0.00018421403289709044
Epoch 249 done with rmse val = 0.000266390004445656, CT = 0.00018421403289709044
Epoch 250 done with rmse val = 0.00026544666474037187, CT = 0.00018421403289709044
Epoch 251 done with rmse val = 0.0002645158886586262, CT = 0.00018421403289709044
Epoch 252 done with rmse val = 0.00026359751170079693, CT = 0.00018421403289709044
Epoch 253 done with rmse val = 0.0002626913711499389, CT = 0.00018421403289709044
Epoch 254 done with rmse val = 0.0002617973060606451, CT = 0.00018421403289709044
Epoch 255 done with rmse val = 0.00026091515724791295, CT = 0.00018421403289709044
Epoch 256 done with rmse val = 0.0002600447672760097, CT = 0.00018421403289709044
Epoch 257 done with rmse val = 0.0002591859804473325, CT = 0.00018421403289709044
Epoch 258 done with rmse val = 0.0002583386427912607, CT = 0.00018421403289709044
Epoch 259 done with rmse val = 0.0002575026020529943, CT = 0.00018421403289709044
Epoch 260 done with rmse val = 0.00025667770768237686, CT = 0.00018421403289709044
Epoch 261 done with rmse val = 0.00025586381082269913, CT = 0.00018421403289709044
Epoch 262 done with rmse val = 0.00025506076429948, CT = 0.00018421403289709044
Epoch 263 done with rmse val = 0.00025426842260922353, CT = 0.00018421403289709044
Epoch 264 done with rmse val = 0.00025348664190814803, CT = 0.00018421403289709044
Epoch 265 done with rmse val = 0.00025271528000088596, CT = 0.00018421403289709044
Epoch 266 done with rmse val = 0.00025195419632915344, CT = 0.00018421403289709044
Epoch 267 done with rmse val = 0.0002512032519603862, CT = 0.00018421403289709044
Epoch 268 done with rmse val = 0.000250462309576342, CT = 0.00018421403289709044
Epoch 269 done with rmse val = 0.0002497312334616681, CT = 0.00018421403289709044
Epoch 270 done with rmse val = 0.00024900988949243176, CT = 0.00018421403289709044
Epoch 271 done with rmse val = 0.00024829814512461533, CT = 0.00018421403289709044
Epoch 272 done with rmse val = 0.00024759586938257274, CT = 0.00018421403289709044
Epoch 273 done with rmse val = 0.00024690293284744975, CT = 0.00018421403289709044
Epoch 274 done with rmse val = 0.00024621920764556583, CT = 0.00018421403289709044
Epoch 275 done with rmse val = 0.00024554456743675876, CT = 0.00018421403289709044
Epoch 276 done with rmse val = 0.0002448788874026923, CT = 0.00018421403289709044
Epoch 277 done with rmse val = 0.0002442220442351269, CT = 0.00018421403289709044
Epoch 278 done with rmse val = 0.00024357391612415431, CT = 0.00018421403289709044
Epoch 279 done with rmse val = 0.00024293438274639717, CT = 0.00018421403289709044
Epoch 280 done with rmse val = 0.00024230332525317384, CT = 0.00018421403289709044
Epoch 281 done with rmse val = 0.00024168062625862997, CT = 0.00018421403289709044
Epoch 282 done with rmse val = 0.0002410661698278381, CT = 0.00018421403289709044
Epoch 283 done with rmse val = 0.0002404598414648664, CT = 0.00018421403289709044
Epoch 284 done with rmse val = 0.0002398615281008178, CT = 0.00018421403289709044
Epoch 285 done with rmse val = 0.00023927111808184153, CT = 0.00018421403289709044
Epoch 286 done with rmse val = 0.00023868850115711838, CT = 0.00018421403289709044
Epoch 287 done with rmse val = 0.0002381135684668217, CT = 0.00018421403289709044
Epoch 288 done with rmse val = 0.00023754621253005553, CT = 0.00018421403289709044
Epoch 289 done with rmse val = 0.0002369863272327722, CT = 0.00018421403289709044
Epoch 290 done with rmse val = 0.00023643380781567148, CT = 0.00018421403289709044
Epoch 291 done with rmse val = 0.00023588855086208283, CT = 0.00018421403289709044
Epoch 292 done with rmse val = 0.00023535045428583343, CT = 0.00018421403289709044
Epoch 293 done with rmse val = 0.0002348194173191038, CT = 0.00018421403289709044
Epoch 294 done with rmse val = 0.00023429534050027373, CT = 0.00018421403289709044
Epoch 295 done with rmse val = 0.0002337781256617599, CT = 0.00018421403289709044
Epoch 296 done with rmse val = 0.00023326767591784887, CT = 0.00018421403289709044
Epoch 297 done with rmse val = 0.0002327638956525265, CT = 0.00018421403289709044
Epoch 298 done with rmse val = 0.000232266690507307, CT = 0.00018421403289709044
Epoch 299 done with rmse val = 0.00023177596736906388, CT = 0.00018421403289709044
Epoch 300 done with rmse val = 0.00023129163435786483, CT = 0.00018421403289709044
Epoch 301 done with rmse val = 0.00023081360081481369, CT = 0.00018421403289709044
Epoch 302 done with rmse val = 0.00023034177728990126, CT = 0.00018421403289709044
Epoch 303 done with rmse val = 0.0002298760755298682, CT = 0.00018421403289709044
Epoch 304 done with rmse val = 0.0002294164084660814, CT = 0.00018421403289709044
Epoch 305 done with rmse val = 0.00022896269020242777, CT = 0.00018421403289709044
Epoch 306 done with rmse val = 0.00022851483600322612, CT = 0.00018421403289709044
Epoch 307 done with rmse val = 0.0002280727622811616, CT = 0.00018421403289709044
Epoch 308 done with rmse val = 0.00022763638658524315, CT = 0.00018421403289709044
Epoch 309 done with rmse val = 0.00022720562758878765, CT = 0.00018421403289709044
Epoch 310 done with rmse val = 0.00022678040507743337, CT = 0.00018421403289709044
Epoch 311 done with rmse val = 0.00022636063993718345, CT = 0.00018421403289709044
Epoch 312 done with rmse val = 0.0002259462541424839, CT = 0.00018421403289709044
Epoch 313 done with rmse val = 0.00022553717074433674, CT = 0.00018421403289709044
Epoch 314 done with rmse val = 0.00022513331385845152, CT = 0.00018421403289709044
Epoch 315 done with rmse val = 0.0002247346086534371, CT = 0.00018421403289709044
Epoch 316 done with rmse val = 0.0002243409813390358, CT = 0.00018421403289709044
Epoch 317 done with rmse val = 0.00022395235915440267, CT = 0.00018421403289709044
Epoch 318 done with rmse val = 0.00022356867035643117, CT = 0.00018421403289709044
Epoch 319 done with rmse val = 0.00022318984420812815, CT = 0.00018421403289709044
Epoch 320 done with rmse val = 0.00022281581096703933, CT = 0.00018421403289709044
Epoch 321 done with rmse val = 0.0002224465018737284, CT = 0.00018421403289709044
Epoch 322 done with rmse val = 0.00022208184914031053, CT = 0.00018421403289709044
Epoch 323 done with rmse val = 0.00022172178593904296, CT = 0.00018421403289709044
Epoch 324 done with rmse val = 0.00022136624639097416, CT = 0.00018421403289709044
Epoch 325 done with rmse val = 0.00022101516555465356, CT = 0.00018421403289709044
Epoch 326 done with rmse val = 0.00022066847941490346, CT = 0.00018421403289709044
Epoch 327 done with rmse val = 0.000220326124871655, CT = 0.00018421403289709044
Epoch 328 done with rmse val = 0.0002199880397288494, CT = 0.00018421403289709044
Epoch 329 done with rmse val = 0.00021965416268340685, CT = 0.00018421403289709044
Epoch 330 done with rmse val = 0.0002193244333142636, CT = 0.00018421403289709044
Epoch 331 done with rmse val = 0.00021899879207147954, CT = 0.00018421403289709044
Epoch 332 done with rmse val = 0.0002186771802654174, CT = 0.00018421403289709044
Epoch 333 done with rmse val = 0.00021835954005599466, CT = 0.00018421403289709044
Epoch 334 done with rmse val = 0.0002180458144420103, CT = 0.00018421403289709044
Epoch 335 done with rmse val = 0.0002177359472505465, CT = 0.00018421403289709044
Epoch 336 done with rmse val = 0.00021742988312644735, CT = 0.00018421403289709044
Epoch 337 done with rmse val = 0.00021712756752187578, CT = 0.00018421403289709044
Epoch 338 done with rmse val = 0.00021682894668594916, CT = 0.00018421403289709044
Epoch 339 done with rmse val = 0.000216533967654455, CT = 0.00018421403289709044
Epoch 340 done with rmse val = 0.0002162425782396478, CT = 0.00018421403289709044
Epoch 341 done with rmse val = 0.00021595472702012796, CT = 0.00018421403289709044
Epoch 342 done with rmse val = 0.0002156703633308036, CT = 0.00018421403289709044
Epoch 343 done with rmse val = 0.00021538943725293568, CT = 0.00018421403289709044
Epoch 344 done with rmse val = 0.00021511189960426818, CT = 0.00018421403289709044
Epoch 345 done with rmse val = 0.00021483770192924322, CT = 0.00018421403289709044
Epoch 346 done with rmse val = 0.00021456679648930202, CT = 0.00018421403289709044
Epoch 347 done with rmse val = 0.00021429913625327254, CT = 0.00018421403289709044
Epoch 348 done with rmse val = 0.00021403467488784433, CT = 0.00018421403289709044
Epoch 349 done with rmse val = 0.00021377336674813056, CT = 0.00018421403289709044
Epoch 350 done with rmse val = 0.0002135151668683189, CT = 0.00018421403289709044
Epoch 351 done with rmse val = 0.0002132600309524101, CT = 0.00018421403289709044
Epoch 352 done with rmse val = 0.0002130079153650465, CT = 0.00018421403289709044
Epoch 353 done with rmse val = 0.00021275877712242903, CT = 0.00018421403289709044
Epoch 354 done with rmse val = 0.00021251257388332423, CT = 0.00018421403289709044
Epoch 355 done with rmse val = 0.0002122692639401612, CT = 0.00018421403289709044
Epoch 356 done with rmse val = 0.0002120288062102183, CT = 0.00018421403289709044
Epoch 357 done with rmse val = 0.0002117911602269005, CT = 0.00018421403289709044
Epoch 358 done with rmse val = 0.00021155628613110693, CT = 0.00018421403289709044
Epoch 359 done with rmse val = 0.0002113241446626891, CT = 0.00018421403289709044
Epoch 360 done with rmse val = 0.00021109469715199948, CT = 0.00018421403289709044
Epoch 361 done with rmse val = 0.00021086790551153077, CT = 0.00018421403289709044
Epoch 362 done with rmse val = 0.00021064373222764608, CT = 0.00018421403289709044
Epoch 363 done with rmse val = 0.00021042214035239896, CT = 0.00018421403289709044
Epoch 364 done with rmse val = 0.0002102030934954447, CT = 0.00018421403289709044
Epoch 365 done with rmse val = 0.00020998655581604136, CT = 0.00018421403289709044
Epoch 366 done with rmse val = 0.0002097724920151413, CT = 0.00018421403289709044
Epoch 367 done with rmse val = 0.00020956086732757258, CT = 0.00018421403289709044
Epoch 368 done with rmse val = 0.00020935164751431057, CT = 0.00018421403289709044
Epoch 369 done with rmse val = 0.00020914479885483848, CT = 0.00018421403289709044
Epoch 370 done with rmse val = 0.00020894028813959786, CT = 0.00018421403289709044
Epoch 371 done with rmse val = 0.0002087380826625278, CT = 0.00018421403289709044
Epoch 372 done with rmse val = 0.0002085381502136929, CT = 0.00018421403289709044
Epoch 373 done with rmse val = 0.00020834045907199982, CT = 0.00018421403289709044
Epoch 374 done with rmse val = 0.00020814497799800127, CT = 0.00018421403289709044
Epoch 375 done with rmse val = 0.00020795167622678846, CT = 0.00018421403289709044
Epoch 376 done with rmse val = 0.00020776052346097005, CT = 0.00018421403289709044
Epoch 377 done with rmse val = 0.00020757148986373826, CT = 0.00018421403289709044
Epoch 378 done with rmse val = 0.0002073845460520211, CT = 0.00018421403289709044
Epoch 379 done with rmse val = 0.00020719966308972075, CT = 0.00018421403289709044
Epoch 380 done with rmse val = 0.00020701681248103685, CT = 0.00018421403289709044
Epoch 381 done with rmse val = 0.00020683596616387508, CT = 0.00018421403289709044
Epoch 382 done with rmse val = 0.00020665709650333973, CT = 0.00018421403289709044
Epoch 383 done with rmse val = 0.0002064801762853103, CT = 0.00018421403289709044
Epoch 384 done with rmse val = 0.00020630517871010124, CT = 0.00018421403289709044
Epoch 385 done with rmse val = 0.00020613207738620412, CT = 0.00018421403289709044
Epoch 386 done with rmse val = 0.00020596084632411218, CT = 0.00018421403289709044
Epoch 387 done with rmse val = 0.0002057914599302263, CT = 0.00018421403289709044
Epoch 388 done with rmse val = 0.0002056238930008415, CT = 0.00018421403289709044
Epoch 389 done with rmse val = 0.00020545812071621404, CT = 0.00018421403289709044
Epoch 390 done with rmse val = 0.00020529411863470776, CT = 0.00018421403289709044
Epoch 391 done with rmse val = 0.00020513186268701977, CT = 0.00018421403289709044
Epoch 392 done with rmse val = 0.000204971329170484, CT = 0.00018421403289709044
Epoch 393 done with rmse val = 0.00020481249474345255, CT = 0.00018421403289709044
Epoch 394 done with rmse val = 0.00020465533641975405, CT = 0.00018421403289709044
Epoch 395 done with rmse val = 0.000204499831563228, CT = 0.00018421403289709044
Epoch 396 done with rmse val = 0.0002043459578823352, CT = 0.00018421403289709044
Epoch 397 done with rmse val = 0.00020419369342484226, CT = 0.00018421403289709044
Epoch 398 done with rmse val = 0.00020404301657258125, CT = 0.00018421403289709044
Epoch 399 done with rmse val = 0.00020389390603628177, CT = 0.00018421403289709044
Epoch 400 done with rmse val = 0.00020374634085047662, CT = 0.00018421403289709044
Epoch 401 done with rmse val = 0.0002036003003684791, CT = 0.00018421403289709044
Epoch 402 done with rmse val = 0.00020345576425743157, CT = 0.00018421403289709044
Epoch 403 done with rmse val = 0.00020331271249342494, CT = 0.00018421403289709044
Epoch 404 done with rmse val = 0.00020317112535668752, CT = 0.00018421403289709044
Epoch 405 done with rmse val = 0.00020303098342684336, CT = 0.00018421403289709044
Epoch 406 done with rmse val = 0.0002028922675782391, CT = 0.00018421403289709044
Epoch 407 done with rmse val = 0.00020275495897533767, CT = 0.00018421403289709044
Epoch 408 done with rmse val = 0.00020261903906818038, CT = 0.00018421403289709044
Epoch 409 done with rmse val = 0.00020248448958791349, CT = 0.00018421403289709044
Epoch 410 done with rmse val = 0.00020235129254238162, CT = 0.00018421403289709044
Epoch 411 done with rmse val = 0.0002022194302117851, CT = 0.00018421403289709044
Epoch 412 done with rmse val = 0.00020208888514440155, CT = 0.00018421403289709044
Epoch 413 done with rmse val = 0.00020195964015237063, CT = 0.00018421403289709044
Epoch 414 done with rmse val = 0.0002018316783075415, CT = 0.00018421403289709044
Epoch 415 done with rmse val = 0.00020170498293738175, CT = 0.00018421403289709044
Epoch 416 done with rmse val = 0.0002015795376209475, CT = 0.00018421403289709044
Epoch 417 done with rmse val = 0.000201455326184914, CT = 0.00018421403289709044
Epoch 418 done with rmse val = 0.00020133233269966554, CT = 0.00018421403289709044
Epoch 419 done with rmse val = 0.00020121054147544413, CT = 0.00018421403289709044
Epoch 420 done with rmse val = 0.00020108993705855697, CT = 0.00018421403289709044
Epoch 421 done with rmse val = 0.00020097050422764027, CT = 0.00018421403289709044
Epoch 422 done with rmse val = 0.0002008522279899809, CT = 0.00018421403289709044
Epoch 423 done with rmse val = 0.00020073509357789316, CT = 0.00018421403289709044
Epoch 424 done with rmse val = 0.00020061908644515114, CT = 0.00018421403289709044
Epoch 425 done with rmse val = 0.0002005041922634758, CT = 0.00018421403289709044
Epoch 426 done with rmse val = 0.00020039039691907565, CT = 0.00018421403289709044
Epoch 427 done with rmse val = 0.00020027768650924066, CT = 0.00018421403289709044
Epoch 428 done with rmse val = 0.00020016604733898873, CT = 0.00018421403289709044
Epoch 429 done with rmse val = 0.0002000554659177638, CT = 0.00018421403289709044
Epoch 430 done with rmse val = 0.0001999459289561853, CT = 0.00018421403289709044
Epoch 431 done with rmse val = 0.00019983742336284737, CT = 0.00018421403289709044
Epoch 432 done with rmse val = 0.00019972993624116895, CT = 0.00018421403289709044
Epoch 433 done with rmse val = 0.0001996234548862916, CT = 0.00018421403289709044
Epoch 434 done with rmse val = 0.00019951796678202677, CT = 0.00018421403289709044
Epoch 435 done with rmse val = 0.0001994134595978502, CT = 0.00018421403289709044
Epoch 436 done with rmse val = 0.0001993099211859437, CT = 0.00018421403289709044
Epoch 437 done with rmse val = 0.00019920733957828326, CT = 0.00018421403289709044
Epoch 438 done with rmse val = 0.00019910570298377271, CT = 0.00018421403289709044
Epoch 439 done with rmse val = 0.00019900499978542306, CT = 0.00018421403289709044
Epoch 440 done with rmse val = 0.00019890521853757598, CT = 0.00018421403289709044
Epoch 441 done with rmse val = 0.00019880634796317097, CT = 0.00018421403289709044
Epoch 442 done with rmse val = 0.00019870837695105633, CT = 0.00018421403289709044
Epoch 443 done with rmse val = 0.00019861129455334242, CT = 0.00018421403289709044
Epoch 444 done with rmse val = 0.00019851508998279703, CT = 0.00018421403289709044
Epoch 445 done with rmse val = 0.00019841975261028204, CT = 0.00018421403289709044
Epoch 446 done with rmse val = 0.0001983252719622313, CT = 0.00018421403289709044
Epoch 447 done with rmse val = 0.00019823163771816844, CT = 0.00018421403289709044
Epoch 448 done with rmse val = 0.00019813883970826418, CT = 0.00018421403289709044
Epoch 449 done with rmse val = 0.00019804686791093334, CT = 0.00018421403289709044
Epoch 450 done with rmse val = 0.00019795571245046967, CT = 0.00018421403289709044
Epoch 451 done with rmse val = 0.00019786536359471906, CT = 0.00018421403289709044
Epoch 452 done with rmse val = 0.00019777581175278985, CT = 0.00018421403289709044
Epoch 453 done with rmse val = 0.00019768704747280034, CT = 0.00018421403289709044
Epoch 454 done with rmse val = 0.00019759906143966191, CT = 0.00018421403289709044
Epoch 455 done with rmse val = 0.00019751184447289877, CT = 0.00018421403289709044
Epoch 456 done with rmse val = 0.00019742538752450217, CT = 0.00018421403289709044
Epoch 457 done with rmse val = 0.00019733968167681945, CT = 0.00018421403289709044
Epoch 458 done with rmse val = 0.0001972547181404775, CT = 0.00018421403289709044
Epoch 459 done with rmse val = 0.00019717048825233975, CT = 0.00018421403289709044
Epoch 460 done with rmse val = 0.0001970869834734958, CT = 0.00018421403289709044
Epoch 461 done with rmse val = 0.0001970041953872844, CT = 0.00018421403289709044
Epoch 462 done with rmse val = 0.00019692211569734811, CT = 0.00018421403289709044
Epoch 463 done with rmse val = 0.00019684073622571965, CT = 0.00018421403289709044
Epoch 464 done with rmse val = 0.00019676004891093927, CT = 0.00018421403289709044
Epoch 465 done with rmse val = 0.00019668004580620325, CT = 0.00018421403289709044
Epoch 466 done with rmse val = 0.00019660071907754166, CT = 0.00018421403289709044
Epoch 467 done with rmse val = 0.00019652206100202708, CT = 0.00018421403289709044
Epoch 468 done with rmse val = 0.00019644406396601127, CT = 0.00018421403289709044
Epoch 469 done with rmse val = 0.00019636672046339167, CT = 0.00018421403289709044
Epoch 470 done with rmse val = 0.0001962900230939057, CT = 0.00018421403289709044
Epoch 471 done with rmse val = 0.00019621396456145302, CT = 0.00018421403289709044
Epoch 472 done with rmse val = 0.0001961385376724454, CT = 0.00018421403289709044
Epoch 473 done with rmse val = 0.0001960637353341835, CT = 0.00018421403289709044
Epoch 474 done with rmse val = 0.00019598955055326035, CT = 0.00018421403289709044
Epoch 475 done with rmse val = 0.00019591597643399088, CT = 0.00018421403289709044
Epoch 476 done with rmse val = 0.00019584300617686733, CT = 0.00018421403289709044
Epoch 477 done with rmse val = 0.00019577063307704001, CT = 0.00018421403289709044
Epoch 478 done with rmse val = 0.00019569885052282302, CT = 0.00018421403289709044
Epoch 479 done with rmse val = 0.0001956276519942243, CT = 0.00018421403289709044
Epoch 480 done with rmse val = 0.0001955570310615003, CT = 0.00018421403289709044
Epoch 481 done with rmse val = 0.00019548698138373405, CT = 0.00018421403289709044
Epoch 482 done with rmse val = 0.00019541749670743669, CT = 0.00018421403289709044
Epoch 483 done with rmse val = 0.0001953485708651722, CT = 0.00018421403289709044
Epoch 484 done with rmse val = 0.00019528019777420463, CT = 0.00018421403289709044
Epoch 485 done with rmse val = 0.0001952123714351675, CT = 0.00018421403289709044
Epoch 486 done with rmse val = 0.00019514508593075562, CT = 0.00018421403289709044
Epoch 487 done with rmse val = 0.00019507833542443771, CT = 0.00018421403289709044
Epoch 488 done with rmse val = 0.00019501211415919097, CT = 0.00018421403289709044
Epoch 489 done with rmse val = 0.0001949464164562561, CT = 0.00018421403289709044
Epoch 490 done with rmse val = 0.00019488123671391303, CT = 0.00018421403289709044
Epoch 491 done with rmse val = 0.00019481656940627677, CT = 0.00018421403289709044
Epoch 492 done with rmse val = 0.0001947524090821132, CT = 0.00018421403289709044
Epoch 493 done with rmse val = 0.0001946887503636744, CT = 0.00018421403289709044
Epoch 494 done with rmse val = 0.00019462558794555306, CT = 0.00018421403289709044
Epoch 495 done with rmse val = 0.00019456291659355614, CT = 0.00018421403289709044
Epoch 496 done with rmse val = 0.00019450073114359688, CT = 0.00018421403289709044
Epoch 497 done with rmse val = 0.0001944390265006049, CT = 0.00018421403289709044
Epoch 498 done with rmse val = 0.00019437779763745503, CT = 0.00018421403289709044
Epoch 499 done with rmse val = 0.00019431703959391295, CT = 0.00018421403289709044
Epoch 500 done with rmse val = 0.00019425674747559896, CT = 0.00018421403289709044
Epoch 501 done with rmse val = 0.00019419691645296833, CT = 0.00018421403289709044
Epoch 502 done with rmse val = 0.0001941375417603091, CT = 0.00018421403289709044
Epoch 503 done with rmse val = 0.0001940786186947556, CT = 0.00018421403289709044
Epoch 504 done with rmse val = 0.00019402014261531915, CT = 0.00018421403289709044
Epoch 505 done with rmse val = 0.00019396210894193425, CT = 0.00018421403289709044
Epoch 506 done with rmse val = 0.00019390451315452054, CT = 0.00018421403289709044
Epoch 507 done with rmse val = 0.00019384735079206058, CT = 0.00018421403289709044
Epoch 508 done with rmse val = 0.0001937906174516924, CT = 0.00018421403289709044
Epoch 509 done with rmse val = 0.0001937343087878174, CT = 0.00018421403289709044
Epoch 510 done with rmse val = 0.00019367842051122298, CT = 0.00018421403289709044
Epoch 511 done with rmse val = 0.00019362294838821926, CT = 0.00018421403289709044
Epoch 512 done with rmse val = 0.00019356788823979046, CT = 0.00018421403289709044
Epoch 513 done with rmse val = 0.00019351323594076026, CT = 0.00018421403289709044
Epoch 514 done with rmse val = 0.0001934589874189706, CT = 0.00018421403289709044
Epoch 515 done with rmse val = 0.00019340513865447433, CT = 0.00018421403289709044
Epoch 516 done with rmse val = 0.00019335168567874122, CT = 0.00018421403289709044
Epoch 517 done with rmse val = 0.00019329862457387675, CT = 0.00018421403289709044
Epoch 518 done with rmse val = 0.00019324595147185398, CT = 0.00018421403289709044
Epoch 519 done with rmse val = 0.00019319366255375815, CT = 0.00018421403289709044
Epoch 520 done with rmse val = 0.00019314175404904345, CT = 0.00018421403289709044
Epoch 521 done with rmse val = 0.00019309022223480254, CT = 0.00018421403289709044
Epoch 522 done with rmse val = 0.0001930390634350473, CT = 0.00018421403289709044
Epoch 523 done with rmse val = 0.00019298827402000233, CT = 0.00018421403289709044
Epoch 524 done with rmse val = 0.00019293785040540933, CT = 0.00018421403289709044
Epoch 525 done with rmse val = 0.00019288778905184316, CT = 0.00018421403289709044
Epoch 526 done with rmse val = 0.0001928380864640396, CT = 0.00018421403289709044
Epoch 527 done with rmse val = 0.00019278873919023333, CT = 0.00018421403289709044
Epoch 528 done with rmse val = 0.00019273974382150744, CT = 0.00018421403289709044
Epoch 529 done with rmse val = 0.00019269109699115297, CT = 0.00018421403289709044
Epoch 530 done with rmse val = 0.00019264279537404026, CT = 0.00018421403289709044
Epoch 531 done with rmse val = 0.0001925948356859987, CT = 0.00018421403289709044
Epoch 532 done with rmse val = 0.00019254721468320833, CT = 0.00018421403289709044
Epoch 533 done with rmse val = 0.00019249992916160053, CT = 0.00018421403289709044
Epoch 534 done with rmse val = 0.00019245297595626867, CT = 0.00018421403289709044
Epoch 535 done with rmse val = 0.00019240635194088876, CT = 0.00018421403289709044
Epoch 536 done with rmse val = 0.00019236005402714897, CT = 0.00018421403289709044
Epoch 537 done with rmse val = 0.0001923140791641891, CT = 0.00018421403289709044
Epoch 538 done with rmse val = 0.00019226842433804847, CT = 0.00018421403289709044
Epoch 539 done with rmse val = 0.00019222308657112393, CT = 0.00018421403289709044
Epoch 540 done with rmse val = 0.00019217806292163556, CT = 0.00018421403289709044
Epoch 541 done with rmse val = 0.0001921333504831017, CT = 0.00018421403289709044
Epoch 542 done with rmse val = 0.00019208894638382245, CT = 0.00018421403289709044
Epoch 543 done with rmse val = 0.0001920448477863717, CT = 0.00018421403289709044
Epoch 544 done with rmse val = 0.0001920010518870969, CT = 0.00018421403289709044
Epoch 545 done with rmse val = 0.0001919575559156278, CT = 0.00018421403289709044
Epoch 546 done with rmse val = 0.00019191435713439249, CT = 0.00018421403289709044
Epoch 547 done with rmse val = 0.00019187145283814153, CT = 0.00018421403289709044
Epoch 548 done with rmse val = 0.00019182884035347993, CT = 0.00018421403289709044
Epoch 549 done with rmse val = 0.0001917865170384065, CT = 0.00018421403289709044
Epoch 550 done with rmse val = 0.0001917444802818609, CT = 0.00018421403289709044
Epoch 551 done with rmse val = 0.0001917027275032777, CT = 0.00018421403289709044
Epoch 552 done with rmse val = 0.00019166125615214827, CT = 0.00018421403289709044
Epoch 553 done with rmse val = 0.0001916200637075888, CT = 0.00018421403289709044
Epoch 554 done with rmse val = 0.00019157914767791627, CT = 0.00018421403289709044
Epoch 555 done with rmse val = 0.00019153850560023076, CT = 0.00018421403289709044
Epoch 556 done with rmse val = 0.00019149813504000442, CT = 0.00018421403289709044
Epoch 557 done with rmse val = 0.0001914580335906774, CT = 0.00018421403289709044
Epoch 558 done with rmse val = 0.0001914181988732601, CT = 0.00018421403289709044
Epoch 559 done with rmse val = 0.0001913786285359418, CT = 0.00018421403289709044
Epoch 560 done with rmse val = 0.0001913393202537057, CT = 0.00018421403289709044
Epoch 561 done with rmse val = 0.00019130027172794983, CT = 0.00018421403289709044
Epoch 562 done with rmse val = 0.00019126148068611466, CT = 0.00018421403289709044
Epoch 563 done with rmse val = 0.000191222944881316, CT = 0.00018421403289709044
Epoch 564 done with rmse val = 0.00019118466209198422, CT = 0.00018421403289709044
Epoch 565 done with rmse val = 0.00019114663012150915, CT = 0.00018421403289709044
Epoch 566 done with rmse val = 0.00019110884679789043, CT = 0.00018421403289709044
Epoch 567 done with rmse val = 0.00019107130997339386, CT = 0.00018421403289709044
Epoch 568 done with rmse val = 0.00019103401752421288, CT = 0.00018421403289709044
Epoch 569 done with rmse val = 0.00019099696735013557, CT = 0.00018421403289709044
Epoch 570 done with rmse val = 0.00019096015737421707, CT = 0.00018421403289709044
Epoch 571 done with rmse val = 0.00019092358554245693, CT = 0.00018421403289709044
Epoch 572 done with rmse val = 0.0001908872498234819, CT = 0.00018421403289709044
Epoch 573 done with rmse val = 0.00019085114820823357, CT = 0.00018421403289709044
Epoch 574 done with rmse val = 0.00019081527870966106, CT = 0.00018421403289709044
Epoch 575 done with rmse val = 0.00019077963936241834, CT = 0.00018421403289709044
Epoch 576 done with rmse val = 0.00019074422822256686, CT = 0.00018421403289709044
Epoch 577 done with rmse val = 0.00019070904336728224, CT = 0.00018421403289709044
Epoch 578 done with rmse val = 0.00019067408289456605, CT = 0.00018421403289709044
Epoch 579 done with rmse val = 0.00019063934492296195, CT = 0.00018421403289709044
Epoch 580 done with rmse val = 0.00019060482759127647, CT = 0.00018421403289709044
Epoch 581 done with rmse val = 0.0001905705290583038, CT = 0.00018421403289709044
Epoch 582 done with rmse val = 0.00019053644750255543, CT = 0.00018421403289709044
Epoch 583 done with rmse val = 0.00019050258112199362, CT = 0.00018421403289709044
Epoch 584 done with rmse val = 0.0001904689281337692, CT = 0.00018421403289709044
Epoch 585 done with rmse val = 0.00019043548677396361, CT = 0.00018421403289709044
Epoch 586 done with rmse val = 0.00019040225529733454, CT = 0.00018421403289709044
Epoch 587 done with rmse val = 0.0001903692319770663, CT = 0.00018421403289709044
Epoch 588 done with rmse val = 0.00019033641510452317, CT = 0.00018421403289709044
Epoch 589 done with rmse val = 0.00019030380298900744, CT = 0.00018421403289709044
Epoch 590 done with rmse val = 0.00019027139395752063, CT = 0.00018421403289709044
Epoch 591 done with rmse val = 0.00019023918635452858, CT = 0.00018421403289709044
Epoch 592 done with rmse val = 0.0001902071785417304, CT = 0.00018421403289709044
Epoch 593 done with rmse val = 0.0001901753688978308, CT = 0.00018421403289709044
Epoch 594 done with rmse val = 0.0001901437558183158, CT = 0.00018421403289709044
Epoch 595 done with rmse val = 0.00019011233771523245, CT = 0.00018421403289709044
Epoch 596 done with rmse val = 0.00019008111301697128, CT = 0.00018421403289709044
Epoch 597 done with rmse val = 0.00019005008016805252, CT = 0.00018421403289709044
Epoch 598 done with rmse val = 0.00019001923762891558, CT = 0.00018421403289709044
Epoch 599 done with rmse val = 0.00018998858387571163, CT = 0.00018421403289709044
Epoch 600 done with rmse val = 0.00018995811740009952, CT = 0.00018421403289709044
Epoch 601 done with rmse val = 0.00018992783670904474, CT = 0.00018421403289709044
Epoch 602 done with rmse val = 0.00018989774032462134, CT = 0.00018421403289709044
Epoch 603 done with rmse val = 0.00018986782678381722, CT = 0.00018421403289709044
Epoch 604 done with rmse val = 0.000189838094638342, CT = 0.00018421403289709044
Epoch 605 done with rmse val = 0.00018980854245443824, CT = 0.00018421403289709044
Epoch 606 done with rmse val = 0.000189779168812695, CT = 0.00018421403289709044
Epoch 607 done with rmse val = 0.0001897499723078649, CT = 0.00018421403289709044
Epoch 608 done with rmse val = 0.00018972095154868327, CT = 0.00018421403289709044
Epoch 609 done with rmse val = 0.00018969210515769056, CT = 0.00018421403289709044
Epoch 610 done with rmse val = 0.0001896634317710572, CT = 0.00018421403289709044
Epoch 611 done with rmse val = 0.00018963493003841132, CT = 0.00018421403289709044
Epoch 612 done with rmse val = 0.00018960659862266854, CT = 0.00018421403289709044
Epoch 613 done with rmse val = 0.00018957843619986505, CT = 0.00018421403289709044
Epoch 614 done with rmse val = 0.00018955044145899276, CT = 0.00018421403289709044
Epoch 615 done with rmse val = 0.00018952261310183698, CT = 0.00018421403289709044
Epoch 616 done with rmse val = 0.00018949494984281642, CT = 0.00018421403289709044
Epoch 617 done with rmse val = 0.0001894674504088262, CT = 0.00018421403289709044
Epoch 618 done with rmse val = 0.00018944011353908214, CT = 0.00018421403289709044
Epoch 619 done with rmse val = 0.00018941293798496858, CT = 0.00018421403289709044
Epoch 620 done with rmse val = 0.0001893859225098874, CT = 0.00018421403289709044
Epoch 621 done with rmse val = 0.00018935906588911003, CT = 0.00018421403289709044
Epoch 622 done with rmse val = 0.0001893323669096313, CT = 0.00018421403289709044
Epoch 623 done with rmse val = 0.0001893058243700256, CT = 0.00018421403289709044
Epoch 624 done with rmse val = 0.000189279437080305, CT = 0.00018421403289709044
Epoch 625 done with rmse val = 0.00018925320386177976, CT = 0.00018421403289709044
Epoch 626 done with rmse val = 0.00018922712354692053, CT = 0.00018421403289709044
Epoch 627 done with rmse val = 0.000189201194979223, CT = 0.00018421403289709044
Epoch 628 done with rmse val = 0.00018917541701307414, CT = 0.00018421403289709044
Epoch 629 done with rmse val = 0.00018914978851362058, CT = 0.00018421403289709044
Epoch 630 done with rmse val = 0.00018912430835663917, CT = 0.00018421403289709044
Epoch 631 done with rmse val = 0.00018909897542840874, CT = 0.00018421403289709044
Epoch 632 done with rmse val = 0.00018907378862558467, CT = 0.00018421403289709044
Epoch 633 done with rmse val = 0.00018904874685507443, CT = 0.00018421403289709044
Epoch 634 done with rmse val = 0.00018902384903391552, CT = 0.00018421403289709044
Epoch 635 done with rmse val = 0.00018899909408915484, CT = 0.00018421403289709044
Epoch 636 done with rmse val = 0.00018897448095772998, CT = 0.00018421403289709044
Epoch 637 done with rmse val = 0.00018895000858635224, CT = 0.00018421403289709044
Epoch 638 done with rmse val = 0.00018892567593139112, CT = 0.00018421403289709044
Epoch 639 done with rmse val = 0.00018890148195876093, CT = 0.00018421403289709044
Epoch 640 done with rmse val = 0.0001888774256438084, CT = 0.00018421403289709044
Epoch 641 done with rmse val = 0.0001888535059712025, CT = 0.00018421403289709044
Epoch 642 done with rmse val = 0.00018882972193482543, CT = 0.00018421403289709044
Epoch 643 done with rmse val = 0.00018880607253766554, CT = 0.00018421403289709044
Epoch 644 done with rmse val = 0.00018878255679171128, CT = 0.00018421403289709044
Epoch 645 done with rmse val = 0.0001887591737178473, CT = 0.00018421403289709044
Epoch 646 done with rmse val = 0.00018873592234575136, CT = 0.00018421403289709044
Epoch 647 done with rmse val = 0.00018871280171379318, CT = 0.00018421403289709044
Epoch 648 done with rmse val = 0.00018868981086893466, CT = 0.00018421403289709044
Epoch 649 done with rmse val = 0.00018866694886663128, CT = 0.00018421403289709044
Epoch 650 done with rmse val = 0.00018864421477073502, CT = 0.00018421403289709044
Epoch 651 done with rmse val = 0.00018862160765339893, CT = 0.00018421403289709044
Epoch 652 done with rmse val = 0.00018859912659498242, CT = 0.00018421403289709044
Epoch 653 done with rmse val = 0.00018857677068395844, CT = 0.00018421403289709044
Epoch 654 done with rmse val = 0.00018855453901682196, CT = 0.00018421403289709044
Epoch 655 done with rmse val = 0.000188532430697999, CT = 0.00018421403289709044
Epoch 656 done with rmse val = 0.00018851044483975807, CT = 0.00018421403289709044
Epoch 657 done with rmse val = 0.00018848858056212176, CT = 0.00018421403289709044
Epoch 658 done with rmse val = 0.00018846683699278038, CT = 0.00018421403289709044
Epoch 659 done with rmse val = 0.00018844521326700624, CT = 0.00018421403289709044
Epoch 660 done with rmse val = 0.0001884237085275694, CT = 0.00018421403289709044
Epoch 661 done with rmse val = 0.00018840232192465453, CT = 0.00018421403289709044
Epoch 662 done with rmse val = 0.00018838105261577895, CT = 0.00018421403289709044
Epoch 663 done with rmse val = 0.00018835989976571167, CT = 0.00018421403289709044
Epoch 664 done with rmse val = 0.00018833886254639382, CT = 0.00018421403289709044
Epoch 665 done with rmse val = 0.00018831794013685947, CT = 0.00018421403289709044
Epoch 666 done with rmse val = 0.00018829713172315872, CT = 0.00018421403289709044
Epoch 667 done with rmse val = 0.00018827643649828077, CT = 0.00018421403289709044
Epoch 668 done with rmse val = 0.00018825585366207836, CT = 0.00018421403289709044
Epoch 669 done with rmse val = 0.0001882353824211936, CT = 0.00018421403289709044
Epoch 670 done with rmse val = 0.00018821502198898416, CT = 0.00018421403289709044
Epoch 671 done with rmse val = 0.0001881947715854512, CT = 0.00018421403289709044
Epoch 672 done with rmse val = 0.00018817463043716752, CT = 0.00018421403289709044
Epoch 673 done with rmse val = 0.00018815459777720724, CT = 0.00018421403289709044
Epoch 674 done with rmse val = 0.0001881346728450763, CT = 0.00018421403289709044
Epoch 675 done with rmse val = 0.00018811485488664349, CT = 0.00018421403289709044
Epoch 676 done with rmse val = 0.00018809514315407306, CT = 0.00018421403289709044
Epoch 677 done with rmse val = 0.0001880755369057579, CT = 0.00018421403289709044
Epoch 678 done with rmse val = 0.00018805603540625326, CT = 0.00018421403289709044
Epoch 679 done with rmse val = 0.0001880366379262121, CT = 0.00018421403289709044
Epoch 680 done with rmse val = 0.00018801734374232075, CT = 0.00018421403289709044
Epoch 681 done with rmse val = 0.00018799815213723544, CT = 0.00018421403289709044
Epoch 682 done with rmse val = 0.0001879790623995201, CT = 0.00018421403289709044
Epoch 683 done with rmse val = 0.0001879600738235843, CT = 0.00018421403289709044
Epoch 684 done with rmse val = 0.00018794118570962287, CT = 0.00018421403289709044
Epoch 685 done with rmse val = 0.00018792239736355526, CT = 0.00018421403289709044
Epoch 686 done with rmse val = 0.00018790370809696678, CT = 0.00018421403289709044
Epoch 687 done with rmse val = 0.00018788511722704973, CT = 0.00018421403289709044
Epoch 688 done with rmse val = 0.0001878666240765459, CT = 0.00018421403289709044
Epoch 689 done with rmse val = 0.00018784822797368939, CT = 0.00018421403289709044
Epoch 690 done with rmse val = 0.00018782992825215042, CT = 0.00018421403289709044
Epoch 691 done with rmse val = 0.00018781172425097993, CT = 0.00018421403289709044
Epoch 692 done with rmse val = 0.00018779361531455455, CT = 0.00018421403289709044
Epoch 693 done with rmse val = 0.0001877756007925227, CT = 0.00018421403289709044
Epoch 694 done with rmse val = 0.00018775768003975095, CT = 0.00018421403289709044
Epoch 695 done with rmse val = 0.00018773985241627157, CT = 0.00018421403289709044
Epoch 696 done with rmse val = 0.0001877221172872302, CT = 0.00018421403289709044
Epoch 697 done with rmse val = 0.00018770447402283476, CT = 0.00018421403289709044
Epoch 698 done with rmse val = 0.00018768692199830437, CT = 0.00018421403289709044
Epoch 699 done with rmse val = 0.0001876694605938196, CT = 0.00018421403289709044
Epoch 700 done with rmse val = 0.00018765208919447286, CT = 0.00018421403289709044
Epoch 701 done with rmse val = 0.00018763480719021953, CT = 0.00018421403289709044
Epoch 702 done with rmse val = 0.00018761761397582973, CT = 0.00018421403289709044
Epoch 703 done with rmse val = 0.00018760050895084087, CT = 0.00018421403289709044
Epoch 704 done with rmse val = 0.0001875834915195105, CT = 0.00018421403289709044
Epoch 705 done with rmse val = 0.00018756656109077, CT = 0.00018421403289709044
Epoch 706 done with rmse val = 0.00018754971707817856, CT = 0.00018421403289709044
Epoch 707 done with rmse val = 0.00018753295889987817, CT = 0.00018421403289709044
Epoch 708 done with rmse val = 0.00018751628597854855, CT = 0.00018421403289709044
Epoch 709 done with rmse val = 0.00018749969774136343, CT = 0.00018421403289709044
Epoch 710 done with rmse val = 0.0001874831936199465, CT = 0.00018421403289709044
Epoch 711 done with rmse val = 0.00018746677305032873, CT = 0.00018421403289709044
Epoch 712 done with rmse val = 0.0001874504354729056, CT = 0.00018421403289709044
Epoch 713 done with rmse val = 0.00018743418033239506, CT = 0.00018421403289709044
Epoch 714 done with rmse val = 0.00018741800707779618, CT = 0.00018421403289709044
Epoch 715 done with rmse val = 0.00018740191516234805, CT = 0.00018421403289709044
Epoch 716 done with rmse val = 0.00018738590404348928, CT = 0.00018421403289709044
Epoch 717 done with rmse val = 0.00018736997318281797, CT = 0.00018421403289709044
Epoch 718 done with rmse val = 0.00018735412204605236, CT = 0.00018421403289709044
Epoch 719 done with rmse val = 0.00018733835010299152, CT = 0.00018421403289709044
Epoch 720 done with rmse val = 0.00018732265682747706, CT = 0.00018421403289709044
Epoch 721 done with rmse val = 0.00018730704169735464, CT = 0.00018421403289709044
Epoch 722 done with rmse val = 0.00018729150419443674, CT = 0.00018421403289709044
Epoch 723 done with rmse val = 0.00018727604380446524, CT = 0.00018421403289709044
Epoch 724 done with rmse val = 0.00018726066001707465, CT = 0.00018421403289709044
Epoch 725 done with rmse val = 0.00018724535232575584, CT = 0.00018421403289709044
Epoch 726 done with rmse val = 0.00018723012022782014, CT = 0.00018421403289709044
Epoch 727 done with rmse val = 0.00018721496322436385, CT = 0.00018421403289709044
Epoch 728 done with rmse val = 0.0001871998808202333, CT = 0.00018421403289709044
Epoch 729 done with rmse val = 0.00018718487252399003, CT = 0.00018421403289709044
Epoch 730 done with rmse val = 0.0001871699378478768, CT = 0.00018421403289709044
Epoch 731 done with rmse val = 0.00018715507630778352, CT = 0.00018421403289709044
Epoch 732 done with rmse val = 0.00018714028742321405, CT = 0.00018421403289709044
Epoch 733 done with rmse val = 0.00018712557071725295, CT = 0.00018421403289709044
Epoch 734 done with rmse val = 0.000187110925716533, CT = 0.00018421403289709044
Epoch 735 done with rmse val = 0.00018709635195120295, CT = 0.00018421403289709044
Epoch 736 done with rmse val = 0.0001870818489548953, CT = 0.00018421403289709044
Epoch 737 done with rmse val = 0.00018706741626469523, CT = 0.00018421403289709044
Epoch 738 done with rmse val = 0.00018705305342110902, CT = 0.00018421403289709044
Epoch 739 done with rmse val = 0.00018703875996803354, CT = 0.00018421403289709044
Epoch 740 done with rmse val = 0.0001870245354527256, CT = 0.00018421403289709044
Epoch 741 done with rmse val = 0.00018701037942577198, CT = 0.00018421403289709044
Epoch 742 done with rmse val = 0.00018699629144105953, CT = 0.00018421403289709044
Epoch 743 done with rmse val = 0.00018698227105574573, CT = 0.00018421403289709044
Epoch 744 done with rmse val = 0.00018696831783022986, CT = 0.00018421403289709044
Epoch 745 done with rmse val = 0.00018695443132812393, CT = 0.00018421403289709044
Epoch 746 done with rmse val = 0.00018694061111622435, CT = 0.00018421403289709044
Epoch 747 done with rmse val = 0.00018692685676448384, CT = 0.00018421403289709044
Epoch 748 done with rmse val = 0.0001869131678459836, CT = 0.00018421403289709044
Epoch 749 done with rmse val = 0.00018689954393690572, CT = 0.00018421403289709044
Epoch 750 done with rmse val = 0.00018688598461650627, CT = 0.00018421403289709044
Epoch 751 done with rmse val = 0.00018687248946708802, CT = 0.00018421403289709044
Epoch 752 done with rmse val = 0.00018685905807397423, CT = 0.00018421403289709044
Epoch 753 done with rmse val = 0.00018684569002548208, CT = 0.00018421403289709044
Epoch 754 done with rmse val = 0.0001868323849128968, CT = 0.00018421403289709044
Epoch 755 done with rmse val = 0.0001868191423304461, CT = 0.00018421403289709044
Epoch 756 done with rmse val = 0.00018680596187527432, CT = 0.00018421403289709044
Epoch 757 done with rmse val = 0.00018679284314741763, CT = 0.00018421403289709044
Epoch 758 done with rmse val = 0.00018677978574977916, CT = 0.00018421403289709044
Epoch 759 done with rmse val = 0.0001867667892881041, CT = 0.00018421403289709044
Epoch 760 done with rmse val = 0.00018675385337095572, CT = 0.00018421403289709044
Epoch 761 done with rmse val = 0.00018674097760969094, CT = 0.00018421403289709044
Epoch 762 done with rmse val = 0.00018672816161843683, CT = 0.00018421403289709044
Epoch 763 done with rmse val = 0.00018671540501406694, CT = 0.00018421403289709044
Epoch 764 done with rmse val = 0.0001867027074161778, CT = 0.00018421403289709044
Epoch 765 done with rmse val = 0.00018669006844706617, CT = 0.00018421403289709044
Epoch 766 done with rmse val = 0.00018667748773170607, CT = 0.00018421403289709044
Epoch 767 done with rmse val = 0.0001866649648977263, CT = 0.00018421403289709044
Epoch 768 done with rmse val = 0.000186652499575388, CT = 0.00018421403289709044
Epoch 769 done with rmse val = 0.00018664009139756255, CT = 0.00018421403289709044
Epoch 770 done with rmse val = 0.00018662773999970995, CT = 0.00018421403289709044
Epoch 771 done with rmse val = 0.00018661544501985688, CT = 0.00018421403289709044
Epoch 772 done with rmse val = 0.0001866032060985756, CT = 0.00018421403289709044
Epoch 773 done with rmse val = 0.00018659102287896262, CT = 0.00018421403289709044
Epoch 774 done with rmse val = 0.00018657889500661766, CT = 0.00018421403289709044
Epoch 775 done with rmse val = 0.0001865668221296232, CT = 0.00018421403289709044
Epoch 776 done with rmse val = 0.00018655480389852372, CT = 0.00018421403289709044
Epoch 777 done with rmse val = 0.00018654283996630546, CT = 0.00018421403289709044
Epoch 778 done with rmse val = 0.00018653092998837645, CT = 0.00018421403289709044
Epoch 779 done with rmse val = 0.00018651907362254652, CT = 0.00018421403289709044
Epoch 780 done with rmse val = 0.00018650727052900754, CT = 0.00018421403289709044
Epoch 781 done with rmse val = 0.0001864955203703142, CT = 0.00018421403289709044
Epoch 782 done with rmse val = 0.00018648382281136454, CT = 0.00018421403289709044
Epoch 783 done with rmse val = 0.00018647217751938084, CT = 0.00018421403289709044
Epoch 784 done with rmse val = 0.00018646058416389093, CT = 0.00018421403289709044
Epoch 785 done with rmse val = 0.0001864490424167094, CT = 0.00018421403289709044
Epoch 786 done with rmse val = 0.000186437551951919, CT = 0.00018421403289709044
Epoch 787 done with rmse val = 0.0001864261124458525, CT = 0.00018421403289709044
Epoch 788 done with rmse val = 0.00018641472357707449, CT = 0.00018421403289709044
Epoch 789 done with rmse val = 0.00018640338502636344, CT = 0.00018421403289709044
Epoch 790 done with rmse val = 0.0001863920964766939, CT = 0.00018421403289709044
Epoch 791 done with rmse val = 0.00018638085761321898, CT = 0.00018421403289709044
Epoch 792 done with rmse val = 0.00018636966812325285, CT = 0.00018421403289709044
Epoch 793 done with rmse val = 0.00018635852769625377, CT = 0.00018421403289709044
Epoch 794 done with rmse val = 0.0001863474360238066, CT = 0.00018421403289709044
Epoch 795 done with rmse val = 0.00018633639279960615, CT = 0.00018421403289709044
Epoch 796 done with rmse val = 0.00018632539771944057, CT = 0.00018421403289709044
Epoch 797 done with rmse val = 0.00018631445048117459, CT = 0.00018421403289709044
Epoch 798 done with rmse val = 0.0001863035507847332, CT = 0.00018421403289709044
Epoch 799 done with rmse val = 0.00018629269833208528, CT = 0.00018421403289709044
Epoch 800 done with rmse val = 0.00018628189282722784, CT = 0.00018421403289709044
Epoch 801 done with rmse val = 0.00018627113397616974, CT = 0.00018421403289709044
Epoch 802 done with rmse val = 0.00018626042148691614, CT = 0.00018421403289709044
Epoch 803 done with rmse val = 0.00018624975506945296, CT = 0.00018421403289709044
Epoch 804 done with rmse val = 0.00018623913443573106, CT = 0.00018421403289709044
Epoch 805 done with rmse val = 0.00018622855929965128, CT = 0.00018421403289709044
Epoch 806 done with rmse val = 0.0001862180293770492, CT = 0.00018421403289709044
Epoch 807 done with rmse val = 0.00018620754438567998, CT = 0.00018421403289709044
Epoch 808 done with rmse val = 0.00018619710404520375, CT = 0.00018421403289709044
Epoch 809 done with rmse val = 0.0001861867080771705, CT = 0.00018421403289709044
Epoch 810 done with rmse val = 0.00018617635620500592, CT = 0.00018421403289709044
Epoch 811 done with rmse val = 0.00018616604815399658, CT = 0.00018421403289709044
Epoch 812 done with rmse val = 0.00018615578365127584, CT = 0.00018421403289709044
Epoch 813 done with rmse val = 0.00018614556242580965, CT = 0.00018421403289709044
Epoch 814 done with rmse val = 0.00018613538420838243, CT = 0.00018421403289709044
Epoch 815 done with rmse val = 0.00018612524873158322, CT = 0.00018421403289709044
Epoch 816 done with rmse val = 0.0001861151557297918, CT = 0.00018421403289709044
Epoch 817 done with rmse val = 0.00018610510493916515, CT = 0.00018421403289709044
Epoch 818 done with rmse val = 0.00018609509609762397, CT = 0.00018421403289709044
Epoch 819 done with rmse val = 0.00018608512894483908, CT = 0.00018421403289709044
Epoch 820 done with rmse val = 0.00018607520322221841, CT = 0.00018421403289709044
Epoch 821 done with rmse val = 0.00018606531867289367, CT = 0.00018421403289709044
Epoch 822 done with rmse val = 0.00018605547504170733, CT = 0.00018421403289709044
Epoch 823 done with rmse val = 0.00018604567207519983, CT = 0.00018421403289709044
Epoch 824 done with rmse val = 0.00018603590952159666, CT = 0.00018421403289709044
Epoch 825 done with rmse val = 0.00018602618713079575, CT = 0.00018421403289709044
Epoch 826 done with rmse val = 0.00018601650465435506, CT = 0.00018421403289709044
Epoch 827 done with rmse val = 0.00018600686184547984, CT = 0.00018421403289709044
Epoch 828 done with rmse val = 0.00018599725845901054, CT = 0.00018421403289709044
Epoch 829 done with rmse val = 0.0001859876942514106, CT = 0.00018421403289709044
Epoch 830 done with rmse val = 0.0001859781689807542, CT = 0.00018421403289709044
Epoch 831 done with rmse val = 0.0001859686824067144, CT = 0.00018421403289709044
Epoch 832 done with rmse val = 0.0001859592342905512, CT = 0.00018421403289709044
Epoch 833 done with rmse val = 0.00018594982439509977, CT = 0.00018421403289709044
Epoch 834 done with rmse val = 0.00018594045248475883, CT = 0.00018421403289709044
Epoch 835 done with rmse val = 0.0001859311183254791, CT = 0.00018421403289709044
Epoch 836 done with rmse val = 0.00018592182168475168, CT = 0.00018421403289709044
Epoch 837 done with rmse val = 0.00018591256233159686, CT = 0.00018421403289709044
Epoch 838 done with rmse val = 0.00018590334003655288, CT = 0.00018421403289709044
Epoch 839 done with rmse val = 0.00018589415457166464, CT = 0.00018421403289709044
Epoch 840 done with rmse val = 0.0001858850057104727, CT = 0.00018421403289709044
Epoch 841 done with rmse val = 0.0001858758932280024, CT = 0.00018421403289709044
Epoch 842 done with rmse val = 0.00018586681690075293, CT = 0.00018421403289709044
Epoch 843 done with rmse val = 0.00018585777650668658, CT = 0.00018421403289709044
Epoch 844 done with rmse val = 0.00018584877182521793, CT = 0.00018421403289709044
Epoch 845 done with rmse val = 0.0001858398026372037, CT = 0.00018421403289709044
Epoch 846 done with rmse val = 0.00018583086872493165, CT = 0.00018421403289709044
Epoch 847 done with rmse val = 0.00018582196987211076, CT = 0.00018421403289709044
Epoch 848 done with rmse val = 0.00018581310586386065, CT = 0.00018421403289709044
Epoch 849 done with rmse val = 0.0001858042764867012, CT = 0.00018421403289709044
Epoch 850 done with rmse val = 0.000185795481528543, CT = 0.00018421403289709044
Epoch 851 done with rmse val = 0.00018578672077867656, CT = 0.00018421403289709044
Epoch 852 done with rmse val = 0.00018577799402776305, CT = 0.00018421403289709044
Epoch 853 done with rmse val = 0.00018576930106782398, CT = 0.00018421403289709044
Epoch 854 done with rmse val = 0.00018576064169223167, CT = 0.00018421403289709044
Epoch 855 done with rmse val = 0.00018575201569569937, CT = 0.00018421403289709044
Epoch 856 done with rmse val = 0.0001857434228742719, CT = 0.00018421403289709044
Epoch 857 done with rmse val = 0.00018573486302531582, CT = 0.00018421403289709044
Epoch 858 done with rmse val = 0.00018572633594751028, CT = 0.00018421403289709044
Epoch 859 done with rmse val = 0.0001857178414408374, CT = 0.00018421403289709044
Epoch 860 done with rmse val = 0.0001857093793065732, CT = 0.00018421403289709044
Epoch 861 done with rmse val = 0.00018570094934727819, CT = 0.00018421403289709044
Epoch 862 done with rmse val = 0.00018569255136678838, CT = 0.00018421403289709044
Epoch 863 done with rmse val = 0.00018568418517020627, CT = 0.00018421403289709044
Epoch 864 done with rmse val = 0.00018567585056389167, CT = 0.00018421403289709044
Epoch 865 done with rmse val = 0.000185667547355453, CT = 0.00018421403289709044
Epoch 866 done with rmse val = 0.00018565927535373834, CT = 0.00018421403289709044
Epoch 867 done with rmse val = 0.00018565103436882682, CT = 0.00018421403289709044
Epoch 868 done with rmse val = 0.00018564282421201976, CT = 0.00018421403289709044
Epoch 869 done with rmse val = 0.0001856346446958321, CT = 0.00018421403289709044
Epoch 870 done with rmse val = 0.0001856264956339841, CT = 0.00018421403289709044
Epoch 871 done with rmse val = 0.00018561837684139256, CT = 0.00018421403289709044
Epoch 872 done with rmse val = 0.00018561028813416256, CT = 0.00018421403289709044
Epoch 873 done with rmse val = 0.0001856022293295792, CT = 0.00018421403289709044
Epoch 874 done with rmse val = 0.00018559420024609916, CT = 0.00018421403289709044
Epoch 875 done with rmse val = 0.00018558620070334287, CT = 0.00018421403289709044
Epoch 876 done with rmse val = 0.0001855782305220859, CT = 0.00018421403289709044
Epoch 877 done with rmse val = 0.00018557028952425125, CT = 0.00018421403289709044
Epoch 878 done with rmse val = 0.0001855623775329013, CT = 0.00018421403289709044
Epoch 879 done with rmse val = 0.00018555449437222989, CT = 0.00018421403289709044
Epoch 880 done with rmse val = 0.00018554663986755434, CT = 0.00018421403289709044
Epoch 881 done with rmse val = 0.00018553881384530775, CT = 0.00018421403289709044
Epoch 882 done with rmse val = 0.00018553101613303144, CT = 0.00018421403289709044
Epoch 883 done with rmse val = 0.00018552324655936683, CT = 0.00018421403289709044
Epoch 884 done with rmse val = 0.00018551550495404836, CT = 0.00018421403289709044
Epoch 885 done with rmse val = 0.00018550779114789553, CT = 0.00018421403289709044
Epoch 886 done with rmse val = 0.00018550010497280569, CT = 0.00018421403289709044
Epoch 887 done with rmse val = 0.00018549244626174646, CT = 0.00018421403289709044
Epoch 888 done with rmse val = 0.00018548481484874848, CT = 0.00018421403289709044
Epoch 889 done with rmse val = 0.00018547721056889793, CT = 0.00018421403289709044
Epoch 890 done with rmse val = 0.00018546963325832944, CT = 0.00018421403289709044
Epoch 891 done with rmse val = 0.00018546208275421896, CT = 0.00018421403289709044
Epoch 892 done with rmse val = 0.0001854545588947764, CT = 0.00018421403289709044
Epoch 893 done with rmse val = 0.00018544706151923872, CT = 0.00018421403289709044
Epoch 894 done with rmse val = 0.00018543959046786286, CT = 0.00018421403289709044
Epoch 895 done with rmse val = 0.0001854321455819188, CT = 0.00018421403289709044
Epoch 896 done with rmse val = 0.00018542472670368267, CT = 0.00018421403289709044
Epoch 897 done with rmse val = 0.00018541733367642978, CT = 0.00018421403289709044
Epoch 898 done with rmse val = 0.00018540996634442807, CT = 0.00018421403289709044
Epoch 899 done with rmse val = 0.000185402624552931, CT = 0.00018421403289709044
Epoch 900 done with rmse val = 0.00018539530814817121, CT = 0.00018421403289709044
Epoch 901 done with rmse val = 0.00018538801697735364, CT = 0.00018421403289709044
Epoch 902 done with rmse val = 0.00018538075088864905, CT = 0.00018421403289709044
Epoch 903 done with rmse val = 0.00018537350973118742, CT = 0.00018421403289709044
Epoch 904 done with rmse val = 0.00018536629335505174, CT = 0.00018421403289709044
Epoch 905 done with rmse val = 0.00018535910161127095, CT = 0.00018421403289709044
Epoch 906 done with rmse val = 0.0001853519343518142, CT = 0.00018421403289709044
Epoch 907 done with rmse val = 0.00018534479142958421, CT = 0.00018421403289709044
Epoch 908 done with rmse val = 0.00018533767269841107, CT = 0.00018421403289709044
Epoch 909 done with rmse val = 0.00018533057801304578, CT = 0.00018421403289709044
Epoch 910 done with rmse val = 0.00018532350722915444, CT = 0.00018421403289709044
Epoch 911 done with rmse val = 0.00018531646020331189, CT = 0.00018421403289709044
Epoch 912 done with rmse val = 0.00018530943679299534, CT = 0.00018421403289709044
Epoch 913 done with rmse val = 0.00018530243685657902, CT = 0.00018421403289709044
Epoch 914 done with rmse val = 0.0001852954602533275, CT = 0.00018421403289709044
Epoch 915 done with rmse val = 0.00018528850684339004, CT = 0.00018421403289709044
Epoch 916 done with rmse val = 0.00018528157648779473, CT = 0.00018421403289709044
Epoch 917 done with rmse val = 0.00018527466904844244, CT = 0.00018421403289709044
Epoch 918 done with rmse val = 0.00018526778438810108, CT = 0.00018421403289709044
Epoch 919 done with rmse val = 0.00018526092237039986, CT = 0.00018421403289709044
Epoch 920 done with rmse val = 0.0001852540828598235, CT = 0.00018421403289709044
Epoch 921 done with rmse val = 0.0001852472657217067, CT = 0.00018421403289709044
Epoch 922 done with rmse val = 0.00018524047082222806, CT = 0.00018421403289709044
Epoch 923 done with rmse val = 0.00018523369802840502, CT = 0.00018421403289709044
Epoch 924 done with rmse val = 0.00018522694720808798, CT = 0.00018421403289709044
Epoch 925 done with rmse val = 0.00018522021822995485, CT = 0.00018421403289709044
Epoch 926 done with rmse val = 0.00018521351096350552, CT = 0.00018421403289709044
Epoch 927 done with rmse val = 0.0001852068252790565, CT = 0.00018421403289709044
Epoch 928 done with rmse val = 0.0001852001610477355, CT = 0.00018421403289709044
Epoch 929 done with rmse val = 0.00018519351814147607, CT = 0.00018421403289709044
Epoch 930 done with rmse val = 0.00018518689643301218, CT = 0.00018421403289709044
Epoch 931 done with rmse val = 0.00018518029579587313, CT = 0.00018421403289709044
Epoch 932 done with rmse val = 0.00018517371610437823, CT = 0.00018421403289709044
Epoch 933 done with rmse val = 0.00018516715723363144, CT = 0.00018421403289709044
Epoch 934 done with rmse val = 0.00018516061905951648, CT = 0.00018421403289709044
Epoch 935 done with rmse val = 0.00018515410145869164, CT = 0.00018421403289709044
Epoch 936 done with rmse val = 0.00018514760430858444, CT = 0.00018421403289709044
Epoch 937 done with rmse val = 0.0001851411274873869, CT = 0.00018421403289709044
Epoch 938 done with rmse val = 0.00018513467087405045, CT = 0.00018421403289709044
Epoch 939 done with rmse val = 0.00018512823434828075, CT = 0.00018421403289709044
Epoch 940 done with rmse val = 0.000185121817790533, CT = 0.00018421403289709044
Epoch 941 done with rmse val = 0.00018511542108200706, CT = 0.00018421403289709044
Epoch 942 done with rmse val = 0.0001851090441046423, CT = 0.00018421403289709044
Epoch 943 done with rmse val = 0.00018510268674111296, CT = 0.00018421403289709044
Epoch 944 done with rmse val = 0.00018509634887482353, CT = 0.00018421403289709044
Epoch 945 done with rmse val = 0.00018509003038990352, CT = 0.00018421403289709044
Epoch 946 done with rmse val = 0.00018508373117120326, CT = 0.00018421403289709044
Epoch 947 done with rmse val = 0.00018507745110428884, CT = 0.00018421403289709044
Epoch 948 done with rmse val = 0.00018507119007543754, CT = 0.00018421403289709044
Epoch 949 done with rmse val = 0.0001850649479716334, CT = 0.00018421403289709044
Epoch 950 done with rmse val = 0.00018505872468056237, CT = 0.00018421403289709044
Epoch 951 done with rmse val = 0.00018505252009060782, CT = 0.00018421403289709044
Epoch 952 done with rmse val = 0.00018504633409084615, CT = 0.00018421403289709044
Epoch 953 done with rmse val = 0.0001850401665710421, CT = 0.00018421403289709044
Epoch 954 done with rmse val = 0.00018503401742164458, CT = 0.00018421403289709044
Epoch 955 done with rmse val = 0.00018502788653378179, CT = 0.00018421403289709044
Epoch 956 done with rmse val = 0.00018502177379925734, CT = 0.00018421403289709044
Epoch 957 done with rmse val = 0.00018501567911054554, CT = 0.00018421403289709044
Epoch 958 done with rmse val = 0.00018500960236078712, CT = 0.00018421403289709044
Epoch 959 done with rmse val = 0.000185003543443785, CT = 0.00018421403289709044
Epoch 960 done with rmse val = 0.0001849975022540001, CT = 0.00018421403289709044
Epoch 961 done with rmse val = 0.00018499147868654693, CT = 0.00018421403289709044
Epoch 962 done with rmse val = 0.00018498547263718935, CT = 0.00018421403289709044
Epoch 963 done with rmse val = 0.0001849794840023367, CT = 0.00018421403289709044
Epoch 964 done with rmse val = 0.00018497351267903917, CT = 0.00018421403289709044
Epoch 965 done with rmse val = 0.00018496755856498422, CT = 0.00018421403289709044
Epoch 966 done with rmse val = 0.000184961621558492, CT = 0.00018421403289709044
Epoch 967 done with rmse val = 0.00018495570155851158, CT = 0.00018421403289709044
Epoch 968 done with rmse val = 0.00018494979846461697, CT = 0.00018421403289709044
Epoch 969 done with rmse val = 0.00018494391217700268, CT = 0.00018421403289709044
Epoch 970 done with rmse val = 0.0001849380425964804, CT = 0.00018421403289709044
Epoch 971 done with rmse val = 0.00018493218962447442, CT = 0.00018421403289709044
Epoch 972 done with rmse val = 0.00018492635316301806, CT = 0.00018421403289709044
Epoch 973 done with rmse val = 0.0001849205331147498, CT = 0.00018421403289709044
Epoch 974 done with rmse val = 0.0001849147293829092, CT = 0.00018421403289709044
Epoch 975 done with rmse val = 0.00018490894187133324, CT = 0.00018421403289709044
Epoch 976 done with rmse val = 0.0001849031704844524, CT = 0.00018421403289709044
Epoch 977 done with rmse val = 0.0001848974151272869, CT = 0.00018421403289709044
Epoch 978 done with rmse val = 0.000184891675705443, CT = 0.00018421403289709044
Epoch 979 done with rmse val = 0.00018488595212510928, CT = 0.00018421403289709044
Epoch 980 done with rmse val = 0.00018488024429305272, CT = 0.00018421403289709044
Epoch 981 done with rmse val = 0.0001848745521166153, CT = 0.00018421403289709044
Epoch 982 done with rmse val = 0.00018486887550371012, CT = 0.00018421403289709044
Epoch 983 done with rmse val = 0.00018486321436281798, CT = 0.00018421403289709044
Epoch 984 done with rmse val = 0.00018485756860298357, CT = 0.00018421403289709044
Epoch 985 done with rmse val = 0.00018485193813381202, CT = 0.00018421403289709044
Epoch 986 done with rmse val = 0.00018484632286546522, CT = 0.00018421403289709044
Epoch 987 done with rmse val = 0.0001848407227086586, CT = 0.00018421403289709044
Epoch 988 done with rmse val = 0.00018483513757465707, CT = 0.00018421403289709044
Epoch 989 done with rmse val = 0.00018482956737527213, CT = 0.00018421403289709044
Epoch 990 done with rmse val = 0.000184824012022858, CT = 0.00018421403289709044
Epoch 991 done with rmse val = 0.00018481847143030834, CT = 0.00018421403289709044
Epoch 992 done with rmse val = 0.00018481294551105296, CT = 0.00018421403289709044
Epoch 993 done with rmse val = 0.0001848074341790541, CT = 0.00018421403289709044
Epoch 994 done with rmse val = 0.00018480193734880343, CT = 0.00018421403289709044
Epoch 995 done with rmse val = 0.00018479645493531838, CT = 0.00018421403289709044
Epoch 996 done with rmse val = 0.00018479098685413912, CT = 0.00018421403289709044
Epoch 997 done with rmse val = 0.00018478553302132502, CT = 0.00018421403289709044
Epoch 998 done with rmse val = 0.0001847800933534516, CT = 0.00018421403289709044
Epoch 999 done with rmse val = 0.0001847746677676071, CT = 0.00018421403289709044
Epoch 1000 done with rmse val = 0.00018476925618138927, CT = 0.00018421403289709044
Epoch 1001 done with rmse val = 0.00018476385851290221, CT = 0.00018421403289709044
Epoch 1002 done with rmse val = 0.0001847584746807532, CT = 0.00018421403289709044
Epoch 1003 done with rmse val = 0.0001847531046040496, CT = 0.00018421403289709044
Epoch 1004 done with rmse val = 0.00018474774820239546, CT = 0.00018421403289709044
Epoch 1005 done with rmse val = 0.00018474240539588868, CT = 0.00018421403289709044
Epoch 1006 done with rmse val = 0.00018473707610511763, CT = 0.00018421403289709044
Epoch 1007 done with rmse val = 0.00018473176025115849, CT = 0.00018421403289709044
Epoch 1008 done with rmse val = 0.00018472645775557167, CT = 0.00018421403289709044
Epoch 1009 done with rmse val = 0.0001847211685403992, CT = 0.00018421403289709044
Epoch 1010 done with rmse val = 0.0001847158925281614, CT = 0.00018421403289709044
Epoch 1011 done with rmse val = 0.00018471062964185418, CT = 0.00018421403289709044
Epoch 1012 done with rmse val = 0.00018470537980494584, CT = 0.00018421403289709044
Epoch 1013 done with rmse val = 0.00018470014294137417, CT = 0.00018421403289709044
Epoch 1014 done with rmse val = 0.0001846949189755435, CT = 0.00018421403289709044
Epoch 1015 done with rmse val = 0.00018468970783232187, CT = 0.00018421403289709044
Epoch 1016 done with rmse val = 0.00018468450943703806, CT = 0.00018421403289709044
Epoch 1017 done with rmse val = 0.0001846793237154787, CT = 0.00018421403289709044
Epoch 1018 done with rmse val = 0.00018467415059388544, CT = 0.00018421403289709044
Epoch 1019 done with rmse val = 0.00018466898999895212, CT = 0.00018421403289709044
Epoch 1020 done with rmse val = 0.00018466384185782187, CT = 0.00018421403289709044
Epoch 1021 done with rmse val = 0.00018465870609808441, CT = 0.00018421403289709044
Epoch 1022 done with rmse val = 0.00018465358264777327, CT = 0.00018421403289709044
Epoch 1023 done with rmse val = 0.00018464847143536285, CT = 0.00018421403289709044
Epoch 1024 done with rmse val = 0.000184643372389766, CT = 0.00018421403289709044
Epoch 1025 done with rmse val = 0.0001846382854403309, CT = 0.00018421403289709044
Epoch 1026 done with rmse val = 0.00018463321051683863, CT = 0.00018421403289709044
Epoch 1027 done with rmse val = 0.0001846281475495004, CT = 0.00018421403289709044
Epoch 1028 done with rmse val = 0.0001846230964689548, CT = 0.00018421403289709044
Epoch 1029 done with rmse val = 0.00018461805720626533, CT = 0.00018421403289709044
Epoch 1030 done with rmse val = 0.00018461302969291748, CT = 0.00018421403289709044
Epoch 1031 done with rmse val = 0.00018460801386081642, CT = 0.00018421403289709044
Epoch 1032 done with rmse val = 0.00018460300964228416, CT = 0.00018421403289709044
Epoch 1033 done with rmse val = 0.00018459801697005702, CT = 0.00018421403289709044
Epoch 1034 done with rmse val = 0.00018459303577728314, CT = 0.00018421403289709044
Epoch 1035 done with rmse val = 0.00018458806599751986, CT = 0.00018421403289709044
Epoch 1036 done with rmse val = 0.00018458310756473117, CT = 0.00018421403289709044
Epoch 1037 done with rmse val = 0.0001845781604132853, CT = 0.00018421403289709044
Epoch 1038 done with rmse val = 0.000184573224477952, CT = 0.00018421403289709044
Epoch 1039 done with rmse val = 0.00018456829969390023, CT = 0.00018421403289709044
Epoch 1040 done with rmse val = 0.00018456338599669568, CT = 0.00018421403289709044
Epoch 1041 done with rmse val = 0.00018455848332229826, CT = 0.00018421403289709044
Epoch 1042 done with rmse val = 0.00018455359160705968, CT = 0.00018421403289709044
Epoch 1043 done with rmse val = 0.000184548710787721, CT = 0.00018421403289709044
Epoch 1044 done with rmse val = 0.00018454384080141028, CT = 0.00018421403289709044
Epoch 1045 done with rmse val = 0.00018453898158564015, CT = 0.00018421403289709044
Epoch 1046 done with rmse val = 0.00018453413307830546, CT = 0.00018421403289709044
Epoch 1047 done with rmse val = 0.00018452929521768085, CT = 0.00018421403289709044
Epoch 1048 done with rmse val = 0.00018452446794241855, CT = 0.00018421403289709044
Epoch 1049 done with rmse val = 0.00018451965119154585, CT = 0.00018421403289709044
Epoch 1050 done with rmse val = 0.00018451484490446298, CT = 0.00018421403289709044
Epoch 1051 done with rmse val = 0.00018451004902094072, CT = 0.00018421403289709044
Epoch 1052 done with rmse val = 0.0001845052634811181, CT = 0.00018421403289709044
Epoch 1053 done with rmse val = 0.00018450048822550005, CT = 0.00018421403289709044
Epoch 1054 done with rmse val = 0.00018449572319495554, CT = 0.00018421403289709044
Epoch 1055 done with rmse val = 0.00018449096833071476, CT = 0.00018421403289709044
Epoch 1056 done with rmse val = 0.00018448622357436728, CT = 0.00018421403289709044
Epoch 1057 done with rmse val = 0.0001844814888678598, CT = 0.00018421403289709044
Epoch 1058 done with rmse val = 0.00018447676415349375, CT = 0.00018421403289709044
Epoch 1059 done with rmse val = 0.00018447204937392344, CT = 0.00018421403289709044
Epoch 1060 done with rmse val = 0.00018446734447215358, CT = 0.00018421403289709044
Epoch 1061 done with rmse val = 0.00018446264939153727, CT = 0.00018421403289709044
Epoch 1062 done with rmse val = 0.00018445796407577387, CT = 0.00018421403289709044
Epoch 1063 done with rmse val = 0.0001844532884689068, CT = 0.00018421403289709044
Epoch 1064 done with rmse val = 0.00018444862251532155, CT = 0.00018421403289709044
Epoch 1065 done with rmse val = 0.00018444396615974334, CT = 0.00018421403289709044
Epoch 1066 done with rmse val = 0.00018443931934723545, CT = 0.00018421403289709044
Epoch 1067 done with rmse val = 0.0001844346820231966, CT = 0.00018421403289709044
Epoch 1068 done with rmse val = 0.00018443005413335936, CT = 0.00018421403289709044
Epoch 1069 done with rmse val = 0.00018442543562378796, CT = 0.00018421403289709044
Epoch 1070 done with rmse val = 0.00018442082644087607, CT = 0.00018421403289709044
Epoch 1071 done with rmse val = 0.00018441622653134505, CT = 0.00018421403289709044
Epoch 1072 done with rmse val = 0.00018441163584224173, CT = 0.00018421403289709044
Epoch 1073 done with rmse val = 0.00018440705432093665, CT = 0.00018421403289709044
Epoch 1074 done with rmse val = 0.0001844024819151218, CT = 0.00018421403289709044
Epoch 1075 done with rmse val = 0.0001843979185728089, CT = 0.00018421403289709044
Epoch 1076 done with rmse val = 0.0001843933642423273, CT = 0.00018421403289709044
Epoch 1077 done with rmse val = 0.00018438881887232204, CT = 0.00018421403289709044
Epoch 1078 done with rmse val = 0.00018438428241175197, CT = 0.00018421403289709044
Epoch 1079 done with rmse val = 0.00018437975480988786, CT = 0.00018421403289709044
Epoch 1080 done with rmse val = 0.00018437523601631033, CT = 0.00018421403289709044
Epoch 1081 done with rmse val = 0.0001843707259809082, CT = 0.00018421403289709044
Epoch 1082 done with rmse val = 0.00018436622465387638, CT = 0.00018421403289709044
Epoch 1083 done with rmse val = 0.0001843617319857142, CT = 0.00018421403289709044
Epoch 1084 done with rmse val = 0.00018435724792722333, CT = 0.00018421403289709044
Epoch 1085 done with rmse val = 0.00018435277242950609, CT = 0.00018421403289709044
Epoch 1086 done with rmse val = 0.00018434830544396372, CT = 0.00018421403289709044
Epoch 1087 done with rmse val = 0.00018434384692229416, CT = 0.00018421403289709044
Epoch 1088 done with rmse val = 0.0001843393968164907, CT = 0.00018421403289709044
Epoch 1089 done with rmse val = 0.00018433495507883985, CT = 0.00018421403289709044
Epoch 1090 done with rmse val = 0.00018433052166191973, CT = 0.00018421403289709044
Epoch 1091 done with rmse val = 0.00018432609651859827, CT = 0.00018421403289709044
Epoch 1092 done with rmse val = 0.0001843216796020313, CT = 0.00018421403289709044
Epoch 1093 done with rmse val = 0.000184317270865661, CT = 0.00018421403289709044
Epoch 1094 done with rmse val = 0.000184312870263214, CT = 0.00018421403289709044
Epoch 1095 done with rmse val = 0.0001843084777486997, CT = 0.00018421403289709044
In [ ]:
plt.figure(figsize=(15,10))
plt.plot(rmse_train, label = 'Training RMSE')
plt.plot(rmse_val, label = 'Validation RMSE')
plt.xlabel('Epoch')
plt.ylabel('RMSE')
plt.legend()
plt.show()
Part 4 - Mini-batch and Stochastic Gradient Descent [4 marks]
Epoch 1096 done with rmse val = 0.0001843040932764087, CT = 0.00018421403289709044
Epoch 1097 done with rmse val = 0.0001842997168009107, CT = 0.00018421403289709044
Epoch 1098 done with rmse val = 0.0001842953482770532, CT = 0.00018421403289709044
Epoch 1099 done with rmse val = 0.0001842909876599596, CT = 0.00018421403289709044
Epoch 1100 done with rmse val = 0.0001842866349050277, CT = 0.00018421403289709044
Epoch 1101 done with rmse val = 0.00018428228996792765, CT = 0.00018421403289709044
Epoch 1102 done with rmse val = 0.00018427795280460073, CT = 0.00018421403289709044
Epoch 1103 done with rmse val = 0.0001842736233712575, CT = 0.00018421403289709044
Epoch 1104 done with rmse val = 0.00018426930162437617, CT = 0.00018421403289709044
Epoch 1105 done with rmse val = 0.00018426498752070094, CT = 0.00018421403289709044
Epoch 1106 done with rmse val = 0.00018426068101724043, CT = 0.00018421403289709044
Epoch 1107 done with rmse val = 0.00018425638207126616, CT = 0.00018421403289709044
Epoch 1108 done with rmse val = 0.00018425209064031072, CT = 0.00018421403289709044
Epoch 1109 done with rmse val = 0.00018424780668216647, CT = 0.00018421403289709044
Epoch 1110 done with rmse val = 0.00018424353015488373, CT = 0.00018421403289709044
Epoch 1111 done with rmse val = 0.00018423926101676928, CT = 0.00018421403289709044
Epoch 1112 done with rmse val = 0.00018423499922638498, CT = 0.00018421403289709044
Epoch 1113 done with rmse val = 0.00018423074474254586, CT = 0.00018421403289709044
Epoch 1114 done with rmse val = 0.00018422649752431892, CT = 0.00018421403289709044
Epoch 1115 done with rmse val = 0.0001842222575310215, CT = 0.00018421403289709044
Epoch 1116 done with rmse val = 0.00018421802472221954, CT = 0.00018421403289709044
Epoch 1117 done with rmse val = 0.00018421379905772655, CT = 0.00018421403289709044
Loop breaks at epoch = 1117
--- Total Training Time: 8.5181405544281 (s) ---
Write a function that performs mini-batch gradient descent until the convergence threshold (CT) is reached. [1]
The inputs of that function are:
Input data (training/validation), Batch size, Learning Rate, Convergence Threshold (CT)
Your function will return the following arrays:
The final weights after training.
The training RMSE at each epoch.
The validation RMSE at each epoch.
An array that has the elapsed time from the start of the training process to the end of each epoch (e.g., if each epoch takes exactly 2 seconds, the
array would look like: [2 4 6 8 ...]).
Notes:
For certain batch sizes, GD might not converge to a solution. For that reason, you need to check the RMSE of the validation/training set at each
epoch, and if it's getting larger and larger, you should stop the training for that case (the design is up to you!).
CT will help you to know when your model is converged.
Important: after each epoch, you need to shuffle the entire training set. This ensures that new mini-batches are selected for every epoch. Hint:
use np.random.permutation .
Let's now use the function to investigate the effect of batch size on convergence. When the batch size is 1, we call that stochastic gradient descent. When
the batch size equals the # of training data, it is full-batch (i.e., all data points are used at every iteration). Anywhere in between is mini-batch (we use
some of the data).
Sweep different values for the mini-batch size (at least 5 values that converge), each time using a learning rate of . Hint: Try batch sizes
that are powers of two (e.g., 2,4,8,16,32,64,128...). These batch sizes fit better on the hardware and may achieve higher performance. [0.5]
Provide the following plots:
1. Plot training and validation RMSE vs. epoch for all the converging batch sizes (some batch sizes might not converge) in a figure. The X-axis is
Epoch # and the Y-axis is RMSE. [0.5]
2. Plot training and validation RMSE vs. time for all the converging batch sizes in a figure. The X-axis is Time, and the Y-axis is RMSE. [0.5]
3. Plot Total training time (y-axis) vs. Batch size (x-axis). [0.5]
Describe your findings, including the main takeaways from each of your plots. [1]
α = 0.01
3
In [ ]:
def rmse(y_pred, y):
return np.sqrt(mean_squared_error(y_pred, y))
In [ ]:
# The function.
def mini_batch_gd(X_train, y_train, X_val, y_val, batch_size, learning_rate, CT):
"""
A function that performs mini-batch gradient descent until
the convergence threshold (CT) is reached.
:param X_train: Training set
:param y_train: Training target
:param X_val: Validation set
:param y_val: Validation target
:param batch_size: batch size
:param learning_rate: learning rate
:param CT: convergence threshold
:return: Final weight, training & validation rmse at each epoch and an array that
has the elapsed time from the start of the training process to the end of each epoch
(e.g., if each epoch takes exactly 2 seconds, the array would look like: [2 4 6 8 ...])
"""
start_time = time.time()
weight = []
train_rmse = []
val_rmse = []
elapsed_time = []
checker = True
epoch = 0
w = np.random.random(X_train.shape[1])/1000
size = X_train.shape[0]
while checker:
shuffle = list(np.random.permutation(size))
X_train = X_train.loc[shuffle]
y_train = y_train[shuffle]
for i in range(0, size, batch_size):
Xtr_b = X_train[i : i + batch_size]
ytr_b = y_train[i : i + batch_size]
yb_pred = np.dot(Xtr_b, w.T)
gd = (1 / len(yb_pred)) * (np.dot(Xtr_b.T, yb_pred - ytr_b))
w = w - learning_rate * gd
weight.append(w)
ytr_pred = np.dot(X_train, w.T)
yv_pred = np.dot(X_val, w.T)
tr = rmse(ytr_pred, y_train)
vr = rmse(yv_pred, y_val)
train_rmse.append(tr)
val_rmse.append(vr)
elapsed_time.append(time.time() - start_time)
epoch += 1
if epoch > 1:
if val_rmse[-1] > val_rmse[-2]:
checker = False
print('Model diverges with batch size = ', batch_size)
if vr <= CT:
checker = False
print('Epoch {} done with val rmse = {}'.format(epoch, vr))
return weight[-1], train_rmse, val_rmse, elapsed_time
# w = np.random.random(len(Xtr_std.columns))/1000
# this_val_rmse = CT + 1
# start_time = time.time()
# while checker and this_val_rmse >= CT:
# for i in range(0, length, batch_size):
# Xtr_b = X_train[i : i + batch_size]
# ytr_b = y_train[i : i + batch_size]
# yb_pred = np.dot(Xtr_b, w.T)
# gd = (1 / len(yb_pred)) * (np.dot(Xtr_b.T, yb_pred - ytr_b))
# w = w - learning_rate * gd
#
# ytr_pred = np.dot(X_train, w.T)
# this_train_rmse = rmse(ytr_pred, y_train)
#
# yv_pred = np.dot(X_val, w.T)
# this_val_rmse = rmse(yv_pred, y_val)
#
# if epoch > 0:
# if this_val_rmse > val_rmse[-1]:
# checker = False
# print('Model diverges with {} batches, rmse = {}'.format(batch_size, this_val_rmse))
# break;
#
#
# weight.append(w)
# train_rmse.append(this_train_rmse)
# val_rmse.append(this_val_rmse)
# elapsed_time.append(time.time() - start_time)
#
# epoch += 1
#
# return weight[-1], train_rmse, val_rmse, elapsed_time
In [ ]:
w, tr, vr, et, ft = mini_batch_gd(Xtr_std, y_train, Xvl_std, y_val, 1024, 0.01, CT)
In [ ]:
w = np.random.random(len(Xtr_std.columns))/1000
w
In [ ]:
vr
Epoch 1 done with val rmse = 0.003005139866288454
Epoch 2 done with val rmse = 0.0024016005535587095
Epoch 3 done with val rmse = 0.002089764884119976
Epoch 4 done with val rmse = 0.0018527807170018932
Epoch 5 done with val rmse = 0.0016583009749093834
Epoch 6 done with val rmse = 0.0014993463875516203
Epoch 7 done with val rmse = 0.0013694680901699893
Epoch 8 done with val rmse = 0.0012489806414372837
Epoch 9 done with val rmse = 0.0011443580476326353
Epoch 10 done with val rmse = 0.00105901593345498
Epoch 11 done with val rmse = 0.000982530006567729
Epoch 12 done with val rmse = 0.0009165491704570156
Epoch 13 done with val rmse = 0.000860923449433651
Epoch 14 done with val rmse = 0.0008095320245033482
Epoch 15 done with val rmse = 0.0007588564859723204
Epoch 16 done with val rmse = 0.0007193619870626984
Epoch 17 done with val rmse = 0.0006674128445893115
Epoch 18 done with val rmse = 0.0006471818409255765
Epoch 19 done with val rmse = 0.000620827357320922
Epoch 20 done with val rmse = 0.0006020466146429733
Epoch 21 done with val rmse = 0.0005820744935565961
Epoch 22 done with val rmse = 0.000559376717457176
Epoch 23 done with val rmse = 0.0005537084063168026
Epoch 24 done with val rmse = 0.0005312511031638866
Epoch 25 done with val rmse = 0.000526350599733706
Epoch 26 done with val rmse = 0.0005135712741971628
Epoch 27 done with val rmse = 0.0005030285882887317
Epoch 28 done with val rmse = 0.0004965557990754817
Epoch 29 done with val rmse = 0.0004923853538942588
Epoch 30 done with val rmse = 0.00048510972610968573
Epoch 31 done with val rmse = 0.0004824990495596328
Epoch 32 done with val rmse = 0.0004806661237223341
Epoch 33 done with val rmse = 0.00047030008240360834
Epoch 34 done with val rmse = 0.0004671347495696984
Epoch 35 done with val rmse = 0.00046422917053656006
Epoch 36 done with val rmse = 0.0004583478564821691
Epoch 37 done with val rmse = 0.00045587629574062873
Epoch 38 done with val rmse = 0.0004547784996181725
Epoch 39 done with val rmse = 0.00045046513473879715
Model diverges with batch size = 1024
Epoch 40 done with val rmse = 0.00045367047664611856
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
Input In [153], in ()
----> 1 w, tr, vr, et, ft = mini_batch_gd(Xtr_std, y_train, Xvl_std, y_val, 1024, 0.01, CT)
ValueError: not enough values to unpack (expected 5, got 4)
Out[ ]:
array([4.77435721e-04, 8.48770491e-04, 5.76293593e-04, 7.10884013e-05,
7.81852714e-04, 1.25206209e-04, 7.63646926e-04, 2.55663921e-04,
3.45216532e-04, 3.85302586e-04, 9.69478615e-04, 2.09460894e-04,
8.24068617e-04, 1.00477485e-04, 6.81952115e-05, 9.74453974e-04,
7.59644019e-04, 3.14274777e-04, 2.30608813e-04, 8.28335036e-04,
4.50288704e-04, 6.28368334e-04, 8.08498068e-04, 9.57210183e-04,
2.13240526e-04, 4.77203740e-04, 1.52530920e-04, 1.20420708e-04,
2.29099251e-04, 6.45630408e-04, 2.07164896e-04, 8.36146472e-04,
2.22975824e-04, 1.74398926e-04, 7.05720577e-04, 8.87613211e-04,
6.54048705e-05, 6.02072282e-04, 4.28403724e-04, 7.81268405e-05,
4.97494013e-04])
Out[ ]:
[0.5656870389029859]
In [ ]:
# batch size = 2 4 8 16 32 64 128 256.
alpha = 0.01
# CT = 1.001 * np.sqrt(mean_squared_error(y_val_pred, y_val)) # same as above
batch_size = [2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096]
final_weigh = {}
rmse_train_dict = {}
rmse_val_dict = {}
elapsed_time_dict = {}
for bs in batch_size:
w, tr, vr, et = mini_batch_gd(Xtr_std, y_train, Xvl_std, y_val, bs, alpha, CT)
final_weigh[bs] = w
rmse_train_dict[bs] = tr
rmse_val_dict[bs] = vr
elapsed_time_dict[bs] = et
Part 5 - Gradient Descent Learning Rate [2 marks]
Use the function that you designed in Part 4 to investigate the effect of learning rate on convergence.
1. Pick 3 batchsize that are not converging with a learning rate of (last part) and try to find a learning rate that results in convergence for
each batchsize. Report your findings in a table. Mention batch size, learning rate, and training and validation RMSE. [0.5]
2. Select the best batchsize based on Part 4's fastest convergence time and sweep the learning rate (at least 10 values, all should converge) while
applying Mini-batch GD.
Plot the training and validation RMSE vs. epoch (x-axis) for all the learning rates that you tried in a figure. [0.5]
Plot the training and validation RMSE vs. time (x-axis) for all the learning rates that you tried in a figure. [0.5]
Describe your findings, including the main takeaways from each of your plots. [0.5]
α = 0.01
In [ ]:
### YOUR CODE HERE ###
Part 6 - Finalizing a model [1 marks]
Based on your findings from the previous parts, pick a model (or combination of model settings) that you think would work best for our dataset and
evaluate it on the test set.
Briefly describe your model selections/settings. [0.5]
Summarize the performance of your model for the task of managing ailerons of an aircraft. [0.5]
In [ ]:
### YOUR CODE HERE ###
Model diverges with 2 batches
Model diverges with 4 batches
Model diverges with 8 batches
Model diverges with 16 batches
Model diverges with 32 batches
Model diverges with 64 batches
Model diverges with 128 batches
Model diverges with 256 batches
Model diverges with 512 batches
Model diverges with 1024 batches
Model diverges with 2048 batches
Model diverges with 4096 batches
Part 7 - Getting started with GitHub [3 marks]
This final part of the project assignment is to be completed independently from Parts 1 - 6. In this part you will be completing some additional coding tasks
using a different dataset and submitting your results on Github. To access this part of the assignment and upload your answers, you will need to use
Github. Please complete the following step-by-step instructions:
1. Create a Github account and install git for Windows or Mac:
https://git-scm.com/download/win (https://git-scm.com/download/win)
https://git-scm.com/download/mac (https://git-scm.com/download/mac)
2. Open this link: https://classroom.github.com/a/ZC8Petsq (https://classroom.github.com/a/ZC8Petsq) to create your assignment repository in GitHub.
You should get a link similar to:
https://github.com/APS-1070-2021/f22-project4-git-*********
This your private repository to get this part questions and upload your answers. Copy this link to the text box below to be graded for this part.
1. Open Git Bash , the app you downloaded in step 0 , and set your Email and username by:
git config --global user.email “”
git config --global user.name “”
2. Create a folder for the course on your computer and cd to that. cd means Change Directory . For example, on a Windows machine, where I
have a folder on "C:\aps1070":
cd c:aps1070
3. Get your assignment by the link you got in step 1:
git clone https://github.com/APS-1070-2021/f22-project4-git-*********
If this is the first time you are using Git Bash, it will ask you to sign in on GitHub via your browser.
4. A new folder should be created in your directory similar to:
C:\aps1070\f22-project4-git-********
This folder has an ipynb notebook which you need to manually upload to colab and answer its questions.
5. After you finished working on this notebook, download the notebook from colab and move it to the directory in step 5.
6. Replace the old notebook with the new one that has your answers. Make sure your completed notebook has the same name as the original notebook
you downloaded.
7. To submit your work, follow:
cd
git add F22_Project_4_git.ipynb
git commit -m “Final Submission”
git push
If you had any problem with pushing your work on GitHub you can try one of following commands:
git push --force
or
git push origin HEAD:main
8. Make sure your submission is ready for grading. Open the private repository link in your browser and make sure you can see your final submission
with your latest changes there. Only you and the teaching team can open that link.
Private Repository Link : --FILL HERE---