python代写-HW1 FRQ
时间:2022-11-01
Regularization
xkcd.com/1725/
Logistics
• HW1 FRQ grades are posted to Canvas
• Survey #2 is posted to Canvas, due Wednesday
• HW2 setup is due Wednesday
• HW2 coding and FRQ are due next Wednesday
Polynomial Regression
javatpoint.com/machine-learning-polynomial-regression
Under/Over Fitting
Under/Over Fitting
https://dziganto.github.io/
Underfitting Overfitting
Overfitting and Variance
Overfitting and Variance
2-degree polynomial8-degree polynomial32-degree polynomial
https://tinyurl.com/cs349regularization
What goes wrong?
• Underfit: too many assumptions
• Hypothesis class doesn’t contain the optimal hypothesis
• (or even a “good” hypothesis)
• Data representation discards essential information
• Overfit: not enough assumptions
• Hypothesis class is “hard to search”
• Learning algorithm is inefficient, can’t optimize
parameters
• Many hypotheses perfectly fit training data
xkcd.com/2048/
How do we stop overfitting?
Recall: Google Flu Trends
Recall: Google Flu Trends
f(x; w) = w0 + w1 * x1 + w2 * x2 + … + wm * xm
Case study: Google Flu Trends
How do we stop overfitting?
Motivating regularization
• Our predictor f with parameters w has loss L:
L(y, f(x; w)) = (y - w•x)2
Motivating regularization
3
Motivating regularization
• What if we add a term to our loss?
L(y, f(x; w)) = (y - w•x)2 + λw2
How do we stop overfitting?
L(y, f(x; w)) = (y - w•x)2 + λw2
λ = 0 1e-7 1
How do we stop overfitting?
L(y, f(x; w)) = (y - w•x)2 + λw2
λ = 0
λ = 1e-7
λ = 1
Regularized Least Squares
How do we stop overfitting?
L(y, f(x; w)) = (y - w•x)2 + λw2
λ = 0
λ = 1e-7
λ = 1
Overfitting and Variance
https://tinyurl.com/cs349regularization

How do we stop overfitting?
L(y, f(x; w)) = (y - w•x)2 + λw2
λ = 0
λ = 1e-7
λ = 1
Regularization Behavior
https://www.cs.toronto.edu/~rgrosse/courses/csc411_f18/slides/lec06-slides.pdf

Regularization + Gradient Descent Demo
https://github.com/nucs349s22/gradient_descent_demo
Summary: regularization
λ = 0
λ = 1e-7
λ = 1
M=9
λ = 0
M = 1
M = 3
M = 9
Dr. Philip Graff and Dr. Jared Markowitz
Recall: What goes wrong?
• Underfit: too many assumptions
• Hypothesis class doesn’t contain the optimal hypothesis
• (or even a “good” hypothesis)
• Data representation discards essential information
• Overfit: not enough assumptions
• Hypothesis class is “hard to search”
• Learning algorithm is inefficient, can’t optimize
parameters
• Many hypotheses perfectly fit training data
No Free Lunch Theorem
• Models make assumptions about the world
• An assumption-less model allows every hypothesis
and make learning difficult
• We make assumptions to “focus” the model
• But wrong assumptions limit the model
• Nothing works best for every problem!
• (Cross-)validation and careful experimentation
can help us refine our assumptions
essay、essay代写