代写CS 5340、代写Java，c++程序

日期：2021-12-04 04:53

Fall 2021 CS 5340/4340 Project 6

Points 100 (UG) or 200 (G)

Due: Dec 9, 11:59 pm

r>This project is on implementing regression with regularization, with the regularization

parameter estimated from cross-validation. Do not use any package, except for the use of

random number generators and matrix operations (if any). That is, do not use any package

where regression, cross-validation, ridge or lasso are implemented as built-in functions. Use the

function y = f(x) = x2 + 10 to create a random sample (xj, yj) (training data) of size 12 (that is, j = 1

to 12) by uniform random sampling of x in -2 <= x <= 10. Obtain another five points (x-y pairs)

(by uniform random sampling of x in the same interval) to be used as the test set. Make sure

that the training set and the test set have no points in common; in the unlikely event of an

overlap, just re-sample.

1. Obtain a non-linear (quadratic) regression of y on x. Do this without regularization.

2. Next, implement linear regression of y on x with regularization (use “ridge” regression,

λ∑

2

), obtaining your λ from three-fold cross validation. Try the following values of λ

and choose one using cross-validation: 0.1, 1, 10, 100.

3. [For the grad section only; this part will require you to look up, read and implement

algorithms not covered in class] Finally, replace ridge in part 2 above with “lasso,”

λ∑ || , and re-do the computation. See the ISLR book for a good description of lasso.

Note that this project uses no single, fixed validation set because 3-fold cross-validation

arranges for multiple validation sets to be implicitly obtained from the training set; that is, we

are not using the traditional three-set (train/validate/test) scenario, but we are using the two-

set (train/cross-validate/test) scenario.

Clearly provide the following in your report as an itemized list (in addition to the source code):

(a) The twelve (x,y) values in the training set and the five (x,y) values in the test set.

(b) The equation of the non-linear regression relation you obtained in part 1; the in-sample

error; and the test error.

(c) The values of λ and the corresponding cross-validation errors (four such pairs) obtained

from the cross-validation phase in part 2 (also in part 3 for the grad section).

(d) The final (best) λ you chose after performing cross-validation in part 2 (and part 3 for

the grad section).

(e) The final equation of the regression relation you obtained after regularization (you

should have re-trained with the final λ on all 12 points in the training set); the

corresponding (final) in-sample error (state how many points were used in the

calculation of this error); and the corresponding test error.

(f) State whether or not you obtained your final (best) λ analytically. Just state (no

explanation).

(g) For ridge (and lasso for the grad section), arrange your three numerical errors (the

cross-validation error, the final in-sample (training) error, the test error) in ascending

order.

Plot your data points and the final solutions (by hand or with a plotting software; no extra

credit for using the software). Specifically, your plot should show 12 + 5 points on the x-y plane

and two (three for the grad section) final solutions (along with their respective equations) – one

curve for regression without regularization and two (three) lines for regression with

regularization – on the same plot.

Note that our original x is one-dimensional here. Take the average of squared differences as the

error.

Submission instructions as before. Please submit a single pdf (or Word or text) file on Canvas. If

you find it hard to merge text/figure and code into a single file, please submit multiple pdf files

(do NOT upload a zip file). Do not just provide a link to your code somewhere on the web.

Please write legibly if you are submitting any hand-written (and scanned) stuff.