MLE 3代做、代写Machine留学生、代做Java，Python，c/c++程序设计

日期：2019-03-14 10:54

Machine Learning Homework 4

2/13/2019

Contents

1. Expectation, Covariance, and Correlation 2

2. Poisson MLE 3

3. Minimum Variance Portfolio 3

4. AIC and BIC in SLR 4

1

1. Expectation, Covariance, and Correlation

Note: this problem introduces a simple form of factor analysis, which is a very important model.

Let Suppose a person is about to take three tests.

Let Yi be the score on the i

th test, i = 1, 2, 3.

Let

Yi = A + Ti

The idea is the A is a general ability and the Ti are test specific abilities.

Let A and all the Ti be independent with

V ar(A) = σ2

A, V ar(Ti) = σ2i, E(A) = μ, E(Ti) = 0

(a)

What is E(Yi), i = 1, 2, 3?

(b)

What is the 3 × 3 variance matrix Σ for Y = (Y1, Y2, Y3)0.

(c)

Supppse μ = 100, σ2

A = 100, and σ2

i = 25, i = 1, 2, 3.

What is the correlation matrix of Y ?

(d)

Suppose we have the same setup as in (c).

What is the expected value and variance of S = a

0Y where a

0 = (1, 1, 1), so that S is the total score?

(e)

Suppose we have the same setup as in (c).

and D = b

0Y where b

0 = (1, .5,.5) so that D is the difference between the score on the first test and the

average score on the other two tests.

What are the mean and variance of D?

(f)

Suppose we have the same setup as in (c).

What is σS,D = cov(S, D)?

What is ρS,D?

(g)

2

Suppose we have the same setup as in (c).

Note that in general

V ar(AY ) = A V ar(Y ) A0

Let A0 = (a, b) and check your previous results by computing A V ar(Y ) A0.

(h) Suppose

Y = μ + β A + T

where Y = (Y1, Y2, . . . , Yp)0, β = (β1, β2, . . . , βp), F is a single random variable with E(F) = 0, V ar(F) = σ2F,

and T = (T1, T2, . . . , Tp) with the Ti

iid, E(Ti) = 0, σ2

i = V ar(Ti) = σ2.

Hence V ar(T) = σ2

Ip where Ip is the p × p identity matrix.

What is V ar(Y ) in terms of β and σ2

2. Poisson MLE

Suppose yi

is a count then a very common model is to assume the Poisson disttribuion:

P(Y = y | λ) = e

, y = 0, 1, 2, . . .

Given Yi ～ P oisson(λ) iid, (that is, Yi = yi), what is the MLE of λ?

3. Minimum Variance Portfolio

Suppose we are considering investing in p stocks where the uncertain return on the i

th stock is denoted by

Ri, i = 1, 2, . . . , p.

Let R = (R1, R2, . . . , Rp)0.

A portfolio is a given by w = (w1, w2, . . . , wp)

0 where wi

is the fraction of wealth invested in asset i.

The {wi} must satisfy Pwi = 1.

The return on the portfolio is then

We want to find the global minimum variance portfolio:

V ar(P), subject toXwi = 1.

If we let ι = (1, 1, . . . , 1)0

, the vector of ones, and V ar(R) = Σ then our problem is.

Find the global minimum variance portfolio in terms of Σ and ι.

3

4. AIC and BIC in SLR

Get the midcity housing data from the Rob data page.

md = read.csv("http://www.rob-mcculloch.org/data/midcity.csv")

head(md)

## Home Nbhd Offers SqFt Brick Bedrooms Bathrooms Price

## 1 1 2 2 1790 No 2 2 114300

## 2 2 2 3 2030 No 4 2 114200

## 3 3 2 1 1740 No 3 2 114800

## 4 4 2 3 1980 No 3 2 94700

## 5 5 2 3 2130 No 3 3 119800

## 6 6 1 2 1780 No 3 2 114600

dim(md)

## [1] 128 8

We will use SLR (simple linear regression) to relate house price to house size.

ddf = data.frame(size = md$SqFt/1000,price=md$Price/1000)

plot(ddf$size,ddf$price)

1.6 1.8 2.0 2.2 2.4 2.6

80 120 160 200

ddf$size

ddf$price

As in the notes let

M1:

Yi = β0 +i,i ～ N(0, σ2), iid.θ1 = (β0, σ).

M2:Yi = β0 + β1xi + i, i ～ N(0, σ2), iid.

θ2 = (β0, β1, σ).

According the the usual regression ouput:

4

lmf = lm(price~size,ddf)

summary(lmf)

##

## Call:

## lm(formula = price ~ size, data = ddf)

##

## Residuals:

## Min 1Q Median 3Q Max

## -46.59 -16.64 -1.61 15.12 54.83

##

## Coefficients:

## Estimate Std. Error t value Pr(>|t|)

## (Intercept) -10.091 18.966 -0.532 0.596

## size 70.226 9.426 7.450 1.3e-11 ***

## ---

## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

##

## Residual standard error: 22.48 on 126 degrees of freedom

## Multiple R-squared: 0.3058, Adjusted R-squared: 0.3003

## F-statistic: 55.5 on 1 and 126 DF, p-value: 1.302e-11

we might prefer M2.

(a)

According to BIC, which model is preffered.

(b)

According to AIC, which model is preffered.

(c)

Using BIC what is the approximate value of P(M2 | the data).