MATH 456 Practice Exam 02 Solutions

No electronic devices, of any kind, allowed out nor on your desk during the exam. If you have such a device visible, I will immediately give you a zero on this exam.

  1. Consider the following data set which measures median housing prices within various towns of the greater Boston area. In order to predict medv the median value of homes in $1,000s, we have the following explanatory variables:
fit <- lm(medv ~ chas + age + crim + rm + rm:age + I(rm^2):age, data = df)
summary(fit)

Call:
lm(formula = medv ~ chas + age + crim + rm + rm:age + I(rm^2):age, 
    data = df)

Residuals:
    Min      1Q  Median      3Q     Max 
-35.392  -2.641  -0.387   1.884  34.202 

Coefficients:
              Estimate Std. Error t value Pr(>|t|)    
(Intercept) -46.171517   6.845592  -6.745 4.26e-11 ***
chas          3.904184   0.938191   4.161 3.73e-05 ***
age           1.521498   0.131585  11.563  < 2e-16 ***
crim         -0.246650   0.029891  -8.252 1.40e-15 ***
rm           11.604344   1.054641  11.003  < 2e-16 ***
age:rm       -0.451271   0.037937 -11.895  < 2e-16 ***
age:I(rm^2)   0.031431   0.003039  10.342  < 2e-16 ***
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

Residual standard error: 5.275 on 499 degrees of freedom
Multiple R-squared:  0.675, Adjusted R-squared:  0.6711 
F-statistic: 172.7 on 6 and 499 DF,  p-value: < 2.2e-16
  1. Write the fitted regression equation for the model above, filling in the coefficients based on the output above.

\[\widehat{medv} = -46.17 + 3.9 \cdot 1_{chas} + 1.5 age - 0.25 crim + 11.6 rm - 0.45 age \cdot rm + 0.03 age \cdot rm^2\]

  1. Interpret, in context of the data, the coefficient on the predictor chas.

We expect the median value of homes across towns in the greater Boston area that are next to the Charles river to cost $3,904 more than the median home value in towns that are not next to the Charles river.

I’ll also accept direct interpretations of the intercept itself, even though it doesn’t make sense.

  1. Interpret, in context of the data, the coefficient on the predictor crim.

Holding all other numeric predictors constant, for every 1 point increase in the per capita crime rate by town, we expect the median home value to decrease by $246.65.

  1. Set up and conclude a hypothesis test for the coefficient on crim using a level of significance of \(\alpha = 0.05\).

\(H_0: \beta_{crim} = 0\)

\(H_1: \beta_{crim} \ne 0\)

\(\alpha = 0.05\)

Because the p-value \(< 0.001 < \alpha = 0.05\), we reject \(H_0\).

  1. Interpret, in context of the data, the conclusion of the hypothesis test.

We have evidence that when the crime rate changes, the median value of homes in towns in the greater Boston area also changes, holding all else constant.

  1. What is the “slope” on rm? Write your answer mathematically.

\(\frac{d\,\widehat{medv}}{d\,rm} = 11.6 - 0.45 age + 0.06 age \cdot rm\)

  1. Write R code, using the function predict, to make a prediction from this model. I’m expecting you to write code that will make reasonable and valid values for all explanatory variables from which to predict medv.
predict(fit, newdata = data.frame(chas=1, age = mean(df$age), crim = mean(df$crim), rm = mean(df$rm)))
       1 
24.75258 
  1. Interpret adjusted \(R^2\) in context of the data.

\(67.1\%\) of the variation in median home value in towns of the greater Boston area is explained by this linear model on the explanatory variables chas, age, crim, rm.

  1. The simplified log-likelihood for (Normal distribution) linear regression based on \(N\) data points \((y_n, \mathbf{x}_n)\) is

\[\ell(\mathbf{\beta} | \mathbf{x}, \mathbf{y}) \propto \sum_{n=1}^N (y_n - (\beta_0 + \sum_{j=1}^J \beta_j x_{n,j}))^2\] where \(\beta\) is vector of length \(J + 1\). Write the math that justifies this expression.

The density function for the Normal distribution is

\[f(x | \mu, \sigma) = (2\pi\sigma^2)^{-1/2}\exp{\{-(x - \mu)^2 / 2\sigma^2\}}\]

For (Normal distribution) linear regression, the mean \(\mu\) becomes a function of the predictors \(x_j\) as \(\mu = \beta_0 + \sum_{j=1}^J \beta_j x_j\). With this substitution for \(\mu\), we use the Normal density function to get the log-likelihood with respect to the coefficients \(\beta\) dependent on the data \((y_n, \mathbf{x}_n)\)

\[\ell(\beta | \mathbf{y}, \mathbf{x}) = \sum_{n=1}^N \log{f(y_n | \beta_0 + \sum_{j=1}^J \beta_j x_{n,j}, \sigma)}\] Filling in the density function gives

\[\ell(\beta | \mathbf{y}, \mathbf{x}) = \sum_{n=1}^N \log{\{(2\pi\sigma^2)^{-1/2} \exp{[-(y_n - (\beta_0 + \sum_{j=1}^J \beta_j x_{n,j}))^2 / 2\sigma^2]}\}}\] Using properties of \(\log\) we find

\[\ell(\beta | \mathbf{y}, \mathbf{x}) = \sum_{n=1}^N \log{\{(2\pi\sigma^2)^{-1/2}\}} - (y_n - (\beta_0 + \sum_{j=1}^J \beta_j x_{n,j}))^2 / 2\sigma^2\] Since we are only interested in the terms involving \(\beta\), the first term with the \(\log\) will go to zero under differentiation with respect to \(\beta\). The denominator of the second term, specifically the \(2\sigma^2\), will also go away when we set the derivative equal to \(0\). So we can use the proportional to symbol \(\propto\) to get

\[\ell(\beta | \mathbf{y}, \mathbf{x}) \propto \sum_{n=1}^N (y_n - (\beta_0 + \sum_{j=1}^J \beta_j x_{n,j}))^2\] 3. Write the simplified log-likelihood above in an R function named ll_linreg. Make sure your function works with the code from question 4.

ll_linreg <- function(beta, data) {
  x <- data$x
  y <- data$y
  yhat <- x %*% beta
  r <- y - yhat
  return(sum(r * r))
}
  1. Answer the following questions based on the R code below.

    optim(init_beta,
          ll_linreg,
          data = data,
          method = "L-BFGS-B")
    X <- model.matrix(fit)
    head(X)
      (Intercept) chas  age    crim    rm   age:rm age:I(rm^2)
    1           1    0 65.2 0.00632 6.575 428.6900    2818.637
    2           1    0 78.9 0.02731 6.421 506.6169    3252.987
    3           1    0 61.1 0.02729 7.185 439.0035    3154.240
    4           1    0 45.8 0.03237 6.998 320.5084    2242.918
    5           1    0 54.2 0.06905 7.147 387.3674    2768.515
    6           1    0 58.7 0.02985 6.430 377.4410    2426.946
    1. Based on the following model matrix, write R code to definte init_beta.
    init_beta <- rnorm(ncol(X))
    1. Write R code to define the variable data.
    data <- list(x = X, y = df$medv)
  2. Consider the following data set which records whether or not students were admitted to graduate school. Short descriptions of the variables are found below.

Answer the following questions based on this logistic regression model and the associated output.

fitl <- glm(admit ~ rank + gpa + gre, data = dfa, family = "binomial")
summary(fitl)

Call:
glm(formula = admit ~ rank + gpa + gre, family = "binomial", 
    data = dfa)

Coefficients:
             Estimate Std. Error z value Pr(>|z|)    
(Intercept) -3.989979   1.139951  -3.500 0.000465 ***
rank2       -0.675443   0.316490  -2.134 0.032829 *  
rank3       -1.340204   0.345306  -3.881 0.000104 ***
rank4       -1.551464   0.417832  -3.713 0.000205 ***
gpa          0.804038   0.331819   2.423 0.015388 *  
gre          0.002264   0.001094   2.070 0.038465 *  
---
Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1

(Dispersion parameter for binomial family taken to be 1)

    Null deviance: 499.98  on 399  degrees of freedom
Residual deviance: 458.52  on 394  degrees of freedom
AIC: 470.52

Number of Fisher Scoring iterations: 4
mgpa <- mean(dfa$gpa)
mgre <- mean(dfa$gre)
predict(fitl, 
        newdata = data.frame(rank = c("1", "2", "3", "4"), 
                             gpa = rep(mgpa, 4), 
                             gre = rep(mgre, 4)),
        type = "response")
        1         2         3         4 
0.5166016 0.3522846 0.2186120 0.1846684 
predict(fitl,
        newdata = data.frame(rank = c("1", "1"),
                             gpa = c(mgpa, mgpa + 1),
                             gre = c(mgre, mgre)), 
        type = "response") %>%
  diff
        2 
0.1882415 
  1. Interpret, in context of the data, the predicted probability for one of the ranks.

When the numeric predictors are set to their mean values, the probability of being admit to graduate school from a rank 1 university is 51.7%.

  1. Based on the coefficient for the predictor rank2, what can you say about the probabilities of being accepted into graduate school relative to rank 1 and rank 3 undergraduate schools?

The probability of being admit to graduate school from rank 2 university is in between the probability of being admit from a rank 1 university and a rank 3 university.

  1. Interpret, in context of the data, the calculated “slope”.

Coming from a rank 1 university, when the student’s GRE score is equal to the mean GRE scores, and the student’s GPA increases by 1 point from the mean of gpa, we expect the probability of being admit to graduate school to increase by 18.8%.

  1. Write R code to make a scatter plot of the variables admit and gpa with the predicted probabilities across gpa based on whichever ranked school you want and a reasonable value for gre.
dfp <- data.frame(gpa = seq(min(dfa$gpa), max(dfa$gpa), length.out = 101),
                  gre = mean(dfa$gre),
                  rank = factor(1))
dfp$phat <- predict(fitl, newdata = dfp, type = "response")

ggplot() +
  geom_point(data = dfa, aes(gpa, admit)) +
  geom_line(data = dfp, aes(gpa, phat))

  1. Recall that the sigmoid function is defined as \(\sigma(x) = 1 / (1 + \exp{(-x)})\). Write mathematically, a predicted probability from this model.

Let \(p\) be the probability of being admit to a graduate school.

\[\hat{p} = 1 / (1 + \exp{\{-(-3.98 + -0.68 \cdot 1_{rank2} - 1.34 \cdot 1_{rank3} - 1.55 \cdot 1_{rank4} + 0.80 gpa + 0.002 gre)\}}\]