The code below imports into R a data set containing the self-reported heights (in feet and inches), lengths of index and pinky fingers (in millimeters), shoe size, and shoe size gender (“m”/“w”) of several statistics students. The code then removes an observation with shoe size gender given as “uk” and adds to the data set a column containing the heights in centimeters.
# import the datalink <-url("https://gregorkb.github.io/data/height_length_index_pinky_shoesize_combined.csv")hl0 <-read.table(link,sep=",",header=T)# remove a "uk" show sizermv <-which(hl0$shmw =="uk")hl <- hl0[-rmv,]# convert ft and in. to centimeter heightshl$height <- (hl$ft*12+ hl$in.)*2.54# view the first few rows of the data framehead(hl)
ft in. ind pnk shoe shmw class height
1 5 4 75 65 8.0 w STAT_516_sp_2026 162.56
2 5 4 70 56 7.0 w STAT_516_sp_2026 162.56
3 5 8 70 52 10.0 w STAT_516_sp_2026 172.72
4 5 5 68 62 7.5 w STAT_516_sp_2026 165.10
5 5 6 71 55 9.5 m STAT_516_sp_2026 167.64
6 6 4 78 65 13.0 m STAT_516_sp_2026 193.04
It is of interest to use the multiple linear regression model to predict the height of a person based on his or her index and pinky finger lengths, shoe size, and shoe size gender “m” or “w”.
1.
Consider the multiple linear regression model which uses all the covariates—the index and pinky finger lengths, shoe size, and shoe size gender—to predict the height of a person.
1.a
Give the critical value for the overall F test at significance level \(\alpha = 0.05\). This the value such that when it is exceeded by the value of the test statistic \(F_{\operatorname{test}}\) we reject \(H_0\) at \(\alpha = 0.05\).
n <-nrow(hl)p <-4alpha <-0.05Fcrit <-qf(1- alpha, p, n-(p+1))
The critical value is 2.51767.
1.b
Fit the model and give the value of the test statistic for the overall F test of significance as well as the p-value associated with it. Give an interpretation of these values.
lm_all <-lm(height ~ ind + pnk + shoe + shmw, data = hl)summary(lm_all)
Call:
lm(formula = height ~ ind + pnk + shoe + shmw, data = hl)
Residuals:
Min 1Q Median 3Q Max
-11.1911 -2.4660 0.2083 3.3303 9.9596
Coefficients:
Estimate Std. Error t value Pr(>|t|)
(Intercept) 127.0611 7.6453 16.620 < 2e-16 ***
ind 0.1489 0.1602 0.929 0.356
pnk 0.1562 0.1503 1.040 0.303
shoe 3.0248 0.3969 7.622 1.64e-10 ***
shmww -7.9098 1.3934 -5.677 3.73e-07 ***
---
Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
Residual standard error: 4.597 on 63 degrees of freedom
Multiple R-squared: 0.8358, Adjusted R-squared: 0.8254
F-statistic: 80.17 on 4 and 63 DF, p-value: < 2.2e-16
The value of the test statistic for the overall F
test is 80.16598, and the p value is 0. We reject the
null hypothesis that all regression coefficients
(apart from the intercept) are equal to zero at every significance level
greater than 0.
Therefore we conclude (at any such significance level),
that not all the regression coefficients are equal to zero; that is,
at least one of the covariates is significantly (linearly) related
to the response.
1.c
Report the variance inflation factor for each of the four covariates.
slm_ind <-summary(lm(ind ~ pnk + shoe + shmw, data = hl))slm_pnk <-summary(lm(pnk ~ ind + shoe + shmw, data = hl))slm_shoe <-summary(lm(shoe ~ ind + pnk + shmw, data = hl))slm_shwm <-summary(lm(shmw =='w'~ ind + pnk + shoe, data = hl))vif_ind <-1/(1-slm_ind$r.squared)vif_pnk <-1/(1-slm_pnk$r.squared)vif_shoe <-1/(1-slm_shoe$r.squared)vif_shoe_wm <-1/(1-slm_shwm$r.squared)# Can also run:# library(car)# vif(lm_all)
The VIFs were 3.249869 for the index finger
length, 2.816658 for the pinky finger length 1.909865 for the
shoe size, and 1.560264 for the shoe size gender. The
VIFs are smaller in this model than in the model with all
four covariates.
2.
Fit a model with all covariates except the shoe size gender covariate.
2.a
Use the full-reduced model F test to test whether the shoe size gender covariate has a nonzero regression coefficient in the full model. Give the value of the test statistic as well as the p value. Interpret the result of your test.
The value of the test statistic is 32.22543 and the
p value is 3.73342e-07. There is sufficient evidence at the 0.05
significance level to conclude that shoe size gender is significantly
related to height.
2.b
Obtain the value of the test statistic \(T_{\operatorname{test}}\) and the p value associated with it for testing \(H_0\): \(\beta_j=0\) versus \(H_1\): \(\beta_j \neq 0\) where \(j\) is the index of the shoe size gender covariate.
We can obtain this from applying the summary()
function to the output of the lm() function. We
find that the T statistic is equal the square root of the
F statistic, but signed in the direction of the
estimated regression coefficient. The p value is the same
as that for the full-reduced model F test.
2.c
Explain the relationship between the value of the test statistic \(F_{\operatorname{test}}\) of the full-reduced model F test when one considers the removal of a single covariate with the test statistic \(T_{\operatorname{test}}\) for testing whether a single covariate is significantly related to the response.
The former is the square of the latter.
3.
Fit a model using only the two shoe size covariates.
3.a
Use the full-reduced model F test to test whether either of the finger length variables has a nonzero regression coefficient in the full model. Give the value of the test statistic as well as the p value. Interpret the result of your test.
The value of the test statistic is 3.179345 and the
p value is 0.04836705. We may reject the null hypothesis at the
0.05 significance level. It appears that that index and pinky finger
lengths, do contribute, when taken together
with the shoe size and shoe size gender, to our
ability to predict a person's height.
3.b
Compute the variance inflation factors of the two variables in this model. Comment on how these compare to their counterparts in the model with all four covariates.
slm_shoe <-summary(lm(shoe ~ shmw, data = hl))slm_shmw <-summary(lm(shmw =='w'~ shoe, data = hl))vif_shoe <-1/(1-slm_shoe$r.squared)vif_shmw <-1/(1-slm_shmw$r.squared)
The VIFs were 1.483552 for the shoe
size and 1.560264 for the shoe size gender.
3.c
Comment on anything else interesting about these two variance inflation factors!
The two VIFs are equal. This will always be
the case when there are only two covariates. The
reason for this is that the VIF is based on the
coefficient of determination in a model in which the
covariate under consideration is the response and
all other covariates are used as predictors. If
there are only two covariates, then when one is put as
the response, there is only one other predictor. Then,
when the other is put as the response, the response
and the single predictor trade places. If one
regresses 'Y' on a single covariate 'x' and then
regresses 'x' on 'Y', one will get the same coefficient
of determination in both regressions.
4.
Fit a model using only the index and pinky finger lengths.
4.a
Give the value of the test statistic and the p value for the full-reduced model F test for testing whether either of the two shoe size covariates has a nonzero regression coefficient in the full model. Interpret the result.
The value of the test statistic is 73.10892 and the
p value is 0. There is strong evidence against the null
hypothesis that both shoe size covariates have regression coefficients
equal to zero.
4.b
Suppose the index and pinky finger measurements were recorded in centimeters instead of millimeters. Describe the effect this would have on the value of the test statistic \(F_{\operatorname{test}}\) in the previous part as well as on the p value.
There would be no effect whatsoever
in the value of the test statistic or the p
value. The only change would be in
the value of the estimated regression
coefficients. The tests for
statistical significance of covariates in
multiple linear regression are invariant to any
shifting or scaling of the covariate and
response values.
5.
5.a
Give the model chosen by backward selection based on the AIC criterion.
Considering the model with shoe and shoe size gender as well as pinky finger
length, none of the Cook's distances are
alarmingly large. Moreover, there do not appear to be any outlying residuals in the residuals versus fitted values plot.
