Create and save the file p4.R on your computer. The file p3.R, which should contain all the required functions.
Some of you will be tempted to use for and while-loops to solve some of the problems below (if you’ve used those before). Please don’t do this – the goal here is to try to use R the way professional data scientists use it, which usually means no loops.
In this problem you will be working with a smaller version of the gapminder dataset. Define
g1952 <- gapminder %>% filter(year == 1952)Plot log10(lifeExp) vs. log10(gdpPercap). Display both a scatterplot and a straight line through the data points. Make sure that you know the syntax for doing this: if you use the lecture notes at first, try to then repeat the same thing without looking at the notes.
Fit a linear regression predicting log(lifeExp) from log(gdpPercap). Fill in the terms in the equation
\[\log(\text{lifeExp}) \approx \text{__} + \text{__}\times \log(\text{gdpPercap})\]
Write a function that takes in a GDP per capita figure, as well the coefficients a_0 and a_1 (such as the ones that you obtained in Problem 1(b)), and returns the predicted life expectancy using those coefficients. Use the facts that \(\exp(\log(z)) = \log(\exp(z)) = z\) and that if you regressed y on x you can make a prediction using \(\hat{y} = a_0 + a_1 x\). Your function should be defined using my.pred <- function(a0, a1, gdpPercap). Use this function to estimate the life expectancy in a country with a GDP per capita of $20,000.
Characterize the relationship between the GDP per capita and the predicted life expectancy. Speicifically, if \(\log(\text{gdpPercap})\) increases by 1, what is the effect on the predicted \(\log(\text{lifeExp})\)? What is the effect on \(\text{lifeExp}\)?
Recall that \(\log(ab) = \log(a) + \log(b)\), \(\exp\log(z) = \log(\exp(z)) = z\), and \(\exp(z) = e^z = 2.7^z\).
Write a function which finds the country for which the prediction made by the regression in Problem 1 was the most accurate. Write a function that finds for which country the prediction was the least accurate. Measure the accuracy as the absolute difference between the predicted lifeExp and the actual lifeExp.
Make a dataframe like g1952, but add the column pred.error, which contains the difference between the predicted log(lifeExp) and the actual log(lifeExp). Do you see any patterns? For which countries are we overpredicting and for which countries are we underpredicting?
In this problem, you will write a function that finds good coefficients for linear regression.
Write a function called my.lm that could be used to find the coefficient of Simple Linear Regression. The function could be used like this
my.data <- data.frame(X = c(1, 2, 3),
Y = c(3.1, 4.9, 7.05))
my.lm(my.data, intercept) # Returns approximately 2, since Y ~ 2*X + 1The function should work as follows. First, generate possible values for the coefficient using
seq(-5, 5, 0.1)## [1] -5.0 -4.9 -4.8 -4.7 -4.6 -4.5 -4.4 -4.3 -4.2 -4.1 -4.0 -3.9 -3.8 -3.7 -3.6
## [16] -3.5 -3.4 -3.3 -3.2 -3.1 -3.0 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.3 -2.2 -2.1
## [31] -2.0 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1 -1.0 -0.9 -0.8 -0.7 -0.6
## [46] -0.5 -0.4 -0.3 -0.2 -0.1 0.0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
## [61] 1.0 1.1 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2.0 2.1 2.2 2.3 2.4
## [76] 2.5 2.6 2.7 2.8 2.9 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9
## [91] 4.0 4.1 4.2 4.3 4.4 4.5 4.6 4.7 4.8 4.9 5.0Then, of those possible coefficients, find the one that produces the smallest sum of squared errors.
Try different inputs, and make sure that the answer you are getting is close to the answer you would expect. Explain to your preceptor how you came up with the different inputs.
Now, write a function that finds both a good intercept and a good coefficient. Hint: use a modification of the function in 3(a) that returns both the coefficient and the sum of squared errors in produces. Then repeatedly use that function for every possible intercept hypothesis.