---
title: "Week 2 Lecture 1"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
```

## Review of vector subsets.

We again define the salary offers:

```{r}
offer <- c(241, 590, 533, 425, 261)
```

Here is a vector of `logical`s, with `TRUE` indicating that the element is greater than 400, and `FALSE` indicating that it's not.

```{r}
offer > 400
```

We can also get this vector:

```{r}
offer < 550
```

We can combine those vector using `&`:

```{r}
offer > 400 & offer < 550
```

Here, `TRUE` corresponds to elements that are both greater than 400 and smaller than 550. We can now retrieve the elements that are between 400 and 550 using

```{r}
offer[offer > 400 & offer < 550]
```

Recall that `FALSE` means "drop" and `TRUE` means keep, so we just keep the elements between 400 and 550 (not including 400 and 500).

What about the elements of offer outside the range of 400..550? One way to go is

```{r}
offer[!(offer > 400 & offer < 550)]
```

The `!` operator turns `TRUE` into `FALSE` and vice versa, so we keep the elements we dropped before, and drop the elements we kept.

Another way to obtain the same result is

```{r}
offer[offer <= 400 | offer >= 550]
```

Note that we are now using `<=` rather than `<`. That because we want `offer <= 400 | offer >= 550` to be true when the element is equal to exactly 400 to be consistent with `offer[!(offer > 400 & offer < 550)]`. (Think this through if it's not obvious.)

## Parallel vectors

Let's go back to `offer`. Actually, those were the average offers to various specialties of doctors:

```{r}
offer <- c(241, 590, 533, 425, 261)
spec <- c("family doc", "cardiologist", "orthopedic", "dermatologist", "psychiatrist")
```

So for example the offers to cardiologist were $590k per year in 2018. Suppose we want to automatically figure out which specialty makes the most money.

So, if we knew it were specialty number 2, we'd go `spec[2]`. But how do we figure out the `2`? 

Let's try another tack. We'd like to compute `spec[c(F, T, F, F, F)]`. Can we compute the logical vector? Here's an idea:

```{r}
offer == max(offer)
```

Combining those ideas, we'll get

```{r}
spec[offer == max(offer)]
```

So how do we get the 2? We can first generate the vector
```{r}
1:length(offer)  # generate c(1, 2, 3, ..., length(offer))
```

And now, we can simply go
```{r}
(1:length(offer))[offer == max(offer)]
```

The following is really unnecessarily complicated, but we could have
```{r}
spec[(1:length(offer))[offer == max(offer)]]
```





## Data frames




Data frames are R's way of storing tables (note that the salary data we had was also actually a table). 


### Defining your own data frames

You can define a data frame using the following syntax:

```{r}
offers <- data.frame(amount = c(241, 590, 533, 425, 261),
                     spec = c("family doc", "cardiologist", "orthopedic", "dermatologist", "psychiatrist"))
```

Note the `amount` and `spec` are the names of *columns* in the new data frame. We define data frames column-by-column.


The value of `offers` will be displayed as follows in the console:

```{r}
offers
```

Let's load a data frame (you must previously have successfully run `install.packages("babynames")`).

```{r}
library(babynames) #Load the data frame babynames into R
```

Here are the first several rows of the data frame

```{r}
head(babynames) # Display the first 5 rows of a data frame
```

We can access, for example, row 2 and column "year" of the table like so:

```{r}
babynames[2, "year"]
```

If we want to access all of row 2, we can omit the second part:

```{r}
babynames[2, ]
```

We can access rows 2 through 6 like so:

```{r}
babynames[2:6, ]
```

We can only take the columns `"n"` and `"year"`, like this:

```{r}
babynames[2:6, c("n", "year")]
```

Finally, if we want a particular column as a vector (rather than as a data frame), we can do the following (make sure to keep track of the quotes)

```{r}
babynames[5:20, ]$name
```


### A note about `$` vs. `[, "colname"]`

Note that sometimes, `df[, "colname"]` will yield a vector (when you are operating on one kind of data frames), and sometimes, it will a data frame. `df$colname` will always yield a vector. So if you want a vector, use the `$` operator.

## Working with data frames

Let's now write code that finds the most common name in 1999

```{r}
babies.1999 <- babynames[babynames$year == 1999 & babynames$sex == "F", ]
max.name.count <- max(babies.1999$n)
(babies.1999$name)[max.name.count == babies.1999$n]
```

Let's make this into a more general function:

```{r}
most.common.name <- function(babynames, year, sex){
  b.year.sex <- babynames[babynames$year == year & babynames$sex == sex,  ]
  (b.year.sex$name)[b.year.sex$n == max(b.year.sex$n)]
}
```


## Pipes and the `tidyverse`

To use pipes, first you need to install and load the `tidyverse` library. Use the following:

    install.packages("tidyverse")
    library(tidyverse)


```{r message = F}
library(tidyverse)
```

### Composition of functions

Let's define two functions:

```{r}
f <- function(x){
  x^2
}

g <- function(y){
  y + 1
}
```


We can compute `f(g(5))` and `g(f(5))`. Note that those are not the same: $f(g(5)) = f(6) = 36$ but $g(f(5)) = g(25) = 26$.
  
Now, we'll compute the same qunatities using pipes:

```{r}
5 %>% f %>% g 
```

This is the same as computing `g(f(10))`. The way to think about it is this: we start with 10, then apply `f` to 10 and obtain `f(10)`, and then apply g to `10 %>% f` (i.e., `f(10)`) and obtain `f(g(10))`.

To compute `f(g(10))`, we can use:

```{r}               
5 %>% g %>% f 
```

This is the same as 

```{r}               
5 %>% g() %>% f()
```

If the only thing that we're sending to `f` is `g(5)`, the parentheses are optional in this notation.

This is useful mostly because it's easier to read something like `x %>% f1 %>% f2 %>% f3 %>% f4` than to read `f4(f3(f2(f1(x))))`.

## Using `tidyverse`/`dplyr` to wrangle data

### `filter`

The `filter` function is used to select rows from a data frame. The following are all equivalent. They are different ways of selecting rows for which the `year` is 1880 **and** `sex` is `"F"`.


`filter(babynames, year == 1880, sex == "F")`

`idx <- (babynames$year == 1880) & (babynames$sex == "F")`
`babynames[idx, ]`

`babynames[(babynames$year == 1880) & (babynames$sex == "F"), ]`

`babynames %>% filter(year == 1880 & sex == "F")`

`babynames %>% filter(year == 1880, sex == "F")`

The last way is probably preferable, though different people might have different preferences.

### `select`

We can use `select` as follows:

```{r}
b.y.n <- babynames %>% select(year, name)
b.y.n[1:20, ]
```

We only kept the `year` and `name` columns here.

### `summarize` and `group_by`

First, let's use `summarize` to compute the average number of babies per name:

```{r}
babynames %>% summarize(b.pername = mean(n))
```

Inside of `summarize`, we can refer to columns in the data frame we are processing by name, without the `$` operator. (And you shouldn't be using the `$` operator.) This is basically the same as not using `summarize` and computing `mean(babynames$n)` -- not that interesting.

The real power of `summarize` is in using `group_by` -- we can group rows and compute a function such as `mean` for every group of rows separately. For example, we could compute the average number of babies per name for different sexes separately.

```{r}
babynames %>% group_by(sex) %>% 
              summarize(b.pername = mean(n))
```

We see that the number of babies per name for `"F"` is much smaller than for `"M"` -- female names are more diverse with fewer babies per name, and male names are less diverse with more babies per name.




