---
title: "Week 10 Lecture 1"
output: html_document
---

```{r setup, include=FALSE}
knitr::opts_chunk$set(echo = TRUE)
library(tidyverse)
```



#### Binomial distribution

* Two outcomes: success/failure
* Examples:
 * Number of times a coin came up "heads" out of 100 trials
 * Number of times we got a "yes" in a poll of 100 people
* Trials are independent
 * Not asking the same person in a poll again and again
 * Not asking the same *kind* of person in a poll again and again
* Count the number of successes

Null hypothesis: *the probability of success is $p$

P-value: If the null hypothesis is true, how often would we observe the outcome that we do?

Outcome example: HTHHTHHHHH
We *summarize* the outcome and say we got 8 heads in 10 trial. This *summary* is called a statistic. Another summary is that out of 10 trials, we got success 80% of the time. 80% is another statistic.

##### Computing the p-value

What is the probability of observing an outcome that's as extreme as 80%, assuming the null hypothesis that $p = 0.6$ is true?

0 1 2 3 4 5 6 7 8 9 10

```{r}
pbinom(q = 4, size = 10, prob = 0.6) + (1 - pbinom(q = 7, size = 10, prob = 0.6))
```

If the null hypothesis were $p = 0.5$, the calculation would be



```{r}
pbinom(q = 2, size = 10, prob = 0.5) + (1 - pbinom(q = 7, size = 10, prob = 0.5))
```

Now, the probability of observing 2 or fewer heads is the same as the probability of observing 2 or fewer tails, *if heads and tails are equally likely*. So we can simplify things and compute

```{r}
2 * (1 - pbinom(q = 7, size = 10, prob = 0.5))
```

In fact, we could use

```{r}
2 * pbinom(q = 7, size = 10, prob = 0.5, lower.tail = F)
```

The "lower tail" is to the left of the `q`, and the "upper tail" is to the right.


##### Power

"Underpowered study" -- too little data is collected to gather evidence against the null hypothesis, even if the null is false

Suppose we toss a coin 100 times. Our null hypothesis is that the coin is fair, but actually, the probability of heads is $0.51$. Could we ever tell?

    OBSERVATION: 45
    EXPECTATION: 50
    
    EXPECTATION - abs(EXPECTATION-OBSERVATION) -- lower extreme value
    EXPECTATION + abs(EXPECTATION-OBSERVATION) -- higher extreme value
    
    Prob(fewer than 46 heads)  (same as fewer than 46 tails)
    
    Prob(fewer than 46 heads) + Prob(fewer than 46 tails)
    
    = 
    2 * Prob(fewer than 46 heads)

```{r}
p.val.r <- function(n.tosses, prob){
  sample1 <- rbinom(n = 1, size = n.tosses, prob = prob)
  p.val <- 2 * pbinom(q = 0.5*n.tosses - abs(0.5*n.tosses-sample1), size = n.tosses, prob = 0.5 )
}

my.exp <- data.frame(p.val = replicate(10000, p.val.r(100, .51)))
ggplot(my.exp) + geom_histogram(mapping = aes(x = p.val), binwidth = 0.05)
```

This looks about the same as when the null hypothesis is exactly true.

```{r}
my.exp <- data.frame(p.val = replicate(10000, p.val.r(100, .50)))
ggplot(my.exp) + geom_histogram(mapping = aes(x = p.val), binwidth = 0.05)
```

But what if we toss the coin 10,000 times?

```{r warning = F}
my.exp <- data.frame(p.val = replicate(10000, p.val.r(10000, .51)))
ggplot(my.exp) + geom_histogram(mapping = aes(x = p.val))
```

It looks like the data is now not consistent with the null hypothesis. Let's make sure that if the null hypothesis is exactly true, things would look different.


```{r warning = F}
my.exp <- data.frame(p.val = replicate(10000, p.val.r(10000, .50)))
ggplot(my.exp) + geom_histogram(mapping = aes(x = p.val))
```


This illustrates the fact that the p-value is *not* the probability of the null hypothesis's being true (it either is or isn't, and usually it isn't). Rather, the p-value measures how consistent the data is with the null hypothesis. 

A lot of the time, a small sample will be consistent with the hull hypothesis just because it's small.


#### Normal approximations

We know that for large enough `size` (number of trials) and `prob` close enough to 0.5, the Binomial distribution can be approximated using the normal distribution.

For example,

> If the true probability of heads is 0.6 and the number of trials is 1000, the probability of q heads or fewer can be approximated using `pnorm(q = q, mean = 0.6*1000, sd = sqrt(1000*0.6*(1-0.6))`

Another way of putting that is that we know that the statistic "Number of heads" is distributed with $N(p\times n, n \times p\times (1-p))$.

We can compute the p-value using this knowledge.

### Normal distribution

The normal distribution $N(\mu, \sigma^2)$ generates real numbers with mean $\mu$ and standard deviation $\sigma$. If we have just one observation in the sample, the observation is the statistic. With a null hypothesis $\mu = 5$ and $\sigma = 1$, we can compute the p-value for the observation $x_1 = 10$ using 

```{r}
pnorm(q = 0, mean = 5, sd = 1) + (1-pnorm(q = 10, mean = 5, sd = 1))
```

Or we could use

```{r}
2 * (1-pnorm(q = 10, mean = 5, sd = 1))
```

#### Multiple observations

For multiple observations, the statistic we can use is the mean of the sample. We know that the mean of the sample $\bar{x}$ is distributed according to 

$N(\mu, \sigma^2/n)$



### Normal distribution, unknown standard deviation

If we have multiple observations, and the standard deviation is unknown, we cannot use the idea above. But we know that we can compute

$T = \frac{\bar{X}-\mu}{S/\sqrt{n}}$

We know that the t-statistic is t-distributed with $n-1$ degrees of freedom.

### Two samples from normal distributions

If we have two samples, we don't have a null hypothesis about the the mean. Instead, we have a null hypothesis about the difference between the means. We know that the t-statistic

$T = \frac{\bar{X_1}-\bar{X_2}}{\sqrt{S_1^2/n_1 + S_2^2/n_2}}$ is t-distributed, with $\nu$ degrees of freedom.