---start---

office hours: wednesday: 11-1; monday: 12-1, 4-5
4044 VHUP 898 5076

So. we ended last time with a one sided rule for hypothesis testing. 
Next: two sided rule.
recall that we had decided to interject a hypothesis testing format on 
top of the puppy pair expt, accepting the null hypothesis as a basis. if 
null = true, %wins for vaccine==%wins for placebo
Ho: ¼+ = ¼- = 1/2

p+ = vaccine wins
p- = placebo wins
ties are thrown out of statistical analysis.

recall that we said 11% of the time we'd see 5 or 6 vaccine wins even if 
placebo and vaccine were equal.

now, usually,these one sided tests aren't used.

if we do NOT accept null hypothesis, we do:

H1: ¼+ > ¼- > 1/2
this assumes that the vaccine is better than the placebo.

in two sided test, null stays the same, but you can use alternative 
hypothesis:

H1: ¼+ ‚ ¼-

which just says that the vaccine is either better or worse than the 
placebo.

we gather evidence against null hoping to support alternative theory 
(state of nature hypothesis)

[he's shading stuff on the graph on the board and talking about alpha 
regions...]


so, now you accept  0 or 1 wins for vaccine or 5 or 6 wins for vaccine 
and you add the probabilities so now instead of .1094 like before, the 
alpha value is .2188

so you can account for possibility of the vaccine making infection MORE 
likely.

now, you could change it to 0 wins or 6 wins, which would make alpha .03
this is an arbitrary decision. why do you do this? well, before you have 
a nearly 22% chance of error, and now you have a 3% chance of error. and 
you want to minimize error, right? so.

if you get into tails of curve, either one or two tails, and you get into 
rejection region, you reject null on basis that alternative is more 
likely, but you COULD BE WRONG and you want to minimize possibility of 
being wrong!

Sabin oral poliovaccine trial: tested in kibbutzim, because people lived 
closely, and transmission mechnism had to do with human feces, and they 
wanted to see how fast this would move...10% of population took vaccine, 
and over time, 90% of people got the immunity. so then they modified it 
so people could "have a choice" so it wasn't communicable so well. but 
unlike other vaccines, this was of course a live vaccine, and they had 
already figured out they would give polio to 6 in 6 million people. now, 
2-3000/million were ALREADY getting it. but you can't guarantee that the 
6 who get vaccine induced polio are in the 2000-3000 who would have 
gotten it anyway. Now, they decided to give it to people anyway. so some 
people refused to take it. now, when testing it originally in israel, 
they used VERY small significance percentages. so the importance of what 
is being tested has to do with how you set the alpha region. 

in a real experiment, this is done by power and sample size, a technique 
used before the experiment is done tofigure out how to set sample size 
and get probabilities.

when drug companies do this, and they do it a lot, every subject included 
in the experiment, especially these days, costs big bucks! average 
5-10,000 per subject is what it costs the company. so you want to 
MINIMIZE the number of subjects while at the SAME time MAXIMIZING chance 
of being RIGHT.

think of this question:

resident, intern, world renowned researchers...they want to know HOW MANY 
SUBJECTS DO I NEED???

how do you answer that?

investigator comes in and tells statistician about the biology and the 
experiment and asks how many subjects do i need.

every once in a while to be funny, Lasker would pretend to make a 
calculation and then say "twenty-two" and the researcher would get up to 
leave,and then he'd say wait! that was a joke! :)

you can't just DO that.

ponder this. you set up an experiment and hope to have results that turn 
out to be worth finding. what is worth finding? (see p 6 handout 2)

fig 3a ¼+ = .5 (no diff between placebo and vaccine)
so if 5 or 6 vaccine wins, you reject null in favor of hypothesis, with 
alpha = .1094

but, given you're using 6 subjects, how good a test is it? well, given 
such a rule, someone could say, what if I were to tell you that we have, 
based on phase I and II trials, a belief that this vaccine will really 
win 80% of the time? Well. in 80% of puppy pairs in which there is no 
tie, the vaccine will win. so, marketing people say, hey. if the vaccine 
works 80% of the time, then ¼+ = 0.8, right? ok. how do you show, 
binomially, the binomial distribution if the split were 80-20? you apply 
the binomial theorem as shown in handout, and the histogram with 
probability distribution is seen on lower part of p 6. 
so, most likely outcome is now 5 wins. ( p5 = .3930, p6 = .2626). Now, if 
you are assuming that this is the truth, that the vaccine WILL win 80% of 
the time. so if you do a WHOLE BUNCH of puppy pair experiments using 6 
subjects in each, how would you show the probability? as seen on p 6. so, 
about 65.6% of the time that you conduct the experiment, you would detect 
5 or 6 wins for the vaccine, and 34.4% of the time you would detect fewer 
than 5 wins. this is what happens when you KNOW the vaccine WILL win 80% 
of the time. now, that doesn't look very good, does it? recall, under the 
null, we saw 5 or 6 wins 11% of the time. now we see it 66% of the time. 
before, vaccine had only 50% chance of winning; now has 80% chance of 
winning.

so, Ho-->null is true, get 5 or 6 wins 11% of time
    H1-->null not true, ¼+ = .8, get 5 or 6 wins 66% of time.

now, we always use the null to test the hypothesis. but to figure out how 
many subjects you need, you need to make a supposition as to how 
effective the product is, so you can estimate sample size.

now, statistician has suggested 6 subjects. marketing says it works 80% 
of time. so then statistician has to figure it out....the question is, if 
you use the experimental procedure proposed under the null when the 
supposition is made, in what % of cases will you correctly reject the 
null? 66%. so. if vaccine really works 80% of time, 20% of time you will 
NOT get a 5 or 6. you will get 0-4 about 34% of the time. 

so. 11% chance of type one error (rejecting null when it is true)
    34% chance of type two error (NOT rejecting null when it is FALSE)

power = 1 - type two error.

so you have a 66% chance of making the correct choice if the vaccine 
works 80% of the time. this is not very good. 
power = 1 - 0.34 = 0.66

you want to MINIMIZE alpha- 11% too high.
now, if you use 6 instead of 5 or 6 as your rejection criterion, you 
reduce alpha to 1.6%, but then beta becomes 75%, and 1 - beta = .2626

this sucks.

so, you need more subjects.  let n=18 puppy pairs.see bottom of p 7 for 
example.
under the binomial null, where n=18 and probability of win = .5:

p (X „ 14/Ho) = .017 + .0031 + .0006 + .0001 + .0000 = .0155
(we select 14,15,16,17,or 18 as compound rejection region)


now, if you raise ¼+ to 0.8, you get .7164 in the equation above. so beta 
is < 0.3, alpha is .0155, and 1 - beta is .72 which is good.

now, if it doesn't work 80% of time...say it only works 60% of the time.

the type one error will not change, that's based on the null. it remains 
.0155. if you recalculate p (X „ 14/H1) using .6, you get .0942, a 
miserable 9% power, and 91% type two error.

HOMEWORK will be given out on wednesday. it will be due the next week.
----end----