PK iA:5! , Problem_Set_5_Inference/01-Poll_Error.en.srt1
00:00:00,000 --> 00:00:04,000
In opinion polls, we often report a margin of error.
2
00:00:04,000 --> 00:00:09,000
Something like this margin of error is the same as the confidence interval width.
3
00:00:09,000 --> 00:00:15,000
Usually, the margin of error in polls is stated at the 95% level.
4
00:00:15,000 --> 00:00:21,000
Consider two candidates. In our poll, candidate A has 55 supporters.
5
00:00:21,000 --> 00:00:26,000
Candidate B has only 45 supporters.
6
00:00:26,000 --> 00:00:29,000
Please tell me what we would report as the results of the poll.
7
00:00:29,000 --> 99:59:59,000
Percentage of support for candidate A plus or minus the margin of error at 95% confidence.
PK iAf* 5 Problem_Set_5_Inference/02-Poll_Error_Solution.en.srt1
00:00:00,000 --> 00:00:05,000
Of course the percentage support for candidate A is 55%.
2
00:00:05,000 --> 00:00:10,000
To compute the margin of error, we will use the following formula.
3
00:00:10,000 --> 00:00:18,000
1.96 the quartile from the normal at 97.5% times the square root of
4
00:00:18,000 --> 00:00:27,000
P times 1 minus P equals n and substituting n and evaluating this expression
5
00:00:27,000 --> 99:59:59,000
the square root of 0.550.45/1001.96 is equal to 9.75%.
PK iACOy y - Problem_Set_5_Inference/03-Sample_Size.en.srt1
00:00:00,000 --> 00:00:08,000
Let' assume we want a lower margin of error to say 5%. How many people should we ask?
PK iAm:3 6 Problem_Set_5_Inference/04-Sample_Size_Solution.en.srt1
00:00:00,000 --> 00:00:08,000
Now, you might say that the answer will depend in part on what we see the response
2
00:00:08,000 --> 00:00:16,000
is being but to simplify this let's assume that 55% still will support candidate A.
3
00:00:16,000 --> 00:00:24,000
Well that won't quite be right. So to answer this question, we'll need to solve this formula for n.
4
00:00:24,000 --> 00:00:28,000
Let's call the margin of error e. We can actually simplify this.
5
00:00:28,000 --> 00:00:41,000
If we just square and divide, we get e²/1.96² is equal to the old variance 0.550.45/n.
6
00:00:41,000 --> 00:00:47,000
We actually know e. We specify that is going to be 0.05.
7
00:00:47,000 --> 00:00:59,000
That gives us, if we multiply through by n, so if we calculate this out we get 380.3.
8
00:00:59,000 --> 00:01:05,000
Now since it's pretty hard to ask a fraction of a person, we actually have to ask a bit more.
9
00:01:05,000 --> 99:59:59,000
We have to ask 381 people.
PK iAϒ) ) - Problem_Set_5_Inference/05-Sensitivity.en.srt1
00:00:00,000 --> 00:00:07,000
In answering how many people we actually need to poll, we made an assumption.
2
00:00:07,000 --> 00:00:11,000
And the assumption more or less is that small changes in p don't matter.
3
00:00:11,000 --> 00:00:18,000
So we don't have to worry about getting it exactly right if we're trying to estimate a sample size.
4
00:00:18,000 --> 00:00:26,000
To prove this to ourselves, let's redo this calculation with p = 0.5 and p = 0.6.
5
00:00:26,000 --> 99:59:59,000
Please enter your values for n here.
PK iA 6 Problem_Set_5_Inference/06-Sensitivity_Solution.en.srt1
00:00:00,000 --> 00:00:05,000
Looking at our solution over here, we can see that the only place this matters
2
00:00:05,000 --> 00:00:11,000
is p(1-p) flows right through to down here.
3
00:00:11,000 --> 00:00:14,000
So we need to change these to numbers.
4
00:00:14,000 --> 00:00:25,000
So this is going to be 0.50.51.96²/0.05²
5
00:00:25,000 --> 00:00:31,000
and for this it will simply be 0.6 times 0.4 times the same thing.
6
00:00:31,000 --> 00:00:46,000
For p = 0.5, this evaluates to 384.16 and for 0.6 it evaluates to 369.79.
7
00:00:46,000 --> 00:00:55,000
Since again we can't have fractional people, the answer is 385 and the answer for 0.6 is 369.
8
00:00:55,000 --> 00:01:01,000
A quick way to see why this changed so little is simply that this is the same.
9
00:01:01,000 --> 99:59:59,000
N 0.50.5 is 0.25. N 0.60.4 is 0.24.
PK iAl , Problem_Set_5_Inference/07-More_Error.en.srt1
00:00:00,000 --> 00:00:04,000
Often we find in reality that the margin of error is wrong.
2
00:00:04,000 --> 00:00:10,000
Which of these reasons could make us understate the true margin of error?
3
00:00:10,000 --> 00:00:18,000
If the poll is taken before the election at a different time, does that change the margin of error?
4
00:00:18,000 --> 00:00:23,000
If some voters are under-represented in our sample, could that increase error?
5
00:00:23,000 --> 00:00:32,000
Similarly, if some in our sample are less likely to vote than others, does that affect our error?
6
00:00:32,000 --> 00:00:39,000
Now, assume that we haven't somehow perfectly matched up the frequency of voting
7
00:00:39,000 --> 00:00:44,000
and their relative representation in sample.
8
00:00:44,000 --> 99:59:59,000
And if the responses differ from what someone intends to actually do, could that cause more error?
PK iAi 5 Problem_Set_5_Inference/08-More_Error_Solution.en.srt1
00:00:00,000 --> 00:00:04,000
Each of these things can cause more errors. The answer is all of them marked.
2
00:00:04,000 --> 00:00:10,000
The poll being taken before the election can certainly affect its accuracy because
3
00:00:10,000 --> 00:00:13,000
sentiments shift over time.
4
00:00:13,000 --> 00:00:15,000
Some being under-represented in the samples.
5
00:00:15,000 --> 00:00:20,000
Certainly, if voters who favor one candidate versus another are under or over-represented
6
00:00:20,000 --> 00:00:22,000
that will change the results.
7
00:00:22,000 --> 00:00:28,000
Similarly if we tend to have people who are more likely to vote
8
00:00:28,000 --> 00:00:31,000
represented the same as those who are less likely to vote,
9
00:00:31,000 --> 00:00:34,000
they're not being over-represented enough.
10
00:00:34,000 --> 00:00:39,000
So these two things ideally should be linked.
11
00:00:39,000 --> 00:00:44,000
And in a well-designed poll, they try to do this although it's hard to do this perfectly.
12
00:00:44,000 --> 99:59:59,000
And responses that differ of course will cause additional error.
PK iA1 - Problem_Set_5_Inference/09-Weight_Loss.en.srt1
00:00:00,000 --> 00:00:03,000
Let's assume we have two different ways to lose weight,
2
00:00:03,000 --> 00:00:07,000
and we want to figure out which one is the most effective.
3
00:00:07,000 --> 00:00:15,000
We have 10,000 people who received Treatment A. Their average loss is 10 pounds.
4
00:00:15,000 --> 00:00:21,000
The standard deviation of their loss is also 10 pounds.
5
00:00:21,000 --> 00:00:25,000
Let's consider a second treatment, Treatment B.
6
00:00:25,000 --> 00:00:32,000
We also applied it to 10,000 people. Our average loss in this case was 20 pounds.
7
00:00:32,000 --> 00:00:35,000
And we also have a standard deviation of 20 pounds.
8
00:00:35,000 --> 00:00:42,000
Our null hypothesis of course is that weight loss from Treatment A equals
9
00:00:42,000 --> 00:00:45,000
the average weight loss from Treatment B.
10
00:00:45,000 --> 00:00:52,000
Our alternative hypothesis is if we are perhaps the providers of Treatment B
11
00:00:52,000 --> 00:00:58,000
is that WB is greater than WA or alternatively that the difference is positive.
12
00:00:58,000 --> 99:59:59,000
With a 5% allowable false positive rate, which hypothesis do we choose to accept?
PK iA+U?Q1 1 6 Problem_Set_5_Inference/10-Weight_Loss_Solution.en.srt1
00:00:00,000 --> 00:00:06,000
To answer this, our variable of interest is the difference of these two things
2
00:00:06,000 --> 00:00:16,000
that has a mean of 10 pounds and its standard deviation is going to be the sum
3
00:00:16,000 --> 00:00:20,000
of this standard deviation and this standard deviation.
4
00:00:20,000 --> 00:00:27,000
Now we need to find out the standard deviation of this. Since variances add, let's work with those.
5
00:00:27,000 --> 00:00:40,000
So the variance of Treatment B is 400/10,000 and the variance of Treatment A is 100.
6
00:00:40,000 --> 00:00:47,000
So if we evaluate this, this will give us the standard deviation of this number.
7
00:00:47,000 --> 00:00:58,000
And it's 0.2236. So we can see the ratio of this to this.
8
00:00:58,000 --> 00:01:06,000
This is much, much greater than the critical value of 1.645.
9
00:01:06,000 --> 00:01:12,000
And so in this case we reject the null hypothesis and therefore accept the alternative.
10
00:01:12,000 --> 00:01:18,000
And there's one particularly interesting feature of these two treatments
11
00:01:18,000 --> 00:01:20,000
as they relate to each other.
12
00:01:20,000 --> 99:59:59,000
Just consider how many people lost weight in each case, and post your thoughts to the forums.
PK iA=~@ @ 2 Problem_Set_5_Inference/11-Hypothesis_Tests.en.srt1
00:00:00,000 --> 00:00:04,000
Now, I'd like to give you a number of situations where you might be able
2
00:00:04,000 --> 00:00:06,000
to use hypothesis tests.
3
00:00:06,000 --> 00:00:11,000
In which of these situations would it be helpful to use a hypothesis test?
4
00:00:11,000 --> 00:00:16,000
Comparing crash rates of two different airlines with 100 flights each.
5
00:00:16,000 --> 00:00:21,000
In case you don't know but I assume most of you do, planes don't crash very often.
6
00:00:21,000 --> 00:00:26,000
Comparing five-year auto repair rates on 100 vehicle samples.
7
00:00:26,000 --> 00:00:31,000
Comparing the heights of the tallest buildings in two different cities.
8
00:00:31,000 --> 99:59:59,000
And determining if a course improved test scores in some sample of students.
PK iA ; Problem_Set_5_Inference/12-Hypothesis_Tests_Solution.en.srt1
00:00:00,000 --> 00:00:05,000
And I would say the answers are comparing auto repair rates.
2
00:00:05,000 --> 00:00:08,000
Those are simply proportions that can be compared to each other
3
00:00:08,000 --> 00:00:11,000
as we've done for many things.
4
00:00:11,000 --> 00:00:13,000
And determining if a course improve test scores.
5
00:00:13,000 --> 00:00:18,000
We would argue that we can't compare crash rates of two airlines with 100 flights
6
00:00:18,000 --> 00:00:21,000
because in general there aren't crashes.
7
00:00:21,000 --> 00:00:24,000
Now in theory, you could say you can use a hypothesis test
8
00:00:24,000 --> 00:00:30,000
if you recognize that it's not normally distributed and do the appropriate
9
00:00:30,000 --> 00:00:34,000
binomial test and figure out that you can't determine anything
10
00:00:34,000 --> 00:00:38,000
but it's generally not adding anything over intuition.
11
00:00:38,000 --> 00:00:43,000
And comparing heights of the tallest building in two cities, that's a deterministic number.
12
00:00:43,000 --> 00:00:48,000
It's either their taller or it's not. There are no random errors. There's no need for a hypothesis test.
13
00:00:48,000 --> 00:00:54,000
That again a more creative application could be that you'd want to use it to
14
00:00:54,000 --> 00:00:58,000
the measurements of the buildings in each of the two cities to determine whether
15
00:00:58,000 --> 00:01:06,000
one was taller but that's only going to be useful if the measurements have the error
16
00:01:06,000 --> 00:01:10,000
on the same order of the difference in buildings which is unusual.
17
00:01:10,000 --> 99:59:59,000
So in practice you would not use a hypothesis test there.
PK iA:J J 4 Problem_Set_5_Inference/13-Large_Sample_Limit.en.srt1
00:00:00,000 --> 00:00:05,000
As the sample size approaches infinity, what happens to the confidence interval width?
2
00:00:05,000 --> 00:00:11,000
As the sample size approaches infinity, the confidence interval width approaches--
3
00:00:11,000 --> 99:59:59,000
0. Some constant that varies based on the problem.
PK iA]O^2 2 = Problem_Set_5_Inference/14-Large_Sample_Limit_Solution.en.srt1
00:00:00,000 --> 00:00:02,000
The answer is 0. See this.
2
00:00:02,000 --> 00:00:08,000
Remember that the width of the confidence interval decreases with the √n.
3
00:00:08,000 --> 99:59:59,000
But as n goes to infinity, the √n goes to infinity and any number divided by infinity is 0.
PK iA;h h 3 Problem_Set_5_Inference/15-Large_Sample_Test.en.srt1
00:00:00,000 --> 00:00:05,000
Consider two marketing campaigns. Campaign A has a response rate of 1%.
2
00:00:05,000 --> 00:00:16,000
And we have a second marketing campaign, Campaign B, that has a response rate of 1.01%.
3
00:00:16,000 --> 99:59:59,000
For each of these, we have 10 million data points. At 95% confidence, is this significant?
PK iAvgW <