t test Type II error %

[obh]

Maybe it might help if I illustrate *why* it's not just a shift.

First, let's start by looking at the test statistic:

t = (xbar1 - xbar2)/(Sp*sqrt(1/n1 + 1/n2)) hopefully the terminology is clear enough here. I'm going to make a slight simplification and just define

SE = (Sp*sqrt(1/n1 + 1/n2))

so that our original reduces to

t = (xbar1 - xbar2)/SE

Great. Now under the null hypothesis that the two means are equal this has a central T distribution with v=n1+n2-2 degrees of freedom right? The main thing to note is that the numerator here (xbar1 - xbar2) has expected value of 0 since under the null hypothesis the means are equal. There are lots of other properties but this is the important one for our purposes.

Now let's consider what happens under the alternative hypothesis

t = (xbar1 - xbar2)/SE

Well the numerator xbar1-xbar2 doesn't have an expected value of 0 anymore. And I'm sure this whole time you've been saying "yes of course and that's why we apply the shift!". But I'm going to show why that isn't quite right. But let's try that approach and see what happens - can we get the expected value of the numerator to be 0? Sort of! We can subtract the expected and then add the expected value right? Let's give it a try

t = ((xbar1 - xbar2) - (mu1-mu2) + (mu1-mu2))/SE
= ((xbar1 - xbar2) - (mu1-mu2))/SE + (mu1-mu2)/SE


Nice. Now we have this broken into two pieces and the first one ((xbar1 - xbar2) - (mu1-mu2))/SE does have an expected value of 0 in the numerator and hey it *does* have a central T distribution. So you might be thinking - yes this is all proving my point because now we have a central T and then we're just adding something to it - that's basically a shift right? Wrong.

Let's look at what we're adding here:
(mu1 - mu2)/SE

The big hiccup here is the SE in the denominator. If that was a constant then this whole term would be constant and the idea of just shifting the T distribution would work perfectly. Unfortunately for us in this situation SE contains the pooled estimate of the standard deviation. Which means that whole term is a random variable itself. And that whole term *isn't* a central T distribution either. If SE was constant (like it is in the case for a Z-test) then this would all work and that's how we go about calculating power for a Z-test. It's just that the math ends up being trickier for this T-test situation.

Fortunately for us though we do have an answer for the overall question: Under the alternative our t-statistic follows a non-central T distribution. Proving that piece is left as an exercise for the reader but hopefully this provided some insight into why we can't just shift the central T to do power calculations here.

Thank you @Dason !

 

[Dason]

If x is distributed Normal, then sample average differences, (x̄ 1 - x̄ 2), are distributed t

That is incorrect. I didn't read through everything but if you're using that assumption then your analysis is wrong.

 

[joeb33050]

Is my analysis now right? I make too many mistakes, thanks for the help.
EXPLANATION: TYPE II ERRORS AND t TESTS

t testing and TYPE II error

The TYPE II or β error is the probability of accepting Ho when Ho is false.

If x is distributed Normal; then (x̄ 1 - x̄ 2) / (s / √ n), is distributed t.

t testing assumes that µ 1 = µ 2

One t test hypothesis is:

Ho: µ 1 = µ 2

Ha: µ 1 > µ 2

Implied is the notion that distribution 2 and µ 2 are the baseline, and that distribution 1 and µ 1 are to be compared with that baseline. In t testing, we design distribution 2, select α, calculate t α, calculate t test, and compare t α and t test. The comparison leads to acceptance or rejection of Ho.



Find/select/calculate n 1, n 2, v, α, tα, x̄ 1, x̄ 2 or µ 2, and s- sometimes pooled s, = s P.

Calculate “t test”.

t test = (x̄ 1 - x̄ 2) / (s /√n 2).

If “t test” ≤ t α; then we accept Ho and reject Ha.

t value units are standard deviations

x value units are ounces or inches or days or…

Degrees of freedom, v, = (n1 -1 + n2 -1) when n > 0. There is one t distribution for each = v.

If v and α are known, t α can be easily looked up using an online calculator.

A t distribution can describe a set of data, sets of x values, such as inches.

t is a number, such as .456. x is a number OF something, such as 2.375 inches.

t α, the t value of α, can be looked up with an online calculator.

x α, the x value of α, can be calculated thus: x α = t α * (σ / √ n).

x α is a number OF something, such as inches or ounces or days, or…

If t test ≤ t α; then we accept Ho and reject Ha.

If x̄ test ≤ x̄ α; then we accept Ho and reject Ha.

Now to the TYPE II error.

If µ 1 ≠ x̄, if µ 1 is replaced by µ β; what is the probability of accepting Ho and rejecting Ha?

We need to design a t distribution with µ β, σ β = σ, and t α β.

We assumed that µ 1 = µ 2, now know that µ 1 is replaced by µ β and that µ β ≠ µ 1 or µ 2.

t = (µ β - x̄ 2) / (σ / √ n)

x α β = x α + (µ β - µ 2)

t α β = x α β * (σ / √ n)

P (t ≤ α β) = TYPE II error % - the probability of accepting Ho when it is false.

Example: t testing TYPE II error

Do red apples weigh the same as yellow apples? We take samples of 16 each, weigh them, and calculate the mean and standard deviation of each, in ounces. Since n = 16, a t test is appropriate.

n1 = n 2 = 16; v = 30

x̄ Y = 3.2 oz; x̄ R = 3.0 oz

s Y = 1.4 oz, s R = 1.0 oz; s P = 1.2 oz, (s P is the pooled value of s.)

α = .05; t α = 1.697

Ho: µ Y = µ R

Ha: µ Y > µ R; a RIGHT TAIL test is required

t test = (x̄ Y - x̄ R) / (s P / √n) = .2 oz / (1.2 oz / 4) = .2 oz / .3 oz =.67

t test, .67 < t α, 1.697; Accept Ho and reject Ha.

x α = x̄ R + (t α * (s P/√n)) = 3.0 oz + (1.697 * (1.2 oz / √ 16)) = 3.0 oz + (1.697 * .3 oz = 3.5091 oz

If x̄ test ≤ 3.5091 oz, Accept Ho and reject Ha.

If µ Y = 3.4 oz = µ β, what is the probability of accepting Ho when it is false, what is the probability of x̄ test ≤ 3.5091 oz, what is P (t test ≤ t β)?



Imagine a v = 30 t distribution with µ = µ β = 3.4 oz and σ = s P = 1.2 oz and n = 16.

The x α value of that distribution is x α β = x α = 3.5091 oz.

(If x̄ test ≤ 3.5091 oz, Accept Ho and reject Ha).

The t value is then t α β = (x α β - µ β)/ (s P / √n) = (3.5091 oz – 3.4 oz) / .3 oz = .1091 oz / .3 oz =.3637.

And P (t test ≤ .3637) = .6407 = the TYPE II error, the probability of accepting Ho when µ Y = 3.4 oz.

(Or, imagine a Normal distribution, µ = 3.4 oz, σ = 1.2 oz, n = 16.

What is the probability of Z ≤ 3.5091 oz?

Z = (3.5091 oz – 3.4 oz) / (1.2 oz / √16) = 1.091 oz / .3 oz = .3767, of course.

P (Z ≤ .3767) = .647, not far from .6407.)

 

[Dason]

I still don't get your entire approach. Why are you doing all of this when the only question that matters is what distribution does the t statistic have when the alternative hypothesis is true.

 

[joeb33050]

The question that matters to me is: "In a t test situation, how can we calculate the TYPE II error?". This leads to POWER, and on. And led me to t vs. Z.

 

[Dason]

Yeah and we told you. You look at the distribution of the test statistic under the specified parameters. But like we mentioned - the t statistic in that situation follows a non central T distribution.

I literally don't care about any of the other distribution stuff you post because it isn't directly looking at what is the distribution of the test statistic which is all that matters.

 

[joeb33050]

In my example above: "P (t test ≤ .3637) = .6407 = the TYPE II error, the probability of accepting Ho when µ Y = 3.4 oz." How far is this from your result?

 

[joeb33050]

Well, I checked the non-central distribution and solved for P (t test </+ t) = .6294; vs. .6407 above. I do not understand what/how to calculate v in the non-central dist, is it v in the t calculation? I will work on more comparisons.