t test Type II error %

fed2

Active Member
#22
In the standard sample size equations out there the non centrality parameter is actually ignored. In fact t is often replaced with normal dist altogether. It is non-central though, but who has the time for that with all these great stats posts too read.
 
#25
Hi Joe,

Usually, you start a question with the data like ave(x1)=39, avg(x2)=44 n1= n2= ..
I assume it should be average, not μ since if you know μ1 and μ2 you wouldn't run the t-test...

Despite the fact that I can't see the input data, it looks like your calculation was correct if H1 distribution would be central t.
not sure because I can't see the question's data
But it should be non-central ...

If you don't believe it... you may try to calculate the beta using simulation, and see what distribution will give you the correct beta...
It is very easy to do a simulation in R
 
#26
Hi obh;
All the data is on the chart, and I'm NOT running a t test, I'm estimating TYPE II error. If we know the sigma, mu's, n's and alpha; we can estimate TYPE II error
I have never found, anywhere, a way to estimate this error, and since 1972 have taught to use a Normal dist. method.
Maybe my method above is new, original, never-done-before. Maybe not. Maybe I'm a poor explainer. Maybe I'm wrong. That's why I'm here.
 

Dason

Ambassador to the humans
#29
So as had been said before it's perfectly possible to do the power calculation. It requires the non-central T distribution to do so. No amount of 'shifting' the T distribution will be able to match the non-central T exactly so you're fighting a losing battle by taking this approach.

Your results might be close in some cases but they won't be exact.
 

Dason

Ambassador to the humans
#30
Maybe it might help if I illustrate *why* it's not just a shift.

First let's start by looking at the test statistic:

t = (xbar1 - xbar2)/(Sp*sqrt(1/n1 + 1/n2)) hopefully the terminology is clear enough here. I'm going to make a slight simplification and just define

SE = (Sp*sqrt(1/n1 + 1/n2))

so that our original reduces to

t = (xbar1 - xbar2)/SE

Great. Now under the null hypothesis that the two means are equal this has a central T distribution with v=n1+n2-2 degrees of freedom right? The main thing to note is that the numerator here (xbar1 - xbar2) has expected value of 0 since under the null hypothesis the means are equal. There are lots of other properties but this is the important one for our purposes.

Now let's consider what happens under the alternative hypothesis

t = (xbar1 - xbar2)/SE

Well the numerator xbar1-xbar2 doesn't have an expected value of 0 anymore. And I'm sure this whole time you've been saying "yes of course and that's why we apply the shift!". But I'm going to show why that isn't quite right. But let's try that approach and see what happens - can we get the expected value of the numerator to be 0? Sort of! We can subtract the expected and then add the expected value right? Let's give it a try

t = ((xbar1 - xbar2) - (mu1-mu2) + (mu1-mu2))/SE
= ((xbar1 - xbar2) - (mu1-mu2))/SE + (mu1-mu2)/SE


Nice. Now we have this broken into two pieces and the first one ((xbar1 - xbar2) - (mu1-mu2))/SE does have an expected value of 0 in the numerator and hey it *does* have a central T distribution. So you might be thinking - yes this is all proving my point because now we have a central T and then we're just adding something to it - that's basically a shift right? Wrong.

Let's look at what we're adding here:
(mu1 - mu2)/SE

The big hiccup here is the SE in the denominator. If that was a constant then this whole term would be constant and the idea of just shifting the T distribution would work perfectly. Unfortunately for us in this situation SE contains the pooled estimate of the standard deviation. Which means that whole term is a random variable itself. And that whole term *isn't* a central T distribution either. If SE was constant (like it is in the case for a Z-test) then this would all work and that's how we go about calculating power for a Z-test. It's just that the math ends up being trickier for this T-test situation.

Fortunately for us though we do have an answer for the overall question: Under the alternative our t-statistic follows a non-central T distribution. Proving that piece is left as an exercise for the reader but hopefully this provided some insight into why we can't just shift the central T to do power calculations here.
 
#31
Maybe it might help if I illustrate *why* it's not just a shift. x̄

First let's start by looking at the test statistic:

t = (xbar1 - xbar2)/(Sp*sqrt(1/n1 + 1/n2)) hopefully the terminology is clear enough here.

I'm going to make a slight simplification and just define

SE = (Sp*sqrt(1/n1 + 1/n2))

so that our original reduces to

t = (xbar1 - xbar2)/SE

Great. Now under the null hypothesis that the two means are equal this = t test has a central T distribution with v=n1+n2-2 degrees of freedom right?

The main thing to note is that the numerator here (xbar1 - xbar2) has expected value of 0 since under the null hypothesis the means are equal. There are lots of other properties but this is the important one for our purposes.

Now let's consider what happens under the alternative hypothesis

t = (xbar1 - xbar2)/SE


Nyet! Here, you decide that x̄ 1 is the WHAT IF value.

The root question is: “What if x 1 = what I call µ β, where µ β ≠ x 2 or µ 2?”

Or:” Given that we know n, s and α; what is P (t ≥ t β), where x α = x α β?”

I use two distributions, you use one, the non-centered t.

I use the µ 2 and µ β distributions.

Take the data for the x 1 and x2 distributions and find t α and x α = t α * s.

x α is going to be some number/units like 34 inches or 11 pints.

The question is” If µ 2 = 10 pints and µ β = 12 pints, what is P (t ≤ t β)?”

You use 1 distribution, I use 2.

So here I stop. I could be wrong, but this/yours does not seem to explain why.

We are talking about different approaches/methods.
 
#33
Maybe it might help if I illustrate *why* it's not just a shift.

First, let's start by looking at the test statistic:

t = (xbar1 - xbar2)/(Sp*sqrt(1/n1 + 1/n2)) hopefully the terminology is clear enough here. I'm going to make a slight simplification and just define

SE = (Sp*sqrt(1/n1 + 1/n2))

so that our original reduces to

t = (xbar1 - xbar2)/SE

Great. Now under the null hypothesis that the two means are equal this has a central T distribution with v=n1+n2-2 degrees of freedom right? The main thing to note is that the numerator here (xbar1 - xbar2) has expected value of 0 since under the null hypothesis the means are equal. There are lots of other properties but this is the important one for our purposes.

Now let's consider what happens under the alternative hypothesis

t = (xbar1 - xbar2)/SE

Well the numerator xbar1-xbar2 doesn't have an expected value of 0 anymore. And I'm sure this whole time you've been saying "yes of course and that's why we apply the shift!". But I'm going to show why that isn't quite right. But let's try that approach and see what happens - can we get the expected value of the numerator to be 0? Sort of! We can subtract the expected and then add the expected value right? Let's give it a try

t = ((xbar1 - xbar2) - (mu1-mu2) + (mu1-mu2))/SE
= ((xbar1 - xbar2) - (mu1-mu2))/SE + (mu1-mu2)/SE


Nice. Now we have this broken into two pieces and the first one ((xbar1 - xbar2) - (mu1-mu2))/SE does have an expected value of 0 in the numerator and hey it *does* have a central T distribution. So you might be thinking - yes this is all proving my point because now we have a central T and then we're just adding something to it - that's basically a shift right? Wrong.

Let's look at what we're adding here:
(mu1 - mu2)/SE

The big hiccup here is the SE in the denominator. If that was a constant then this whole term would be constant and the idea of just shifting the T distribution would work perfectly. Unfortunately for us in this situation SE contains the pooled estimate of the standard deviation. Which means that whole term is a random variable itself. And that whole term *isn't* a central T distribution either. If SE was constant (like it is in the case for a Z-test) then this would all work and that's how we go about calculating power for a Z-test. It's just that the math ends up being trickier for this T-test situation.

Fortunately for us though we do have an answer for the overall question: Under the alternative our t-statistic follows a non-central T distribution. Proving that piece is left as an exercise for the reader but hopefully this provided some insight into why we can't just shift the central T to do power calculations here.
Thank you @Dason !
 
#34
Do red apples weigh the same as yellow apples? We take samples of 16 each, weigh them, and calculate the mean and standard deviation of each, in ounces. Since n = 16, a t test is appropriate.

n1 = n 2 = 16; v = 30

x̄ R = 3.0 oz; x̄ Y = 3.2 oz

s R = 1.0 oz; s Y = 1.4 oz, s P = 1.2 oz

α = .05; t α = 1.697

Ho: µ Y = µ R

Ha: µ Y > µ R; a RIGHT TAIL test is required

t test = (x̄ Y - x̄ R) / (s P / √n) = .4 oz / (1.2 oz / 4) = 1.2 oz / .3 oz = .4

t test, .4 < t α, 1.697; Accept Ho and reject Ha.

x α = x̄ R + (t α * (s P/√n)) = 1.697 * (1.2 oz / √ 16) = 3.0 oz + (1.697 * .3 oz = 3.5091 oz

If x̄ test ≤ 3.5091 oz, Accept Ho and reject Ha.

If µ Y = 3.4 oz = µ β, what is the probability of accepting Ho when it is false, what is the probability of x̄ test ≤ 3.5091 oz, what is P (t test ≤ t β)\



Imagine a v = 30 t distribution with µ = µ β = 3.4 oz and σ = s P = 1.2 oz and n = 16.

The α value of that distribution is x α β = x α = 3.5091 oz.

(If x̄ test ≤ 3.5091 oz, Accept Ho and reject Ha).

The t value is then t α β = (x α β - x̄ β)/ (s P / √n) = (3.5091 oz – 3.4 oz) / .3 oz = .1091 oz / .3 oz =.3637.

And P (t test ≤ .3637) = .6407 = the TYPE II error, the probability of accepting Ho when µ Y = 3.4 oz.

Or, imagine a Normal distribution, µ = 3.4 oz, σ = 1.2 oz, n = 16.

What is the probability of Z ≤ 3.5091 oz?

Z = (3.5091 oz – 3.4 oz) / (1.2 oz / √16) = 1.091 oz / .3 oz = .3767, of course.

P (Z ≤ .3767) = .647, not far from .6407.