+ Reply to Thread
Page 2 of 2 FirstFirst 1 2
Results 16 to 28 of 28

Thread: Hypothesis-test: difference between paired means

  1. #16
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means




    Also on the site I found this analysis, and IMO this can not be right:

    "State the hypotheses. The first step is to state the null hypothesis and an alternative hypothesis.

    Null hypothesis: μ >= 110
    Alternative hypothesis: μ < 110

    Note that these hypotheses constitute a one-tailed test. The null hypothesis will be rejected if the sample mean is too small.

    Formulate an analysis plan. For this analysis, the significance level is 0.01. The test method is a one-sample t-test.

    Analyze sample data. Using sample data, we compute the standard error (SE), degrees of freedom (DF), and the t-score test statistic (t).

    SE = s / sqrt(n) = 10 / sqrt(20) = 10/4.472 = 2.236
    DF = n - 1 = 20 - 1 = 19
    t = (x - μ) / SE = (108 - 110)/2.236 = -0.894

    where s is the standard deviation of the sample, x is the sample mean, μ is the hypothesized population mean, and n is the sample size.

    Since we have a one-tailed test, the P-value is the probability that the t-score having 19 degrees of freedom is less than -0.894."

    When calculating the t-value the analysis uses the "the hypothesized population mean" but there is no such thing, the hypothesis is an interval. Also, I see no difference in the p-value calculation from the case where they usedd the null hypothesis mean = 110 and there has to be a difference, right?

  2. #17
    Devorador de queso
    Points: 95,940, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,937
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Hypothesis-test: difference between paired means

    I don't really want to get into the whole theory of it all right now but it's just a likelihood ratio test: http://en.wikipedia.org/wiki/Likelihood-ratio_test

    You would use the "most likely" value of the parameter under the null hypothesis when calculating your p-value. If the sample mean in your case was above 110 then it boils down to the exact same thing you would do if you specified your null as \mu = 110.
    I don't have emotions and sometimes that makes me very sad.

  3. #18
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means

    Quote Originally Posted by Dason View Post
    Well it doesn't really change much of anything if the test statistic falls in the region that is more likely under the alternative hypothesis.
    i would still see two problems:
    1. how to define a t-value in this case?( x-mu)/s will not work because we have no defined mu.
    2. how to calculate the probability of the statistic when we have an infinite number of possibilities (one for each possible value of mu) for the mean, each of those contributing a small amount to the probabilty?

    regards
    rogojel

  4. #19
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means

    hi,
    that means that from the calculations POV it actually does not matter whether one specifies a constant as the null or a semi-infinite interval? this is really counter-intuitive, I would definitely expect different p-values.

    From a theoretical perspective there is a huge difference IMHO: Imagine I wanted to sell homeopathic medicine and set up a test to prove that my medicine works. I set up zhe test with the null being - my effect is greater or equal to zero. Then I ran the test, with the exact same calculations as if the null was : the effect is zero. I then find that I can not reject the null - under the constant null this means that I can not make the claim that my treatment works. Under the interval null the exactly same test means that my opponents can not make the claim that it does NOT work.

    what do you think?

  5. #20
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means

    Quote Originally Posted by Dason View Post
    I don't really want to get into the whole theory of it all right now but it's just a likelihood ratio test: http://en.wikipedia.org/wiki/Likelihood-ratio_test

    You would use the "most likely" value of the parameter under the null hypothesis when calculating your p-value. If the sample mean in your case was above 110 then it boils down to the exact same thing you would do if you specified your null as \mu = 110.
    hi Dason,
    just a last thought. I do not know the theory, but just from a practical point of view, how can the most likely value be the lower edge of the interval? I make the hypothesis that the mean is greater or equal to a value D. What assumptions do I need to make to have the most likely value be equal to D?

  6. #21
    Devorador de queso
    Points: 95,940, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,937
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Hypothesis-test: difference between paired means

    If I tell you that the true mean is less than or equal to 110 and then tell you that the sample mean is 120 - what would you 'best guess' be for the true mean? It would be 110 of course.
    I don't have emotions and sometimes that makes me very sad.

  7. #22
    Devorador de queso
    Points: 95,940, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Posting AwardCommunity AwardDiscussion EnderFrequent Poster
    Dason's Avatar
    Location
    Tampa, FL
    Posts
    12,937
    Thanks
    307
    Thanked 2,630 Times in 2,246 Posts

    Re: Hypothesis-test: difference between paired means

    Quote Originally Posted by rogojel View Post
    hi,
    that means that from the calculations POV it actually does not matter whether one specifies a constant as the null or a semi-infinite interval? this is really counter-intuitive, I would definitely expect different p-values.

    From a theoretical perspective there is a huge difference IMHO: Imagine I wanted to sell homeopathic medicine and set up a test to prove that my medicine works. I set up zhe test with the null being - my effect is greater or equal to zero. Then I ran the test, with the exact same calculations as if the null was : the effect is zero. I then find that I can not reject the null - under the constant null this means that I can not make the claim that my treatment works. Under the interval null the exactly same test means that my opponents can not make the claim that it does NOT work.

    what do you think?
    I think that you might be getting confused as to what we can say when we do hypothesis tests. If we fail to reject the null that doesn't mean we think the null is true. Plus we typically don't expect the opponents of homeopathic medicine to prove that it doesn't work - it's up to the supporters of homeopathic medicine to prove that it does work (good luck with that!).
    I don't have emotions and sometimes that makes me very sad.

  8. #23
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means

    Quote Originally Posted by Dason View Post
    I think that you might be getting confused as to what we can say when we do hypothesis tests. If we fail to reject the null that doesn't mean we think the null is true.
    Hi Dason,
    of course not. IMHO it means that we can not contradict the statement that it is false, so it might be true. For any quack this could be enough - so,ething like " the experts checked my statement and could not prove me wrong ".

    But my main point is that we can not have two contradictory conclusions for the same test, and this is what we seem to have here:

    - the test mechanics are exactly the same whether I am using the constant or the interval null hypothesis, and the p value is the same too.
    - for the constant null the conclusion will be that we can not prove that the method works
    - for the interval null (with the exact same data and calculation) the conclusion is that we cannot prove that the method does not work.

    This just can not be right I believe,

    regards
    rogojel

  9. #24
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means

    Quote Originally Posted by Dason View Post
    If I tell you that the true mean is less than or equal to 110 and then tell you that the sample mean is 120 - what would you 'best guess' be for the true mean? It would be 110 of course.
    With this logic if the sample mean is even slightly less then the limit, 109,99 say then the most likely value is the sample mean, and because I am using the most likely value to calculate the t, my t- value is always zero?

    This would leadt to a really simple test, no matter what my significance is, never reject the null if the sample mean is below or equal the limit. Does it really work like this?

  10. #25
    Points: 44, Level: 1
    Level completed: 88%, Points required for next Level: 6

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Hypothesis-test: difference between paired means

    Thanks for all your answer to this discussion! Unfortunately I didn't get the answer to my doubt below !!!

    Where theoretically (I think) I wrong?

    Thanks!

    Quote Originally Posted by solimyr View Post
    Dear All,
    I have a doubt about the theory of matched pair t-test. I have the difference between two means and I wanna do an hyphothesis-test analysis. I have the difference-mean, the difference-standard deviation and the degree of freedom.

    Letís start that I wanna say that the difference is equal to 0. So my null-hypotheses says that the difference is equal to 0. The alternative hypothesis says no. Itís a two tail problem. The significance level is set to 0.05 (alfa).

    So, happens that if P-value is greater than alfa then I cannot reject the null hypothesis. If P-value is less than alfa then I reject the null hypothesis. This is clear!

    Now, I wanna say that the difference is greater equal than D. The alternative hypothesis says that the difference is less than D (One tail problem). The significance level is set again to 0.05.

    So, happens like before. Looking at the P-value I can/cannot reject the null hypothesis.

    My doubts comes out now. I have made some tests and I see that for less value of D I get rejected the null hypothesis. Increasing D, Iím increasing the P-value. The fact is that if I continue to increase D for very big values, I will always be able to NOT reject the null hyphotesis! Doesnít exist a D value big enough to reject the null hypothesis!

    Iím guessing that I use the one problem tail just to find the lower/upper limit where I can not reject the null hypothesis! Is this correct or I wrote something wrong in my explanation?

    Thanks!
    Solimyr

  11. #26
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means

    hi solymir,
    yes, the discussion developed its own momentum . For your question, could you post some stats output, with your results?

    The way I now understand this to work, if your sample mean is less the the D the test should reject the null. Are you sure your null is NOT that the value is less then or equal D? In that case it will behave exactly as you describe it.


    regards
    rogojel

  12. #27
    Points: 44, Level: 1
    Level completed: 88%, Points required for next Level: 6

    Posts
    6
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: Hypothesis-test: difference between paired means

    Quote Originally Posted by rogojel View Post
    hi solymir,
    yes, the discussion developed its own momentum . For your question, could you post some stats output, with your results?

    The way I now understand this to work, if your sample mean is less the the D the test should reject the null. Are you sure your null is NOT that the value is less then or equal D? In that case it will behave exactly as you describe it.


    regards
    rogojel
    Yeah! Thank you! I will show 2 experiments.

    Let's start with a two tails problem. I have two experiments with these value:
    1. A => Mean = 91.133 ; (mean over 100 run)
    2. B => Mean = 91.753 ; (mean over 100 run)

    The difference is equal to 0.62 and the standard deviation of the difference values is equal to: 1.83.

    I will set my Ho (null hypothesis) equal to D=B-A=0 (two tail problem). I have followed the formulas on this website (link) to compute the t-value. I get that the t-value is equal to 3.38. The P-value, with 99 degrees of freedom, is equal to 0.001. I have set the significance level equal to 0.05 and so I have to reject the null hypothesis that the difference D is equal to 0.

    I computed also the confidence interval (CI) and I get that in the 95% of the cases, the difference D is between 0.25 and 0.98. Furthermore, if I keep the significance level 0f 0.05, just if I set the Ho equal to the difference D that fall within this interval I couldn't reject the null hypothesis. And If I set the D equal to the limits of the CI, I get a P-value equal to 0.05 that means that I couldn't reject the null hypothesis with a significance level of 0.05.

    I think, this it is clear. Please tell me if I write some wrong and/or state something wrong in term of theory.

    Now, let's move to the second case, or in other words, in one tail problem.

    Let's start with a two tails problem. I have two experiments with these value:
    1. A => Mean = 91.133 ; (mean over 100 run)
    2. B => Mean = 80.339 ; (mean over 100 run)

    The difference is equal to 10.79 and the standard deviation of the difference values is equal to: 2.42.

    I will set my Ho (null hypothesis) equal to D=A-B >=10 (two tail problem). Using the same formulas of before I compute the t-value. I get that the t-value is equal to 3.26. The P-value, with 99 degrees of freedom, is equal to 0.00007. Using again a significance level equal to 0.05, I have to reject the null hypothesis that the difference D is greater than 0.

    The CI in this case has just the lower limit that is equal to 10.39. And so, again, if I set D equal to the lower limit, I get the P-value equal to 0.05 and so, I couldn't reject that the Difference is greater than 10.39.

    Here comes my misunderstanding. If now I set D equal to 50, I get a P-value equal to 1 and so, I couldn't reject the null-hypothesis that state that the difference D is greater than 50. This obviously is wrong, also looking at the experiments, there aren't experiments with this difference but all are lower and so, to me, this seems wrong because there are experiments with a value less than 50.

    Maybe I have to see this in another way. I don't know! I'm pretty confused!!! Probably I state wrongly my phrase! I don't know!

    Thank You All!!!

  13. #28
    TS Contributor
    Points: 12,287, Level: 72
    Level completed: 60%, Points required for next Level: 163
    rogojel's Avatar
    Location
    I work in Europe, live in Hungary
    Posts
    1,471
    Thanks
    160
    Thanked 332 Times in 312 Posts

    Re: Hypothesis-test: difference between paired means


    hi solymir,
    can you give us the detailed calculation?

    to my mind it should look like like this:

    1. Sample difference 10,something
    2. Null hypothesis: difference >= 50 = D.
    3. t-value: (actual - D )/SE= -30'something -is this right?

    4. the p-value should be pretty much 0, not 1. Where is the difference?

    regards
    rogojel
    Last edited by rogojel; 07-13-2013 at 06:01 AM. Reason: clarifying

+ Reply to Thread
Page 2 of 2 FirstFirst 1 2

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats