+ Reply to Thread
Results 1 to 6 of 6

Thread: multiple proportions from one sample- sig testing

  1. #1
    Points: 2,176, Level: 28
    Level completed: 18%, Points required for next Level: 124

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    multiple proportions from one sample- sig testing




    Greetings all,
    I have a question about analysing responses from a multiple choice question.
    A lot of what we do in our analysis is say which option is picked most often (and therefore ranked #1) out of a multiple choice question.
    E.g. which is your favourite colour?

    -red -40%
    -blue -35%
    -green – 25%

    Red ranks first. But if the margin of error is +/- 5 %, the difference between red and blue is not actually statistically significant, right?

    So my question is, what test do you use to show which proportions are significantly higher, aside from eyeballing it? One of my coworkers uses a z-test for proportions, but that is for 2 independent samples... (http://www.dimensionresearch.com/res...ors/ztest.html)
    But these aren’t independent samples, right? They’re the same sample!
    But it’s not like it’s a pair-wise design either....
    Chi – square also works to show that there is a statistical difference between the 3 options, but doesn’t show which ones are different: red and blue don’t have to be significantly different to get an overall significant difference.
    Right?
    This has actually been driving me nuts.
    Is a z-test for proportions legitemate? I've read through a bunch of textbooks and websites and can't find anything that deals with this particular problem.
    Thanks!
    -Tony
    Last edited by TonyW; 11-08-2010 at 11:38 AM.

  2. #2
    Points: 5,115, Level: 45
    Level completed: 83%, Points required for next Level: 35

    Location
    Bangalore
    Posts
    135
    Thanks
    3
    Thanked 30 Times in 29 Posts

    Re: multiple proportions from one sample- sig testing

    Quote Originally Posted by TonyW View Post
    Greetings all,
    I have a question about analysing responses from a multiple choice question.
    A lot of what we do in our analysis is say which option is picked most often (and therefore ranked #1) out of a multiple choice question.
    E.g. which is your favourite colour?

    -red -40%
    -blue -35%
    -green – 25%

    Red ranks first. But if the margin of error is +/- 5 %, the difference between red and blue is not actually statistically significant, right?

    So my question is, what test do you use to show which proportions are significantly higher, aside from eyeballing it? One of my coworkers uses a z-test for proportions, but that is for 2 independent samples... (http://www.dimensionresearch.com/res...ors/ztest.html)
    But these aren’t independent samples, right? They’re the same sample!
    But it’s not like it’s a pair-wise design either....
    Chi – square also works to show that there is a statistical difference between the 3 options, but doesn’t show which ones are different: red and blue don’t have to be significantly different to get an overall significant difference.
    Right?
    This has actually been driving me nuts.
    Is a z-test for proportions legitemate? I've read through a bunch of textbooks and websites and can't find anything that deals with this particular problem.
    Thanks!
    -Tony
    Did you get any solutions...i am looking solution for a similar problem!
    Let me know if you have figured it out..

  3. #3
    Points: 2,176, Level: 28
    Level completed: 18%, Points required for next Level: 124

    Posts
    3
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: multiple proportions from one sample- sig testing

    Hi, no, haven't heard anything... what, three years later?!
    how about you? any luck?

  4. #4
    TS Contributor
    Points: 22,410, Level: 93
    Level completed: 6%, Points required for next Level: 940

    Posts
    3,020
    Thanks
    12
    Thanked 565 Times in 537 Posts

    Re: multiple proportions from one sample- sig testing

    Actually you got a multinomial sample. So in such case you may also use normal approximation - of course in this time the two sample proportions are not independent anymore as there is a covariance term in the standard error:

    \hat{SE}[\hat{p}_1 - \hat{p}_2] = \sqrt{\frac {\hat{p}_1(1 - \hat{p}_1)} {n} + \frac {\hat{p}_2(1 - \hat{p}_2)} {n} - \frac {\hat{p}_1\hat{p}_2} {n}}

    Of course under the null hypothesis H_0: p_1 = p_2 you could also use the pooled estimate \frac {\hat{p}_1 + \hat{p}_2} {2} as before.

  5. #5
    Points: 3,311, Level: 35
    Level completed: 74%, Points required for next Level: 39

    Posts
    46
    Thanks
    3
    Thanked 5 Times in 5 Posts

    Re: multiple proportions from one sample- sig testing

    Sorry to bump such an old thread, but I got interested in a similar problem. After googling for hours I found an online calculator to compare percentages from one sample: http://www.quirks.com/resources/Calculator.aspx.
    I then spend another few hours trying to figure out how this online calculator calculates and finally found a formula for the variance that will give the same results as in the online calculator at: http://www.stat.wmich.edu/s160/book/node64.html
    I noticed the result will be different than the formula given by BGM. A question that arises is now, which one is correct?

  6. #6
    Points: 2, Level: 1
    Level completed: 3%, Points required for next Level: 48

    Posts
    1
    Thanks
    0
    Thanked 0 Times in 0 Posts

    Re: multiple proportions from one sample- sig testing


    I believe the equation you found is correct. BGM's is correct except the covariance wasn't added twice (it should have a 2 in it). This can be seen in the covariance matrix. Where it will be (for two variables x and b)

    |x^2|xb|
    |bx|b^2|

    xb and bx terms are the same and so would be equivalent to 2xb.

    ... if that makes sense.

    Though I do agree with BGM that if you were making a pooled estimate (based on H0: p1=p2) then you'd use the alternative for a pooled estimate.

+ Reply to Thread

           




Similar Threads

  1. testing for significant diffs between proportions
    By TonyW in forum Psychology Statistics
    Replies: 3
    Last Post: 03-26-2011, 04:03 AM
  2. Significance testing for proportions (NPS)
    By Kevin_ind in forum Statistical Research
    Replies: 2
    Last Post: 03-07-2011, 05:17 AM
  3. Replies: 5
    Last Post: 08-28-2010, 08:46 PM
  4. Sample Proportions and Such
    By twistedlogic149 in forum Statistics
    Replies: 0
    Last Post: 03-13-2009, 12:23 PM
  5. Hypothesis testing for population proportions
    By Rojo in forum Psychology Statistics
    Replies: 1
    Last Post: 01-14-2006, 11:52 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats