+ Reply to Thread
Page 1 of 2 1 2 LastLast
Results 1 to 15 of 16

Thread: G*Power Sample Size Calculation

  1. #1
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Question G*Power Sample Size Calculation




    Hi Everyone,

    First, let me say that I am thrilled to find this forum! It has been a while since I have taken Stats I and II, and I will be the first to admit that I struggled my way through it then. I have a project that is going to require some data analysis coming up, and I was wondering if anyone would be willing to look over my sample size calculation and tell me if I am on the right track? Here's what I have:

    Design: 2x3x2 Three-way factorial ANOVA
    Population: Unknown (est. is greater than 100,000). Single group.
    Confidence Interval: 95%
    Effect Size: Small or Medium

    I am using G*Power 3.1.3, and the information below is what I have entered in the fields. I have colored the areas that I am really not certain about, but if you see anything else that looks out of place, please let me know.

    Test Family: F Tests
    Statistical Test: ANOVA, Fixed effects, special, main effects, and interactions
    Type of Power Analysis: A priori

    Effect size: .10
    error prob: .05
    Power (1-b error prob): .8
    Degrees Freedom: 2
    Number of groups: 12

    Thanks,

    Lady M

  2. #2
    R must die
    Points: 24,194, Level: 94
    Level completed: 85%, Points required for next Level: 156
    Awards:
    Activity Award
    noetsi's Avatar
    Posts
    4,425
    Thanks
    265
    Thanked 704 Times in 676 Posts

    Re: G*Power Sample Size Calculation

    Are you doing a fixed effect ANOVA or is your analysis using random effects? I did not see that in your comments (although most use fixed effects). Remember if you are using a priori this is supposed to occur before you have seen the data (a point I consider artificial but the field does not). Is it reasonable that the effect size is .1 (I don't know why you decided on this value).
    "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise." John Tukey

  3. The Following User Says Thank You to noetsi For This Useful Post:

    LadyM (07-12-2012)

  4. #3
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: G*Power Sample Size Calculation

    Hi,

    Thanks for the response

    This is social science research (survey method) and I am using the variables of introvert/extrovert (2 levels), motivation category (3 levels), and gender (2 levels) on time. Thus, I would think they would all fall in the category of "fixed."

    I have not seen the data yet. I know I will need to do a post test once it is collected. I chose .1 for the effect size because I thought that a smaller effect was better, but I need to calculate the sample size correctly before I can see if that will even be feasible. If its not, then I am most likely looking at a .25 for a medium effect, correct? I think that medium is the standard. I also noticed that the power defaults to .95, I wonder if I should use that rather than a .8? Any thoughts?
    Last edited by LadyM; 07-12-2012 at 05:10 PM.

  5. #4
    Phineas Packard
    Points: 9,305, Level: 64
    Level completed: 85%, Points required for next Level: 45
    Lazar's Avatar
    Location
    Sydney
    Posts
    903
    Thanks
    148
    Thanked 245 Times in 222 Posts

    Re: G*Power Sample Size Calculation

    Is not personality and motivation typically measured using continuous scales? If you have taken a continuous scale and made it into a categorical one this can have a range of negative implications for your analysis.

  6. The Following User Says Thank You to Lazar For This Useful Post:

    LadyM (07-12-2012)

  7. #5
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: G*Power Sample Size Calculation

    Quote Originally Posted by Lazar View Post
    Is not personality and motivation typically measured using continuous scales? If you have taken a continuous scale and made it into a categorical one this can have a range of negative implications for your analysis.
    In this particular survey, the personality variable will be measured using 1 self-report item. As far as the motivation piece, the empirical tool that I am using defines a primary motivation (there are 3 types) based upon a point system. I feel that this is best described by the author of the measure. According to Yee (2007), "Because standardized scores and effect sizes (based on continuous variables) are less-interpretable than percentages (based on categorical variables), a different way of understanding this data is presented here. The “primary motivation” for each player was inferred from their scores. A respondent was assigned a primary motivation if there was no close secondary motivation (primary * .75 > secondary). 57% of players were assigned a primary motivation based on this criteria. This is a somewhat lax criteria but serves the purpose of providing an easier interpretation of the data".

    If I may ask, what types of implications do you think that I could be looking at here?

  8. #6
    Phineas Packard
    Points: 9,305, Level: 64
    Level completed: 85%, Points required for next Level: 45
    Lazar's Avatar
    Location
    Sydney
    Posts
    903
    Thanks
    148
    Thanked 245 Times in 222 Posts

    Re: G*Power Sample Size Calculation

    Loss of power and bias and the two main ones. Is the personality item from one of the 5 item big-five measures? Is so, is it not on a five point scale?

    EDIT:

    This puts it nicely

    The fact that some people murder doesn't mean we should copy them. And murdering data, though not as serious, should
    also be avoided.
    -- Frank E. Harrell (answering a question on categorization of continuous variables in survival modelling)
    R-help (July 2005)

  9. #7
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: G*Power Sample Size Calculation

    No, it is a self-classification based on a description from existing studies. There is no scale on that item, its either introvert or the extrovert. How does the continuous vs. categorical data impact sample size?

  10. #8
    Phineas Packard
    Points: 9,305, Level: 64
    Level completed: 85%, Points required for next Level: 45
    Lazar's Avatar
    Location
    Sydney
    Posts
    903
    Thanks
    148
    Thanked 245 Times in 222 Posts

    Re: G*Power Sample Size Calculation

    Not sure I know what you mean by "how does it impact sample size". Do you mean how does it impact power?

  11. #9
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: G*Power Sample Size Calculation

    Yep, that is what I am wondering. Would I need to make additional adjustments to my sample size calculation to increase the power somehow?

    Here is what I am looking at right now as my preliminary sample size calculation:

    Effect size: .25 (I think you are correct that .1 may not be feasible)
    error prob: .05 (Standard)
    Power (1-b error prob): .95 (The higher the power, the better right?)
    Degrees Freedom: 2 (calculated by (2-1) * (3-1)* (2-1) =2)
    Number of groups: 12 (calculated by 2x3x2)

    I put this into G*Power and came up with: Total sample size 251

    Does that sound about right?

  12. #10
    R must die
    Points: 24,194, Level: 94
    Level completed: 85%, Points required for next Level: 156
    Awards:
    Activity Award
    noetsi's Avatar
    Posts
    4,425
    Thanks
    265
    Thanked 704 Times in 676 Posts

    Re: G*Power Sample Size Calculation

    This is social science research (survey method) and I am using the variables of introvert/extrovert (2 levels), motivation category (3 levels), and gender (2 levels) on time. Thus, I would think they would all fall in the category of "fixed."
    Fixed means, commonly, that you are using all possible levels of a variable and that you are only interested in the specific levels you measure. Random effects involve sampling a group of levels from a wider subset of effects - when what you are really interested in is the larger subset not just what you sampled. I would guess you are doing fixed from your description.

    I have not seen the data yet. I know I will need to do a post test once it is collected. I chose .1 for the effect size because I thought that a smaller effect was better, but I need to calculate the sample size correctly before I can see if that will even be feasible. If its not, then I am most likely looking at a .25 for a medium effect, correct? I think that medium is the standard. I also noticed that the power defaults to .95, I wonder if I should use that rather than a .8? Any thoughts?
    A priori has more power, you don't have to do post hoc test if you do a apriori calculation (although commonly this is ignored). You should base your effect size on what you think actually exists in the population. If you don't know that using a conservative number is your best alternative. Is there any literature such as a meta analysis or a collegue's research to suggest what the effect size is?

    Each specialization has its own rules for what a small, medium, and large effect size is. Cohen's suggestions are commonly utilized.
    "Far better an approximate answer to the right question, which is often vague, than an exact answer to the wrong question, which can always be made precise." John Tukey

  13. #11
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: G*Power Sample Size Calculation

    Quote Originally Posted by noetsi View Post
    A priori has more power, you don't have to do post hoc test if you do a apriori calculation (although commonly this is ignored).
    That is good to know!

    I'm a little boggled by the effect sizes. I have not been able to locate a meta-analysis, so I am going to have to go with Cohen's d. However, my textbook says .2 is small, .5 is medium, and .8 is large. G*Power says .1 is small, .25 is medium, .5 is large. When I google Cohen's d, I get a table that says .1 is small, .30 is medium, and .5 is large. Now that is just plain confusing lol. Is this what you meant by differences in specialization?

  14. #12
    King of all Drama
    Points: 11,484, Level: 70
    Level completed: 59%, Points required for next Level: 166
    spunky's Avatar
    Location
    vancouver, canada
    Posts
    1,533
    Thanks
    97
    Thanked 344 Times in 278 Posts

    Re: G*Power Sample Size Calculation

    Quote Originally Posted by LadyM View Post
    Is this what you meant by differences in specialization?
    the problem with effect sizes is that what Cohen intended to act as a general guideline and a mere set of suggestions became dogma to many people and now it gets thrown around as if it were the word of God ( much like the whole p < .05 thing). for instance, in areas like mine where a lot of research is correlational, we're quite happy with an effect size of 0.5. if we find something around the 0.8 range we even get suspicious because it's simply too high... or we ignore it. it's important that one knows his/her area enough to know which effect sizes are in which range... or, to be honest, if at some point this becomes too much of a hassle just take the Cohen (1988) citation and run with it. it's such a well-known reference and so many people use it that you just can't be all that wrong by using it as well...
    for all your psychometric needs! https://psychometroscar.wordpress.com/

  15. #13
    Points: 83, Level: 1
    Level completed: 66%, Points required for next Level: 17

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: G*Power Sample Size Calculation

    Quote Originally Posted by spunky View Post
    it's important that one knows his/her area enough to know which effect sizes are in which range... or, to be honest, if at some point this becomes too much of a hassle just take the Cohen (1988) citation and run with it. it's such a well-known reference and so many people use it that you just can't be all that wrong by using it as well...
    That sounds like a good idea! Since G*power seems to be the most conservative estimate, I am going to compare that with Cohen (1988) and see how it stacks up. Thanks!

    *Update* I looked it up and what I found when I googled (Wikipedia) was identical to what Cohen (1988) reports. I'm not a fan of Wiki at all, but perhaps it is useful sometimes!
    Last edited by LadyM; 07-14-2012 at 12:47 PM.

  16. #14
    TS Contributor
    Points: 9,832, Level: 66
    Level completed: 46%, Points required for next Level: 218
    Karabiner's Avatar
    Location
    Schalke 04, Germany
    Posts
    1,496
    Thanks
    27
    Thanked 372 Times in 354 Posts

    Re: G*Power Sample Size Calculation

    I'm a little boggled by the effect sizes. I have not been able to locate a meta-analysis, so I am going to have to go with Cohen's d. However, my textbook says .2 is small, .5 is medium, and .8 is large. G*Power says .1 is small, .25 is medium, .5 is large.
    That's not for d, I suppose. Perhaps for f?
    When I google Cohen's d, I get a table that says .1 is small, .30 is medium, and .5 is large.
    Definitely not. That's for r, not for d.

    Kind regards

    K.

  17. #15
    Beep
    Points: 60,970, Level: 100
    Level completed: 0%, Points required for next Level: 0
    Awards:
    Discussion EnderPosting AwardCommunity AwardMaster TaggerFrequent Poster
    Dason's Avatar
    Location
    Ames, IA
    Posts
    11,038
    Thanks
    260
    Thanked 2,134 Times in 1,815 Posts

    Re: G*Power Sample Size Calculation


    Quote Originally Posted by Karabiner View Post
    That's not for d, I suppose. Perhaps for f?

    Definitely not. That's for r, not for d.

    Kind regards

    K.
    And of course it depends on the discipline you're in as well...
    Morte a tutti i raptors
    001100010010011110100001101101110011

+ Reply to Thread
Page 1 of 2 1 2 LastLast

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats