Power Analysis and Sample Size determination help!

#1
Hi all!

I am new here so I hope my post is appropriate!

I have searched the web and this forum and haven't been able to figure out how exactly I need to perform a power analysis for sample size determination (although I admit I didn't scroll through ALL of the results to my search on this forum)...

I am working on an assignment for an upper year undergraduate psychology course for which I need to write a hypothetical research proposal.

I am trying to determine my sample size and understand that I will need perform perform a power analysis and a sample size analysis (a priori).

I would need to perform a 2 way ANOVA so I found this link explaining how to perform a power analysis for 2 way ANOVA in SAS...
https://support.sas.com/documentation/cdl/en/statug/63033/HTML/default/viewer.htm#statug_glmpower_a0000000155.htm

I am wondering what data I use to perform this power analysis. Do I select data from relevant literature on my research topic? What if the standard deviation is not reported? Do I create a set of hypothetical data?

Again I apologize if this post is not appropriate. I am not very strong in stats so this is probably a stupid question!

Thanks in advance for any help you may be able to provide!

-C
 

ted00

New Member
#2
I am wondering what data I use to perform this power analysis. Do I select data from relevant literature on my research topic? What if the standard deviation is not reported? Do I create a set of hypothetical data?
Hi
your post is completely appropriate, no need to apologize.
You're correct: you'd pull information from previous literature in your field and/or from experts to gain the values needed for a power/sample size analysis. Regarding whether you know the correct value to use for certain parameters, such as standard deviation, I'd say that in practice we rarely know exactly what the inputs should be, but only have to give our best educated guesses. There are online calculators available that will give you graphs of how sensitive the power and sample size figures are to the choices of standard deviation, etc. You might find one or more there helpful, not sure.
 
#3
@ted00 thank you so much for your response! It is greatly appreciated and very helpful.

I suppose it just seems strange to me that values that would be necessary to conduct power analyses such as standard deviation, in this case, are not reported in the literature.
What is common practice when these values aren't reported and the body of literature in the area is small, so there is no good way of estimating these values? Do you email the researchers and ask for the values? Again I am extremely naive so this might be a silly question!

@ted00 thank you also for the link... it looks extremely useful! Unfortunately I don't think I saw anything calculators for 2 way ANOVA :(

Thanks again!!!

-C
 

ted00

New Member
#4
true, no 2-way anova there yet, but should be in the future, not that that's much help to you at the moment

This is probably a good excercise your prof has you guys doing, b/c it seems you're running into some real-world issues
I've done power/sample size for my job, and I've seen it done different ways
A lot of times, if it's important enough, people will conduct pilot studies
I actually ran a pilot survey for my undergraduate thesis that turned out to be the final analysis b/c the population turned out to be very homogeneous
There are work projects I've been involved in where pilot studies are planned from the beginning for the very reasons you state -- not a lot of knowledge yet
And in those pilot studies, still there's the question of sample size
Best we can sometimes do is come up with the best guess we can and assess sensitivity to that guess
That's why having graphs like that can come in so handy, to me anyway

Edit: I'll add that you can get graphs illustrating sensitivity from SAS, too, I believe
 
#5
@ted00 great to hear that 2 way ANOVA calculators should be found on the website in future! I will keep checking back as hopefully I will have more power analyses to conduct in my career... (however this remains to be seen haha)..

Thank you so much again for your response... you have no idea how helpful this is! I would have been completely lost!

I will certainly account for piloting in my "research proposal" :)

Again thank you so much!!!!!!!!!!!!!!!!!!!!!
 

ted00

New Member
#6
it's no problem, best luck & I'm sure you'll do well on your project
Also, if you keep checking back here someone might be able to give insight on doing it in SAS if that's what you're looking for