Does anyone know how to calculate the sample size of my serial mediation model with Monte Carlo Power Analysis?

#1
Does anyone know how to calculate the sample size of my serial mediation model with Monte Carlo Power Analysis? let's say I want to achieve a medium effect. I am using a tool called "Monte Carlo Power Analysis for Indirect Effects, but I am not quite sure how do I read the research result, such as LL, UL, power. Here I also attached the research result as well as the setting that I am using .
Monte Carlo Tool: https://schoemanna.shinyapps.io/mc_power_med/
 

Attachments

hlsmith

Less is more. Stay pure. Stay poor.
#2
Thanks for sharing. This is not may area - interesting looking tool - I only looked at the image. It doesn't look like you put any corr values into the tool? Not looking at the documentation, definitions perhaps may be:
LL = lower limit; power = power; UL = upper limit. So it could be generating power estimate per sample size and providing precision bounds around power based on the simulations.
 
#3
Thanks . you are very helpful.
Oh, corr values, you meant effect size, right? but as you can see, there are many blanks provided below, do I have to input effect size to all the blanks below. Let's say I want to achieve a small effect size , like 0.2 in this model, do I have to input 0.2 in all the blank that provide below ? and then type calculate power? and how about the standard deviation? some people told me I should set my SD as 90 with 500 samples, I am not so sure if that is true. but I set it as 90, and tried to run the model again, and I posted my result here too, do you think my calculation is ok? according to the result, 450 samples are required if I need to achieve 0.81 power with my model? I understand you don't use this software, but perhaps you may know something about that I guess?
 

Attachments

spunky

Can't make spagetti
#4
Thanks . you are very helpful.
Oh, corr values, you meant effect size, right? but as you can see, there are many blanks provided below, do I have to input effect size to all the blanks below. Let's say I want to achieve a small effect size , like 0.2 in this model, do I have to input 0.2 in all the blank that provide below ? and then type calculate power?
Are your effect sizes of interest the inter-variable correlations? Or the indirect effect coefficients? If you are interested only in the power to detect correlations of 0.2 you are doing it correctly. If you are interested in the indirect effect path coefficient as an effect size, then that is not correct.


and how about the standard deviation? some people told me I should set my SD as 90 with 500 samples, I am not so sure if that is true. but I set it as 90, and tried to run the model again, and I posted my result here too, do you think my calculation is ok? according to the result, 450 samples are required if I need to achieve 0.81 power with my model? I understand you don't use this software, but perhaps you may know something about that I guess?
What about them? Mediation models come form the analysis of covariance structures. What that app is trying to do is establish the covariance structure that would result in your model of interest. You'll have to figure those on your own (previous research, informed guess, etc.)

Why would you set your SD=90? What is the rationale behind it?

Your calculations may or may not be OK depending on what you are trying to do. But at first glance it does not seem to me that you are doing what you think you need.
 
#5
Are your effect sizes of interest the inter-variable correlations? Or the indirect effect coefficients? If you are interested only in the power to detect correlations of 0.2 you are doing it correctly. If you are interested in the indirect effect path coefficient as an effect size, then that is not correct.




What about them? Mediation models come form the analysis of covariance structures. What that app is trying to do is establish the covariance structure that would result in your model of interest. You'll have to figure those on your own (previous research, informed guess, etc.)

Why would you set your SD=90? What is the rationale behind it?

Your calculations may or may not be OK depending on what you are trying to do. But at first glance it does not seem to me that you are doing what you think you need.

Oh, I forgot to tell you my hypotheses. My research contains 5 hypotheses. Given my hypotheses showed on below, may I know if I should choose the inter-variable correlations or the indirect effect coefficients? and how could I calculate my sample size of my research with this app? Sorry to bother you again, many many thanks!

H1-X has a positive relationship to Y
H1a-: X has a positive relationship to M1

H1b-: M1 has a positive relationship to M2
H1c-: M2 has a positive relationship to Y


H2: M1 and M2 would mediate the relationship between X and Y
 

spunky

Can't make spagetti
#6
Oh, I forgot to tell you my hypotheses. My research contains 5 hypotheses. Given my hypotheses showed on below, may I know if I should choose the inter-variable correlations or the indirect effect coefficients? and how could I calculate my sample size of my research with this app? Sorry to bother you again, many many thanks!

H1-X has a positive relationship to Y
H1a-: X has a positive relationship to M1
H1b-: M1 has a positive relationship to M2
H1c-: M2 has a positive relationship to Y


H2: M1 and M2 would mediate the relationship between X and Y
This is a good start, but there is not enough information here to do a power analysis. When you say:

H1-X has a positive relationship to Y you also need to specify how you expect X and Y to be correlated. You have to give the app a correlation for every pair of variables in your model and you have to give it the standard deviations. So you need to decide on which numbers to plug into the app.

Once you have decided which correlations and standard deviations to plug in, the app will do the rest because you've already established a mediation model (this seems like an R-based implementation of the PROCESS SPSS macro). So at least you won't have to calculate the indirect effects by hand, lol.
 
#7
This is a good start, but there is not enough information here to do a power analysis. When you say:

H1-X has a positive relationship to Y you also need to specify how you expect X and Y to be correlated. You have to give the app a correlation for every pair of variables in your model and you have to give it the standard deviations. So you need to decide on which numbers to plug into the app.

Once you have decided which correlations and standard deviations to plug in, the app will do the rest because you've already established a mediation model (this seems like an R-based implementation of the PROCESS SPSS macro). So at least you won't have to calculate the indirect effects by hand, lol.
Many thanks again! May I know how to look for effect size and SD? Do I look for them from the previous related research ? but I meant this is a new research, I meant so far I only know the effect size and SD for two variables based on the previous literature. Since this is a new research, what should I do if I couldn't manage to find the rest of them ? can I use other ways to look for effect size and SD apart from looking for previous literatures?
 

spunky

Can't make spagetti
#8
Many thanks again! May I know how to look for effect size and SD? Do I look for them from the previous related research ? but I meant this is a new research, I meant so far I only know the effect size and SD for two variables based on the previous literature. Since this is a new research, what should I do if I couldn't manage to find the rest of them ? can I use other ways to look for effect size and SD apart from looking for previous literatures?
Previous research is your only answer. This is not something you can calculate, you need to take it from somewhere and you need to be able to justify it.
 
#9
Previous research is your only answer. This is not something you can calculate, you need to take it from somewhere and you need to be able to justify it.
Many thanks , you have been super helpful !! Could I just ask you one more questions? It seems that the model on the app can only include serial mediating model with two mediators in it. As you can see in the attached photos, what if I add one more variable (Z) to form the model with two more hypotheses (H3 and H4) ? Could I still calculate the sample size with this app? Many many thank!

H3: Y would mediate the relationship between X and Z
H4: M1, M2 and Y would mediate the relationship between X and Z
 

Attachments

spunky

Can't make spagetti
#10
Many thanks , you have been super helpful !! Could I just ask you one more questions? It seems that the model on the app can only include serial mediating model with two mediators in it. As you can see in the attached photos, what if I add one more variable (Z) to form the model with two more hypotheses (H3 and H4) ? Could I still calculate the sample size with this app? Many many thank!

H3: Y would mediate the relationship between X and Z
H4: M1, M2 and Y would mediate the relationship between X and Z
Well, does the app support this model? What the app is doing is taking mediation models from the (long) list of models from the PROCESS macro http://www.processmacro.org/index.html

If you have the additional info you need (covariances and standard deviations) as well as that model in the drop-down menu, sure. It should work.
 
#11
Well, does the app support this model? What the app is doing is taking mediation models from the (long) list of models from the PROCESS macro http://www.processmacro.org/index.html

If you have the additional info you need (covariances and standard deviations) as well as that model in the drop-down menu, sure. It should work.
thanks again, but I am not quite sure what you meant. I meant , for example, only Two Serial Mediators model can be chosen in the app (https://schoemanna.shinyapps.io/mc_power_med/) as you can see in the attached photo, but H4 (M1, M2 and Y would mediate the relationship between X and Z) includes three mediators (M1, M2 and Y). In this case, how do I estimate the sample size? or do I have to use other apps to do that ?
 

Attachments

#13
Oh yes, yes. I can see that. I didn't know the app so I didn't know they only offer a limited number of models.

Then you're out of luck. You'll probably have to approximate power by specifying your model a path analysis model and conduct a Monte Carlo simulation. Here's something to get you started:

https://www.tandfonline.com/doi/abs/10.1207/S15328007SEM0904_8
But may I know how to figure out the effect size for my study ? people have been telling me I should refer to previous literature to look for effect size? but the problem is that if it is a new study, how can I refer to effect sizes from previous study? For example, if I want to know the relationship between education level (IV) and student stress ( DV), do I need to look for previous studies on stress only , and from which I get the effect size? or do I need to look for the studies with same IV and DV of my study to get the effect size? but the studies with same IV and DV do not exist, because every study is suppose to be new and innovative, right?
 

spunky

Can't make spagetti
#14
But may I know how to figure out the effect size for my study ? people have been telling me I should refer to previous literature to look for effect size? but the problem is that if it is a new study, how can I refer to effect sizes from previous study? For example, if I want to know the relationship between education level (IV) and student stress ( DV), do I need to look for previous studies on stress only , and from which I get the effect size? or do I need to look for the studies with same IV and DV of my study to get the effect size? but the studies with same IV and DV do not exist, because every study is suppose to be new and innovative, right?
People tell you you should be looking at the literature because nobody is ever working in something so new as to claim there is no previous literature on it. Sure, a few details about your study will be new to make it interesting, but unless you completely invented a new scientific field on your own, there is always previous literature. Look for studies that look like yours even if they don't use the same variables or the same populations.
 
#15
People tell you you should be looking at the literature because nobody is ever working in something so new as to claim there is no previous literature on it. Sure, a few details about your study will be new to make it interesting, but unless you completely invented a new scientific field on your own, there is always previous literature. Look for studies that look like yours even if they don't use the same variables or the same populations.
Thanks, you are being super helpful!