Interpreting a moderation effect

[Mariana Ramos]

Yesterday I submitted my firts thread but I learnt that it was too vague. I did my work but now I have some doubts interpreting a moderation effect.

Based on what I' ve learnt in my readings, I've developed a regression model, with the main VIs (VI1 and VI2) and the interaction term (Vi1 * Vi12). I have centered all the VIs (all continuous).

The final results are:
R2: 0.106 (adjusted R2=0.074)
Anova F 3.320 (p=0.024)

B (VI1) = -0.098 (p=0.080)
B VI2 = 0.090 (p=0.211)
B VI1*VI2 = -0.102 (p=0.052)

I would like to know if we can consider this result as a moderation effect. I know that the significance is quite borderline (0.052); but, on the other hand, there is quite a difference when we consider the interaction term.
Can anyone help interpreting this moderation effect?

 

[mandy_a41]

Interpreting Moderator effects in Multiple Regression

Hi Mariana,

The best way to interpret interaction effects in moderated multiple regression is to plot them. Keep in mind that this result is not *statistically significant* therefore should be interpreted with caution.

The method to plot an interaction is described in detail in the reference given at the end of this post and will provide you (or anyone else) with more in-depth information than I will provide here.

I suggest excel for this as other statistical packages seem to have problems and are not as intuitive.

You need to calculate three values for the IV that you want to act as the moderator (IV2). Deciding on cut-points can be relatively arbitrary or if you have meaningful points within your data I suggest you use those. The common method to decide on cut-points is 1SD above the mean, the mean and 1SD below the mean. Alternatively you can use a low value, the mean value and a high value. Note that the mean value will be 0 as you are using centered values.

Your excel spreadsheet should then be set up to have all possible X values (or at least the X values that you are interested in) in one column that you will refer to:

Predicted Y values
X Zlow Zmean Zhigh
1
2
3
4
5

The predicted Y values are given by using the following formula:
[B1+(B3*Z)]*X + [(B2*Z) + B0]

Where:
B0 = the constant (or intercept)
B1 = the unstandardised B coefficient associated with IV1
B2 = the unstandardised B coefficient associated with IV2
B3 = the unstandardised B coefficient associated with IV1*IV2
Z = your chosen value for the moderator

Once you have finished all you need to do is graph the information!!!


Cohen, J., Cohen, P., West, S., & Aiken, L. (2003). Applied multiple regression/correlation analysis for the behavioural sciences. Mahwah, NJ: Lawrence Erlbaum Associates Inc.

 

[CB]

The final results are:
R2: 0.106 (adjusted R2=0.074)
Anova F 3.320 (p=0.024)

B (VI1) = -0.098 (p=0.080)
B VI2 = 0.090 (p=0.211)
B VI1*VI2 = -0.102 (p=0.052)

I would like to know if we can consider this result as a moderation effect. I know that the significance is quite borderline (0.052); but, on the other hand, there is quite a difference when we consider the interaction term.
Can anyone help interpreting this moderation effect?

I think it was Jacob Cohen who asked whether God truly loves the 0.05 greater than the 0.06 Remember that alpha cutoffs are somewhat arbitrary. We all follow along and use 0.05 because everyone else does, but a p value of 0.052 is not somehow vastly superior to one of 0.048, say. Regarding statistical significance as black or white is a srsly bad idea. In this case, I'd say the interaction effect is worth interpreting - point out the p value, sure, but don't ignore it just because it's a tiny smidgen over 0.05. Hell, if you use one fewer decimal point it WILL be 0.05!

I guess my one question would be sample size - if your sample size is smallish, the p value being over 0.05 could be the result of insufficient statistical power rather than the effect not being strong enough (which you could also discuss).

 

[Shi_Lei]

I always round the p value on the first decimal. You still have a very high likelihood that there is something systematical going on.
BTW, do you have a directed hypothesis for the interaction term? If yes, a cut-off at p<=0.1 (one-tailed test) would even be appropriate.