Power Analysis for Factorial ANOVA Design?

[firefly2442]

Hello,

I have the following experimental design:

Dependent Variable: time taken for task
Independent Variables: task difficulty(2), device configuration(5)

Participants are randomly assigned to one task difficulty (easy or hard). They then go through all 5 device configurations (which are randomized for order) and the time taken for each task is recorded. In addition, for each device configuration, they do that task twice in a row.

I have pilot data from 8 participants. I'm trying to do a power analysis and figure out roughly how many participants I will need for the full study. I'm trying to use this software:

http://www.stat.uiowa.edu/~rlenth/Power/index.html

Based on my design, I think it should be a nested factorial design (ANOVA). From there, I guess I can put in the standard deviations and wanted power level to identify how many participants will be needed.

Does this sound like I am using the correct ANOVA in the tool?

Does anyone know what the detectable contrast value that can be set is in the tool?

Am I on the right track?

Thanks in advance.

 

[jeromyanglim]

I would describe your design as a 2 by 5 mixed ANOVA with 2 levels of the between subjects factor (task) and 5 levels of the repeated measures factor (device configuration).

I'd recommend checking out G Power 3. It supports many different statistical models, several different kinds of power analysis, and assists with effect size calculation.

An important point to note is that power is not a property of your study. In your study you will have multiple hypotheses. At the very least you will have the main effect of task, the main effect of device configuration, and the interaction effect. Each of these may have different statistical power.
This issue is described more in a very similar question on power analysis in mixed anova available here.

 

[firefly2442]

Thank you, that is very helpful. In GPower3, I am uncertain about what to use for the effect size. Some websites suggest picking values based on "small", "medium", and "large" differences. However, this seems quite arbitrary. In GPower3, there is a sidebar tool for calculating (estimating?) the effect size. There appear to be multiple options for calculating effect size, one using the variances, one using the standard deviation and means, and one using partial n^2 ( partial eta-squared). Most material seem to suggest eta-squared as a standard way of determining effect size, however, there appears to be confusion on not only eta-squared vs. partial eta-squared but also the equations that are used:

https://www.msu.edu/~levinet/eta squared hcr.pdf

In addition, GPower3 lists different options for how effect size is calculated (estimated?) based on the test picked.

Repeated Measures Between Factors ANOVA:
-effect size from means
-effect size from variances
-direct using partial eta-squared

Repeated Measures Within Factors ANOVA:
-effect size from variances
-direct using partial eta-squared

Repeated Measures Within-Between Interactions ANOVA:
-effect size from variances
-direct using partial eta-squared


Does anyone have suggestions on what to do in this case? Thank you in advance.