ANOVA on three columns?

[nik]

Hi all! I am new to this forum so hopefully this will be a success!

I have some data in excel from 3 experiments that are run independent from each other (they do not influence one another). Each experiment has 10 runs each. All the data across the three sets of experiments (i.e. 30 runs) have the following structure:

Time(sec), value1
0.5, 0.34
12.5, 0.39
15.30, 0.65

The time column is in seconds and the value1 column contains values from a range -1 to 1. What I would like to do is carry out a proper statistical significance test comparing the end-values from each set of experiments. So for example, from experiment 1 I will have 10 end values and the same from experiment 2 and 3.

I expect the first two sets of 10 numbers will be from around the same distribution (i.e., no significant difference) and that there will be a significant difference between their
distributions and the distribution from the values of experiment 3.

I was thinking to use an ANOVA test as it allows multiple columns to be considered. So I would:
1. State the Research Hypothesis
2. State the Null Hypothesis
3. Select a probability of error level (alpha value of 0.05 for example)
4. Select and compute the test for statistical significance

Could you help me out by describing whether this is appropriate?

 

[obh]

Hi Nik,

You mention the time. Are the observations independent?
Does the result of 10 seconds depend on the 9 seconds result?

 

[nik]

Hi!

So I have 3 groups. In each group I have 10 values, each from an independent experiment. so that value within its own 'experiment run' is the result of all other values over time only in that run.

Let me show you what I have done:



In this case, there are 30 vals in each group, not 10. But the principle is the same. I have applied ANOVA to the groups to investigate whether there is any significant difference in the average value per each group - and it turns out that there is (as I expected). Then, I used a Post-hoc test with a bonferroni corrected alpha value to investigate which exact group carries this sigificant difference - and it turns out to be group 3.

My question is - is this a sound and methodological statistical process to investigate something like this?

 

[obh]

Hi Nik,

If it meets the ANOVA assumptions it should be okay.
But the Bonferroni correction is too conservative.
It is usually better to do the Tukey HSD test.

 

[nik]

Hi obh,

What do you mean by 'conservative'?

And thank you so very much!

 

[obh]

It reduce the alpha too much. Hence you may not reject incorrect h0. Usually there is no pure independency between the columns.

 

[gw1]

Hey, I like the way the data has been presented. I notice there's lots of alpha and no mention of power (1 - beta); worth considering with alpha a-priori for sample size estimates, and post hoc to analyse closer differences like between groups one and two. i.e., the long-run likelihood of accepting the null between groups when the alternative is true.

 

[obh]

You should calculate the test power before running the ANOVA test, based on the effect size you want to identify.
Running Tukey HSD will not give different accepted/rejected results, in your specific case correct alpha is 0.00166 and Group2-Group3 p-value is 0.01558.
But if the p-value was 0.00169 you might not reject the H0 but reject it with Tukey HSD.

What statistical program did you use?
The following link will give you the test power and the Tukey HSD results, and the R code.

https://www.statskingdom.com/180Anova1way.html