# Thread: A simple case that doesn't seem that simple

1. ## A simple case that doesn't seem that simple

I've been trying to find the best solution for my experiment. I don't like that much what my colleagues are using for the statistic test here. So I would really appreciate some help.

Case: Let's assume this, in vitro assays cells, with 3 groups (a control group, a group with "drug A" added, a group with "drug B" added). I want to see if there is an effect by using either drug. They are unrelated and I'm NOT looking for combined effects.

I do the experiment once, let's say with triplicates, so I get a bar graph, 3 bars, n=3 each. So far so good. So to analyze this, some do t-tests (control vs A and control vs B) but some do ANOVA. Since I don't want to compare A with B, but both with the control should I really do the ANOVA or just stick with 2 t-tests and see if any has an effect?
Ok, this was point number one.

Now point number two:

I repeat the experiment 2 more times. So I'll have the same thing as before but now 3 graphs. My major problem is: how do I analyze the 3 assays as a whole? Do I just analyze each case independently and decide if there's an effect? Or is there a correct way to pool the 3 assays to get one final graph and one final statistical analyzes?
Simply adding all the results from each group and calculating the mean seems just wrong, as well as grouping all the controll data, assume 100% and calculate percentage. But how exactly can I solve this?

Thank you so much in advance.

2. ## Re: A simple case that doesn't seem that simple

Can someone give me a hand, please?

Thank you.

3. ## Re: A simple case that doesn't seem that simple

For the first part you can do Dunnett's test for comparison of several treatments to control. Most software can do that.

Now you have already done the experiments but there is a “suggestion” to use more observations in the control, since every treatment is compared to the control. If the number of treatments+control is k then you should use sqrt(k) times more observations in the control experiment as compared to in each treatment. If the number of treatments and control are 9 then you should have sqrt(9) = 3 times more units in the control group. In this case k=3 so with 3 observations in each treatment group you could be advised to use 3*sqrt(3) or about 5 observations in the control group.

(The remaining point I leave to the other visitors here ....)

4. ## Re: A simple case that doesn't seem that simple

Thank you for your help on my topic.

There's still something I'd like to ask. With dunnet's multiple comparison test I can compare each treatment with the control but I still have an issue here. Doing the ANOVA it tells me the means are different, but if I use t-test the test each treatment only one is significantly different from the control, if I do the ANOVA + Dunnet it tells me that both are significantly different from the control.

So a question that remains is: what is the correct way?

1- Should I just do t-tests and that's it?
2- Do the ANOVA and then the t-tests?
3- Do the ANOVA and then Dunnet comparison?

(2 and 3 assuming ANOVA shows means are different).

In this example case that I just told you I get:
ANOVA summary
F 11,05
P value 0,0023
P value summary **
Are differences among means statistically significant? (P < 0.05) Yes
R square 0,6676

Dunnett's multiple comparisons test Mean Diff, 95% CI of diff, Significant? Summary

Control vs. 1 1,027e+009 8,779e+007 to 1,966e+009 Yes *
Control vs. 2 1,628e+009 7,424e+008 to 2,513e+009 Yes **

Unpaired t test (Control vs 1)
P value 0,0621
P value summary ns
Significantly different? (P < 0.05) No
One- or two-tailed P value? Two-tailed
t, df t=2,218 df=7

Unpaired t test (Control vs 2)
P value 0,0026
P value summary **
Significantly different? (P < 0.05) Yes
One- or two-tailed P value? Two-tailed
t, df t=4,315 df=8

5. ## Re: A simple case that doesn't seem that simple

I would in general prefer the Dunnett's test.

Maybe this example is a rare situation were gaining a few more degrees of freedom (df) actually matters. With t-test and df=7 give a p-value 6%. With the Dunnett's test the p-value becomes less than 5% and thus significant. I believe that by using the Dunnett procedure the pooled standard deviation will be based on a larger df, say 7+8+7. That will give a lower critical value.

Often people say: “By increasing the number of observations I gain df”. Well, that's true but the important thing is that the “n-value” increases. Look at this formula: t * s/sqrt(n)
By increasing the sample size the critical t-value come close to 2 even for quite small df-numbers. But the important thing is that the sqrt(n) increases so that precision improves.

In the above example it seems like by pooling the standard deviation in the Dunnett's procedure just enough df is gained to make it statistically significant.

But I would still be a little bit sceptical since the high values for the means (1,027*10^(+009)) could suggest to try a log-transformation to maybe stabilize variances.

And also, how many actual observations were there really?

Originally Posted by srd
I do the experiment once, let's say with triplicates, so I get a bar graph, 3 bars, n=3 each.

6. ## Re: A simple case that doesn't seem that simple

Per condition, 3 observations. For each observation I actually count the cells twice (from the same source solution) and average the value to get the "observation", so 3 values.

My mind is till stuck in the first problem. Why use a ANOVA and not 2x t-test, if the 2 test conditions have nothing to do with each other and all that matters is comparing each with the control condition?

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts