choosing the right test to better answers the research questions

#1
Hello,

My research is about associative learning in worms, I train worms to associate two stimulus and measure their change in behavior
specifically, I expose the worms to chemicals called DA (attractive odor) or to DA together with HCl (aversive stimulus). the behavior test is called chemotaxis test, in which we measure the worms attraction toward DA, the assumption is that worms that where exposed to both DA and HCl will be less attracted to DA following the training, in this figure i'm testing different dilution of DA in the test (all 3 set of columns underwent the same training), and as you can see the smaller the dilution both DA and DA+HCl are Skewed toward more positive values of chemotaxis index (CI) because the DA concentration is higher - the attraction force is greater. my question is which analysis will better help me determine the best DA dilution in the test? me and my PI have difference of opinion regarding the analysis that best suited here. 1) two-way anova followed by multiple comparisons which to my understanding is the default analysis in this case. the problem is in my opinion is that one of the assumption of ANOVA is that distributions have the same variance, because anova create a sort of average variance which all data in compered against. I tested the variance using Levene's test and found p-value equals 0.180 so that means that technically I can assume equal variance. 2) perform two sample equal variance t-test between each pair, here the result is different as I get the smaller p-value for 1:100 dilution (probably because this dilution produce better homology in the data)
I think that even though ANOVA is the standard way to analyze similar data in this case it not right because it Skewed the p-value toward 1:1K dilution probably because this dilution gives the highest difference between the means (but not significantly) compared to the other dilutions, but in this specific case because I'm trying to determine the optimal dilution that will be most effective in the chemotaxis test, isn't it wrong to use anova that average the variance across dilutions and basically nullified the objective? won't a more right way will be to use series of unrelated t-test which will use each dilutions variance independently?

for your convenience I'm attaching the raw data and the two example of analysis
any insight will be most appreciated
thank you
Netanel
 

Attachments

Karabiner

TS Contributor
#2
I tested the variance using Levene's test and found p-value equals 0.180 so that means that technically I can assume equal variance.
Definetly not. You just weren't able to reject the null hypothesis of equal variance,
but the reason was not that the equal variance assumption was correct. Instedd,
your test had very poor power to detect unequal variances, due to very small
sample sizes.
2) perform two sample equal variance t-test between each pair, here the result is different as I get the smaller p-value for 1:100 dilution (probably because this dilution produce better homology in the data)
p-values are not indicators for effect sizes.
I'm trying to determine the optimal dilution that will be most effective in the chemotaxis test
I am not sure that I understand how you determine "most effective". Most effective = the largest
mean difference between conditions across the three dilutions?

With kind regards

Karabiner
 
#3
Thank you for your replay

regarding your last comment,
I am not sure that I understand how you determine "most effective". Most effective = the largest
mean difference between conditions across the three dilutions?
please ignore this part "I'm trying to determine the optimal dilution that will be most effective in the chemotaxis test"
based on the data and your understanding of the figure purpose, which of the two analysis whould you recommend and why? or maybe a different analysis ?

With gratitude
Netanel
 
#5
My research question was in which of the three dilutions I will get the largest mean difference between conditions, but I didn't see any significant difference, I do see that smaller dilution i.e. 1:100 gives smaller variance across data points, so I thought I can focus on that.
Am I wrong to think that in this specific case ANOVA Isn't right because it averages the variance? regardless of the research question
 

Karabiner

TS Contributor
#6
You do a 2-factorial analysis of variance, and if the interaction effect is not statistically significant,
then you have no evidence that the difference between conditions is different between dilutions.
Since the power of your test is poor due to small sample size, the issue is more or less undecided.
Moreover, the variances are very different between cells, which might influence the results. So
maybe you should consider to transform your dependent variable (if it makes sense scientifically,
e.g. often a log scale is more appropriate for a dependent variable), or apply robust standard errors
and/or you have a look at Bruce Weaver's contribution here.

With kind regards

Karabiner