Two girls one bonferroni - adjustment according to which families?

#1
Hey guys
We are two girls in need of some statistical guidance in our bachelor project in medicine. As it is the easter holidays, our advisor is not available, and the due date is coming up fast...
So here’s the deal:

We have 17 subjects in a crossover design of three groups:
· Challenge agent (CA)
· CA+ compound of interest (CI)
· CI
- Our data proves this has no physiological effect, and is thus not used in statistical comparison between the CA and CA+CI


The physiological effect of [CI+CA] will be compared to the effects of [CA] to see if the [CI] can decrease the physiological effects of the [CA].

Three continuous visual analog scales (VASs) were used for assessing subjective effects (pain and irritation).
The dataset for each VAS consisted of 600 timepoints which were condensed to 11 timepoints (including 0) by taking the mean of all seconds +-10s from every minute.
The VAS data is not normally distributed, even after transformations.

We would like to describe how the curve for each VAS for each treatment evolves
· First change from 0 (to detect delays in [CA+CI] response compared to [CA])

· Significant differences from timepoints to timpepoints (to detect peaks and other changes)


For each VAS we would like to compare [CA+CI] to [CA] at different timepoints.
· To detect delays in responses

· To detect overall less response

· To detect lower peak response

· Other differences


For all three VASs, we compared all condensed timepoints within each treatment with Wilcoxon signed rank tests. ie. CI_0 vs CI_1; CI_0 vs CI_2 ... CI_1 vs CI_2: CI1_1 vs CI_3 ...

We also compared the condensed timepoints between groups.
All analysis is done with SPSS

With 11 timepoints that is 55 comparisons. We see a clear tendency in our output corresponding with what we see on our graphs.
In SPSS, the output is unadjusted. How should we adjust our data?
With bonferroni 0.05/55 as a significance level kills all our significances.
Can it be true that taking fewer timepoints in the comparisons will change the strength of the adjustment to e.g. 0.05/6 if we were to examine only four timepoints? In that way, as we have continuous data, we could select any timepoint that we deem interesting and only compare the one timepoint across the two treatments which gives us only one comparison, and we can keep our significance level at 0.05 (as bonferroni 0.05/1)

We found the adjustment of our dataset very confusing. Are we dividing with the right thing when making the bonferroni adjustment? Or should we instead divide by number of comparisons made at each timepoint, i.e. dont adjust?

We have also calculated area under curve (AUC) on the continuous data, which also shows the same tendencies as the VAS data when comparing intervals across treatments.

We get that if testing 1000 aspects of changes you have to make adjustments as some of them are bound to be type I errors without, but if our data supports each other, does that not count for something?
And how many comparisons should one adjust with? If making multiple comparisons on blood pressure in the same study, will those multiple comparison affect the adjustments made on VAS data?
One could argue then that all the multiple comparisons performed by the researcher his whole life should be taken into account – cause he was bound to find something someday!

Wow, that got quite long! We hope some of you made it to here and maybe even have some comments that could clear up some things for us. Anything is MUCH appreciated!

Best regards,
Eva and Anne-marie