Unsure if I need to transform my data or use a p-value correction for multiple comps.

krts

New Member
#1
Hello all,

Right off the bat I'd like to say that I'm pretty new to statistics and am having to employ some for analysing the results of a listening test I conducted for my masters thesis (in Environmental Acoustics), so apologies if I struggle to explain myself clearly!

Essentially, my listening test involved subjects listening to seven different stimuli and adjusting their levels so that they give a perception of equal annoyance relative to a reference stimuli.

So, my results have the adjustment levels (in dB) of 15 participants, for each of the 7 stimuli.

My objective is to find out how combined noise characteristics affect annoyance - i.e if someone adjusts a stimuli with a tone by -3dB and another with an impulsive characteristic by -6bB (against the reference) then how much do they adjust the stimuli that includes both the impulsive and tonal characteristic combined?

This has really glossed over my experiment and some fundamental acoustic concepts that you probably wouldn't need to know especially, but if any more info is needed or you're simply interested please let me know...

I'm using a combination of Matlab and Excel for analysing my results.

After plotting the means and errorbars for 95% confidence (some of which are quite large - I've attached the plot) I performed a one-way ANOVA on all of the results, which showed there was significance somewhere within them with a p-value of 0.010234!

I followed that up with a multcompare analysis in Matlab, (which essentially is performing Tukey's HSD test), which showed no significance between groups (stimuli)! This conflicts what the ANOVA was saying, sort of. Though I have a limited understanding that they're kind of looking at different things - so it is possible!

My supervisor and I were a bit confused by this, so did a bit of reading and some of the assumptions were not met for the ANOVA and Tukey's test - turns out that 16 of the 42 condition comparisons do not have equal variance. Also tested for equal distribution for each stimulus using the Lilliefors test and 4 of the 7 conditions were shown to not be normally distributed. A QQ plot in Matlab (attached) indicated the same thing for all results across all stimuli.

So, I was then going to look at doing a t-test for unequal variance but realised I couldn't due to the normal distribution assumption not being met. So then I figured perhaps a non-parametric t-test equivalent would be the way to go; perhaps the Mann-Whitney test, however I then stumbled across this when learning about non-parametric tests:

"Don't be too quick to switch to using the nonparametric Kruskal-Wallis ANOVA (or the Mann-Whitney test when comparing two groups). While nonparametric tests do not assume Gaussian distributions, the Kruskal-Wallis and Mann-Whitney tests do assume that the shape of the data distribution is the same in each group. So if your groups have very different standard deviations and so are not appropriate for one-way ANOVA, they also should not be analyzed by the Kruskal-Wallis or Mann-Whitney test."

I took a look at the standard deviation from the scores of each stimuli and they do vary quite a bit - 3.2, 2.7, 2.5, 4.3, 5.3, 4.7 and 5.2. So I assume that I can't use nonparametric tests after all?

I was looking at transforming my data, though after chatting with my supervisor today he suggested that I look at performing t-tests with a p-value correction for multiple comparisons? But surely that wouldn't be appropriate as the assumptions for the t-test still are not met?

If I did need to transform the data it probably wouldn't be wise to do a log transformation as the adjustments are in a dB (hence logarithmic) scale anyway!

Any pointers as to where to go from here (if you've survived reading for this long) would be greatly appreciated! Also, please let me know if any further information is needed or if I've not explained myself very well!

I hope this is the correct forum to post in as well, though there is a possibility this might need to be moved to the applied statistics forums - hopefully the mods can move it there if that's the case.

Thank you in advance for any help or suggestions.
 

Karabiner

TS Contributor
#2
Re: Unsure if I need to transform my data or use a p-value correction for multiple co

So, my results have the adjustment levels (in dB) of 15 participants, for each of the 7 stimuli.
Do you have 15 subjects, each measured 7 times? Or do you have
7 groups, each comprising of 15 subjects?

With kind regards

K.
 

krts

New Member
#3
Re: Unsure if I need to transform my data or use a p-value correction for multiple co

Do you have 15 subjects, each measured 7 times? Or do you have
7 groups, each comprising of 15 subjects?

With kind regards

K.
Hello Karabiner,

I have 7 groups, each comprising of 15 subjects.

Best wishes,

Chris
 
#4
Re: Unsure if I need to transform my data or use a p-value correction for multiple co

Do you have different individuals in each of the group, so that you have 7*15=105 individual, or did the 15 individuals test all the seven groups?

It seems like you have 2^3 =8 factorial design with three factors (tone, no tone) and (impulse, no impulse) and (intermittent, not intermittent) and where the 8th level is the base level of (no tone, no impulse, not intermittent). Is that correct? So that the seven "groups" are the three main effects, the three two-factor interactions and the three factor interaction. Your diagram looks like that.

Did you do the experiment that the test person started with say 60 dB of disturbance at base level (no tone, no impulse, not intermittent) and then at the "tone" experiment the subject decreased the noise from 60 dB to 56 dB thus the effect of noise for that subject was 4? Is that the way it was done?

If the data have this factorial design it might be that you can gain precision by only estimating the three main effects and not all seven effects. ( So that the combined effect of "tone" and "impulse" would be 4 + 4.5 which is the estimated main effects (by looking at the diagram).
 

krts

New Member
#5
Re: Unsure if I need to transform my data or use a p-value correction for multiple co

Do you have different individuals in each of the group, so that you have 7*15=105 individual, or did the 15 individuals test all the seven groups?

It seems like you have 2^3 =8 factorial design with three factors (tone, no tone) and (impulse, no impulse) and (intermittent, not intermittent) and where the 8th level is the base level of (no tone, no impulse, not intermittent). Is that correct? So that the seven "groups" are the three main effects, the three two-factor interactions and the three factor interaction. Your diagram looks like that.

Did you do the experiment that the test person started with say 60 dB of disturbance at base level (no tone, no impulse, not intermittent) and then at the "tone" experiment the subject decreased the noise from 60 dB to 56 dB thus the effect of noise for that subject was 4? Is that the way it was done?

If the data have this factorial design it might be that you can gain precision by only estimating the three main effects and not all seven effects. ( So that the combined effect of "tone" and "impulse" would be 4 + 4.5 which is the estimated main effects (by looking at the diagram).
Hi Greta - the 15 individuals tested all 7 groups. And yes, you're essentially right.

Basically there is a British standard for the assessment of industrial noise. Within it a character correction is applied to a noise source that contains tonal, impulsive or intermittent characteristics. The standard stipulates that where a noise source contains more that one characteristic - say tonal and impulsive - then a tonal and an impulsive character correction is to be applied to the noise source and then linearly added together. But there seems to be no scientific basis for this, so I am aiming to investigate whether or not this linear addition of character penalties are appropriate.

So, my reference sound consisted of a clip of a broadband fan noise, which I then edited to introduce characteristics. All stimuli were then loudness matched.

So you have:

1) Tonal
2) Impulsive
3) Intermittent
4) Tonal and intermittent
5) Impulsive and intermittent
6) Tonal and impulsive
7) Tonal, impulsive and intermittent

So yes, I am not particularly interested in the level adjustments of the individual characteristics per se, but more as to whether or not subjects make an adjustment of 'x' for the tonal characteristic and an adjustment of 'y' for the impulsive characteristic leads them to making an adjustment of 'x + y' for the tonal and impulsive characteristics combined.

I have uploaded a quick plot of my raw results, and as you can see the sensitivity to noise varies greatly from participant to participant. Considering what I'm actually trying to find out I'm also wondering if I ought to normalise the results - a was thinking of the Xnorm = (X - Xmin) / (Xmax - Xmin), thus bringing all the participants' results to the same scale, but my supervisor seemed to not think it appropriate. However to my mind it ought to still be useful to determine whether the linear addition of multiple characteristics is appropriate and ought to make the analysis of the data easier?

Apologies if I'm going on a bit with superfluous information!

Best wishes,

Chris