Thread: Which test to use and Bonferroni - Helping a Beginner

1. Which test to use and Bonferroni - Helping a Beginner

Hello all,

I am looking for some help with the statistical analysis part of my masters project. The course is a practical one with very little by way of maths or stats in it, so this is a new adventure for me.

My test
3 methods of recording sound were used to record 4 pieces of music.

For each piece of music, the three methods are being compared against eachother in the following way.

A vs. B, A vs. C, B vs. C.

Each pairwise comparison is called a section in my test.

This comparison is then being applied to the other pieces of music. This means that there are 12 sections to the test in total.

In each pairwise comparison section, participants were asked:
• What sound they preferred
• What sound they thought had more of W
• What sound they thought had more of X
• What sound they thought had more of Y
• What sound they thought had more of Z

Where W, X, Y and Z are just attributes of sound. The preference is the main focus of the test. Suppose a result of 95 or even 99% is established for preference. Any significant results in the attributes section could indicate the a correlation between preference and a certain attribute. This indication can then be investigated in further work as my project can only deal with so much.

Each participant essentially has 12 votes to cast in the test. Each participant is casting 4 votes on an A vs. B pair and so on. I have 20 participants and in total I have 20 votes per musical piece. *

*This then works out as 40 votes per song as two songs were used to supply the 4 pieces, 2 per song. When grouped together, the significance between 2 songs will be examined.

The Stat Tests
As a stat newbie, I went to the library and got some applied nonparametric stat books by P. Sprent and W. J. Conover. They have helped along with some videos from YouTube.

One Proportion Test
Sources

I have built this calculation into Excel, and it works great except that I can not figure out how to calculate the p value at the very end.

Chi-Squared
Sources
http://www.wikihow.com/Calculate-P-Value

When trying to figure out how to calculate the P value, I came across this link which explained everything very well. I set it up in Excel both "manually" with the formula and by using the built in Excel formula. They both give me the same p-value. This is good as by using MiniTabs One Proportion test, which supplies you with a Z value and actual p-value, the p-values all match up. As do the Z values where appropriate.

The Sign Test
Sources
Applied Non-parametric Statistical Methods by P. Sprent
Practical Nonparametric Statstics by W. J. Conover

I started out with this, and got it working in Excel however, I can only get the same Z value as the one proportion test, which we know matches my Chi Square test, when the normal approximation of the binomial distribution is not used. The P. Sprent book tells me to use this approximation when the sample is larger than 20, which is always is. Using the approximation is what MiniTab is telling me it is doing when calculating the figures that I am referencing against what I am doing in Excel.

Does this mean the Chi Squared and One Proportion are wrong or that I made a mistake in the Sign test calculations? Perhaps, this is just because they are different tests which are better or worse than each other?

On that note, should one test be used over another in my project write up?

Bonferroni

I have been advised on another forum which has a maths section that I need to use this correction in my test. A correction of 15 was what was quoted. This takes into account the 15 questions across the 3 sections required to test the first music piece.

However the correction is taking a 3 of the 5 significant results away. Without knowing to what level I should apply the correction, I could find all my results get turned from significant into parity, which I would not like at all!

I hope I made some sense, I have come on leaps and bounds since I started this a couple of days ago and I am really happy I have worked out three tests which all give the same results.

All the best!

Attached please find a copy of my stat calc spreadsheet, I hope it is fairly self explanatory. Though I will say that a set of test results from 20 participants is provided which I have been working off and you should only need to type in the sample size and first answer number for it all to work.

2. Re: Which test to use and Bonferroni - Helping a Beginner

I probably won't be of much help here anyway,
but it isn't at all clear what your research question
is and what you actually want to or need to compare
and/or to test.

Is this hidden in here
Where W, X, Y and Z are just attributes of sound. The preference is the main focus of the test. Suppose a result of 95 or even 99% is established for preference. Any significant results in the attributes section could indicate the a correlation between preference and a certain attribute. This indication can then be investigated in further work as my project can only deal with so much.
or here
When grouped together, the significance between 2 songs will be examined.
- but what exactly do you mean by this?

With kind regards

K.

3. Re: Which test to use and Bonferroni - Helping a Beginner

Originally Posted by Karabiner
I probably won't be of much help here anyway,
but it isn't at all clear what your research question
is and what you actually want to or need to compare
and/or to test.

Is this hidden in here
or here

- but what exactly do you mean by this?

With kind regards

K.
Hi there,
Thanks for the reply. I'll be happy to clear it up for you.

The test consists of 12 pairwise comparisons where two audio samples are presented to test participants individually. 5 questions will be asked which are:

1) which do you prefer and
2-5) which has more of W, X, Y and Z respectively.

Where W, X, Y and Z are just attributes of sound. The preference is the main focus of the test. Suppose a result of 95 or even 99% is established for preference. Any significant results in the attributes section could indicate the a correlation between preference and a certain attribute. This indication can then be investigated in further work as my project can only deal with so much.
What the quote is saying is that establishing whether or not there is a significant difference in preference between the recording techniques/methods is the primary objective.

The additional attribute questions can then be analysed. An attribute question would be worded "Which recording is the most W", for example. A (in)significant result in one of the attributes can then be seen as an indicator or reason behind the preference result. Since establishing the the relationship between an attribute and preference is not the main scope of the project (finding the preference is) then further work could investigate this relationship. It is not much use finding the preference if possible reasons behind it are not established as well.

When grouped together, the significance between 2 songs will be examined.
All this means, with consideration to what is being talked about in that part of the post, is that there are 4 pieces of music being compared. The 4 pieces are sourced from different parts of 2 songs. So, 2 pieces belong to song 1 and 2 belong to song 2. The songs are different in energy and power. One delicate, another is fast paced. The difference in preference and attributes will be compared on this level as well as overall.

The reason it is mentioned is that I wanted to give a full outline of the test construction. It means that 20 participants will have cast 20 votes per audio piece. 20 + 20 means that those participants will have given 40 votes per song. This type of breakdown allows for a song versus song comparison as well as a piece by piece.

 Tweet

Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts