Seek proper test

#1
What statistical test should I use if I have two groups who separately viewed one of two texts describing an ojbect, then rate their feelings about the object? I hope to show significance for one text on the rating. n=100 for each group.

Each group answered a set of identical questions about their impression of the object.

Thank you.
 
Last edited:
#2
Hello, gregapan! It depends on the questions, whether or not to summarize answers, range of values, type of distribution. May you provide more information?
 
#3
Thank you. I will try to clarify. The answer choices for the questions are in basic Likkert style:
  • I strongly agree
  • I somewhat agree
  • I neither agree nor disagree
  • I somewhat disagree
  • I strongly disagree
So I am comparing case A and case B in terms of the level of agreement. Case A has more people saying "strongly agree" for some questions, for example. I am looking for statistical significance betweeen the cases. I have paired sets of data like this:
Group A Group B
27.2% 19.4%
45.6% 35.2%
24.3% 36.1%
2.9% 8.3%
0.0% 0.9%
 
#4
So you want to compare the two different texts? h0: no difference between the text? h1: significant difference? looks like chi-square test.
but occurrence must be >=5, so you can group categories (or maybe use Fisher's Exact Test.)
 
#5
Thank you, obh.

The texts describe a place. The survey takers read one of the texts, then rate their interest in the place. The questions mostly relate to the place, not the text itself. (Like, "Do you have a positive impression of this place after reading the text?")

So I'm looking for significant difference between the two groups' impression of the place, which will indicate the effect of text A vs. text B on their impression.

Can you explain why you recommend chi-square over t-test for this, and clarify what you mean about the occurrence?

Thank you!
 
#6
So you want to understand if there is a significant difference between feeling toward 2 places?

You use t-test to compare averages of two samples, for example, compare the average weight of potatoes from 2 fields.
When you assume the variables are normally distributed and don't know the standard deviation.

In this case, the data doesn't distribute normally...you can't do average to the marks

occurrence say count.

sorry chi-square can be used for nominal data (like England, France, Australia) when you can't order the groups, and you compare the count for each group, but in your case you can order the groups (as you did), So mayby Mann-Whitney U test is better?
 
#7
No, I am comparing the averages of two samples. (You saw my sample numbers above? It's like that.)

And it's their feelings towards one place, not two. Each group read a different description of the place. So I am comparing the effect of the different text on their feelings. The numbers above represent their answers to a question about their feeling. Same question for both groups.

I wouldn't use an unpaired t-test?
 

Karabiner

TS Contributor
#8
Seemingly you have 2 independent groups and want to compare their reactions,
which were measured on a 5-point ordinal scale. Comparing 2 groups with regard
to an ordinal dependent variable can be done using the Mann-Whitney U-test.

With kind regards

Karabiner
 
#9
I think, Mann-Whitney test would be suitable method for these data. But you should describe your data as you wrote, numbers of participants and percentages for each answer. Averages (means) don't have sense for such data.
 
#10
Hi Greg,

If the marks will be numbers 1-100, and the distribution would be normal (or symmetric, near normal) you would be able to use the t test, and calculate the average of these 2 groups.

For example
Group A: 74.275
Group B: 65.925

But the data is not normally distributed, and no symmetric (left/negative skewed), and you use only 5 categories (I strongly agree, ...., I strongly disagree)
So even if you translate the ordinal groups to values (100,75,50,25,0 as I did above) I suspect the t-test won't be good enough

Regards,
O


 
#11
Thank you, everyone, for your kind responses. I sincerely appreciate the help.

I must apologize, as I misstated something above. Not enough sleep. o_O

I am not comparing averages. I meant to say that I am comparing those percentages across the columns for groups A and B.​

The key is that each group read something different. I am looking for significance in the difference between the two groups at each level of the Likert, so that we can know whether the choice of reading material had significant effect on the respondents' Likert rankings.

How interested are you in this place after reading?
Reading A group Reading B group​
I am definitely interested 27.2% 19.4% significant difference here?
I am somewhat interested 45.6% 35.2% or here?
I am neither interested nor 24.3% 36.1% etc.
uninterested
I am somewhat uninterested 2.9% 8.3%
I am definitely uinterested 0.0% 0.9%

I have several questions like this sample above.

The Mann-Whitney looks like a possible fit for what I am doing, but I am not sure. Does everyone concur on that one?
 

Karabiner

TS Contributor
#12
I am looking for significance in the difference between the two groups at each level of the Likert,
This is not how it is done. You have a measurement on an ordinal level, so you look whether
the ratings in one group tend to be higher than those in the other group. But this is never done
level-by-level. The appropriate descriptive statsistc would be the median for each group, not
the percentages for each level (although the U-test does not precisely compare medians).

With kind regards

Karabiner
 
#13
This is not how it is done. You have a measurement on an ordinal level, so you look whether
the ratings in one group tend to be higher than those in the other group. But this is never done
level-by-level. The appropriate descriptive statsistc would be the median for each group, not
the percentages for each level (although the U-test does not precisely compare medians).

Karabiner
That surprises me, because I have read many studies where signficance is found for some differences, but not others. So is it because I have scaled intervals?
 
#14
We can't use median for such data, because usually medians are equal but M-W test shows statistical significance. And this example is the same. I would say that the distributions are different and text A better than text B. Another way is aggregate values, for example, first two and others and make Chi-square test for a table 2x2.
 

Karabiner

TS Contributor
#15
That surprises me, because I have read many studies where signficance is found for some differences, but not others. So is it because I have scaled intervals?
I don't know what sense it should make to perform test a test on each level of an ordinal scaled variable.

With kind regards

Karabiner
 

Karabiner

TS Contributor
#16
We can't use median for such data, because usually medians are equal but M-W test shows statistical significance. And this example is the same. I would say that the distributions are different and text A better than text B. Another way is aggregate values, for example, first two and others and make Chi-square test for a table 2x2.
We can use the median as descriptive statistic (as I wrote before) very well. It is the appropriate statistic for the central tendency of ordinal scaled variables.

Transforming ordinal scaled variables into categorical variables destroys information. There is neither a reason nor a justification for it. Moreover, it could look a bit strange to construct categories AFTER inspection of the results.

With kind regards

Karabiner
 

Karabiner

TS Contributor
#19
Sorry, I do not understand your question. I am not used to the idea to perform statistical tests of significance first,
and then to decide which descriptive statistics to display, depending on the results of the statistical test.

With kind regards

Karabiner
 
#20
I meant that if you use median as descriptive statistic and M-W test for comparison groups by categorical data with such small amount of values, you will often have situation in which medians are equal and the test shows statistical significance.