Research problem with variables

Ivaaa

New Member
Hi everyone,
I'm doing my master thesis about non material motivation factors and i want to see if there is significant difference between two age groups. I have 16 factors, when I do statistics in SPSS i got results separately. For each of those variables (16 results). 8 od 16 show significant difference. But my hypothesis is to see if there is difference in non material factors for two groups. So I need one result. I made new variable by conputevariable (all/16). On that way s. difference is very strong (p<0.001) but how is it possible if i watch them separately, only 8 of them are significant. Have I done something incorrectly?

obh

Well-Known Member
Hi Ivaaa,

I don't know what test did you run, and if the 16 factors are on the same scale? but I will give you a simple extreme example.
When you are not sure you should look for a simple extreme example to improve your intuition

Factor1:
Group1: 1,2,1,2,1,2,1,2
Group2: 3,4,5,3,4,5,3,4

Clearly there is a significant difference (pval=0.00002)

factor 2:
Group1: 1,1,1,1,1,1,1,1
Group2: 1,1,1,1,1,1,1,1

Clearly there is no significance level (pval=1)

So if you take the average

factor (1+2)/2:
Group1: 1,1.5,1,1.5,1,1.5,1,1.5
Group2: 2,2.5,3,2,2.5,3,2,1.5

Clearly the average is also significant.

So the question why do you expect the average not to be significant???
I can find you another extreme example, that the non-significance factor has a large variance, and the average won't be significant.

But what if the (all/16). wasn't significant?
You may say that there is a significant difference between the group in some factor and no sig difference in other, why should you take the average? (unless all the factors describe what you want to compare, say aspects of the same factor)

Ps when you do 16 different tests, you should use a smaller significance level. (see multiple comparisons)

Ivaaa

New Member
Hi obh and thank you for your response.But I think I maybe didn't explain well my problem. I used likert scale for all 16 items (flexibile work time, good relationships etc). Because my data is not normal and distribution nither, I used Mann Whitnet U test. (im also not sure if that is ok because my N is 362) my hypothesis is there is significant difference in motivation preferences between gen Y and generation X. So only way to check it was to make a new variable (all/16). That shows significant difference (p<0.01)and that's great, but I expected bigger difference when I tested tham separately. So I'm asking myself if computing variables was even the right option. Because if average (all/16) shows so big difference, I think when I watch items separately there should be more significant values then just 8 from 16? And also If i count p values (p1+... +p16) /16 a dont get p value from computed variable. Is that normal?

Last edited:

Ivaaa

New Member
I needed to compute avarage only because of hypothesis. I dont know better way to prove it.

obh

Well-Known Member
What is the sample size of each group? 362?
How many values did you use on the Likert scale?

So all the factors related to motivation? can you give an example of your factors?

I already showed you that the average of the p-values will not be the p-value of the average...
In my example, the average of the p-values is (pval=0.00002+1)/2=0.500001
While the p-value of the average is 0.00002

Ivaaa

New Member
What is the sample size of each group? 362?
How many values did you use on the Likert scale?

So all the factors related to motivation? can you give an example of your factors?

I already showed you that the average of the p-values will not be the p-value of the average...
In my example, the average of the p-values is (pval=0.00002+1)/2=0.500001
While the p-value of the average is 0.00002
I have 186 millenials and 176 people from X generations. 16 questions, in every one i used 7 point likert scale. I think i figured out where is problem. In compute variable spss rounded my responses(if avg is 5.6 spss rounded it to 6, 2.3 to 2 etc).

obh

Well-Known Member
I think i figured out where is problem. In compute variable spss rounded my responses(if avg is 5.6 spss rounded it to 6, 2.3 to 2 etc).
I don't understand the problem you figured.

Mann Whitney U test compares the rank, not the averages.
But it doesn't really matter the main disadvantage of the MWU test is the test power, but your sample size is large, so you shouldn't have a power problem.

So all the factors related to motivation? can you give an example of your factors?

Ivaaa

New Member
Sallary, relationship with management, relationship with collegues, flexible work time, work for home, progress at work. Those are some examples

obh

Well-Known Member
So there is no real connection between the factors, say each stand by itself represent different aspect of life

Ivaaa

New Member
Yes. But I need to see if there is significant different in motivation preferences between millenials and older generation (gen X) i cant confirn this if i do test for every factor separately.

obh

Well-Known Member
But it is okay to say that there are significant differences in factors 1,2,8,7 but not in factors 3,4,5,6

Especially if this is the status...

Ivaaa

New Member
Well, I hope so, i need to mail my mantor. That would solve my problem.

obh

Well-Known Member
Since you are doing multiple testing, please don't forget to take a smaller significance level.

obh

Well-Known Member
Hi Ivaaa,

Bonferroni correction (0.05/16=0.003125) / Sidak (0.003201) are very conservative, assuming all the tests are indepenent.

I assume you may also consider a multivariate test like Hotelling's T-squared test

Ivaaa

New Member
Thank you. I have just one more question, because im not really educated in statistics, I'm learning it cross my master thesis. I already told you I have 16 non material motivation factors. But I have 13 material too(like car, phone, payed vacation etc, also 7 point Likert scale) Distribution also isnt normal. What should I do If I want to test hipothesis: Both generations (younger and older) are more motivated with non-material factors than material.

obh

Well-Known Member
Hi Ivaaa,

So you want to compare the average of the non-material factors to the average of the material factors?

Ivaaa

New Member
Yes, if that is even comparable. I want to prove that both generations prefer non material factors more than material. But I dodnt know If I even need test for that. Because I already know that non material factor have better average score than material (on 7 point likert scale average for non mat is 6, and for material its 5,26). Maybe thats enough for proving hipothesis?

obh

Well-Known Member
Hi Ivana,

If you use a sample, you can't be sure the result is not a random result and in the next sample it won't be an opposite result, for this, you use a statistical test that will calculate the probability that your results doesn't represent the population, say the probability you got such extreme result under the assumption that the null assumption (H0) is correct. (p-value). If this p-value is very small (common number is less than 0.05) then you assume that you can reject the H0.

So you may use the MWU test.
I assume you may also use the two-sample t-test, as the sample size is large.

Ivaaa

New Member
I tried both test for every case and they show the same result (reject Ho). Distribution for non material factors is not normal, thats for sure. But for material k s test shows its not normal and shapiro wilk opossite. But I think I'll use MW any way