What statistical analysis to use for a ordinal scale questionnaire?

[swetha]

Dear Experts,

Im a noob when it comes to statistics. So please pardon my lack of knowledge in this post.

I need to do a statistical analysis on a set of questionnaires. There are 3 sets of questionnaires for 3 different populations (Human Resource Reps, Managers, Employees). My hypothesis is "An ERP implementation helped to increase work quality, efficiency, productivity and accountability across HR business processes in an organisation." Dependent Variables being - work quality, efficiency, productivity and accountability. I have a set of questions in each questionnaire which attributes to each of these variables. I'm using a 5 point ordinal scale for measuring response.

What statistical analysis could I use for proving my hypothesis right or otherwise? I came across chi-squared test but there is no link between two variables in my case..or so I think

I also read up about factor analysis and conjoint analysis, but could not seem to fit my data there..or maybe I'm wrong.

Would you please guide me? Thanks a ton!

- Nisha

 

[Karabiner]

If you want to compare the groups with respect to how
they responded on the ordinal scale(s), then you can
perform Kruskal-Wallis H-tests or Mann-Whitney U-tests
with "poulation" as grouping variable. If you want
to analyse whether the responses to certain questions
are correlated with each other, then you can use the
Spearman rank correlation coefficient rho.

With kind regards

K.

 

[swetha]

Dear Karabiner

I do not want to compare the three groups (Managers, HR Reps, Employees), neither do I want to analyse correlation between responses.

I'm using questions associated with each variable in my questionnaires-
Variables: Work Quality, Efficiency, Productivity, Accountability

Questions (examples):

1. How easier is it to analyse the data in the ERP system for decision making?
1. Far more easier compared to before
2. It helps to an extent
3. No effect
4. I have to put in more effort to understand the data

2. Does the ERP system affect business performance positively inside the company?
1. No, it is doing more harm
2. No effect
3. Yes, a little bit
4. Yes, it had a great effect

3. Did the ERP system help make you more integrated with other departments and teams?
1. Yes, very much!
2. Yes, a little
3. No effect
4. It caused problems with others!

To each response, there will be a score attached. No effect or negative effect responses will be where the hypothesis is proven wrong. Positive effect responses indicate hypothesis is correct. I have been reading up meanwhile and came across 'One Sample median test'. Should this be sufficient to establish from a statistical view-point, whether the hypothesis is correct or not?

Much thanks for your response above!

- Nisha

 

[Karabiner]

With the median test, you can analyse whether the responses
within your sample are from a population where the median
response is above (or below) some threshold, which is to be
defined by you.

For example, if you assume that a median response
of < 3 to the question "1. How easier is it to analyse the data
in the ERP system for decision making?" supports your hypothesis,
then you can do it this way.

Mind that you have to define by yourself when you consider the
hypothesis as supported, and that a median of < 3 (for example)
does not automatically mean that all people are supposed
to find it easier, it's just the median.

With kind regards

K.

 

[swetha]

That makes sense! Thank you soooo much...:tup:

 

[Karabiner]

By the way, as an additional optional you could define a percentage
of postive responses which has to be reached as support for the hypothesis,
e.g. "If > 70% in the population say 'Far more easier compared
to before' or at least 'It helps to an extent', then my hypothesis
is supported". You could define new variables (e.g. response
1, 2 versus 3, 4) and test this new dichotomous variables
against your threshold (in the example: 0.70), using Binomial test.

 

[swetha]

You're a savior! That last piece of advice really helped...

 

[noetsi]

It might be noted, although this is one of the most disputed topics in stats as far as I know, that you can use interval level statistics with ordinal level data (as the DV) if you can assume the difference between each level is the same. In that case you can calculate a meaningful mean. This is done quite often....and the debate rages around it

 

[tantsevat]

This may or may not be helpful, but I'm not certain that you really need to do inferential tests. Descriptive statistics might be enough (e.g., 78% of respondents indicated that the ERP helped a little or had a great effect on productivity, whereas 8% reported that it harmed productivity). That might be enough. As Karabiner pointed out, you're the one who has to decide what level of agreement constitutes support for the hypothesis.

As a side issue, I'd run some statistical tests to examine the reliability of the scale, if you're using total scores on the scales in your analysis. For example, you'd want to run a Cronbach's alpha and a factor analysis or principal components analysis on all the scales to ensure that all your items are tapping into the same latent construct (in other words, are all your items measuring the same thing?). If you have a latent construct (like productivity), and all of your items capture it well, then people who respond 5 on one item should generally be responding pretty high on other items. If that's not the case, you've got a problem. The FA/PCA will be able to give you some indication of how many latent constructs your items are tapping into (depending on what options you choose for the analysis) and identify if you have some problematic items (i.e., items that do not correlate well with the other items) that you could choose to remove them from your analysis.

Hope that helps!