Small sample and a discussion re: relevant analytical methods

#1
I have a small sample that I have analyzed with various methods, however I want to have a discussion with smarter people than I if there are additional tests that I have overlooked.

Context: I am studying what management practices help employees feel better after a service failure. Basically, imagine you work for a company and there is a failure which means you have to listen to customers complain all day long. Well, management can do some things prior to this which may help you “recover” from such a situation. For example, training you what to do if there is a failure.

Data: I had access to a small company where there was an employee group I was interested in. There are 50 people in this group. Sent a survey and got 43 responses, but 18 surveys were incomplete so I discarded them. My total sample size is 25.
The survey used 5-point Likert-scale questions and common demographic questions. There are 8 constructs from the literature and I used Cronbach’s alpha to test these.

Methods: I used Mann-Whitney and Kruskall-Wallis tests for differences testing (e.g. against the demographic questions). I used chi-square and Spearman’s Rho for association testing (e.g. 7 constructs against employee satisfaction with management’s practices). I did get some relevant results, however I feel that my analysis is rudimentary.

I am not a statistician by training and have taught myself the basics. I considered trying out structural equation modeling, but realize that the small sample size is a hindrance. I have considered filling in the incomplete surveys using the means from the 25 respondents, but don’t know if that would make a difference.
So, are there any tests or additional methods that I could consider?
 

hlsmith

Not a robit
#2
Seems like you did as much at you can given the small sample limiting the efficiency of more complex procedures. The only other thng I may recommend would be using exact options on your tests. Typically with small samples they perform better. For example, instead of the chi-square use Fisher's exact test, which is usually recommended if you have any contingency cell with a count of 5 or less. In addition, Wilcoxon and Kruskal tests have exact versions.
 
#3
Thanks for the reply. Seems as if I did understand what I was reading.

Is there an acceptable way to include the rejected responses from the incomplete surveys? I seem to recall that filling in omitted responses with the mean value of those who responded would be an option. I guess it would have to an integer value, but might that work?

This would increase the sample to 43, which may allow for more robust analytics?

Or, will that "dilute" the results and make them unusable?
 

noetsi

Fortran must die
#4
To me this involves one of the anomolies of statistics (or my understanding of it anyhow).:p You need a certain sample size to calculate given methods. But you actually have a very high preportion of the total possible responses in this case. Far more than normal analyst would have. Even if everyone has replied with full responses, you would have had only 50 cases when many methods (like regression for example) commonly stress a need for a hundred or more.

I wonder if there are statistics that work on the proportion of the total population reported rather than cases (I don't know any but I am sure there must be some).