Minimum sample size for calculation of guttamns lower bounds?

BOAZ

New Member
#1
Minimum sample size for calculation of Guttmans lower bounds?

Hi there,

I am in the process of assessing the internal consistency of two tests, test A & B. Both tests are measuring the same thing so I am also conducting a split half reliability analysis. All participants have taken both tests. Cronbach's alpha and split-half coefficients for each analysis are below:

Test A = .542
Test B = .721
Spearman Brown = .204
Guttman's = .204

The main problem is that the sample size (N=10) is too small to produce truly representative indicators of reliability. A power analysis using PASS (1, 2, 3) shows that the minimum sample sizes to achieve a Cronbach's alpha of 0.7 and 0.5 are 16 and 54 respectively (K=32). Comparison of alphas requires a sample size of 164 in each group!

Analyses using Guttman's lower bounds produces the following Lambda 2:

Test A = .679
Test B = .809

Obviously I would like to quote these figures in addition to Cronbach's alpha, especially given the current debate regarding the popularity of alpha despite it's shortcomings (see Sjitsma 2009 - sorry no ref!). However, I cannot find any literature regarding appropriate/minimum sample sizes.

Your input would be much appreciated!

References
1.Bonett, Douglas. 2002. 'Sample Size Requirements for Testing and Estimating Coefficient Alpha.' Journal of Educational and Behavioral Statistics, Vol. 27, pages 335-340.
2.Feldt, L.S.; Woodruff, D.J.; & Salih, F.A. 1987. 'Statistical Inference for Coefficient Alpha.' Applied Psychological Measurement, Vol. 11, pages 93-103.
3.Feldt, L.S.; Ankenmann, R.D. 1999. 'Determining Sample Size for a Test of the Equality of Alpha Coefficients hen the Number of Part-Tests is Small.' Psychological Methods, Vol. 4(4), pages 366-377.
 
Last edited:

spunky

Can't make spagetti
#2
uhmm... guttman's lower bounds sorta fell in disuse quite a few years ago because the communalities of factor analysis can be transformed to work as more accurate lower bounds to the true reliability...

.... would you feel comfortable with a simulation approach to this problem?? both cronbach's alpha and many of the reliability indeces used in psychometrics have some devilishly complicated sampling distributions, so i find it a little hard to believe that people would continue to do research on this area (SPSS i know uses montecarlo simulations to figure out a lot of the power stuff...)
 

BOAZ

New Member
#3
Best internal consistency test for small sample (N = 10)

Thanks Spunky. I guess my main concern at the moment is not so much finding the correct sample size, but instead to report the best possible indicator of internal consistency given the very small sample size that has already been used. Unfortunately I am not in a postion to re-do the study with a larger sample. At this stage I can only report the cronbach's alphas stated above, and comment that the sample size was too small for them to be truly represenative. Would you recommend omitting a discussion of Guttman's Lambda, i.e. is it no longer taken seriously amongst researchers? This sub-study is being published as a poster.

I am a bit out of my league when it comes to communalities of factor analysis. I've tried to read up on this look and for a procedure in SPSS but I am unsure of the next step. Can you recommend a test within SPSS or do you think it is best to stick with cronbach's alpha? I should mention at this point that the data is dichotomous, and of the 32 items in Test A SPSS removed 18 from the scale due to zero variance.
 

BOAZ

New Member
#4
Best internal consistency test for small sample (N = 10)

And just another thought. Tests A & B have been used as a pre (A) and post (B) test to measure the same scale. Is it dubious that the cronbach's alphas are so different (.542 vs .721)? The spearman brown co-effienct is also very low, but as mentioned previously N = 164 was a realistic requirement for this test. The tests do seem to be equivalent... a Wilcoxin signed ranks test showed no significicnat difference on test score (Z = -.634, P = .536) and order of test administration (AB vs BA) was not a significant factor on a one way ANOVAs fo reach test (P =.496, and P =.436).