Generalizability of results / External validity

[Renāta]

Hi everyone,

Employment of probability sampling to draw a sample from the population in a quantitative study (cross-sectional design, data collected via surveys) is one of the criteria needed to make inferences to the population.

What other criteria are in play? How to determine, where the results are generalizable to the population.

Thanks in advance for your help

Reference to the relevant article or book will be as well much appreciated.

Cheers,
Renata

 

[noetsi]

The only other criteria I know for external validity is that you have to actually randomly sample the population you are interested in. This comes into play, practically if not usually commented on in theory, in two ways. First you may not have any way to know what the true population is or what their address is. It is very rare you have a way to know or to contact the true population - so even if you can randomly sample you can't randomly sample the real population. This is an issue of sampling frame.

Second is that virtually never will you get a full response. Indeed response rates below 50 percent (often well below) are increasingly common. In my area of research it is often 15 percent. When you get a low response rate those who respond may vary from those that do not which will reduce the external validity - sometimes seriously even if you randomly sample. This is an issue that often gets ignored in discussions of random sampling.

 

[hlsmith]

Another key structural concept is that you have to ensure internal validity before you ever think about external validity. So are you really collecting what you intended to collect.

 

[victorxstc]

What other criteria are in play? How to determine, where the results are generalizable to the population.

Well there are several factors than can favor the generalizability (external validity). For example broader Age ranges of the subjects, sampling from more centers, sampling from a broader range of socioeconomic statuses, sampling from a broader range of ethnicities, sampling from a broader range of businesses, etc. etc... See the pattern? If you sample from a broader range of a characteristic continuum (whatsoever), your study has a better external validity.

 

[Renāta]

Hi All,

Thanks a lot for the input! Much appreciated!

 

[noetsi]

Victor's comments above are effectively methods to address what I was discussing above. When you don't have the sampling frame, the complete list of the entire population or at least how to contact them, you have problems with randomly sampling the population. So you try to extend who you sample to partially address this [cluster sampling is another variation of this].