+ Reply to Thread
Results 1 to 4 of 4

Thread: How meaningful is to pool data from different surveys on the same target population?

  1. #1
    Points: 1,017, Level: 17
    Level completed: 17%, Points required for next Level: 83

    Posts
    13
    Thanks
    3
    Thanked 0 Times in 0 Posts

    How meaningful is to pool data from different surveys on the same target population?




    HI everybody, this is a very theoretical question for me but I would like to find an answer as I might want to use this in the (near) future.
    In many cases same questions are used in different surveys which are targeted for the same (or almost the same) population (and all designed to give national estimates). However, as most surveys have long time periods between waves, sometimes I would like to compare results from different data sources which, perhaps, have data from two consecutive years.
    I have done some basic reading (Korn & Graubard 1999; Thomas 2007) but in many cases they provide examples on how to pool data from different waves of the same survey and, sub-sequentially, how to normalize the weights.
    However, many papers open to the possibility to pool data from different data sources, bearing in mind that populations must be comparable and there should be no evident differences in terms of sampling strategy/survey methodology.

    But what if surveys have been conducted on the same population and the questions asked are pretty much the same but with the two surveys being conducted with a (slightly) different sampling strategy (e.g. three sampling units vs two sampling units, but both surveys are country representative and have similar sample size)? Is it still meaningful to pool data together? and if so, how should the weights be adjusted? In same cases I read that authors advised to rescale the weights according to the new sample size. Not sure if it is the case in this situation as well though. What should you recommend?

    Thanks in advance

  2. #2
    Fortran must die
    Points: 58,790, Level: 100
    Level completed: 0%, Points required for next Level: 0
    noetsi's Avatar
    Posts
    6,532
    Thanks
    692
    Thanked 915 Times in 874 Posts

    Re: How meaningful is to pool data from different surveys on the same target populati

    I don't know the answer to this although I suspect meta-analysis is something you should look at. Simply asking the same population does not mean you can compare across eras because populations change. Asking questions of white males in 1960 would likely have led to very different answers (and ways of understanding the questions even) than in 1968 even with exactly the same question. People interpret reality through a culture and against the political reality of a given time which can have dramatic impact on results.
    "Very few theories have been abandoned because they were found to be invalid on the basis of empirical evidence...." Spanos, 1995

  3. #3
    Omega Contributor
    Points: 38,432, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    7,006
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: How meaningful is to pool data from different surveys on the same target populati

    So you don't have a unique ID to match individual responses? What is your hypothesis?
    Stop cowardice, ban guns!

  4. #4
    Points: 1,017, Level: 17
    Level completed: 17%, Points required for next Level: 83

    Posts
    13
    Thanks
    3
    Thanked 0 Times in 0 Posts

    Re: How meaningful is to pool data from different surveys on the same target populati


    I am aware of possible bias but in many cases the time span between two surveys would be 1-2 years.
    I excluded the meta-analysis option as I would be interested in time trends on the same population.
    As we are talking about different surveys I wouldn't be able to link data longitudinally but I would like to look at changes over time (given that in theory each sample would be representative of the population at that specific time). I do not have a specific question yet but for instance I might want to look at physical activity in people over 65 yo in the UK using different data sources which would give me the possibility to have at least two data points within 5 years. This to see whether some policies put in place have an impact on this.

    I would certainly consider each result carefully given the different types of bias, but I am not sure whether this can be done if the sampling method is slightly different even if the population target is the same. Also, I have doubts about how to re-scale weights when pooling different data sources together. Normally I would either divide the weights by N or I would divide the weights by the mean of the weights in each sub-sample before pooling data together.

+ Reply to Thread

           




Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats