+ Reply to Thread
Results 1 to 10 of 10

Thread: multilevel modelling: predicting random effects for new clusters

  1. #1
    Points: 196, Level: 3
    Level completed: 92%, Points required for next Level: 4

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    multilevel modelling: predicting random effects for new clusters




    Hi:

    I have a query I'm hoping someone may be able to help with. I've chatted to a few people and looked at texts and the literature but haven't yet found a clear answer.

    The general question is:

    Is it possible to predict point estimates (and standard errors) for random effects (i.e. intercepts, slopes) for 'new' clusters? Where 'new' means clusters not included when the model was estimated. Or, is it necessary to use the 'average' random effect (which is 0) with the variance estimated when fitting the model for any new clusters?

    Here are some details of my specific situation:

    I'm fitting a 'growth curve' model (i.e. using year as a predictor) where level 1 is measurement occasion and level 2 is country (using logistic regression for an aggregated binary outcome (i.e. proportions) under a binomial distribution). The model also includes additional predictors. To fit the model I have historical observations for a sub-set of all countries. The random effects are being estimate at country-level.

    After fitting the model, I want to predict the outcome variable for future years for a larger set of countries than is being used to fit the model.

    This is straightforward for countries that were used to fit the model as I can estimate the country-level random effects. But, I'm not sure how to proceed for countries not included when fitting the model. Presumably the random effects for these countries will lie somewhere within the distribution of random effects estimated when fitting the model.

    And the specific question:

    Is it possible to predict point estimates (and se) for the random effects (i.e. intercepts/slopes) for countries that weren't included when fitting the model?

    [The reason I'm in this situation is that historical observed data for fitting the model is only available for some countries, while future projections are available for all countries].

    Or, do I need to assume that the random effects in these countries are the 'average' value with a variance as estimated by the model?

    Any advice would be greatly appreciated. (I'm using Stata)

    Cheers
    S

  2. #2
    R purist
    Points: 35,103, Level: 100
    Level completed: 0%, Points required for next Level: 0
    TheEcologist's Avatar
    Location
    United States
    Posts
    1,921
    Thanks
    303
    Thanked 607 Times in 341 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Quote Originally Posted by sigmundo View Post
    Is it possible to predict point estimates (and se) for the random effects (i.e. intercepts/slopes) for countries that weren't included when fitting the model?

    No there is no predictive power in random effects. Random effects estimate the variability. All you can do is include the country level variation to adjust your prediction intervals for each country.

    Sounds like you are actually interested in predictions at the country level, so you should include country level predictors in your fixed effects.

    Your question does sound like a classic point #6 in our guidelines.
    The true ideals of great philosophies always seem to get lost somewhere along the road..

  3. The Following User Says Thank You to TheEcologist For This Useful Post:

    trinker (04-25-2015)

  4. #3
    Points: 196, Level: 3
    Level completed: 92%, Points required for next Level: 4

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Thanks Ecologist - much appreciated

    That's right my goal is to make predictions for my outcome of interest at country-level for some countries that were included when fitting the model as well as for some countries that weren't. The model includes fixed effects for country level predictors and country-level random effects (intercept and slope for year).

    It seems to me that:
    (i) For countries included when fitting the model, the correct path is to predict the country-level outcome in some future year using BLUPs (or similar) to estimate the point estimates (and standard error) for the random intercept and slope for each country, and combine these with the future values for the predictor variables (which have fixed effects). I know how to do this.

    (ii) For countries not included when fitting the model, I'm not sure of the correct path. I would like to estimate the random effects for these countries but I don't know of a way to do it. Maybe it's possible using known values for the predictor variables for the fixed part of the model?. My guess is that the best I can do is use the average values for the slope and the intercept (which in effect is the fixed part of the model) and use the estimated variances for these to capture the likely range of the outcome.

    From the first reply, I'm thinking that using the average slope and intercept is all I can do. Is that right?

    Thanks again.

  5. #4
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    TE,

    How about if you had a cross-level interaction term (one variable at level one and one at level two)? Would it have predictive power or not? My question is ignoring the OP's cluster question.
    Stop cowardice, ban guns!

  6. #5
    Points: 196, Level: 3
    Level completed: 92%, Points required for next Level: 4

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Hi again. Thanks for raising some issues hlsmith. Interested to know what people's thoughts are.

    One thing I didn't mention earlier is that I did find one paper which proposed a method to do exactly what I'm trying to do.
    http://www.sciencedirect.com/science...67947313001369

    Essentially it suggests that after fitting the multilevel model, a series of regressions equations are developed in which the cluster specific random effects (estimated using BLUPS or similar) are the outcome and various known cluster specific variables are used as the predictors. After fitting these equations, they are then driven by the known predictors for the 'new' clusters to estimate the random effects for these clusters. While I could try this, the paper isn't from a 'big' journal and hasn't been cited much so I'm not sure if it really is a good method... If anyone has any thoughts on this it'd be great to hear them.

    CHeers

  7. #6
    R purist
    Points: 35,103, Level: 100
    Level completed: 0%, Points required for next Level: 0
    TheEcologist's Avatar
    Location
    United States
    Posts
    1,921
    Thanks
    303
    Thanked 607 Times in 341 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Quote Originally Posted by sigmundo View Post
    Essentially it suggests that after fitting the multilevel model, a series of regressions equations are developed in which the cluster specific random effects (estimated using BLUPS or similar) are the outcome and various known cluster specific variables are used as the predictors. After fitting these equations, they are then driven by the known predictors for the 'new' clusters to estimate the random effects for these clusters. While I could try this, the paper isn't from a 'big' journal and hasn't been cited much so I'm not sure if it really is a good method... If anyone has any thoughts on this it'd be great to hear them.
    I'm not too surprised it's not in a high journal, it's nothing new and a little silly, but from what I gather (haven't read the paper), they say fit the mixed model, extract the random effects, then fit a model with a predictor to those random effects - with which you try to predict the random effects. Yes, this is very valid. I've seen people do it.

    Now it's silly because you can also add the predictor you are using as a fixed effect and achieve the same goal directly (you explain more variance in your mixed model and hopefully have smaller variance in your random effects).

    So, this is in fact nothing else than I originally suggested to you, just doing it indirectly.

    There are cases when I would opt to do a post-hoc regression on the random effects. In a situation where adding another predictor in the fixed effects would lead to 3 or 4 way interactions - i.e. a model that is hard to interpret.
    The true ideals of great philosophies always seem to get lost somewhere along the road..

  8. The Following User Says Thank You to TheEcologist For This Useful Post:

    sigmundo (04-27-2015)

  9. #7
    R purist
    Points: 35,103, Level: 100
    Level completed: 0%, Points required for next Level: 0
    TheEcologist's Avatar
    Location
    United States
    Posts
    1,921
    Thanks
    303
    Thanked 607 Times in 341 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Quote Originally Posted by hlsmith View Post
    TE,

    How about if you had a cross-level interaction term (one variable at level one and one at level two)? Would it have predictive power or not? My question is ignoring the OP's cluster question.
    He hlsmith,

    Let me see if I understand you. Are you suggesting this model:

    Y_{ij} = \beta_0 + \beta_1 X_i + \beta_2 X_i Z_j + \mu_j + \epsilon_{ij}

    Where Y is a vector of response variables, X a vector of independent predictors at level i (lower level), Z predictors at level j (higher level), \mu_j is a random effect at level j and \epsilon_{ij} is the residual error?

    In this case \mu_j, by itself, still has no predictive power.
    Last edited by TheEcologist; 04-25-2015 at 05:08 AM.
    The true ideals of great philosophies always seem to get lost somewhere along the road..

  10. #8
    Points: 196, Level: 3
    Level completed: 92%, Points required for next Level: 4

    Posts
    9
    Thanks
    2
    Thanked 0 Times in 0 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Thanks a lot Ecologist - very helpful and very much appreciated.

    Cheers

  11. #9
    R purist
    Points: 35,103, Level: 100
    Level completed: 0%, Points required for next Level: 0
    TheEcologist's Avatar
    Location
    United States
    Posts
    1,921
    Thanks
    303
    Thanked 607 Times in 341 Posts

    Re: multilevel modelling: predicting random effects for new clusters

    Quote Originally Posted by TheEcologist View Post
    He hlsmith,

    Let me see if I understand you. Are you suggesting this model:

    Y_{ij} = \beta_0 + \beta_1 X_i + \beta_2 X_i Z_j + \mu_j + \epsilon_{ij}

    Where Y is a vector of response variables, X a vector of independent predictors at level i (lower level), Z predictors at level j (higher level), \mu_j is a random effect at level j and \epsilon_{ij} is the residual error?

    In this case \mu_j, by itself, still has no predictive power.
    So can I assume this was what you meant hlsmith?
    The true ideals of great philosophies always seem to get lost somewhere along the road..

  12. #10
    Omega Contributor
    Points: 38,334, Level: 100
    Level completed: 0%, Points required for next Level: 0
    hlsmith's Avatar
    Location
    Not Ames, IA
    Posts
    6,998
    Thanks
    398
    Thanked 1,186 Times in 1,147 Posts

    Re: multilevel modelling: predicting random effects for new clusters


    Forgive me for not typing the model out, but yes I believe that is it. So my question was if you can predict with a term that is an interaction of a level 1 and level 2 term, that was included as a random effect. I would guess that would also make interpretations on the two first order terms in the interaction not able to predict as well since they are now conditional on the random effect.

    I am pretty sure the answer is no, the interaction random effect just helps explain variance.
    Stop cowardice, ban guns!

+ Reply to Thread

           




Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts






Advertise on Talk Stats