Covariance Matrix in HLM? Or Deviance Calculations?

[StudentShreen]

Hello, I'm a grad student working on converting a mentor's dissertation into a publication. We are looking at treatment outcomes (psychological dysfunction using the OQ-45) in an eating disordered population using HLM (hierarchical linear modeling). The predictors are feedback group and various measures on the EDI.

The problem is that the analyses were conducted two years ago and my mentor is having difficulty locating the original data. I am working with the tables in the document, but I am missing deviance statistics. I would also like to rerun the models to double check the data. Does anybody know whether you can calculate deviance statistics by hand? Or could I possibly use the covariance/correlation matrices in the paper to run it again? I have only run HLM with raw data, and I didn't think you could use matrices. However, I know you can do SEM with just matrices, and latent growth curve modeling is an SEM technique that's nearly identical to HLM, so I thought it was worth a try.

Please let me know if you can help! We are excited about the data, but we know that deviance statistics are important for reviewers. Thank you in advance!

[noetsi]

Some people use HLM to mean regressions where one set of parameter's are nested inside another. Another term for this is multiway analysis. Others mean a specialized form of linear regression where variables are added in blocks and a F test tells if you have added explanatory power by adding them. It would help commentators if you explained which you meant by HLM (or something entirely different).

Deviance statistics in the first form of HLM I noted are used to determine if one nested model is better than another. Is that what you meant? If your mentor did a disertation on them, wouldn't one of his disertation committee members have the raw data? Or maybe the IRB board he had to submit his proposal to?

You can run the correlation/covariance matrix with SEM. HLM uses weighted least squares to do the calculations and I would think it would have to have raw data to do that (although that may not be the case). I have never seen the correlation matrix used with HLM. It is true that the results of HLM and SEM time series methods are similar. But that does not mean the way the results are calculated are.

[spunky]

what kind of fit statistics do you have access to? if you have something like Akaike's Information Criterion (AIC), Bayesian Information Critertion (BIC), the log-likelihoods, etc. you should be able to back-transform those and obtain the deviance from them...

[SiBorg]

what kind of fit statistics do you have access to?

With R you can have access to them all!

I just wish I understood them all properly..... :-$

[spunky]

With R you can have access to them all!

I just wish I understood them all properly..... :-$

i knoooooowww!!! but i mean, the good thing is as long as you know a few of them you're on the safe side

the thing i was referring to is that the OP is re-running an analysis using data that is no longer available. i just wanted to know which fit statistics (s)he has access to from those previous analyses to see if (s)he can re-derive the deviance from them...

[StudentShreen]

Unfortunately, I only have access to the R^2s and the individual parameter t-tests and standard errors. I only have what she actually published in the dissertation tables. You said you could get the AIC and BIC and deviance from R? My mentor is looking for the raw data--of course this would be the best way to proceed. The HLM I'm speaking of is Hierarchical Linear Modeling, which allows you to nest data from one level (like between students) within another level (like between schools) and discover the variance in intercepts (or scores) and slopes (or rate of change) at the different levels. It's a super fancy method, which is good for publication, but unfortunately also makes it complicated and hard to find info on. :/

[SiBorg]

the thing i was referring to is that the OP is re-running an analysis using data that is no longer available.

Sorry Spunky didn't notice that! My bad...

[spunky]

Unfortunately, I only have access to the R^2s and the individual parameter t-tests and standard errors. I only have what she actually published in the dissertation tables. You said you could get the AIC and BIC and deviance from R? My mentor is looking for the raw data--of course this would be the best way to proceed. The HLM I'm speaking of is Hierarchical Linear Modeling, which allows you to nest data from one level (like between students) within another level (like between schools) and discover the variance in intercepts (or scores) and slopes (or rate of change) at the different levels. It's a super fancy method, which is good for publication, but unfortunately also makes it complicated and hard to find info on. :/

yes, i'm familiar with HLMs (or linear mixed models which is their real name but i guess that's a minor point). in any case, AICs, BICs, deviances, etc. are all some transformation of the loglikelihood on which the (usually restricted) maximum likelihood solution settles on. it may not be exact but transforming one to the other should give you a very good idea of what the deviance is. now, you mentioned you have s. do you know how they were calculated? there are about 7-8 different s for linear mixed models (at leas that i know of) because no one agrees on only one. a few of those are derived from the likelihood. depending on which ones they are you may/may not be able to get the deviance back.

i'm starting to thing that probably re-doing this thing as a latent growth curve under a SEM context might be worthwhile...

[StudentShreen]

Spunky--you're lovely! Sigh. I'm a little annoyed about this project, honestly. My mentor did a fine job of analyzing the results, but she did not report the actual statistics as well as I would have liked. :/ It's really not her fault, though. She had to teach herself HLM and no one told her what to report. I rechecked--she doesn't actually report which R-squares they are. She just discusses the effect sizes. . . I think your idea of redoing it under LGC in SEM would definitely be an option. I was thinking of that; I just didn't want to have to rewrite the whole method section. . .

[noetsi]

A practical lesson. Never get rid of your source data. Ever.

[StudentShreen]

Thanks, I'll remember that.

[noetsi]

I speaketh from painful experience.....

I am surprised that no one on the doctoral committee had the source data themself.