# Thread: New normed Goodness of Fit Measures ?

1. ## New normed Goodness of Fit Measures ?

Hello everybody!

I am currently working on a robust regression problem, where Model R^2 and Spearman correlation recommend different models.

Are there new normed Goodness of Fit-/ Association-/ Correlation Measures away from the ones mentioned before (scale of target: interval)? With normed I am referring to at least a normed maximum fit (like 1 in R^2 and Correlation).

Best case they will also work for robust problems.

Thanks
Consuli

2. ## Re: New normed Goodness of Fit Measures ?

Hmm, can you provide more details!

Would a grouping of predictions (e.g., by decile) vs. observed values be of any benefit?

3. ## Re: New normed Goodness of Fit Measures ?

Originally Posted by hlsmith
Hmm, can you provide more details!

Would a grouping of predictions (e.g., by decile) vs. observed values be of any benefit?
No, it does not.

4. ## Re: New normed Goodness of Fit Measures ?

Originally Posted by consuli
Hello everybody!

I am currently working on a robust regression problem, where Model R^2 and Spearman correlation recommend different models.

Are there new normed Goodness of Fit-/ Association-/ Correlation Measures away from the ones mentioned before (scale of target: interval)? With normed I am referring to at least a normed maximum fit (like 1 in R^2 and Correlation).

Best case they will also work for robust problems.

Thanks
Consuli
Can you explain more about why you want a normed measure? Why not do model selection via AIC, BIC, etc?

5. ## Re: New normed Goodness of Fit Measures ?

Originally Posted by CowboyBear
Can you explain more about why you want a normed measure? Why not do model selection via AIC, BIC, etc?
Sure.

I want to compare machine learning models (decision tree, neural network) with different kinds of regression models including interaction terms (OLS, GLM, Quantile, M-Estimator). Especially I want to know, to which proportion a simpler model (lets say OLS regression) has got the overall explanation of the best fitting modell. Thus, I need a normed measure.

This all on robust data problems
(target interval/ratio scale), error NOT following a classical theoretical distribution. As far I know, AIC and BIC make use of Maximum Likelihood, which is based always based on a theoretical distribution. Thus I am questioning wether they are reliable on robust data problems. Usually they are not recommended for robust problems.

Maybe there are no new normed goodness of fit measures.

Compare

6. ## Re: New normed Goodness of Fit Measures ?

Is this on simulated data or real data?

7. ## Re: New normed Goodness of Fit Measures ?

I like to use Omega-naught-squared from Xu (2003). I think this might help a little bit. The measure of goodness of fit is a bit more robust than classical coefficients of determination.

Here is the reference:

Xu, R. (2003). Measuring explained variation in linear mixed effects models. Statistics in Medicine, 22(22), 3527-3541.

8. ## Re: New normed Goodness of Fit Measures ?

Originally Posted by CowboyBear
Is this on simulated data or real data?
Both. I am using the robust datasets shipped in the R packages robustbase, robust and quantileregression. Further I am generating robust data by simulation.

9. ## Re: New normed Goodness of Fit Measures ?

Originally Posted by the42up
I like to use Omega-naught-squared from Xu (2003). I think this might help a little bit. The measure of goodness of fit is a bit more robust than classical coefficients of determination.

Here is the reference:

Xu, R. (2003). Measuring explained variation in linear mixed effects models. Statistics in Medicine, 22(22), 3527-3541.
Are you shure this omega-naught-squared is a general goodness-of-fit measure? In R it is located in the "Time-Domain Deconvolution of Seismometer Response" package (TDD). Compare https://cran.r-project.org/web/packages/TDD/TDD.pdf

Looks strange to me.

10. ## Re: New normed Goodness of Fit Measures ?

Originally Posted by consuli
Both. I am using the robust datasets shipped in the R packages robustbase, robust and quantileregression. Further I am generating robust data by simulation.
For the simulated datasets you can assess how well the different methods do more directly - e.g., just estimate the bias of coefficients, calculate the mean squared deviation between the estimates of each parameter and its true value to estimate efficiency, etc. Not so sure about the real datasets..

 Tweet

#### Posting Permissions

• You may not post new threads
• You may not post replies
• You may not post attachments
• You may not edit your posts