Not Quite Quadratic Regression


New Member
Hello all,

I predict an u-shaped relation between two variables, y and x. If I perform a quadratic linear regression, i.e. use a model like

y = b1 + b2x + b3x^2 + error,

then the coefficients b2 and b3 are not significant. However, if I change the exponent to something less than 2, e.g. 1.5, I obtain significance. In other words a model like

y = b1 + b2x + b3x^1.5 + error,

yields significant estimates of b2 and b3. The curvature is still quite marked using the exponent of 1.5. I can even use an exponent of 1.1 and obtain significance and a nice shape. But of course I dont think I can simply choose the exponent based on what gives me significance. Or can I?

I have tried to run a non-linear regression where the exponent was a free parameter, but although it tend to yield an exponent around 1 to 2, everything became highly insignificant.

I have also tried to use a splines as well as a piecewise constant formulation, but again the results are less than ideal.

I am wondering if it would be considered bad form to use an exponent of e.g. 1.5 based on the fact that it yields significance? It would appear a little arbitrary in my eyes, since then I could just as well choose an exponent of 1.1.

The non-linearity is rather apparant in a scatterplot (although extremely noisy), and the problem mostly shows up when controlling for other covariates and a simple graphical/nonparametric approach is unfeasible.

I am very interested in suggestions as how to approach this problem. Needless to say, I have been searching high and low for an answer before posting here.

Thank you in advance!
Last edited:
If you pass a specification test (e.g. Ramsey) I don't see how it's different from selecting ^2 "arbitrarily".

I doubt there's anything special about ^2.

But this is an interesting question nonetheless when there's a large interval over the reals that the exponent could assume where you both pass the Ramsey's RESET test and still get significance... Very interesting question.


New Member
Thanks for your reply, derksheng.

You have a good point: ^2 is just as arbitrary as any other reasonable exponent. In that case, my problem becomes one of motivating how I land on e.g. ^1.5 or ^1.1 and why I choose one above the other.

Thank you also for your suggestion of running a RESET test. I will certainly try that!

Further input is welcome!
When I first read this I thought that maybe Aikaike Information Criterion (AIC) could be used to evaluate which model works best. Since there are different number of estimated parameters in the two models (x^2 versus x^p) one need to discriminate between them. And AIC tries to make a trade-off between good-fit and model complexity (the number of parameters).

In a prediction problem cross validation can be used.

When I read derkshengs comment (thanks for that) I came to think of fractional polynomials. To change the title:

“Not quite quadratic regression, but maybe fractional polynomials”.

One extreme is to give the exponent a fixed values, like x^2.

Another extreme is to let the exponent be estimated freely, like x^p.

An in-between case is fractional polynomials.

In fractional polynomial: a model:
FP = b0 + b1*x^p1 + b2*x^p2 + b3*x^p3 +…+ bm*x^pm

Where the p1, p2,… are chosen from a finite set of values like (-2, -1, -0.5, 0, 0.5, 1, 2, 3) (where “x^0” are supposed to mean log(x))

So fractional polynomials are just using a few possible exponents. It becomes a quite simple model.

(I stop here because otherwise someone opens this post and think “it is as long as a thesis” and closes.)
How is the model estimated? Eehh, don’t remember. I guess with some iterative procedure to get “good fit”.

(Royston Altman are connected to fractional polynomials)

Fractional polynomials are like a modern variant of the Box-Tisdell from the 1960ies.)

You can estimate fractional polynomials in Stata and I believe so in several packages in R.

Other smoothing possibilities are splines in generalized additive model (gam). In R this can be estimated in the packages gam and mgcv (which stand for m-something general cross validation). (And other smoothing possibilities are like lowess and distance weighted least squares.)
I haven’t really used fractional polynomials myself (but I have use gams).

I haven’t seen it written but I believe someone said to me that fractional polynomials are “about as good” as gam:s. (This is terribly imprecise! Apologize! I also apologize for my even worse than usual English! When I am saying this I am also apologizing if I have typed someone’s name incorrectly.)


New Member
Wow, thank you very much, GretaGarbo! I think the method of fractional polynomials pretty much solves my problem by serving a formal procedure to select the best model with possibly non-integer exponents.

I thank you both for your help! I am sure future "googlers" will also appreciate your responses.

I am still interested in any further input.