Number of IV's in multiple regressions: How many is 'too many'?

#1
Hello

I need to run a number of individual regressions and am trying to decide how to break them down.

I was wondering whether there is a limit to the number of continuous and dichotomous IV's that I can include in a multiple linear regression or a multiple logistic regression?

If there is no limit per se, is there an optimum number?

Sorry if this is a silly question! I am new to regression analysis and there seems to be a lot to get my head around, and I want to avoid making any silly mistakes by planning it as best I can!

Any advice is much appreciated - thank you!
 
Last edited:

JohnM

TS Contributor
#2
There really isn't a general rule for this, but, the more variables you add to the model, the higher the R-squared will go, but it doesn't necessarily mean that the variables are really adding significant information.....

One way to get at this, for a particular model, is to add IV's one at a time to the model to see if they significantly increase the R-square (a technique known as step-wise).

This link covers the general approach: http://en.wikipedia.org/wiki/Stepwise_regression
 
#3
Hi Gemma7,

The absolute maximum number of IVs that you can have in regression equation is limited by your sample size. With one IV, you have n-2 degrees of freedom, and ever additional IV uses up another. After that you run out of degrees of freedom.

But long before you run out of df, you will have inadequate power to detect any significance. So it does depend on your sample size, but I don't know of any formula for a number of subjects per variable.

But other than power, if you get too many variables, you start (practically, but not necessarily theoretically) running into problems of multicollinearity and not adding any predictability to the model (use adjusted R-square as R-square will keep going up).

The best approach for deciding what variables to put in is theory. Think about what you are trying to test. The simplest model that tests your research question is the best.

And don't worry about making mistakes, silly or not. Planning is great, but you'll still make mistakes. It's how you learn it, though.

Karen