Thanks for your reply Dason.

I would like to give an actual example which made me wondering about the choice of reference categories.

I run a linear regression model which also includes 2 factor variables as IVs. One of the factor variables measures respondents level of education (primary, secondary, tertiary).

In a pre-regression step, I run correlation analyses which indicate that primary school correlates negatively with the DV and tertiary education correlates positively with the DV. Secondary schholl does not correlate significantly with the DV.

Hence I was wondering which of the three categories is best used as reference category in the multiple regression model. I thought of using secondary school since this category is not significantly related to the DV.

Does that make sense?