What you're doing is called overparameterization. You lose an extra degree of freedom, but the interpretations become easier. Using (k-1) indicator variables for a categorical variable with k levels makes the interpretation of the coefficients relative to the level that you didn't code.
Overparameterization is very common. It isn't used often in OLS (because the interpretations are simple), but in generalized linear models it's used more often than not.