How do you determine which factors/interactions to remove from a model? Should I be dropping higher-order interactions from my model if they aren't significant and aren't increasing my R-square/F-ratio? Asking specifically for an ANCOVA that I'm running (two categorical IV's and one covariate) but am curious about for other models as well (for example a generalized linear model). Can we run a number of models and decide what to include based on on AICc? In the case of GLM's, can we simply rely on selection methods (i.e., Lasso, elastic net, etc.) to remove factors/interactions? Or are we going to have to use our background knowledge of the system to determine what is important (and should be included) and what might simply be "nuisances" (and can be removed if they're not adding anything to the model)?