I'm taking a regression course this semester and I have a question regarding when to drop the interaction terms.

Suppose we have a regression model like this:

Y = a + b1*X1 + b2*X2 + b3*X3 + b4*X1_X3 + b5*X2_X3

where Y is the response variable (say, the outcome of a disease),

X1 and X2 are both indicator variables for the same factor (for example, there are three BMI categories, and I choose the third category to be the baseline group)

X3 is another variable ( eg. age)

X1_X3 is the interaction between X1 and X3

X2_X3 is the interaction between X2 and X3

After running the regression model on a data set, if we found the beta coefficient for X1_X2 is non-significant (p-value > 0.5) but the beta coefficient for X1_X3 is significant ( P-value < 0.5), should we drop X1_X3 only or both X1_X3 and X2_X3?

My question here is when the two interaction terms are actually between the same two factors ( just different indicator variables for different levels of the same variable), should we treat these interaction terms as a whole or treat them separately?

As we know, for main effect, we cannot drop X1 only and leave X2 in the model. I'm not sure if this is still the case regarding interaction terms!

Thank you!