A constant will not change, ever.
A variable is always subject to change.
If you're talking about 1 particular house at a given moment in time, yes, the number of windows is a constant. But a homeowner always has the option to add or remove windows, right?
If you're talking about the number of windows in a certain population of houses, then it's a variable.
Do you have some specific questions around this?