A bigger house and an earlier retirement can compete for the same dollars, even when the connection does not appear on a mortgage statement. The extra bedroom, larger kitchen, finished basement, or oversized yard may require more than a higher monthly payment.
The real tradeoff includes property taxes, insurance, maintenance, utilities, furnishing costs, and the retirement savings that those dollars could have supported. That makes this less of a housing decision and more of a choice about how someone wants to spend future money and future time.