Enough people will. That much is already clear - look how the huge number of foreclosures have driven down house prices. Enough of anything causes change.
All true on that side of the equation, but there is also protecting the value of your money to consider.
You are aware that a) not everyone lives in covenant property (in fact, most don't) and b) that doesn't help the value of your real estate stay up, right?
The problem as I see it is that much of the "I've got mine, you get yours" sentiment is predicated upon not accepting the fact that bad things that happen to others can cause you problems too.
Also, there is a lot of 'well you should have planned better' sentiment - that's as may be, but what's done is done, eh? You don't want to come right out and say that if I haven't enough money to retire, and I have no other means of support, then I should (and will) starve to death. Now, I haven't pushed you too hard on this, but if one is going to pursue your line of 'only me' reasoning, one has to go to this logical conclusion.
And again, I don't hold that you owe me a debt, or that you have a moral obligation to me (or those like me, etc). I point out that if there are a million people like me, sick, broke, and facing starvation, we'll tear your house down and eat you and your family if we have to. J/K, but what I mean is that yes, enough people like me bloody will affect the economy and the economy is a boat we all float in. You really don't want me busting holes in the bottom of my end of the boat - your side sinks too.