" I don't remember in the constitution where it said that right was limited to those owning a house on land."
That'd be the same document that says a number of things about who can and cannot vote, right? Written largely by and for white male landowners?
WIth of course no mention of vagrancy, which was a crime in all of the States until what, the 70's ? No home, no property, no dollars in your pocket? Yes, you could be arrested for that.