There are some inaccuracies/missed parts here:
One of the main differences was that clothing was expected to last longer. Heck, I haven't heard of hand-me-downs in a while now, but you used to have multiple generations of hand-me-downs in the past.
Two, it was more of the social expectation (in the west at least) that you'd dress up to go out (and moreso to be photographed). These outfits are overrepresented in surviving photos.
Third, yes, clothing was expensive, but that was a function of the expected life of the clothing (plus labor). Plus, clothing held value, so you could reasonably resell it at a relatively high value still. And, it's reasonably a higher priority purchase (as opposed to walking outside with your arse out). But no, it was not _extremely_ (read: prohibitively) expensive to obtain a couple basic outfits (leisure, work) leading up to the industrial revolution, due to the aforementioned reasons. The biggest disparity would usually be in winter, where people would lack the heavier layers that wealthier people would have, and in the condition of clothing (many generations of use vs. selling old clothing and obtaining new clothing).
For what it's worth, I'm not a clothing historian, but I did want to push back on the notion that people were walking around in rags and tatters and that a full outfit was only for the (relatively (relative to what?)) wealthy. I did also make a broad assumption here that we're referencing the rise of the suit leading up to, and during the advent of, the industrial revolution, and not the preceding periods with tailcoats and frocks and such (late 1800s vs late 1700s/early 1800s).