A good fit is truly the most important factor in dressing well. That's why it can be so important, especially in thrift shops, to ignore numbers and tag sizing most of the time.
Pictured above are three similar cable knit sweaters from Brooks Brothers, 1 "S", 1 "M", and 1 "L". Common sense would have you believe that there is these sweaters would fit three different men of different builds. But you'd be wrong. Despite the complete difference in sizing, these three sweaters are roughly the same size. Each one fits me (42 reg/36 waist) reasonably well, with only the "L" being slightly fuller all around.
S,M,L, or "alpha sizing" as it's known, is at best ambiguous. Numeric sizing is perhaps less so, being based on an actual measurement, but even that can be unreliable given the increased proliferation of "vanity sizing" (you know, but jeans that measure 37 inches with a tape measure but say 34 on the tag). The only real way to get clothes that fit is to try things on. It either fits or it doesn't. In thrift shops, this can be more difficult as dressing rooms can frequently be either gross or non-existent. For the die hard, I recommend taking measurements of a pair of pants and a jacket that fit you well, and bringing a tape measure along on your hunting trips. And don't let vanity get in your way. Why leave behind a stunning British tweed simply because the tag says 44 and you insist you only wear 42?
Size matters...the actual size, not the one written on the tag.
p.s. new stuff in the Shop, including the three sweaters pictured in this post.