Soft tissues are important, but are mostly a result of age and health. Once these are gone, it is obviously true that bone structure matters less. No eighty year old woman will look good, regardless of her chiseled bone structure. Face lifts, hair transplants, fake tan spray, hair color, etc are really just ways to retain a youthful look. In the end, we all get old and ugly if we're so lucky to make it that far. So when talking good looks, it is assumed the soft tissues are good, and talking about these things as though they define good looks isn't accurate. A guy with great hair, skin and whatever won't look good with a recessed chin, period. But a bald, pale guy can definitely look good with a solid skull structure.
Also, I think most plastic surgery disasters are the result of messing with soft tissues, but I digress.
Bone structure, which is the foundation, is the primary factor. Ugliness occurs when something is deficient in the bone structure, and in rarer cases overgrown (or in the case of the nose needing rhinoplasty). This is why most the most bang-for-your-buck procedures involve bone restructuring or artificial bone (e.g. chin implant).
Also note that people who age well have great bone structure.
Look at your chin, jawline, cheeks, forehead, and skull in general. Compare it to great looking people and you'll be able to see where you fall short if you do.
I will agree most people don't understand why they're ugly, he'll even some on forums such as this one.