I'm going to make a claim that I know a lot of people are going to disagree with, but here goes: anyone can look great in a hat. Yep, I know there are many naysayers out there who claim that they can't wear them, but just hear me out. There are really only two things standing between you and the perfect chapeau; one is that the selection of styles in many places is woefully sparse, and the other is that you probably don't really understand what works for the shape of your face.
If you haven't done so already, I highly suggest that you figure out what shape your face is. This will give you a lot of clues as to which styles will work and which won't. For instance if your face is rectangular or oblong, you'll need to stay away from styles with a high crown and go for those with a low crown and straight, wider brims; if your face is square then you should avoid cloche-type hats that are worn pulled down low, and look for styles worn back and away from the face, such as a pillbox.
As for the problem of selection, that's a tougher one to deal with. Many stores only carry a handful of standard shapes, and if none of those happen to suit you then I can see how you'd write off hats altogether. Adding to the dilemma is that a lot of vintage hats were made in sizes considerably smaller than the average modern head, which means that many of those great second hand finds won't fit. More and more though, I'm starting to see a wider variety of hats creep into circulation, so perhaps it's time to give them another chance.
My strategy in learning which styles look good on me has been to scour hat shops whenever I come across them and try on as many styles as I can. Over the last few years of doing this I've learned that styles with narrower brims are better for my face, and that they often look better tilted to one side or even worn sideways. A couple of times I've removed the trimming and reattached it so that I could wear the hat turned 90 degrees from the way it was intended. The purple peacock feather hat I made at my recent millinery class is in fact trimmed for a sideways orientation. My point is that you should be creative and try out each hat in different positions on your head before deciding it doesn't work. Tilt it forward, push it back, flop it over to one side, spin it around; a hat is not "supposed" to be worn in any particular way except the one that suits YOU.
I'm nearly finished, but I'll just go out on a limb and make one more radical claim: if you're going to own only one hat that isn't something completely utilitarian like a knit beanie, make it a black beret. A simple beret has so many applications and looks good on pretty much everybody. Again, don't write them off simply because you've been faced with a limited selection; there are a multitude of different variations on the beret if you just keep an eye out for them. Some are as wide as a dinner plate and flop over to one side; others are tiny and perch high up on the head. Some are made of soft knit fabrics that drape down flat, others are made of stiff felt that holds a more structured shape. Whatver the variation, a black beret is classic, timeless, and chic. Of the 30+ hats I own, this is my most frequently-worn, for looks both casual and dressy.
Hat: From ADS in San Francisco
Dress and belt: The Red Dress Shoppe
Scarf: A gift from my sister
Cardi (worn under the dress): A cast-off from my sister
Leggings: American Apparel