You don’t often find retirees in small towns beating their heads against the wall trying to figure out where to live when they retire. That’s because they’re happy where they are. They’ve grown up in the town they’re living in and don’t plan on leaving. Their families and lifetime friends live nearby. In fact, many people move back to their small towns when they retire.

If this sounds like you, you’re very lucky. Because out there, in the larger world, retirees are bopping around states and cities like Goldilocks, hoping to find something that feels just right.

Sometimes people move to Florida but end up feeling too far from their children and grandchildren and they end up moving home again. Or they move halfway back to states like Tennessee. Or they build their dream house in the hills of northern Georgia only to realize they have no friends and find it harder to make them. Or, a story I’ve heard often, they move to Texas or California to be near their kids who have moved away, only to find the kids are busy with their own kids and so the family life they had hoped to create again doesn’t always meet their expectations.

It’s rough out there. And the high cost of housing isn’t helping.

Some folks I know in Jamestown are snowbirds, and if you can afford New York State real estate taxes and rising HOA fees in Florida, good for you. It’s the best of both worlds in my opinion, minus the hurricanes. Tired of the snow? Head to your condo in Fort Lauderdale. Afraid of Florida’s summer heat? Head back to Chautauqua Lake. But that’s not an option for everyone.

I know a New England couple who couldn’t agree on where to retire so he’s living in a cabin in Montana by himself and she’s still in New England.

A lot of people tout The Villages in Central Florida as the perfect retirement place. It’s a 55+ retirement community, and pictures show happy retirees zipping around quaint streets in golf carts, headed to concerts and movie theaters, or dinner with friends. One recent study by Biz Rate named The Villages the most optimal spot for the middle class to spend retirement.

The Villages isn’t so much a retirement community as an empire, reports the New York Times, with a collection of dozens of neighborhoods covering more than 32 square miles spread over three counties. It boasts more than 60,000 households and is constantly expanding. “Dump trucks and excavators swarm the developing areas, and new buildings spring up practically overnight. Housing prices are out of control, but the Villages was the fastest growing metro area over the past decade.”

The folks there seem to be having a ball. The retiring baby boomers who move there are redefining aging. They come for the golf and the concerts, the softball and rowdy card games and even polo. There are concerts and casino nights and the Senior Olympics. They drink, party, sing karaoke, play boozy bingo games, and line dance.

What’s not to love?

Like any place, The Villages has its downside, too, including a lack of diversity and an overall conservative bent, politically. But the housing prices are much lower than the rest of Florida and your entertainment costs are $200 a month. There’s no HOA.

Another problem retirees are running into, beyond location, is downsizing. They hope to cash out their family house but are finding housing prices so high, buying a smaller home costs as much or more as the house they’re selling. Where do they go? Economists say more and more retirees are aging in place–in other words not moving at all. And builders aren’t building a lot of entry-level housing, period. But sometimes that so-called entry-level housing is actually what older homeowners are looking for, right? Smaller homes, less space, fewer stairs. Those entry-level homes are what young families need, too, so there’s a bottle neck in real estate that is causing all kinds of heart ache and headaches.

Some retirees are finding living overseas to be cheaper, and places like France, Columbia and Greece are seeing more and more American retirees coming to their shores. Mexico, too, is a haven for people who want to live cheaply in a beautiful place.

As for us? Me and my husband? It’s hard to define our perfect place. Is there a magical land that boasts 75 degrees year-round with year-round hunting, everything walkable and with good hospitals, and no traffic jams? Can it have a Wegman’s and lots of fresh food grown nearby? Low property taxes, great people, honest politicians, no gossip, no traffic lights and low utility bills?

It sounds a bit like heaven. And I’m fine waiting to relocate there.