Minimizing Early Withdrawals in Early Retirement // Hang On to As Much As You Can for Later

We are planning a two-phase retirement: a more basic lifestyle in phase one (what the Elephant Eaters have dubbed “dirtbag millionaires”), supported by our taxable investments and rental income, followed by a more baller traditional retirement, funded by our more gold-star worthy 401(k)/tax-advantaged funds.

One of the things we consistently get questions about is why we’re taking the two-phase approach, especially with well-funded retirement accounts already in place, instead of starting to tap those funds now, and already being able to retire. Of course, what they’re really asking is Isn’t it all just one big pool of money? You can certainly choose to see it that way, but I’ll be crystal clear on where we stand on the question:

No. We don’t think it’s a good idea to see taxable and tax-advantaged funds as one big pool of money.

And we definitely don’t think it makes sense to apply the 4 percent rule to your total balance to figure out what you can spend in your early retirement years, before you can access your tax-advantaged funds without penalty. (We don’t think you should be using the 4 percent rule at all, actually, but that’s another post.)

I know them’s fighting words for some, and I can already hear the imaginary retorts about all the ways someone can access tax-advantaged funds early. “But the Roth conversion ladder!” And “But the substantially equal periodic payments in rule 72t!” Yep, those are things. But we have three big reasons why we aren’t relying on them to fund the early part of our early retirement:

We massively disagree with the conventional wisdom that people naturally spend less as they get older. 

Or even that it’s a fine idea to plan on level spending over your lifetime.

While it’s true that research shows people DO spend less in older age, that research never asks if that’s by choice or if they are forced to spend less because of financial hardship. That’s like asking Irish people during the potato famine if they find themselves eating fewer potatoes, and assuming that they must just be avoiding carbs. Matt at the Resume Gap wrote an excellent post on the question that I wholeheartedly agree with, and he makes the great point that many people in their 70s and beyond simply can no longer afford their pre-retirement lifestyle, which skews the scale.

But in addition to that, it’s still a huge X factor what health care will cost in the future, and not just in the U.S. Most of the countries that provide free health care are also facing demographic challenges that may force greater cost-sharing in the coming years, as more people age out of the workforce and fewer young people enter it to support them. (So don’t get too comfortable, all you folks feeling smug about your better-than-America health care. Or do. Because we’re still totally jealous.)

But we think it’s a mistake to sell out our future selves to escape the workforce a little earlier, and we’re leaving our true retirement money alone so we can both have a less-dirtbag lifestyle and have a bigger safety net for health care expenses and whatever other unknowns might be lurking out there.

Tax rules can and do change, which could spoil your early withdrawal plans.

The backdoor Roth, for example, has been targeted for years as a loophole that needs closing. This applies to the “mega backdoor Roth” too. Given how little people save for retirement, even when they have tax-advantaged vehicles like IRAs and 401(k)s available to them, and especially given the ethnic and racial disparities in savings rates and the ways some say 401(k)s really just help the rich get richer, it’s not at all unthinkable that Congress might rethink our tax laws that pertain to retirement, especially over a long time horizon like many of us have in our early retirements. The current system isn’t actually working, after all, to prepare Americans for retirement, so a responsible Congress (I know, I know) would take a long, hard look at this. For those of us who are the lucky few benefiting from IRAs and 401(k)s, it’s dangerous to be wholly dependent on rules that could change.

Relying on Roth conversions and rule 72t increase the temptation to accidentally sell out future you.

There is nothing inherently wrong with using currently available tax rules to access money you’ve saved, even if the law intended that money to be for your later years. The problem is that relying too heavily on your tax-advantaged funds for early retirement makes it easy to accidentally spend some of the money older you needs later. The money you could convert or withdraw early, tax free, might be money that age 60+ you needs more than age 30/40/50-something you needs it (see the first reason above), but you can’t go back and fix that later on if you’ve already spent the money in your younger years. And by the time you realize that you should have saved that money for later instead of converting it or withdrawing it early, you’ll have a much harder time earning more income than you would have in your 30s/40s/50s.

(And if you say, “But X blogger has more than they need in their future funds, so I think I’ll be fine,” or “But based on historical averages, we’ll have plenty left by the time we hit 60,” don’t fall prey to recency bias. Instead, read this post. And note that everyone currently writing about Roth conversion ladders is still a 30- or 40-something, not a 60- or 70-something who can affirm that they still have enough money left for their later years.)

All of this really boils down to:

Don’t steal from future you. Love future you.

Give future you the gift of no money stress.

 

More Cautions Against Early Withdrawals

I love that Brandon the Mad Fientist did a detailed, side-by-side analysis of different early withdrawal strategies: the Roth conversion ladder, the substantially equal periodic payments (SEPP) in rule 72t and just sucking it up and paying the 10 percent IRS penalty for early IRA or 401(k) withdrawal. And he found that you could very well be best off paying the penalty vs. all the other complicated strategies over a five-year time horizon, or worst case would only be slightly worse off paying penalties than jumping through the conversion or SEPP hoops. And if you’re penalty-averse and don’t mind running things past a tax accountant or three, your next best option is to go the SEPP route. We’ve filed away the surprising conclusion about the penalty route in our “in case we need another contingency” mental filing cabinet. But as for SEPPs and the Roth conversion plan, we see a few other important downsides worth noting, in addition to all that stuff above about selling out future you:

Forced to sell when you’d rather not — With a SEPP, you must withdraw equal amounts each year, even if your investments are tanking and you’d rather not sell shares. Even if selling shares will create sequence of returns risk that could haunt you long term. And if you’re reliant on a Roth conversion for your cash flow, you might be forced into the same situation, even though there are no tax rules forcing you to sell shares in a down year. Selling shares in a crash vs. sitting tight and living off your multi-year cash cushion could easily be the difference between your money lasting through your lifetime and it coming up short. (Note: this is different from doing a SEPP or Roth conversion just because you want to get those funds out of restricted vehicles and you have room in your tax cap. If you’re just doing that, you could sell at a loss and rebuy at the same low price in an unrestricted vehicle. But we’re talking about when you actually need that money for your cash flow, so can’t rebuy shares.)

Tax (and maybe health care) implications in high earning years — Also with a SEPP, you have to keep withdrawing those equal amounts each year, even if you earn a bunch of money unexpectedly, which could have big tax implications. (We’re not anti-tax, but a lot of these strategies lean heavily on tax minimization.) Worse, if any part of the Affordable Care Act survives, earning just a little over a certain threshold could force you over the ACA subsidy cliff, and cost you far more in health care costs than you gain in income. Impossible to know if any part of that will apply down the road, though.

Do This Instead: The Two-Phase Retirement Plan

Everyone has a different comfort level with this stuff, and ultimately, you should do what lets you sleep best at night, not what some strangers on the internet are doing. But with our belief that we’d all be wise to build our retirement plans around conservative projections and pessimistic forecasts, we think it makes the most sense to build out a two-phase retirement that’s minimally — or at least not wholly — reliant on early withdrawals from your tax-advantaged accounts.

The two biggest reasons why someone would not want to do a two-phased approach are:

1. The math is more complicated. There’s no easy 25x.
2. You have a lot in your 401(k) and IRA and don’t want to wait until you have a large taxable balance saved up as well.

Anyone willing to do the hard work of saving for early retirement is up to the challenge of doing a little more math, especially because it means building a way more souped-up spreadsheet that you can brag about to your friends (because we’re all nerds here), and you can live with the fact that we’re talking about delayed gratification.

Related post: How We Calculated Our Numbers for Each Phase of Early Retirement

If going the two-phased route feels like one-more-year syndrome — forcing yourself to work longer than you otherwise might — consider reframing it around these questions:

If your retirement plan doesn’t allow for the possibility that your spending might need to increase significantly in your later years, is it really a solid plan?

Or if you’ll only be able to increase spending when you’re older if the markets are exceptionally kind to you, is it still a solid plan? 

It’s not one-more-year syndrome if you’re just truly not ready to retire yet, if your plan is only solid if nothing bad happens when you’re older, or if your contingency plan is to get all your health care abroad. (What happens if you’re in a car accident in the U.S., or you have a sudden aneurysm, and can’t ask the ambulance to just drop you off at a hospital in Thailand? If your whole plan would be sunk then, it’s not a finished plan. Time to keep revising.)

Early withdrawals could still have a place in that finished plan, so long as they don’t put future you in a bad spot. Like if you are aiming to stay in a certain tax bracket each year, and you come in $5,000 under the max some years? That might be a great time if the markets are doing well to convert $5,000 to Roth to pad your cash cushion or otherwise bolster your investments. A few years with small conversions likely won’t make a meaningful difference to your “real” retirement fund, but relying on big conversions year after year, especially when markets are down, will always come at a cost.

Share Your Thoughts!

I know plenty of folks will disagree on this position, so let’s hear it! This discussion is always incredibly informative, and lots of people value getting to read different views to inform their own plans — so don’t hold back. What’s your retirement plan built on, and — more importantly — why? What are we not thinking about in our calculations that you’re accounting for? For folks on board with the two-phase plan, what were your deciding factors? Let’s dig into the details in the comments!