Think twice before borrowing from 401(k) for a home down payment, advisors say

Accessing retirement savings can be helpful for a subset of potential buyers, but there are some important caveats.

The average price for U.S. houses sold during the second quarter was nearly $500,000 — that, coupled with the highest mortgage interest rates in over 20 years, could make a 401(k) loan appealing for buyers.

Whether that’s a good idea is another matter. In the best-case scenario, financial advisors say: It’s complicated. For most people, their answer tends to be: Don’t even think about it.

“This is a very challenging market for home buyers and retirement savers alike. But often overlooked are the trade-offs when purchasing a home, as the average home price today mortgaged over 30 years at 6% totals $925,000,” said Laura Lynch, also known as “The Tiny House Advisor,” who specializes in financial planning for people interested in tiny houses. “And this does not include the carrying cost of owning that home, such as the taxes, insurance and maintenance,” Lynch said in an email.

Deciding whether a 401(k) loan could be the right choice should include consideration of a client’s values, she said. “Many are prioritizing experiences over things. Others may prefer the stability and status that a home may provide.”

Historically, 401(k)s have seldom been used to help with down payments. Data from retirement record keeper Alight Solutions show that only about 1% of account owners borrow from 401(k)s to pay for housing, with average balances of about $20,000. Early withdrawals for that purpose are even less common, at about a quarter of a percent of participants, the firm said.

Figures from Vanguard’s How America Saves Report 2023 show that 12% of 401(k) participants had loans of any kind outstanding as of the end of 2022.

But it’s hard to know precisely how many 401(k) savers borrow from their accounts for the purpose of buying a home, said Mike Shamrell, vice president of thought leadership for Fidelity’s workplace investing business. Unlike participants making hardship withdrawals, those borrowing from their 401(k)s don’t need to specify why they need the money, he said.

However, based on a 2021 survey by Fidelity of people who had taken out 401(k) loans, 21% said they did so for home purchases, he said.

In general, 401(k) loans has been trending downward for at least two reasons: Participants are becoming more educated about the downsides of borrowing; and plan sponsors have put limits on the number of loans participants can take, Shamrell said. It used to be common for 401(k) borrowers to have five or six loans out, but many plans now limit active loans to one, he said.

If nothing else, the rising cost of homeownership is making some first-time buyers balance financial priorities. And saving for retirement — something that seems far away from many younger workers — could take a backseat.

“Rising housing costs have impacted a lot of people’s ability to save for retirement. Home prices have not materially dropped since the boom of the pandemic, while mortgage rates have rocketed to levels not seen in 20 years. This, coupled with recent slowing wage growth, has pushed home affordability out of many people’s reach,” Derek Williams, wealth advisor at Veratis Advisors, said in an email. “Growing families or new job opportunities have forced them into the real estate market, and they’re finding that their options are bleak. They’re having to sacrifice other things to afford their home, and for many that might be retirement saving.”

Williams, like many advisors, recommends against borrowing from a 401(k) in most cases. Having a loan means that the assets are out of the market, potentially hurting returns that would benefit from years of compounding. Further, most plans require 401(k) loans to be repaid in full when an employee leaves a company — and that can trigger taxes and penalties for borrowers who don’t have the money to do so. Thirty-nine percent of plans in Vanguard’s record-keeping business allow terminated participants to continue their loan repayment.

But in addition to those issues, 401(k) loans don’t address the problem of monthly cash flow, and they can lead people to become serial borrowers, making it harder to save in the future, Williams noted.

Using 401(k)s to help make down payments should be considered a last resort, said Nicole Sullivan, co-founder of Prism Planning Partners.

But “advantages to a 401(k) loan are that it does not count toward your debt-to-income ratio, will not impact your credit score, and won’t impact your ability to qualify for a mortgage,” Sullivan said in an email.

People can borrow up to 50% of their vested balances, with a maximum of $50,000, she noted. “I strongly encourage anyone contemplating a 401(k) loan to check with their HR department and 401(k) provider prior to starting a loan, as all plans have unique stipulations.”

One group that has been affected significantly by the runaway housing market is new parents, said Kelly Palmer, founder of The Wealthy Parent.

“A new child is often an impulse for purchasing a new home but between rising childcare costs and retirement savings something has to give,” Palmer said in an email. “I have seen parents pausing contributions to their retirement plans in favor of affording a larger home often with the hope they can refinance in the future … As long as there is a tangible plan in place to get back to saving for their retirement goals, I encourage families to consider all their options, including a period of negative cash flow.”

There has been a positive side to rising mortgage rates, said Herschel Clanton, principal of Chancellor Wealth Management. Because existing homeowners with low rates have been encouraged to stay put, they are able to save more for retirement, he said.

“The biggest impact of the increase in mortgage rates has been to encourage people to stay in their homes,” Clanton said in an email. “This eliminates the ‘casual’ buyer that just wants to upgrade their housing. Using a 401(k) to fund a down payment is a terrible idea.”