Hypothetical returns utilized by a host of new exchange traded funds are reigniting a debate about the use of back-tested performance data.
While hypothetical returns can be useful for comparative purposes, they can also be misleading, critics say.
Many of the new ETFs are based on customized indexes that have no real-time performance history, so most providers show hypothetical numbers. And in a highly competitive market, those numbers are almost universally eye-catching.
A few examples of 10-year back-tested annualized returns, as of March, on some broadly based indexes: The PowerShares Dynamic Market Portfolio ETF from PowerShares Capital Management LLC of Wheaton, Ill., shows 10-year returns of 8.47%, versus 3.5% for the Standard & Poor's 500 stock index; the PowerShares FTSE RAFI U.S. 1000 Portfolio shows an 7.1% annualized return versus 3.5% for the S&P 500; and the WisdomTree Dividend Index, which is used by the WisdomTree Total Dividend Fund from WisdomTree Investments Inc. of New York, produced 6.39% compared to the Russell 3000 Index's 3.88%.
"ETFs rely on those hypothetical numbers to sell their new products," said Richard Ferri, founder and chief executive officer of Portfolio Solutions LLC of Troy, Mich., who manages about $850 million in index funds for individual clients.
He said he doesn't show prospective clients any of his own performance data because their expectations may be skewed. Mr. Ferri said he probably loses business as a result.
"Every index provider faces a choice," said Luciano Siracusano, director of research at WisdomTree. "They can not show hypothetical performance and let investors make a decision in a vacuum, or [they can] show back-tested information," he said. "It's pretty standard practice" to provide hypothetical results, Mr. Siracusano said.
A back test is just one tool in evaluating whether a strategy makes sense for a client, said Christian Magoon, president of Claymore Securities Inc., a Lisle, Ill., ETF provider.
"It would be a mistake to look only at a hypothetical back test," he said. "You could potentially have a great back test on stocks with three or more vowels in their names."
Observers warn that hypothetical results are meaningless unless third-party researchers can validate the findings. "The real issue is transparency and how easy it is to replicate" a new index, Mr. Siracusano said. "We have a very transparent back-testing process."
WisdomTree's methodology is available on the company's website and is simple enough for others to replicate, said Stuart Bell, a company spokesman.
Some of the new indexes used by ETFs are actually active strategies that can't be replicated, Mr. Ferri said. The RAFI fundamental indexes from Research Affiliates LLC in Pasadena, Calif., as well as some indexes used by PowerShares, have proprietary rules that can't be tested, he said.
PowerShares officials were traveling last week and unavailable for comment.
RAFI fundamental indexes have been validated by the London-based FTSE Group, Wilshire Associates Inc. of Santa Monica and Nomura Asset Management Co. Ltd. of Tokyo, said David Hennessy, managing director of marketing at Re-search Affiliates.
Claymore has transparent indexes as well as some that use proprietary formulas, Mr. Magoon said.
With doubts about whether new indexes add value, investors can "trust them and try [the product], or wait and see" how it actually performs, he said. Most wait and see.", said Howard Bandy, president of Blue Owl Press Inc. of Sioux Falls, S.D., a consultant on trading systems.
"Advisers usually want to see how an index performs in the real world," he said. "A lot of money usually doesn't come in until a fund has real-world performance numbers."
The hypothetical returns shown in ETF marketing materials almost always show returns superior to a benchmark.
That's because an index with poor-performing hypothetical data wouldn't see the light of day, critics said. "It's all too easy to engineer an attractive back test that ... provides an incomplete picture of what the future is likely to hold," said Jeff Ptak, director of ETF research at Morningstar Inc. of Chicago.
"Providers are likely to be more interested in a back test that tells a happy story," Mr. Ptak wrote in an e-mail. "If I claim that I can provide alpha, I'm going to have a back-tested result that shows that."
"Back-tested indexes are the ETF world's answer to incubator funds," Mr. Ferri said, referring to small portfolios run by open-end-fund companies and brought to market only if they have impressive records.
ETF providers do disclose that the returns are hypothetical and that investors may not achieve the results shown.
But critics say the disclosures are inadequate. The disclosures are "in the micro print" of fund materials, Mr. Ferri said, in contrast to the more prominent mountain charts.
Index providers need to make clear how much of the past performance is back-tested and how much of the result is from real time, Mr. Siracusano said.
WisdomTree is "very clear about showing that distinction," he said.
In contrast, active ETFs that are not based on an index don't have hypothetical returns. "They have to wait three years like an open-end fund company does" to build the beginnings of a performance record, Mr. Ferri said. That wait hurts sales of active ETFs, he said.
Lack of a track record is probably a factor in holding back sales of some of the active ETFs on the market, Mr. Ptak wrote.
But other factors, such as investor uncertainty about active ETFs, lack of knowledge about the managers running them and strategies that aren't unique, are probably larger hurdles to overcome for active ETFs, he wrote.
E-mail Dan Jamieson at [email protected].
