Latest ranking of the robos still ruffling feathers

With the goal of becoming the go-to resource on digital advice platforms, Backend Benchmarking has a new report that ranks 10 of the largest robo advisers.

The research firm is best known for its quarterly “Robo Report,” which analyzes the allocation and performance of the portfolios of each digital adviser. The new ranking adds in a qualitative review of the platform, services and features to determine a more comprehensive score.

(More: Top 10 robo-advisers)

Backend Benchmarking limits the report to firms with two years’ worth of collected data. Vanguard’s Personal Advisor Services took home the honor of “best overall Robo,” with Betterment coming in as the runner-up, and SigFig earning an honorable mention.

Acorns came in 10th place, but earned a runner-up nod for “best robo for first-time investors,” coming behind Betterment.

(More: How robos weathered Q1 market volatility)

The qualitative side of the report looked at financial planning, user interface, access to live advisers, transparency and conflicts of interest, and account minimums. For the qualitative portion, Backend Benchmarking used a patent-pending “normalized benchmarking” method that controls for differences in equity allocation.

“We wanted a way to distill a lot of different data on the top 10 robo advisers into a simple scoring system that made it easier for investors to draw on conclusions,” David Goldstone, a research analyst at Backend, said in a statement.

Several of the digital advice firms featured applauded Backend Benchmarking for going beyond returns and taking a bigger look at the value offered.

“I think it really helps consumers think through the providers and the different pros and cons,” said Mike Sha, CEO and co-founder of SigFig.

However, the firms still take issue with including performance as part of the ranking. A chief complaint is that two years is far too short a time frame for a fair analysis of performance.

(More: Wealthfront boots researchers off platform for violating terms of service)

“I understand why they include it as a pretty meaningful part of the score. Ironically, SigFig has always been at the top of their lists, but I actually think it’s pretty noisy,” Mr. Sha said. “Past performance is not predictive of future performance, but fees and tax efficiency are consistent.”

Mr. Goldstone responded to the criticism by saying this is only the first edition of the ranking, and that results will evolve over time as the methodology is perfected and more data is collected.

“I would ask back to them, ‘is the expectation that no one judges their performance for the first five years?’” Mr. Goldstone said. “If we’re not using returns on real portfolios, what should we be using? We agree two years is short, and we are open to other suggestions.”

Joe Ziemer, Betterment’s vice president of communications, added that Backend’s researchers don’t utilize the platforms the way a typical Betterment customer does, such as using direct deposits and getting optimization across taxable and tax-advantaged accounts.

Another criticism was that Backend penalized robos for allocating portfolios to cash. Ken Schapiro, Backend’s founder, said that the report only devalued cash if it didn’t generate a “reasonable rate of return,” which the firm set at 1%.

Mr. Ziemer and Mr. Sha both felt the analysis on cash holdings was fair.

“An investment portfolio is not the place for cash,” Mr. Ziemer said, referencing a blog post Betterment published on the topic. “A firm is going to have cash in the portfolio to make money.”

The company plans to release new rankings every six months in addition to their quarterly reports. The hope is that by providing more information and transparency to consumers, digital advice firms of all kinds will have to step up their game.