Classical mean-variance optimization is a quantitative tool that many investment advisers use to build portfolios for clients. The goal of MV optimization is to find portfolio weights that optimally diversify risk without reducing expected return and ultimately automate the asset management process. The procedure is based on the pioneering work of Harry Markowitz, the Nobel-winning economist who is widely recognized as the father of modern portfolio theory. While the aim is admirable, the results so far are disappointing.
Anyone who spends time working with most commercially available optimizers usually reaches the conclusion that classical MV optimization fails to live up to its promise. One problem is that it is highly unstable; even small changes in your forecasts of risk or return can often result in totally divergent recommended portfolios. Not being 100% certain of your information, which of the divergent portfolios should you recommend to your client?
Another problem is that MV optimization produces biased portfolios. MV optimized portfolios are "error maximized"; the optimizer overuses the information you give it, ending up with portfolios that are non-intuitive and have little, if any, actual investment value. That's why optimized portfolio return is, on average, an overestimate and why the portfolios that are based on this biased information typically do not perform well. Despite the sophistication of the underlying mathematics and ideas, advisers quickly conclude that the process is somehow critically flawed.
The conventional way of dealing with biases in MV optimized portfolios is to constrain the optimization or manage the inputs so that the investor has a portfolio that you think is appropriate. But this doesn't solve the problem. In this case, optimizers produce predefined portfolios and little more than a scientific veneer for an ad hoc process.
It is natural for investors to blame the problems of MV optimization on flawed inputs. Certainly, it is hard to argue against trying to improve the inputs. As a result, investment institutions typically focus the bulk of their human and capital resources on improving the reliability of forecasts of asset risks and returns. In doing this, however, they often ignore the optimization technology used to transform that information into an investment portfolio. At the end of the day, good inputs are no better than bad ones if the portfolios that represent the information have little real investment value.
Our research shows that the focus on developing inputs and ignoring the optimizer is counterproductive. MV optimization typically creates overestimates of portfolio return (relative to risk) and inferior investment portfolios whatever the quality of the information in the optimization inputs. This bias seriously limits the investment value of MV optimization for many financial planning and asset management objectives, including multiperiod cash flow forecasting.
One way to understand what is happening is that classical optimizers assume that all the information you include in the optimization is certain. This leads to the root cause of the problem: Investor uncertainty is not captured in MV optimization. In simple terms, the model has no option for including any level of analyst uncertainty in the optimization process.
But investors know that their information has uncertainty. What a good investor expects to see in a portfolio optimization and what actually is computed is often very different.
The necessary solution is to incorporate forecast uncertainty in portfolio optimization. Such a process would see the investment world as it really is - in shades of gray rather than black or white. At New Frontier Advisors, we use Monte Carlo simulation to generate hundreds of hypothetical scenarios relative to your inputs to define MV optimized portfolios that reflect forecast uncertainty.
The logic of MV optimization is seductive, but this is mostly an illusion that is all too apparent in the investment period. As used currently, MV optimization has largely a marketing, rather than investment, function.
The demonstrable biases in MV optimization indicate that even the most sophisticated analysts rely largely on their intuition when developing recommended investment portfolios.
This may give rise to a significant fiduciary concern, since technology is now available to improve investment performance and to avoid the unreliability of the ad hoc process underlying many recommended portfolios.
MV optimization is an important idea with many potential investment benefits. But the nearly 50-year-old promise of better-diversified portfolios, improved investment performance, and automatable asset management is likely to be achieved only when uncertainty is integrated into portfolio optimization.
Richard Michaud is president and chief investment officer at Boston-based New Frontier Advisors LLC, whose patented technology investment advisers and institutional investors use for portfolio optimization, asset allocation and portfolio re-balancing.