At Research Affiliates, we believe that backtests can deliver valuable insights into the behavior of rules-based strategies, especially those targeting well-established and accepted factor exposures. That said, we also know that backtesting is a great way to maximize positive error through the combination of selection bias and data mining. Care and attention need to be applied to the construction of backtests and the interpretation of their results to establish realistic forward-looking return expectations (Harvey and Liu, 2015). So, assuming the backtest has not been data mined too aggressively, what are the levels of excess return we can plausibly expect to earn in the future with live assets over a realistic holding period?
To gain an understanding of realistic levels of outperformance achieved over various investor holding periods, we surveyed US mutual fund data over the past 40 years from January 1, 1979, through December 31, 2018. We used the Morningstar survivorship bias–free database of US equity open-end mutual funds, excluding index funds, which survived for at least one calendar year. For funds with multiple share classes, we chose the share class with the longest history. The total sample includes 4,463 funds. We simply measured the after-fee returns of each fund relative to the S&P 500 Index in order to gauge each fund’s ability to outperform the market.
We chose to measure each fund relative to the S&P 500 rather than against the fund’s stated benchmark or Morningstar-assigned benchmark. We did this for the following reasons:
- First, we want to use the same benchmark used in the shark-jumping claim.
- Second, the goal of our research is to understand the historical ability of funds to beat the market rather than to beat their benchmark.
- Third, we make the assumption that investors in US equity mutual funds primarily care about beating the S&P 500 and they have different beliefs as to the best way to do this: choosing to invest in small-cap funds, mid-cap funds, or growth/value (or other factor)-tilted funds within any size category.
- Fourth, fund managers have discretion as to what they list as their benchmark in their prospectus and they could be motivated to pick an easy benchmark to beat.
- Finally, measuring each fund against the S&P 500 allows our results to have a clean and easy interpretation.
Because the newer methods of strategy construction are better than what fund managers had at their disposal in the past, some may very well counter that using historical mutual fund returns to understand the plausible outperformance of smart beta indices is an apples-to-oranges proxy. We concur that history is full of innovations, that often the new methods are better, and that hundreds of billions of dollars have poured into these new methods. No guarantee exists, however, that today’s new methods will perform better than the best strategies in the past and indeed many won’t add value at all.
In fact, many smart beta strategies replicate successful strategies of active managers. For example, Frazzini, Kabiller, and Pederson (2018) found that nearly all of Warren Buffet’s public stock performance at Berkshire Hathaway can be explained by exposure to the quality, value, and low beta factors.4 As Towers Watson (2013) (now Willis Towers Watson, the consulting firm that coined the term smart beta) stated: “Smart beta is simply about trying to identify good investment ideas that can be structured better…. Smart beta strategies should be simple, low cost, transparent, and systematic.”
Smart beta proponents may claim the structured-better element will lead to better performance than what mutual funds have historically earned, at least partially through lower fees. Smart beta portfolio management is closer to indexed portfolio management and therefore can (and arguably should) be priced closer to index fund fees.5 In contrast, active mutual fund managers need to pay for analysts and portfolio managers and, of course, pass along this overhead cost through higher management fees.
Effective smart beta strategies likewise require much skilled analysis, but are able to bypass the high costs of forecasting and of ongoing fundamental monitoring. It is reasonable to expect lower management fees for these strategies because of their rules-based implementation. A well-designed smart beta strategy should have low transaction costs given that it typically has broad stock holdings and minimal turnover. Unfortunately, the expectation of lower transaction costs is not always realized. We assert the true skill in smart beta investing is in balancing the intended factor exposure while minimizing transaction costs and other forms of implementation shortfall (Israel, Jiang, and Ross, 2017, and Chow et al., 2011).
Nonetheless, analyzing the historical performance of actively managed mutual funds in order to understand the distribution of future plausible live performance offers two advantages versus simply looking at backtested results. First, historical fund returns are net of transaction costs and management fees, which most smart beta backtests blissfully ignore. Although smart beta strategies are likely to have lower management fees and, for well-designed strategies, lower transaction costs, these costs aren’t zero—yet this is exactly what is implied in backtests!
Second, mutual fund managers do not need to publish their methodologies. A mutual fund manager can find a profitable anomaly and not disclose it. In contrast, smart beta index strategies typically have a rulebook that explains the method used for stock selection, the rebalancing dates, and so forth. Savvy investors therefore can determine ahead of time which stocks these strategies will trade, when they will trade, and in what quantities.
McLean and Pontiff (2016) examined the out-of-sample performance of 97 equity factor strategies and found that post-publication premiums declined by an average of 32% versus the published figure. Our own work covering the most widely used smart beta strategies found that out-of-sample excess return following publication falls far short of the in-sample published results (Li and West, 2017, and West and Hsu, 2018).6
Considering these various points, we believe the pros easily outweigh the cons of extrapolating the mutual fund experience to smart beta backtests. Again, our intention is to look at plausible outperformance over realistic investor holding periods.
We analyzed the frequency of mutual fund outperformance over 1-, 2-, 3-, 5-, and 10-year periods at various levels of outperformance, ranging from just barely beating the market (> 0%) to beating it handedly (> 5% annualized). We are able to make two major observations: 1) most mutual funds underperform the market regardless of time period, and 2) the win rates of star performers subsequently collapse.
Most Mutual Funds Underperform the Market
Our first observation is self-evident and well-trodden territory. Most mutual funds underperform the market regardless of time period. Only 43.28% of the 1-calendar-year fund periods beat the market (i.e., a 43.28% win rate) based on 53,127 observations. That figure drops slightly when we extend the analysis to 3-calendar-year periods; only 41.61% of the 44,568 observations are winners, earning an average annualized excess return of −0.52% a year. When the performance period is extended to 10 years, the win rate actually improves slightly to 46.19% with only a −0.09% a year shortfall. This is largely due, however, to survivorship bias creeping into the results.