- Performance backtests are often used as evidence to “prove” a smart beta strategy is “better” than its competitors. In our view, careful attention must be given to these claims because backtested results are easily data mined.
- The historical distribution of live fund performance is a useful guide to gauge what long-term smart beta strategy returns are truly plausible. We look at 40 years of mutual fund returns and find that consistent outperformance is elusive.
- If performance claims seem too good to be true, they probably are. Over longer horizons, every strategy will encounter both favorable and unfavorable market conditions.
The number of strategic beta exchange-traded products (ETPs) has exploded in recent years. According to Morningstar, in 2018 nearly 1,500 strategic beta ETPs, representing approximately $800 billion in total assets, existed worldwide, with 132 new products coming to market in that year alone.1 More importantly, the total number of smart beta indices published by index providers and available for tracking has grown even faster, with institutional investors executing these strategies in separate accounts to the tune of billions of dollars.
Investors’ appetite for smart beta strategies is not abating. A recent FTSE Russell smart beta survey revealed that “78% of survey respondents have implemented, are currently evaluating, or plan to evaluate a smart beta strategy” (FTSE Russell, 2019, p. 4). With so much competition and such large numbers of assets at stake, the need for smart beta providers to differentiate their strategies as being “better” is becoming increasingly more intense.
So, what is the means to proving a strategy is better? A performance backtest, of course! Regardless of whether delivery is through an ETP wrapper or an institutional account, nearly all smart beta strategies are touted as having impressive (albeit backtested) excess returns. Many of these backtests have only 10 to 15 years of history and cover a very limited set of market environments. The pressure to generate ever-better backtests may also be a response to recently disappointing results of many smart beta strategies.2
The confluence of shorter time horizons, increased competition, and recent underperformance may well have led to smart beta backtests “jumping the shark,” that is, reporting utterly implausible return outcomes. Jumping the shark refers to the outlandish storyline of the 1970s American television sitcom Happy Days that followed one of the show’s main characters, Fonzie, on a California vacation as he accepted a dare to jump over a caged man-eating shark while on water skis. Now viewed as a pejorative term, jumping the shark refers to outrageous plot devices designed to generate attention.
It is our understanding that at least one factor strategy provider is claiming a 4% annualized excess return over the last 10 years, without incurring a single calendar year of underperformance versus the cap-weighted index. This and similar claims by other providers, like the Fonz’s biker-turned-stunt-skier routine, definitely get attention, but are they plausible? In this article, we examine live US mutual fund track records over the last 40 years to gain an intuition into how much outperformance is reasonable for the most superior smart beta strategy (or for that matter, active manager). We call this return “plausible outperformance.”3
Plausible Outperformance: The Mutual Fund Experience
At Research Affiliates, we believe that backtests can deliver valuable insights into the behavior of rules-based strategies, especially those targeting well-established and accepted factor exposures. That said, we also know that backtesting is a great way to maximize positive error through the combination of selection bias and data mining. Care and attention need to be applied to the construction of backtests and the interpretation of their results to establish realistic forward-looking return expectations (Harvey and Liu, 2015). So, assuming the backtest has not been data mined too aggressively, what are the levels of excess return we can plausibly expect to earn in the future with live assets over a realistic holding period?
To gain an understanding of realistic levels of outperformance achieved over various investor holding periods, we surveyed US mutual fund data over the past 40 years from January 1, 1979, through December 31, 2018. We used the Morningstar survivorship bias–free database of US equity open-end mutual funds, excluding index funds, which survived for at least one calendar year. For funds with multiple share classes, we chose the share class with the longest history. The total sample includes 4,463 funds. We simply measured the after-fee returns of each fund relative to the S&P 500 Index in order to gauge each fund’s ability to outperform the market.
We chose to measure each fund relative to the S&P 500 rather than against the fund’s stated benchmark or Morningstar-assigned benchmark. We did this for the following reasons:
- First, we want to use the same benchmark used in the shark-jumping claim.
- Second, the goal of our research is to understand the historical ability of funds to beat the market rather than to beat their benchmark.
- Third, we make the assumption that investors in US equity mutual funds primarily care about beating the S&P 500 and they have different beliefs as to the best way to do this: choosing to invest in small-cap funds, mid-cap funds, or growth/value (or other factor)-tilted funds within any size category.
- Fourth, fund managers have discretion as to what they list as their benchmark in their prospectus and they could be motivated to pick an easy benchmark to beat.
- Finally, measuring each fund against the S&P 500 allows our results to have a clean and easy interpretation.
Because the newer methods of strategy construction are better than what fund managers had at their disposal in the past, some may very well counter that using historical mutual fund returns to understand the plausible outperformance of smart beta indices is an apples-to-oranges proxy. We concur that history is full of innovations, that often the new methods are better, and that hundreds of billions of dollars have poured into these new methods. No guarantee exists, however, that today’s new methods will perform better than the best strategies in the past and indeed many won’t add value at all.
In fact, many smart beta strategies replicate successful strategies of active managers. For example, Frazzini, Kabiller, and Pederson (2018) found that nearly all of Warren Buffet’s public stock performance at Berkshire Hathaway can be explained by exposure to the quality, value, and low beta factors.4 As Towers Watson (2013) (now Willis Towers Watson, the consulting firm that coined the term smart beta) stated: “Smart beta is simply about trying to identify good investment ideas that can be structured better…. Smart beta strategies should be simple, low cost, transparent, and systematic.”
Smart beta proponents may claim the structured-better element will lead to better performance than what mutual funds have historically earned, at least partially through lower fees. Smart beta portfolio management is closer to indexed portfolio management and therefore can (and arguably should) be priced closer to index fund fees.5 In contrast, active mutual fund managers need to pay for analysts and portfolio managers and, of course, pass along this overhead cost through higher management fees.
Effective smart beta strategies likewise require much skilled analysis, but are able to bypass the high costs of forecasting and of ongoing fundamental monitoring. It is reasonable to expect lower management fees for these strategies because of their rules-based implementation. A well-designed smart beta strategy should have low transaction costs given that it typically has broad stock holdings and minimal turnover. Unfortunately, the expectation of lower transaction costs is not always realized. We assert the true skill in smart beta investing is in balancing the intended factor exposure while minimizing transaction costs and other forms of implementation shortfall (Israel, Jiang, and Ross, 2017, and Chow et al., 2011).
Nonetheless, analyzing the historical performance of actively managed mutual funds in order to understand the distribution of future plausible live performance offers two advantages versus simply looking at backtested results. First, historical fund returns are net of transaction costs and management fees, which most smart beta backtests blissfully ignore. Although smart beta strategies are likely to have lower management fees and, for well-designed strategies, lower transaction costs, these costs aren’t zero—yet this is exactly what is implied in backtests!
Second, mutual fund managers do not need to publish their methodologies. A mutual fund manager can find a profitable anomaly and not disclose it. In contrast, smart beta index strategies typically have a rulebook that explains the method used for stock selection, the rebalancing dates, and so forth. Savvy investors therefore can determine ahead of time which stocks these strategies will trade, when they will trade, and in what quantities.
McLean and Pontiff (2016) examined the out-of-sample performance of 97 equity factor strategies and found that post-publication premiums declined by an average of 32% versus the published figure. Our own work covering the most widely used smart beta strategies found that out-of-sample excess return following publication falls far short of the in-sample published results (Li and West, 2017, and West and Hsu, 2018).6
Considering these various points, we believe the pros easily outweigh the cons of extrapolating the mutual fund experience to smart beta backtests. Again, our intention is to look at plausible outperformance over realistic investor holding periods.
We analyzed the frequency of mutual fund outperformance over 1-, 2-, 3-, 5-, and 10-year periods at various levels of outperformance, ranging from just barely beating the market (> 0%) to beating it handedly (> 5% annualized). We are able to make two major observations: 1) most mutual funds underperform the market regardless of time period, and 2) the win rates of star performers subsequently collapse.
Most Mutual Funds Underperform the Market
Our first observation is self-evident and well-trodden territory. Most mutual funds underperform the market regardless of time period. Only 43.28% of the 1-calendar-year fund periods beat the market (i.e., a 43.28% win rate) based on 53,127 observations. That figure drops slightly when we extend the analysis to 3-calendar-year periods; only 41.61% of the 44,568 observations are winners, earning an average annualized excess return of −0.52% a year. When the performance period is extended to 10 years, the win rate actually improves slightly to 46.19% with only a −0.09% a year shortfall. This is largely due, however, to survivorship bias creeping into the results.