Measuring the “Skill” of Index Portfolios

By Jason Hsu Vitali Kalesnik

FEBRUARY 2014 Read Time: 10 min

Key Points

  • Expected return is the return of an average stock plus the return due to the investor’s skill.
  • Traditional indexing does not come up to the expected return of uninformed investors.
  • Active managers evaluated against cap-weighted indices have to surmount the benchmark’s return drag before they can add value.

The gently satirical variety show “A Prairie Home Companion,” long running on public radio in the United States, has made the fictional town of Lake Wobegon famous. The residents of Lake Wobegon believe that in their town “all the women are strong, all the men are good looking, and all the children are above average.” But the townsfolk are outlandishly optimistic. Clearly, we recognize that everyone can’t be above average; indeed, we would generally suspect naiveté when met with such unrealistic positivity. Nonetheless, when it comes to investing, most of us live in Lake Wobegon; we believe we can select winning stocks and top quartile managers. Most of us think we are above average.1  Indeed, the irony is that it is the select few who aspire to achieve middling results—the 20% of investors who index—who achieve consistent net-of-fees outperformance against Lake Wobegon investors.

When conducting manager searches, we hope to find skilled managers who know which stocks are misvalued and can translate this information into meaningful outperformance. Inevitably, however, when choosing money managers about half of us will be below average. Those who accept that it is hard to find managers who consistently outperform opt for index investing. A passive index does not have the informational advantage that an active manager might possess, but at least the investors will earn a market return without the high cost of active management.

In this issue, we are uninterested in rehashing the old topic of the benefit of index investing. Instead we question one of the key tenets of index investing.

Is it, in fact, true that traditional index investors have earned returns that can be reasonably earned by uninformed investors? Is it true that by investing in the cap-weighted index we receive the average return associated with an uninformed selection of stocks? Surprisingly, we find that traditional indexing—and active managers who hug the benchmark (closet indexers)—deliver below-average returns.2 They systematically deliver returns that are inferior to the expected returns of uninformed investors.

Expected Returns
Uninformed investors cannot distinguish between good stocks and bad ones, and their portfolio returns will deviate from the return of an average stock in a random manner. In comparison, informed investors invest more in stocks that are likely to perform well, and avoid those that are likely to perform worse. There is an intuitive mathematical identity which captures this idea and decomposes any portfolio’s expected return into the sum of two components:

Covariance measures how much two variables move together: If the strategy assigns more weight to better-performing stocks, covariance is positive; if the strategy assigns more weight to worse-performing stocks, it is negative; and if the stock assignment is random, this term is zero. The covariance of stock weights and returns reflects how informed the investor is. In other words, it captures the investor’s skill.

The relationship above is widely used in the financial literature to estimate the skill of managers.4 The expected return of a portfolio is the sum of the return for an average stock and the return due to the investor’s skill. The first component is quite intuitive. The return of an average stock is the expected return of the uninformed investor who selects stocks haphazardly. An informed investor would invest more in stocks expected to have higher returns and less in stocks expected to have lower returns. The better the investor is able to predict stocks’ relative returns, and the more conviction with which these views are expressed in the portfolio, the higher the portfolio return.

The “Skill” of an Index Portfolio
Recognizing that they are unlikely to select skilled money managers, many market participants choose index-based alternatives which presumably deliver the return that would be similar to a portfolio of stocks chosen by an uninformed investor; that is, the return of the average stock. Choosing an index spares investors from paying high fees to unskilled managers.

The return decomposition that we introduced in the previous section can help us determine whether index investing offers expected returns equivalent to the returns that uninformed investors can, in principle, achieve. We can simulate index weights and check the level of skill represented by the covariance between the index constituents’ weights and subsequent returns. If the index does indeed deliver a return matching that of an uninformed manager, then the covariance term showing skill will be zero. We will examine the following five options, four smart beta indices and one traditional cap-weighted index:

  • An equal-weighted strategy;
  • The average of 100 portfolios of 30 randomly selected stocks;
  • A minimum variance strategy;
  • A fundamentally weighted strategy, where the weights are proportional to company size as represented by financial accounting measures; and
  • A cap-weighted index.

Some of these options have been around for many years and have large amounts of assets under management; others are younger and only now gaining widespread acceptance. The first four indices fall into the category of smart betas, strategy indices which do not base weights on price-related measures such as capitalization.5 Ironically the category of smart beta also includes the portfolios of randomly selected stocks. Table 1 shows the return decomposition for the five strategies.

Most of the strategies examined here do not exhibit skill at a level that is statistically different from zero. This is what one would expect of rules-based strategies which trade only on publicly available information. The one exception is the cap-weighted index. It has a statistically significant negative covariance term. In the U.S. market, over the period 1962 through 2012, the negative return translates into approximately 200 bps of drag per year.

Cap-weighted indices are the most commonly used option for passive investing, yet the expected return of a cap-weighted index is lower than that of an approach in which stocks are randomly selected and weighted. That’s the news from Lake Wobegon. But what accounts for this outcome?

Benchmark Choice and Performance Timeframe
Listeners perceive that the residents of Lake Wobegon are, in many respects, limited, quirky, and decidedly below average. But the residents see themselves, and all their children, as better than mediocre. People are prone to avoid facing reality by spinning compelling but nonetheless fanciful narratives.6 When it comes to investment management, some investors believe, “We are well aware of the inferiority of the cap-weighted index approach. That is exactly why we use active management!” Are they right? Does opting for active management automatically free us from the curse of capitalization weighting?

In theory, when we hire active managers, we want them to buy stocks which are more likely to perform well and avoid stocks whose prospective performance is poor. In principle we expect the active portfolio to deviate substantially from the benchmark. In other words, to deviate with conviction.

In practice, however, we fire active managers if they underperform the cap-weighted benchmark over a three-year period. Our trigger-happy behavior creates incentives for managers to avoid trades with long payoff periods and the corresponding risk of short-term underperformance. For rational managers with sizeable assets under management, hugging the benchmark in pursuit of a lower tracking error than the competing managers is one effective strategy to avoid outsized underperformance and the associated termination.

In summary, regardless of what we say, what we do incentivizes managers to hew to the cap-weighted benchmark and not to enter profitable trades that induce higher tracking error and/or are risky in the short term. This means that the average active manager will have a portfolio very similar to the cap-weighted index, and, accordingly, he or she will experience the same return drag. The better active managers will add value relative to the cap-weighted benchmark, but their starting point means that they have to overcome the return drag before they can begin to exhibit outperformance that can match the results of naïve weighting schemes like equal or even random weighting. In contrast, any portfolio which employs mechanical rebalancing rules to create target weights that are unrelated to prices will give investors a head start against active managers.

Suppose you want to hire an active manager but avoid the return drag associated with the cap-weighted benchmark. What should you do? The answer might be simpler than you think. Start with incentives. Tell your active managers you don’t take tracking error into account. Better yet, tell your active managers you do take tracking error into account and expect it to be large. Managers whose tracking error is small aren’t active; they don’t have their own opinions, or don’t hold them strongly. Similarly, lengthening the evaluation period—and sticking to it—will enable managers to take advantage of value bets with long-term payoffs.

The skill of the informed fund manager is investing in stocks that are likely to perform better than average and avoiding those that are likely to perform worse. Investors and consultants devote considerable resources to deciding whether or not a manager is skillful. Ironically, though, when it comes to passive investing, they completely abandon the framework of performance evaluation; they simply accept on faith that the standard cap-weighted benchmark is the optimal “uninformed” portfolio. But what if they were to examine the “management” skill of index portfolios in the same way they examine the results of active managers?

Our analysis shows that cap-weighted indices amazingly have negative skill as measured by the standard analytics used to judge active managers. Cap-weighting systematically allocates larger weights to overpriced stocks and smaller weights to underpriced stocks. Active managers benchmarked to cap-weighted indices have to overcome this return drag before they can start adding value. The same analysis applied to smart beta indices reveals that they are not plagued by the same problem. Smart beta investing, an alternative to negatively skilled passive management, can also complement unencumbered active managers.

Featured Tags

Learn More About the Authors

Partner & Senior Advisor
Director, Head of Equity Research, Research Affiliates Global Advisors (Europe) Limited

1. That experts tend to overestimate their ability to make predictions and estimates is well established. Kahneman (2011), in particular, discusses the commonplace trait of “overconfident optimism” among experts (see especially pages 261–265). Dunning and Kruger (1999) present experimental evidence that inexpert individuals are doubly encumbered: they overestimate their ability and lack the metacognitive ability to realize it. Further research on the Dunning–Kruger effect (Ehrlinger et al., 2008) indicates that, in viewing their performance too favorably, the inexpert do not underestimate others’ abilities; rather, they fail to recognize their own errors.
2. By definition, broad cap-weighted indices match the return of the market portfolio; that is, the aggregate return of all the stocks held at market weights. In other words, traditional indices deliver the market-weighted average return. They do not, however, deliver the return of the average asset.
3. Equation (1) can be derived trivially by noting the definition of covariance:  cov[a,b] = E[ab] – E[a]E[b].

Ex ante the weights wi and returns ri are unknown and represent IID random variables. It may be counterintuitive to think of portfolio weights as random variables. A useful analogy may be to try to compare the performance of different portfolios ten years from now. At the present moment both future weights and future returns represent random variables. All we know is the statistical distribution of weights and returns in different portfolios. In the beginning of the investment period, at time t1, the weights of the portfolio come to be specified. The returns are realized in the end of the investment period, at time t2. The expectation and covariance are both cross sectional operators here. For the last step in the equation we used the fact that ex ante E[wi] = 1⁄n.
4. See, for instance, Grinblatt and Titman (1993).
5. The designation “smart beta” was coined by Towers Watson consultants. The term is not meant to be derogatory to providers of traditional indices whose returns are driven by market beta. The originators’ intent was to suggest that investors would benefit from strategies that tap multiple sources of return.
6. Tuckett and Taffler (2012) report that storytelling is one of the most important ways fund managers deal with the anxiety of making investment decisions under conditions of uncertainty. In particular, managers tell stories to explain their successes and failures in ways that tend to preserve or enhance their self-esteem. The authors do not say that fund managers confabulate, but they suggest that the plausibility and coherence of their narratives may be valued more than their accuracy.

Ehrlinger, Joyce, Kerri Johnson, Matthew Banner, David Dunning, and Justin Kruger. 2008. “Why the Unskilled are Unaware: Further Explorations of (Absent) Self-Insight Among the Incompetent.” Organizational Behavior and Human Decision Processes, vol. 105, no. 1 (January):98–121.

Grinblatt, Mark, and Sheridan Titman. 1993. “Performance Measurement without Benchmarks: An Examination of Mutual Fund Returns.” Journal of Business, vol. 66, no. 1 (January):47–68.

Kahneman, Daniel. 2011. Thinking, Fast and Slow. New York: Farrar, Straus, and Giroux.

Kruger, Justin, and David Dunning. 1999. “Unskilled and Unaware of It: How Difficulties in Recognizing One’s Own Incompetence Lead to Inflated Self-Assessments.” Journal of Personality and Social Psychology, vol. 77, no. 6 (June):1121–1134.

Tuckett, David, and Richard J. Taffler. 2012. Fund Management: An Emotional Finance Perspective. Charlottesville, VA: Research Foundation of CFA Institute.