1. Over 20 years ago, Bernstein (1995) alluded to the revolution in technology available for the task of performance measurement as he highlighted the proliferation of questionable bogeys (or benchmarks) by which investors gauge performance and underscored the fact that the difficulties in telling luck and skill apart make performance measurement a less-than-ideal pursuit. The reader will note I echo some of his themes in this article. Although I do not discuss benchmark selection, I refer the reader to Bernstein’s apt use of analogy in capturing the challenge of choosing the right bogey. Drawing a parallel between benchmark selection and congressional hearings during the McCarthy era, he reminds us of the line, “Who will investigate the man who’ll investigate the man who’ll investigate me?” We miss Peter Bernstein’s insights and writings.
2. The evolution of computational and information technology has produced over the last few decades several obvious large-scale benefits for investors, beyond making performance measurement a simple task. First among them is the ability to scale up investment insights via quantitative methods. That being said, as discussed by Treussard and Arnott (2017), the dark side of this evolution in data processing has been the unleashing of careless backtesting, upon which live strategies are built. Again, replacing careful analysis and theory with mindless computation and data processing (aka outsourcing the hard work to the data) can easily lead quants astray.
3. Duke Professor and Senior Advisor to Research Affiliates Cam Harvey makes this distinction very aptly in his 2017 Presidential Address to the American Finance Association (Harvey, 2017). Imagine that a musicologist correctly distinguished 10 out of 10 pages of music as being written either by Mozart or by Haydn. Also imagine that a kindergartener calls 10 coin flips correctly. Would you be as convinced of the child’s skills as you would be of the musicologist’s? I would hope the answer is “no” because you have a prior on the skills brought to the task, as opposed to being simple luck. Priors aren’t perfect (who knows, maybe the five-year-old is a budding fortune teller…), but they are very helpful.
4. The presence of human capital in investment management should enable professionals to make better decisions relative to nonprofessionals because the professionals should have the necessary information on which to base decisions; that is, professionals should have conditional expectations that are refined relative to the unconditional expectations of lay people (Treussard, 2011).
5. Brightman, Masturzo, and Treussard (2014) articulated Research Affiliates’ most foundational investment belief: Long-horizon mean reversion is the source of the largest and most persistent active investment opportunities.
6. We encourage interested readers to spend time learning more about noise in the seminal work by Black (1986).
7. Silver (2012) made the point (in a chapter titled “How to Drown in Three Feet of Water”) that it is critically important to communicate uncertainty by being explicit about the range of reasonable deviations around a point-estimate prediction. Otherwise, it is too easy for undue precision in forecasts to turn into “being wrong” nearly all the time and causing people to react to the resulting perception of incorrectness. But oddly, confidence intervals are rarely provided, presumably because it undermines the irrational desire to believe that experts are precisely right and that uncertainty can be managed. Silver quotes Jan Hatzius, chief economist at Goldman Sachs, who said: “Why do people not give confidence intervals? Because they’re embarrassed. I think that’s the reason. People are embarrassed.” If this is the case, investment professionals with fiduciary responsibilities must put aside embarrassment and be more explicit about what they can and cannot know.
8. Another very sensible objective of performance analysis is investment-tilt analysis rather than pure performance measurement, in which we study the extent to which a strategy’s or manager’s performance can be explained by well-researched and easily attainable investment styles or factors, such as value, size, and the like. Investors are well advised not to pay “active”-level fees when styles and factors are accessible at a fraction of the cost, where active fees may be reserved for the portion of performance in excess of the factor-based returns.