Callan is committed to original, path-breaking research to help educate the financial industry and serve our clients. That commitment began with the firm’s founding and continues today with a research operation that spans every major asset class and investing strategy.
One of the latest examples of our research stemmed from an attempt by Greg Allen and Ivan “Butch” Cliff to help institutional investors with a challenge they face in evaluating managers: “survivorship bias.” This refers to the fact that typically the results over a period of time for a universe only reflect the outcome for the “survivors,” not the total universe of members at the start of the measurement period. In other words, it is the logical error of focusing only on the things that survived a process while ignoring the things that did not.
This bias has real-world implications outside of investing. In a famous example, World War II aircraft designers decided to study the planes that returned from bombing missions over the European Theater to improve survival rates, by examining the flak damage and bullet holes in those aircraft. At first they planned to reinforce those areas. But they realized they needed to add armor in the places that were not hit in the survivors, especially the engines, on the theory that those were the places where damage brought down the aircraft that did not return from missions.
For investors survivorship bias is important because it skews the analysis of investment managers, resulting in both an overly pessimistic view of the skill of managers whose strategies have survived over time and an overly optimistic view of the effectiveness of a class of investment strategies.
Solving the problem of accounting for the track records of those managers who did not endure over the entire time period is no easy mathematical task. Working with Walter Meerschaert, our manager of information technology, Allen and Cliff developed a technique they call “SUBICO,” for SUrvivorship BIas COrrection. Using sophisticated mathematics, it “fills in” the records of the missing managers. And it avoids the problems of some simpler techniques by correcting for bias across the full distribution of results (i.e., from the 1st to the 99th percentile) and can be applied to other return metrics, such as the Sharpe ratio and correlation. Other techniques may provide a rough sense of the median return but cannot offer results allowing that range of evaluation tools.
FYI
Survivorship bias: The logical error stemming from the results over time for a universe only reflecting the outcome for the “survivors,” not all the members at the start.
