The Reality of Models
The Reality of Models
February 2017
Checking the weather? Looking at a map of the world to plan your next vacation? Guess what—you’re using a model. While models can be useful for gaining insights that can help us make good decisions, they are simplifications of reality.
One example of a model is a weather forecast. Using data on current and past weather conditions, a meteorologist makes a number of assumptions and attempts to approximate what the weather will be in the future. This model may help you decide if you should bring an umbrella when you leave the house in the morning. However, as anyone who has been caught without an umbrella in an unexpected rain shower knows, reality often behaves differently than a model predicts it will.
In investment management, models are used by investors to gain insights that can help inform investment decisions. Financial researchers are frequently looking for new models to help answer questions like “What drives returns?” These models are often touted as being complex and sophisticated and incite debates about who has a “better” model. Investors who are evaluating investment strategies can benefit from understanding that the reality of markets, just like the weather, cannot be fully explained by any model. Hence, investors should be wary of any approach that requires a high degree of trust in a model alone.
THE MODEL, THE USER, AND THE APPLICATION
Just like with the weather forecasts, investment models rely on different inputs. Instead of things like barometric pressure or wind conditions, investment models may look at variables like the expected return or volatility of different securities. For example, using these sorts of inputs, one type of investment model may recommend an “optimal” mix of securities based on how these characteristics are expected to interact with one another over time. Users should be cautious though. The saying “garbage in, garbage out” applies to models and their inputs. In other words, a model’s output can only be as good as its input. Poor assumptions can lead to poor recommendations. However, even with sound underlying assumptions, a user who places too much faith in inherently imprecise inputs can still be exposed to extreme outcomes.
Nobel laureate Robert Merton offered some useful insights on this topic in an interview with David Booth, Chairman and Co-CEO of Dimensional Fund Advisors. “You’ll often hear people say, during the [financial] crisis or something, ‘There were bad models and good models.’ And someone will say, ‘Is yours a good model?’ That sounds like a good question, a reasonable question. But, actually, it isn’t really well-posed. You need a triplet: a model, the user of the model, and its application. You cannot judge a model in the abstract.” (For a video of the interview, please click the following link: Models Interview.)
We believe bringing financial research to life requires presence of mind on behalf of the user and awareness of a model’s limitations in order to identify when and how it is appropriate to apply that model. No model is a perfect representation of reality. Instead of asking “Is this model true or false?” (to which the answer is always false), it is better to ask, “How does this model help me better understand the world?” and “In what ways can the model be wrong?”
“THE EARTH IS ROUND,” INVESTING, AND THE JUDGMENT GAP
Consider the shape of the earth. One simple model describes the earth as a round sphere. While this is a good approximation, it is not completely accurate. In reality, the earth is an imperfect oblate spheroid—fatter at the equator and more squashed at the poles than a perfect sphere. Additionally, the surface of the planet is varied and changing: There are mountains, rivers, and valleys—it is not perfectly smooth. So how should we judge the model of “the earth is round”? For a parent teaching their child about the solar system or for a manufacturer of globes, assuming the earth is a perfect sphere is likely a fine application of the model. For a geologist studying sea levels or NASA engineers launching an object into space, it is likely a poor model. The difference lies in the user of the model and its application.
In investing, one should pay similar attention to the details of user and application when a model informs real-world investment decisions. For example, for investors in public markets, the efficient market hypothesis (EMH) is a useful model stating that asset prices reflect all available information. This model helps inform investors that they can rely on prices and that it is not worth trying to outguess the ones set collectively by millions of market participants. This insight has been confirmed by numerous studies on investment manager performance.[1] In applying this model to real-world investment solutions, however, there are several nuances that a user must understand in order to bridge the gap between theory and practice. Even if prices quickly reflect information, one should not assume that the EMH protects investors from making investment mistakes. Rigorous attention must be paid to trading costs and to avoid trading in situations when there may be asymmetric information or illiquidity that might disadvantage investors. To quote Professor Merton again, successful use of a model is “10% inspiration and 90% perspiration.” In other words, having a good idea is just the beginning. Most of the effort is in implementing the idea and making it work. In the end, there is a difference between blindly following a model and using it judiciously to guide your decisions. By employing sound judgment and thoughtful implementation, we believe it is more likely that outcomes will be consistent with a user’s expectations.