Book note: Fooled by Randomness

Preface

Quality of one’s knowledge or skill may not be as good as they claim. The core probabilistic thinking is to consider that alternative outcomes could have taken place and that the world could have been a different place. Skills matter less on a random environment (such as a stock market) VS other fields such as dentistry. A proper attire, showing up on time, working hard… are necessary and counts toward success but does not cause success. For example, does your trip to go and buy a lottery ticket cause you winning? No, but you still had to go out and buy the ticket to win. Professions such as dentistry are different because it is not as random (i.e. one had to possess a certain level of intelligence to go to college, get accepted into a dentistry school, pass the exam, practice for a long time… all of which shows that the dentist isn’t playing the game of randomness). → that is why you see outsized profits from Wall Street and startup world but not in dentistry because it’s non-random outcome allows only up to a certain profit. Logic does not require empirical verification. It is a mistake to use statistics without logic but logic does not require statistics.

Part 1- Solon’s warning

1. If you are so rich, why aren’t you so smart? Mild success can be explainable by skills and labour. Wild success is attributable to variance (luck and randomness play a large role). Lucky fools do not recognise that they are lucky — they act as if they deserve the money. To claim true skill, consistency matters (example on why dentists are truly rich because they can re-produce the wealth over multiple tries).
2. A bizarre accounting method. A performance cannot be judged by a result but by costs of the alternative (an alternative history). Russian roulette example — if a 25-year-old plays the game once a year, there is a very slim chance that he will live to see his 50th birthday. However, if there are a million 25-year-olds who play the game, it is almost certain that there will be an extremely wealthy few. A $10m earned by playing the Russian roulette or through hard work in dentistry are identical in accounting perspective. But if you take the alternative history into an account, we can readily recognise that the one who played Russian roulette was ‘extremely lucky’ whereas the dentist had more skill than luck. In reality, random events are worse than the Russian roulette in three senses: a) the fatal bullet is delivered much more infrequently (i.e. we may see a bull market for over a decade), b) the odd of risks are not so visible and c) there is an ingratitude factor in warning people about something abstract (something that did not happen). Rational thinking has very little to do with risk avoidance — in other words, decisions tend to be made by emotions and we rationalise with logic. For example, when asked to buy travel insurance that pays for death in any scenario, most people will decline. On the other hand, if presented as insurance that pays in the scenario of a terrorist attack, more people will pay the premium.
3. A mathematical meditation on history. Hindsight bias — when you look at a result in the past, it always look deterministic. The truth is, a mistake is not something to be determined after the fact, but in the light of the information until that point. Those who are good at predicting the past (i.e. explaining the past event by fitting in a logic) will think that they are good at predicting the future too. The truth is that we live in a world where truly important events (not noise) are not predictable. Mean reversion applies to our lives too — lucky idiots will eventually revert to its long term properties (i.e. lottery winners, gamblers…etc) whereas a skilled one who was unlucky will eventually rise up commensurating to its skill level. (ergodicity). The wise man listens to meaning, the fool only gets the noise. (i.e. listening to the motivational speeches…etc). Information and noise are different. Information requires time to be substantiated (for instance, to achieve a 15% return with 10% variability). If this same trait is scaled at looking at portfolio performance on a daily basis, you would achieve success 54% of the time VS 93% of the time if you looked at the portfolio performance once a year. In other words, over a short time increment, you only observe the variability, not the returns (as if you are zoomed into the Michelangelo’s David sculpture and notice the little chips and holes rather than looking at the masterpiece when zoomed out. This is why reading/focusing too much on daily news will emotionally drain you and not give you much meaningful information).
4. Survival of the least fit — can evolution be fooled by randomness? There is a difference between wealth reached from above and wealth reached from below. Always have a margin of error baked in your investment strategy and be aware not to be married to a single strategy. Times change and things change.
5. Skewness and asymmetry — The frequency or probability of the loss is totally irrelevant. It needs to be judged in connection with the magnitude of the outcome. In other words, think in terms of an expected outcome. Therefore, profiting from rare events could make sense because those rare events typically present an outsized profit. Also, this is another reason why selling options may be a poor strategy (despite the fact that 90% of the options lapse) because one event could wipe out all premiums earned. Also, excluding an outlier event may be a mistake because it could have an outsized impact on the outcome. History teaches us that things that never happened before do happen. Avoid relying on too much of naïve empiricism that consists of learning from casual historical facts, disregarding outliers. Rare events are rare because they are unexpected.
6. The problem of induction — Empirical observation can lead you astray. No amount of observations of white swans can allow the inference that all swans are white, but the observation of a single black swan is sufficient to refute that conclusion. In conclusion, extreme empiricism, drawing a conclusion from competitive nature (my house is bigger than yours), in an absence of logic, can be explosive because one bad event could wipe all out. A theory is never right — we will never know if ALL swans are white.
   A theory cannot be verified — Using basketball as an example, past data has a lot of good in it, but it is the bad side that is bad. Staying open to positiveness is preferable (i.e. if you believe in God and if God does exist, you win. If God doesn’t exist, you lose nothing. So a position to be in is to believe in God.) Similarly, use statistics to make bets, not to manage risk because empirical data’s downside is in its unknowns which makes it ill-equipped for risk management. Instead, make bets using some statistical data and manage risk using stop-loss (a pre-determined exit point or a built-in margin of error).

Part 2 — Monkeys on typewriters

How much can past performance be relevant in forecasting future performance? Good past performance may indicate someone’s skill but it is weakened by two factors 1) the random nature of the profession/industry and 2) the number of participants (sample size). The greater the sample size, the greater the likelihood of one of them performing in a stellar manner just by luck. Therefore, one’s performance itself has little information and needs to be viewed in relation to the initial size of the population.

1. Too many millionaires next door. A wife who compares her husband income to that of their neighbour’s in 5th Avenue is miscomputing the probabilities. In other words, what she is seeing is a mere comparative failure. This creates a social treadmill effect where you become rich, move into a rich neighbourhood and then become poor again. The theory on delaying consumption and accumulation of wealth will make one a millionaire is weakened by two things 1) survivorship bias — examples exclude those who accumulated the wrong thing (i.e. people who accumulated stocks that went bust or who accumulated Nikkei?) and 2) the problem of induction — the successful accumulators did so during a bull market and assumes such bull market will continue.
2. It is easier to buy and sell than fry an egg in real life, the larger the deviation from the norm, the larger the probability of it coming from luck rather than skill. Nobody accepts randomness in one’s success, only for failures. Data snooping — fitting the rule on the data. The more I try, the more likely I will find something that works. (i.e. we could probably find a data set that shows a correlation between S&P500 return and weather in Mongolia). Even worse, those ‘rules’ are plenty enough that some of the ones we use apply today may be a victim of survivorship bias. Real randomness does not and need not look random!
3. Loser takes all — on the nonlinearities of life. Life is unfair in a nonlinear way — a small advantage can make a disproportionate payoff or no advantage at all could lead to a bonanza. (in making a sandcastle, the final addition of a grain of sand results in a disproportionate result of toppling the entire castle). Our brain is not cut out for nonlinearities. It and our emotion are designed for linear causality (i.e. 1 hour of work should produce a proportionate amount of result, applied same to everyone. This is not true as some people’s work such as Bill Gates, produced a disproportionate amount of result). In reality, success is rarely a positive linear progression but usually a nonlinear step. Unfortunately, most people lack mental stamina and give up before the rewards. The nonlinearity is behind more things than one might suspect, driving bipolarity of our society: better to have a few enthusiastic followers than a hundred mediocre.
4. Randomness and our mind: we are probability blind. Bounded rationality — humans can’t optimise every single decision because otherwise, we will be spending a disproportionate amount of time to complete a task. Therefore, we have learned to ‘suffice’ and therefore we are inevitably bounded by rationality. A test of the disease has 5% false positives and the disease strikes 1/1,000 of the population. People are tested at random. If a patient is diagnosed positive at the test, then what is the probability the patient being stricken with the disease? It is 2% because 999 people do not have the disease and 5% of them (50 of them) would have been given a false positive. So, 1/51 is 2%. Most people (even doctors), when given this question, answer 95%. In reality, there is rarely a single causality, and it is challenging to isolate one. This is called multivariate analysis. Therefore, it is better to make a decision based on future variance rather than trying to pinpoint an exact future state. For instance, if I am travelling to Barcelona, knowing the temperature will be 20 degrees is weaker than knowing that the temperature will be 20 degrees with +/- 10 degrees. In the case of the latter, I am more certain to bring a jacket (or make a more confident bet).

Postscript

Three afterthoughts

1. The inverse skills problem — the higher up the corporate ladder the lower the evidence of contribution. There are so many CEOs making decisions, some of them are bound to make a good decision. A link between a CEO’s decision and the result of the company is tenuous because the CEO’s decision is one of the thousands made to make something work. In this regard, CEOs (unlike entrepreneurs), are empty suits. Unfortunately, shareholders bare the risk of such empty suits and executives get the rewards. This is investor’s and therefore a corporate governance issue. In short, shareholders are fooled by randomness.
2. On some additional benefits of randomness. A little bit of uncertainty allows us to be a satisficer than a maximiser, giving us more wriggle room in our life and ultimately making us happier. (catching subway in NYC example where a strict time table results in the unhappiness of a diner). For example, writing is much more fun when you are not bound by a deadline or the length it has to be. The unpredictability helps creativity. (another is would you like to know the exact date/time of your death?) Unpredictability is a strong deterrent (in case of conflicts between nations, personal life…etc because no one can guess what your next move might be).