June 24, 2017

Hedge Funds Get Worthless Research

The Crystal Ball by John William Waterhouse

Nate Silver's new book questions financial research's predictive powers.

If hedge funds get worthless research, why would they pay so much for it? Hedge funds are brokerage houses’ best clients. SAC Capital, for instance, is known to pay some of the highest trading and brokerage fees. In return, they expect the best equity research on the street. Goldman Sachs and other prime brokers often provide a high level of personal service and have even been accused of providing top clients with new information first.

Nate Silver asserts that all this predictive research is worthless in his new book, “Signal and the Noise: Why So Many Predictions Fail — but Some Don’t”. Leonard Mlodinow sums up Silver’s book on meritless professional prediction in The New York Times.

Studies show that from the stock pickers on Wall Street to the political pundits on our news channels, predictions offered with great certainty and voluminous justification prove, when evaluated later, to have had no predictive power at all.

Today the data we have available to make predictions has grown almost unimaginably large: it represents 2.5 quintillion bytes of data each day, Mr. Silver tells us, enough zeros and ones to fill a billion books of 10 million pages each. Our ability to tease the signal from the noise has not grown nearly as fast. As a result, we have plenty of data but lack the ability to extract truth from it and to build models that accurately predict the future that data portends.

Healthily peppered throughout the book are answers to its subtitle, “Why So Many Predictions Fail — but Some Don’t”: we are fooled into thinking that random patterns are meaningful; we build models that are far more sensitive to our initial assumptions than we realize; we make approximations that are cruder than we realize; we focus on what is easiest to measure rather than on what is important; we are overconfident; we build models that rely too heavily on statistics, without enough theoretical understanding; and we unconsciously let biases based on expectation or self-interest affect our analysis.

Regarding why models do succeed, Mr. Silver provides just bits of advice (other than to avoid the failings listed above). Mostly he stresses an approach to statistics named after the British mathematician Thomas Bayes, who created a theory of how to adjust a subjective degree of belief rationally when new evidence presents itself.

Suppose that after reading a review, you initially believe that there is a 75 percent chance that you will like a certain book. Then, in a bookstore, you read the book’s first 10 pages. What, then, are the chances that you will like the book, given the additional information that you liked (or did not like) what you read? Bayes’s theory tells you how to update your initial guess in light of that new data. This may sound like an exercise that only a character in “The Big Bang Theory” would engage in, but neuroscientists have found that, on an unconscious level, our brains do naturally use Bayesian prediction.

A week later, The New York Times Book Review published another review by Noam Scheiber.

Nate Silver has lived a preposterously interesting life. In 2002, while toiling away as a lowly consultant for the accounting firm KPMG, he hatched a revolutionary method for predicting the performance of baseball players, which the Web site Baseball Prospectus subsequently acquired. The following year, he took up poker in his spare time and quit his job after winning $15,000 in six months. (His annual poker winnings soon ran into the six-figures.)

His book, “The Signal and the Noise,” is largely about evaluating predictions in a variety of fields, from finance to weather to epidemiology.

As science, this investigation is wholly satisfying. As a literary proposition, it’s a bit disappointing. It’s always more gripping to read about how we might achieve the improbable than why we can’t. “Freakonomics” read like a series of detective stories. Silver’s volume is more like an engagingly written user’s manual, with forays into topics like dynamic nonlinear systems (the guts of chaos theory) and Bayes’s theorem (a tool for figuring out how likely a particular hunch is right in light of the evidence we observe).

There are nuances in scientific and financial data — to say nothing about how we discuss the data in the context of a raging political debate — that people spend their careers assimilating.

Statistics can dazzle with their aura of authority, yet reality is relentlessly messy. Genuine understanding, as even Silver knows, is more than a numbers game.

Investors should wonder what their fees are being used for if the directed brokerage for research yields no return. Silver’s book may assert that hedge funds get worthless research. In which case, institutional investors have a business model to reconsider.

Speak Your Mind