Sunday, February 10, 2013

James Weatherall: Physics of Wall Street

When an arrogant would-be physicist and philosopher writes about trillions whose transfers he can't trace or understand...

I know James Weatherall primarily as a student of mine who was always too sloppy and lazy to learn the subject properly but who was also breathtakingly self-confident, ambitious, and eager to talk about the questions he considered most important.

At various points of time, he would team up with various hardcore crackpots to assault state-of-the-art theoretical physics. But his ambitions are also behind the new book that was brought to my attention by a mathematical physicist in Munich (let me know if you want your name to be here!).

If I summarize the book, in Physics of Wall Street, Weatherall claims that "physicists" invented all the modern financial tools but these tools have been known for thousands of years at the same time; advanced algebra is needed to calculate the inflation indices but almost everyone should learn those; and all knowledge of the financial markets' working boils down to the random walk.

The book must be an inconsistent mixture of incoherent babbling about everything financial that Weatherall doesn't understand but it has gotten some excited reviews and appraisals both by crackpots – such as Horgan and Smolin – as well as people with some distinguished record – Galison and Brown of Microsoft.

Edward Frenkel, a Berkeley mathematician, wrote one positive review of the book:
Don’t Let Economists and Politicians Hack Your Math (Slate)
In this short text, Frenkel focuses on the Consumer Price Index (CPI). The basic claim in the book – endorsed by Frenkel – is that the right calculation of CPI is a complicated thing, politicians may easily abuse it to achieve their goals, and everyone should learn advanced algebra so that we don't get fooled.

Well, in some respects, I agree. One may design complicated formulae to quantify inflation based on advanced maths. I have been playing with such things, too. We've been discussing some of those matters with a vice-governor of the Czech National Bank during a lunch two years ago or so.

For example, should volatile goods such as food and energy be included in the price index? Because of their large short-term fluctuations, one may justify the answer "No" in order to make the price index ("core inflation") more stable. However, when the food and energy prices keep on increasing for five years, it becomes wrong to pretend that they don't exist or they are an irrelevant short-term fluctuation. After some time, such trends become damn real. I think that the Czech central bankers – and many others – are doing a mistake when they neglect such things.

However, there are lots of other questions to be included. Should the real estate prices be included in the price index? People buy them once a life or so and their price really matters. It's not just about things' being included in a basket. It's also "whose basket we should be considering", "which goods and papers are independent enough to be included [whether 'derivatives' of a sort should be there]", and – indeed – "how to correctly account for the (sometimes radically) changing composition of the human consumption, changing consumers' tastes, and so on." Intertemporal comparisons always have lots of questions.

But what Frenkel seems to deny is that there isn't really any "only correct and unique" formula for the CPI. People may propose various formulae that may reflect their preferences concerning the spending of money and their deeper or shallower understanding of linear (and "not so linear") algebra. However, there will always be many possibilities how to calculate the CPI. When we're relying on some statements about the CPI or laws that depend on the value of the CPI, we're assuming that we roughly know how the quantity is defined. Of course that a change of the algorithm to calculate the CPI potentially means a trick how to transfer trillions of dollars or how to obey political promises that would otherwise be unrealistic. But that just shouldn't be shocking. We may always compare the estimates or projections of the CPI according to two different methodologies and figure out what it means. At the end, which of the CPI formulae is being used doesn't really matter much as long as the rule is stable or as long as those who change the formulae pay the costs and have the responsibility.

Frenkel also mentions Weatherall's claim that the calculation of the CPI is exactly equivalent to a gauge theory. As far as I can say, this meme has been copied right from crackpot Lee Smolin.

To summarize, I found Frenkel's review unusually shallow, uncritical, and kind of uninspired. That's totally different from the anti-review by Aaron Brown, a real-world achieved risk manager:
'The Physics of Wall Street': The Most Arrogant Book in the World? Part 8
The previous 7 parts of this (so far) 8-part "introduction to finances based on the debunking of Weatherall's errors" are linked to in the anti-review above. New episodes will appear in the future.

Brown considers Weatherall's book to be "perhaps the most arrogant one in the history", primarily because Weatherall claims that the modern finances were "invented by the physicists". While I am close to claims that various things were invented by the physicists, I realize that such claims are usually historically wrong or at least dependent on the definition of a "physicist".

If we adopt a highly inclusive definition of a physicist, someone who thinks quantitatively about the real world, then it's probably true that the modern finances were built by the "physicists" except that I think that Aaron Brown is a much better physicist according to this definition than James Weatherall.

(One of Weatherall's motivations is to fight against the claims that the recent downturn was "caused by the physicists-turned-traders". He wants to claim that physicists were mostly successful and we need more physics, and not less, at the Wall Street. I have mixed feelings about what he means and whether I agree. I would agree that smarter folks and traders with physics-like tools would be fine. On the other hand, I may disagree with him which ideas are right and physics-like and I think that the papers etc. that many – possibly ordinary – people are relying upon should not demand too much mathematical sophistication of the users; they should be more resilient and idiot-proof, not more sensitive.)

Let me summarize the content of Aaron Brown's anti-review. Or at least some of it.

In Part I, Brown discusses the "arrogance of the physicists" as well as the historical facts about the transition from the gold standard economy to the borderless economy based on abstract papers and derivatives. Who did it? And so on.

In Part II, we learn how incredibly narrow-minded and naive Weatherall's ideas about the "models behind finances" are. He essentially uses one stochastic model for everything – the random walk – and he claims that all the finances are about the prediction of individual prices that ultimately boil to the random walk.

This is a particular extreme viewpoint that throws most of the baby out with the bath water. I wouldn't associate this narrow-mindedness with taking a "physicist's viewpoint". In fact, Brown's attitude is much more physical. He emphasizes that certain things are predictable and follow patterns and he offers an analogy with thermodynamics – I usually like those – and one of its conclusions is that to try to predict the individual prices is, just like to try to predict positions of the individual gas molecules, completely missing the point.

Also, he questions Weatherall's brutal idealizations and declares that much of the interesting science only occurs when one considers the incompleteness of the information; the impact of the transactions on the market itself; and the price of the transactions.

In his mostly history-based and people-based treatment, Weatherall lionizes folks like Jim Simons whose trading strategies are kind of based on the random walk – they're mostly about the high-frequency trading. But despite their producing lots of profit that is kind of reliable and independent of the evolution of the economy, I would say that it would be very bad if all investors were trying to be like this. Dealing with longer timeframes – in which the random walk loses relevance – is essential for profits for many traders except for a selected groups; it must be the bulk of profits, it is the bulk historically, and one needs to admit that there's a lot of wisdom about those, too. Weatherall's approach is fully analogous to an analysis of Feynman's path integral in which everything except for the kinetic terms is dropped. (These are my comments that I formulate as criticisms of Weatherall's opinions. He would probably disagree.)

In Part III, the roulette wheel plays an important role. Although Weatherall apparently tries to dismiss its importance, Brown presents it as the ultimate toy model for the modern traders. Brown says that there typically exists an algorithm to make profit due to a "complementarity" in the casinos. The casino owners either guarantee that the different numbers are uniform. But to achieve so, they must make the marble behave rather predictably and deterministically. Or they throw it non-deterministically but then there will be patterns and non-uniform probabilities of different outcomes. In either way, one will be able to make profit.

This presentation by Brown is very interesting but it betrays the fact that he is mainly a practitioner attempting to make a profit. There are lots of important things in the financial world that behave according to some rules even though the rules don't allow you to make much profit. What I want to say that proper economics can't be just about the "imperfections of the prices" but also about the right behavior of the "perfect markets". In my eyes, Brown seems to downplay the latter.

However, he realistically argues that the modern financial methods weren't invented by physicists – theoreticians – but rather by "engineers" who made many trials and errors. He mentions the construction of suspension bridges as an analogy. I think that Brown is right but I would still insist on the claim that to make a profit is something entirely different than to properly understand what's going on. Many people have made a profit because of some coincidences and good luck, thanks to some semi-legal and semi-ethical if not unethical or illegal deeds, or thanks to some understanding of a very special pattern that was relevant once and that doesn't imply that they understand the bulk of the financial world and its rules. One must be careful not to declare "lucky engineers" as the ultimate scientists understanding the truth.

In Part IV, we learn much more about the ways to profit from the roulette wheel – a toy model of the financial world. Again, we learn that either deterministic predictions or non-uniform probability distributions (patterns) yield methods to make a profit.

Brown criticizes Weatherall's excessive focus on the logarithm of prices which is a technicality (although a natural one) and emphasizes that much of the science is about the information content of the observed or predicted price movements and this content is independent of the redefinitions of variables. The main point – completely misunderstood by Weatherall – is that the efficient market hypothesis is the opposite or complementary tool to the random walk models. They give you the opposite approaches to study what's going on. Their domains of validity are almost always non-overlapping. Weatherall stupidly thinks that they're the same thing. That's a manifestation of Weatherall's obsession with the flawed idea that "finances are about random walks all the way down", using the metaphor with the turtles.

In Part V, Aaron Brown exposes some Weatherall's (and others') misconceptions about the meaning and purpose of derivatives. Weatherall thinks that they're all about hedging against price risk. In contrast with Weatherall's model's predictions, derivatives are typically more expensive than the thing they try to "insure" and they're bought and sold by people who don't have to understand complicated financial mathematical formulae.

Part VI replaces Weatherall's naive "crops producers and buyers" model of derivatives by a more appropriate one. Brown's model is really opposite in many of its predictions relatively to Weatherall's model. Weatherall claims that the producer wants to get insured against the drop of the prices so he should really "short" the commodity he is producing. In the real world, however, he does the opposite. He doubles his exposure to the price risk by making "reserves" which is important because it helps him to achieve steady running of the business and that's what matters most. Brown says a lot of things about what may happen, why, how it is solved in the real world, and so on.

In Part VII, Brown says that the liquidity (the cash that is almost immediately available) determined the events in 2007-2008, not the value of things. Weatherall has missed that point, Brown claims. Weatherall made sign errors when he discussed whether hedge funds borrowed from Bear Stearns. The hedge funds were borrowers, not lenders! So when the hedge funds pulled out of the business, it was the opposite of "run on banks", not a classic run on banks as Weatherall claims. In the same episode, Brown offers pretty much the opposite answers to Weatherall's answers to the question why Bear Stearns needed its hedge fund clients etc.

The point – and I am sure that Brown is right about this point and Weatherall is wrong – is that the immediate events are caused by short-term dynamics (physics analogy: short-distance physics is fundamental). Brown says that "the modern financial system may appear to be buying and selling securities, but it’s more accurate to describe it as trading promises of future cash flows." This is a very important point. The securities are just pledges of the deals (and cash flows) in the future but when something breaks now, within a day, it's the cash flows that go awry now, not the future ones. So if the price of securities affects things, it's only via its influence on people's and companies' immediate decisions on cash flows done now.

Brown says that the genius of the paper money is that they create currency and credit at the same moment.

Part VIII, so far the most recent one, discusses the numerous consequences of the replacement of credit-based money with the liquidity-based money. Brown tells us that in contrast with Weatherall's misconceptions, the physical delivery of goods and services isn't the point, mark-to-market payments are the point. People buy futures contract whose prices are correlated with the things they want to sell – so they're effectively selling their products for future prices. They don't need to depend on the actual other party – they may sell the future pledges to others (or buy them from others). Unless the business is just being liquidated, most of the positions cancel and only the small perturbations around this almost exactly canceling base is what the producers and companies are tracing and making profit of.

The modern liquidity-based money invented between 1965-1980 is supposed to be analogous to the futures contracts. Derivatives took off. In the 1980s, the bank capital was redefined. In the credit-based paradigm, it was the equity contributed by founders plus profit. In the liquidity-based modern paradigm, it's the ability to cover daily changes in prices of banks' assets and liabilities. So the possibility of partners' default is treated as a uniform risk that is spread over days and some money is separated to cover these losses. The institution goes out of business when it's unable to maintain a margin but at that moment, no one loses anything because the assets exceed liabilities at that point. The bank capital requirements have been rebuilt around conditions about their short-term ability to pay in various scenarios (i.e. liquidity) rather than inequalities constraining their credit.

In the future episodes, Brown will discuss new kinds of derivatives that were created and new ways how they began to be traded etc.

I can see why Brown is right about many points. But there are also many points in which I am no expert because I haven't studied the newest types of derivatives etc. – and surely have zero experience in trading them. That's almost certainly the case of James Weatherall, too. His chutzpah needed to write a book about something that he demonstrably can't understand is as amazing as his arrogance needed to make far-reaching statements about string theory or quantum gravity.

And that's the memo.


  1. You've missed off one major part of inflation. It's tax. Tax is the price of government services. If tax goes up, and services are cut, its inflation.

    Not surprisingly that governments don't include the inflation they generate in their inflation measure.

  2. with more physicists or less physicists in the trading banks the crisis would have happened and would not have been different.
    more physicists are needed and a smaller government. physicists have nothing to do with the crisis besides the fact that most of them have socialist ideas and are too much pro government.

  3. Hi Lubos,

    I don't want to scare you again so I will try to be clear on what I am saying without introducing my bias?:)

    First off there was a concern about what the economy was doing and the attempts to deal with this with respectively with science views? How it could be dealt with? How could anyone having that science basis not have an opinion about it, as to how the economy had developed, and what is leading in terms of inflation that is not counted toward the effect it could have on the economy? A lot of questions for it was compared to a "crisis of sorts" and the sound of it may be familiar under that heading?:)

    What part Benoit had to play can come into it later. His association with Taleb is an important one.


  4. Statistical and applied probabilistic knowledge is the core of knowledge; statistics is what tells you if something is true, false, or merely anecdotal; it is the "logic of science"; it is the instrument of
    risk-taking; it is the applied tools of epistemology; you can't be a modern intellectual and not think probabilistically—but... let's not be suckers. The problem is much more complicated than it seems to the casual, mechanistic user who picked it up in graduate school. Statistics can fool you. In fact it is fooling your government right now. It can
    even bankrupt the system (let's face it: use of probabilistic methods for the estimation of risks did just blow up the banking system).THE FOURTH QUADRANT: A MAP OF THE LIMITS OF STATISTICS [9.15.08] By Nassim Nicholas Taleb

    There is always somebody who is the teacher and from them, their is a progeny. It would not be right not to mention John Archibald Wheeler. Or not to mention some of his students.

    Notable students
    Demetrios Christodoulou
    Richard Feynman
    Jacob Bekenstein
    Robert Geroch
    Bei-Lok Hu
    John R. Klauder
    Charles Misner
    Milton Plesset
    Kip Thorne
    Arthur Wightman
    Hugh Everett
    Bill Unruh

    One always hopes that one's progeny excels from what they have been taught. Sometimes, it is not recognize for what has been accomplished or not. Should one not hope to see that what materializes becomes let's say a Georg Riemann who understood well what was to follow from his teacher/adviser Carl Gauss?


  5. Lubos, calling a past student of yours sloppy and lazy is highly unprofessional. A professor has absolutely no right to say negative things like that about their students, can you not grasp the concept of confidentiality ? What next, will you be announcing to the world the grades he got in the course(s) you taught him? Not cool at all.

  6. let me put this simply,

    mathematical models are as useful in predicting the economy as they would have been in predicting the success of Disco in the 70s.

  7. Physicists in finance sector have about a 50-50 chance of being successful - which are much much much better odds than when trying to come up with a model for Nature. (In modern banking/finance almost any model, even badly conceived ones, by an idiot, probably has some chance of some success)

    Considering physicists in finance are paid far better than their heroic fellows in pure science,why would anyone do pure science?

    Weatherall was on BBC Radio 4 Start The Week Programme today ( ) , a single poor lady (Tiffany Jenkins) had to combat three blokes against the idea that physics/mathematics can describe human behaviour in any useful manner. She is of course correct, this was debunked by Popper et al as Historicism many years ago, but the dream of a science of the evolution of society never seems to die.

    The problem is, any finance/economic/sociological model will always fail once some jerk with enough power exerts his free-will - you can't model that.

    I was at J P Morgan in 2011 when Mortgage securities were introduced for the first time - no one seemed to care what they were (we just had to create trading programs for them) - just another derivative, based on mortgages. :-O

    You see, you can model for human greed but you can't model for human idiocy.

  8. Look, the Wiener process is fundamental here, and since no one has even mentioned sausages it's quite clear that all concerned here know Jack Schitt about modern financial systems. QED

    P.S. I can get hold of some horse-meat ones going cheap right now if any of you are interested. Pukka crap EU paperwork too, no problem. Mega-bulk purchase discounts available. Go long in sausages, I say!

  9. I work in finance and was a theoretical physicist once.

    Like Aaron Brown, I find it tedious and arrogant that weatherall assumes that the whole of Wall St. is founded on physics. This is just bollocks. Samuelson, Markovitz, Black, Scholes, Merton were not physicists.

    I hired many physicists who have come from academia into finance. The ones I did not hire were the ones who assumed arrogantly that they knew it all rather than listen. A lot of them were French by the way so they were double arrogant ;-)

    So many of them go to work for hedge funds where they search around looking for intellectual problems which look like physics problems. And then they write papers talking about critical exponents and phase transitions and self-organised criticality. Every bimodal system is an Ising model. Or a field theory. In the end it all fizzles out to nothing. It's like the old saying "to a carpenter with a hammer every problem looks like a nail" (or something to that effect). I do not mean to say that physicists have nothing to say, but a bit of humility is needed. For instance it must be said that physicists have an almost shockingly low knowledge of modern time series analysis.

    The problem with finance is that it is not physics. In some ways it is more like engineering. There are no rigid laws. It is fuzzy. And as Brown shows in his critique, these require you to ask questions and to understand why rather than to assume. Also one small misunderstanding can lead you to a conclusion which is opposite to the truth as Weatherall does in the Bear Stearns case. Detail is key. And making large sweeping statements in the way Weatherall does is a major risk when you are a novice.

    So I would ask physicists to read his book with caution.