Friday, January 25, 2013

Weinberg's evolving views on quantum mechanics

Cool anniversary (1/25): CERN discovered the W-boson (UA1 experiment) exactly 30 years ago, two months after their first W candidate; there was a press conference. Via Joseph S.

A pulsar with a button periodically switched by ET aliens in between two regimes, to broadcast a binary message to us, was found. This answers the question "Where are they?" The new question is "What are they talking about?" Via The Register.
Lectures on Quantum Mechanics by one of the world's most achieved living physicists may be grabbed from the bookshelves; click at the link on the left side.

Aside from the Weinbergization of lots of the usual technical topics you expect in similar textbooks, there is also a section, Section 3.7, dedicated to the interpretations of quantum mechanics.

One may see that Weinberg's views have changed. Unfortunately, the direction of the change may be associated with the word "aging".

Lots of web pages such as a Facebook Weinberg fan page and John Preskill's blog (comments) quoted the most characteristic sentences in the book:
My own conclusion (not universally shared) is that today there is no interpretation of quantum mechanics that does not have serious flaws, and that we ought to take seriously the possibility of finding some more satisfactory other theory, to which quantum mechanics is merely a good approximation.
I think that I remember the Czech translation of Weinberg's "Dreams of a Final Theory" that I bought in 1999. At that time, I was already completely certain that quantum mechanics – in the founders' (refined or unrefined) interpretation – made a complete sense and it was a complete theory linking observations to mathematical objects and able to make (probabilistic) predictions.

And my memory indicates that Weinberg just confirmed my conclusion that the universal postulates of quantum mechanics were exact. They had to be a final answer to the "foundational questions" and they couldn't be deformed.

By now, Steven Weinberg has revised his opinions and – because "no interpretation looks good enough" to him – he believes that quantum mechanics isn't exact, isn't the final story, and so on. Too bad. The basic axioms and postulates of quantum mechanics are rather crisp and easy and if there is a problem with them, I wonder why Steven Weinberg didn't analyze them (rather important questions in science) and didn't articulate their hypothetical problems about 30 or 40 years ago when no one had doubts that his brain was among the 10 most penetrating and reliable physics brains in the world.

However, there are also comments about the foundations of quantum mechanics in his new book that are right on the money. In particular, Weinberg says that all attempts to derive Born's rule for the probabilities out of something "more fundamental" seem to involve circular reasoning.

Circular reasoning is found everywhere – for example, in the very meme that the proper Copenhagen-like interpretations of quantum mechanics are "incomplete". Whenever people say such a thing, it's because they first convince themselves that there must be a whole skyscraper of mechanisms that explain the rules of quantum mechanics using "something more fundamental". Then they search for possible forms of this "more fundamental skyscraper" and when they do it well, they find out that none of the candidates really works. Therefore, they conclude that there's a problem with the interpretations of quantum mechanics – even though their failure only actually proves that there is a problem with the assumption that there is something else and "deeper" to be found about the foundations of quantum mechanics.

But let me return to Weinberg's more specific claim, "published derivations of Born's rule from something else involve circular reasoning". I want to mention one example I was forced to get familiar with, Brian Greene's "The Hidden Reality" that I translated to Czech. In general, it's a very good book about all types of parallel universes one may encounter in physics. The book has many flaws, too. In particular, the whole quantum mechanical chapter is a deeply misguided promotion of the Many Worlds Interpretation and "realism" in quantum mechanics where pretty much every sentence is invalid even though all the misconceptions are presented with Brian's extraordinary clarity. But there are also some would-be "new discoveries" in the chapter that aren't new at all and that don't prove or derive what Brian claims to be proven or derived. I hope it's OK to copy a part of the end note #9 for Chapter 8:
Over the years, a number of researchers including Neill Graham; Bryce DeWitt; James Hartle; Edward Farhi, Jeffrey Goldstone, and Sam Gutmann; David Deutsch; Sidney Coleman; David Albert; and others, including me, have independently come upon a striking mathematical fact that seems central to understanding the nature of probability in quantum mechanics.
As I will discuss in some detail, this list is totally misleading because the "finding" is a general property of all probabilities and it was totally comprehensible to the founding fathers of quantum mechanics, too. For example, if you watch the lecture Quantum Mechanics In Your Face by Sidney Coleman – one of the physicists in Brian's list – he explains this very same argument (without some of the philosophical conclusions that don't follow from the argument) but he also tells you, at the very beginning, that nothing in his lecture is new – except for his style of presentation of these things.
For the mathematically inclined reader, here’s what it says: Let \(\ket\psi\) be the wavefunction for a quantum mechanical system, a vector that’s an element of the Hilbert space \(\HH\). The wavefunction for \(n\) identical copies of the system is thus \(\ket\psi^{\otimes n}\). Let \(A\) be any Hermitian operator with eigenvalues \(\alpha_k\), and eigenfunctions \(\ket{\lambda_k}\). Let \(F_k(A)\) be the “frequency” operator that counts the number of times \(\ket{\lambda_k}\) appears in a given state lying in \(\HH^{\otimes n}\). The mathematical result is that\[

\lim_{n\to \infty} [F_k(A) {\ket\psi}^{\otimes n}] = \abs{ \braket{\psi}{\lambda_k} }^2 {\ket\psi}^{\otimes n}.

\] That is, as the number of identical copies of the system grows without bound, the wavefunction of the composite system approaches an eigenfunction of the frequency operator, with eigenvalue \( \abs{ \braket{\psi}{\lambda_k} }^2\).
But this "discovery" is a trivial consequence of probability theory. It doesn't depend on any new fact about quantum mechanics. At most, it is one consistency check you can make if you want to verify that the probabilities predicted by quantum mechanics are consistent with the usual rules of probability theory.

While the formalism above may look intimidating, its essence is completely simple. It says that if some property is predicted to appear with probability \(p\) and you repeat the same experiment \(n\) times, then the probability is nearly 100 percent that the property will appear in \(pn\pm 5\sqrt{pn}\) cases, i.e. in \(pn\) cases with an error margin that becomes tiny, relatively speaking, as you send \(n\to\infty\). The number \(5\) meant that I wanted a 5-sigma certainty that we will be inside the interval. So by combining \(n\to\infty\) independent propositions of the same form with the same probability to be true \(p\) and by counting, you may construct propositions about the number that are almost certainly true.

This fact is true pretty much by the definition of probabilities: it's what the notion of probabilities means according to the frequentists. In other words, you may prove it by deducing the binomial distribution simply from the distributive law applied to the power \([p+(1-p)]^n\) and from a separation of different powers of \(p\). And this simple claim that is true by the definition of probabilities – and that was true even in statistical physics applied to models of classical physics – was pretty much just translated to the notation of quantum mechanics. Claims about the numbers' having a value were translated to eigenvalue equations; the calculated probabilities were translated, via Born's rule, to the squared absolute values of the complex probability amplitudes.
This is a remarkable result.
No, it's not. Again, it's at most one of the trivial consistency checks one can make to verify that the probabilities predicted by quantum mechanics are compatible with the rudimentary, general, frequentist properties of the notion of probability. Also, one doesn't have to make infinitely many consistency checks one by one. Instead, one can verify that all axioms of general probability theory are satisfied by the quantum-predicted probabilities which implies that all conceivable consistency checks like that would work.
Being an eigenfunction of the frequency operator means that, in the stated limit, the fractional number of times an observer measuring \(A\) will find \(\alpha_k\) is \( \abs{ \braket{\psi}{\lambda_k} }^2\) – which looks like the most straightforward derivation of the famous Born rule for quantum mechanical probability.
As Weinberg correctly said about similar "derivations" in general, this derivation boils down to circular reasoning. Brian Greene concluded that an identity for the limit of some ket vector acted upon by some operator implies that we may say something certain about the number \(pn\) of repetitions of the same experiment, with \(n\) repetitions in total, when a property was satisfied.

But in this claim, it's being assumed that the (nonzero) deviations of the state vectors in the sequence from the ultimate limit of the sequence "don't matter" when \(n\) is large. However, for any finite \(n\), they do matter. If you want to calculate how large \(n\) has to be for your confidence level to exceed a certain threshold (e.g. to discuss the reasonable deviation away from \(pn\) that you may expect, and it is of order \(\sqrt{pn}\)), you will need to know and use Born's rule for the probabilities \(0\lt p\lt 1\) of individual repetitions of the experiment, anyway.

At most, Brian "derived" Born's rule for propositions whose \(p=1\) – by assuming a "simpler, more plausible, special form" of Born's rule for such \(p=1\) propositions – while he allowed himself to be sloppy about the quantification of the small differences of \(p\) from one and about the rigorously required calculation how small they actually are (and whether they are small). At any rate, the reasoning is circular. One may use the general Born's rule for any value of \(p\) and derive the same thing, or one may use the special Born's rule (eigenvalue equation) for propositions that happen to have \(p=1\) and derive the same thing (which is useless at the rigorous level, however, because no nontrivial propositions with \(p=1\) exactly may be constructed out of propositions with general values of \(p\)).

It shouldn't be surprising that all derivations of Born's rule for the probabilities have to be circular – simply because Born's rule is a fundamental and concise enough postulate of quantum mechanics. Probabilities play a fundamental role in quantum mechanics so they can't honestly be derived from anything simpler or more fundamental. If someone has managed to run through consistency checks such as Brian's consistency check, it should assure him that the way how quantum mechanics incorporates probabilities is totally smooth, natural, and internally consistent. It should weaken his or her attempts to "fight" against the foundations of quantum mechanics. And because things would fail to work if anything were "deformed" away from the rules of quantum mechanics, the consistency suggests that the universal laws of quantum mechanics can't be deformed at all. Too bad that so many people – including Brian – try to interpret the success of these consistency checks exactly in the opposite way!
From the Many Worlds perspective, it suggests that those worlds...
I decided to terminate this quote because it's getting preposterous at this point. Brian effectively tries to argue that the trivial translation of the frequentist definition of probabilities to the formalism of quantum mechanics proves the Many Worlds Interpretation. It surely doesn't. At most, Brian attempted to present a "story" whose goal is to demonstrate the compatibility of the quantum mechanical probabilities with the Many Worlds paradigm; even if true, this compatibility would be very far from "proving" the Many Worlds paradigm.

But the truth is that these two things aren't even compatible. The Many Worlds paradigm isn't compatible with the very fact that individual questions usually have probabilities \(p\) that are strictly in between \(0\) and \(1\) and that are, by the way, almost always irrational numbers. This can't really be achieved with "many worlds" at all. For any finite number of many words that are "equally likely", the probabilities will be rational (repetitions of the same world are needed to allow a trivial and generic situation, namely that \(p\) differs from \(1/2\) at all). And if you pick infinitely many worlds, in an attempt to approximate an irrational value of \(p\), the probabilities will be indeterminate form of the type \(\infty/\infty\), so they will be ill-defined.

This has to be combined with all the other severe diseases of the Many Worlds paradigm – no sensible or natural rule "when" the worlds should split and why, failure to obey conservation laws, failure to acknowledge that an arbitrarily large yet finite system always has a nonzero chance to "recohere" so the apparently irreversible "splitting of the worlds" should really never occur, and so on. If I summarize the flaws, the Many Worlds Interpretation contradicts the fundamental fact that the conditions for two possibilities to be already "decohered" are intrinsically subjective conditions, depending on the observer's choice of questions (and her choice of the set of consistent histories), her desired accuracy and confidence level, and other things. A fundamental, undebatable goal of any version of the Many Worlds Interpretation is to make these intrinsically subjective and fuzzily defined "events" look objective and this basic confusion of subjective and objective facts about the real world makes all conceivable mutations of the Many World Interpretation deeply flawed.

In Brian's comments about "derivations of the probabilities", and many similar philosophically oriented remarks about quantum mechanics, the illogical and flawed steps appear pretty much in every step. If Brian were a student who would use so many sloppy steps and incorporated so many logical errors in a calculation of some technical issue that isn't connected with philosophy and widespread misconceptions, he would just fail the exam and that would be the end of the story.

But because the misconception that the probabilities in quantum mechanics (and, more generally, postulates of quantum mechanics) aren't fundamental and exact is so incredibly widespread, all these numerous errors just "don't matter". Brian's end notes – and tons of articles with similarly flawed content – are viewed as OK simply because there are always lots of prejudiced people who feel "certain" about the totally invalid assumption of "realism behind quantum mechanics" and who are therefore ready to forgive an arbitrary number of mistakes as long as the basic spirit of the conclusion agrees with their prejudices.

As I have written many times, this belief in realism is analogous to any other religious belief. People's rational thinking simply gets turned off as soon as they hit questions that could threaten some opinions and assumptions that they view as fundamental for their world view. That's a pity. The advocates of "realism behind quantum mechanics" are running an industry of arguments that is analogous to creationism and its claims that Darwin's evolution has lethal flaws.

Now, Steven Weinberg apparently knows and admits that none of the existing attempts to derive quantum mechanics from something "more fundamental" is more than an example of circular reasoning or, if you want one words instead of two, gibberish. But the older Weinberg is still prejudiced that there should be some "less quantum" foundations beneath the quantum phenomena although this more general prejudice is still scientifically unjustifiable and ultimately wrong.


  1. "Nobody understands quantum mechanics" is not a correct statement, there are people who do understand it correctly ... ;-)

  2. One may see that Weinberg's views have changed. Unfortunately, the direction of the change may be associated with the word "aging". hahahaha.

    i don't know much about quantum mechanics but in case you are right the same could be true for Gerard 't Hooft. although in 't Hooft's case i think
    he knows about the points you raised. he is probably also gambling so in case he is right it would be something he would be recognised for.

    in case he is wrong he was on the top already so it wouldn't change something. 't Hooft maybe thinks that he does not have enough time to do more work on quantum mecahnics and get better proofs for what he wants but in case he is right he will stay in history.

  3. That's a relief!

  4. Luboš,

    I hesitate as it's a minor point, but the expression pn ± 5√[pn] should be pn ± 5√[p(1-p)n] since the variance of the binomial distribution is p(1-p)n, not pn, and similarly for all the other expressions involving √[pn].

    Using √[pn] as an approximation is fine but only if p is sufficiently small.

  5. Dear Lubos,

    if gravity in the real world is holographic, then, in principle, isn't it possible that a nonlocal hidden variable theory lives on the other side?

  6. Maybe Weinberg’s confusion is a product of aging but it is not an inevitable consequence of getting old. In three years I will be as old as Weinberg is now and I am certain that I will still be with Lubos on this issue.

    Quantum mechanics is not just another theory such as classical mechanics. Mordehai Milgrom could have been right in suggesting that Newtonian mechanics might require modification in order to explain galactic rotation rates. That would have been extremely unpleasant but not unbelievable. On the other hand, any modification QM seems to be unbelievable, in my view. If anyone ever falsifies QM experimentally, I, like Arkani-Hamid, will just kill myself.

    QM and Boltzmann’s statistical mechanics are completely rigid and cannot be modified without abandoning reason itself. It is a bit easier to see in the case of Boltzmann but it is just as true of QM.

  7. In case some of you are confused by Lubos' neat but short(-tempered) rebuttal see this 23-page paper, which says pretty much the same thing in a long-winded precise, polite manner ;-)

    To be fair to Brian Greene, he does mention the flaw in the argument in the second half of the note.

    re Weinberg's comment (I posted the quote a couple months back when his book appeared on amazon with 'Look Inside' enabled) - I don't think his suggestion is at all controversial, and in fact I think he may well be proved correct.

  8. Weinberg is wrong. There can't be any reality deeper than quantum mechanics. I'd like to consider a really dumb analogy, which doesn't really reflect the true math behind quantum mechanics but which completely grasps the essence of Bell's Theorem. Imagine two friends who see two different roads ahead of them. The task of the game is for the two friends to devise a strategy to always assure that each one takes one of the roads, and the other friend takes the other road. Now, suppose that these friends win every single time. You might suppose that they had the following strategy: they decide ahead of time that one friend will take road A, and the other friend will take road B. WRONG! And this is Bell's Theorem. A pre-existing set of instructions telling the friends how to behave will fail under any conditions. This has nothing to do with non-locality. Bell's Theorem simply states that pre-existing instructions cannot exist in any theory.

  9. wow what a great Feynman video on exactly this. Excellent.

  10. Nice. I prefer to say the same thing in the context of the free will theorem.

  11. Great, Gene, but please don't join Nima's harakiri bandwagon! There could be some flawed experiment and some mobs demanding both of you realize your plan.

  12. Dear Lubos

    Could You please give us your opinion on this paper:

  13. It's almost certainly bullshit, one can't "circumvent" complementarity by a simple thing like using a photon pair, but apologies, I don't want to spend hours with this stuff that is almost certainly bullshit, so if someone requires that I spend hours with this stuff for me to comment on it, then please consider my answer as "no comment".

  14. Thank you anyway. "No comment" is also a kind of comment on the other hand:)

    Here'a a bonus:

  15. Luboš,

    That QM violates Bell's Inequality rules out any interpretation in terms of local realism. Adherence to locality then implies we must dump the realism. I'm happy to accept that implication, and its consequence, even though the latter takes a bit of getting used to as it cuts against the grain of one's natural instincts. It just shows the grain was wrong.

    I'm not a physicist but I do understand a little of QM, including its Hilbert-space formulation in terms of observable operators acting on state vectors evolving in time. However, I know nothing at all about the relativistic treatment of QM even though I am familiar with special relativity.

    The point is I'd be grateful if you could expound a little on the locality aspect and how adhering to it—instinctively desirable though it be—is perfectly fine. In particular, dumping the realism gets us out of the bind of accepting the conjunction of locality and realism in the assumption of local realism in a purely logical manner but it still leaves the question—for me, at least—of the status of locality on its own and how it is consistent with QM. Could you say a few words on that please and maybe recommend a book?

  16. I know this is addressed to Lubos, but I wanted to comment. Did you read the text below? The friends cannot pre-decide which roads will be chosen. In fact, Bell desperately tried to stress this during his career, but most people still don't get Bell's theorem.

    Now, you ask about locality. If you and your friend did not decide ahead of time which roads to take, but that the decisions were always correlated, you might suspect, in the words of John Bell, that "something is going on behind the scenes".

    You asked for a reading. Please see none other than Bell himself

  17. Thanks. I have seen similar elucidations but not that particular one. I've heard of Bertlmann's socks though — many years ago, in some publication by David Mermin I'm sure.

    From your link: "Of course, mere correlation between distant events does not by itself imply action at a distance, but only correlation between the signals reaching the two places." [p. 4]
    No, but without invoking realism it's not clear to me what I should think of such a correlation — just accept it, enquire no further and get on with it, presumably. Don't get me wrong — I'm not advocating realism here, only asking what I need to think.

    Incidentally, "For we do not allow the first factor to depend on a, nor the second on b." [p. 13] seems to be typo as surely he means it the other way round and needs to swap a and b here?

    The "ENVOI" [p. 15] is interesting.

    "It is notable that in this argument nothing is said about the locality, or even localisability of the variables λ. These variables could well include, for example, quantum mechanical state vectors, which have no particular localisation in ordinary space time." [p. 15].
    Do we now know if there are such quantum mechanical state vectors?

    "Even if one is obliged to admit some long-range influence, it need not travel faster than light — and so would be much less indigestible." [p. 15]
    OK, but what sort of influence would that be, and what use would it be in explaining apparent influences travelling faster than light, which potentially is what we're talking about in Aspect's experiment? Or have I misunderstood Bell here?

    "Thirdly, it may be that we have to admit that causal influences do go faster than light. The role of Lorentz invariance in the complete theory would then be very problematic. An "ether" would be the cheapest solution. Bt the unobservability of this ether would be disturbing. So would the impossibility of "messages" faster than light, which follows from ordinary relativistic quantum mechanics in so far as it is unambiguous and adequate for procedures we can actually perform. The exact elucidation of the concepts like "message" and "we", would be a formidable challenge."
    The interesting bit there for me is the sentiment embodied in "ordinary relativistic quantum mechanics in so far as it is unambiguous and adequate for procedures we can actually perform." I wasn't aware there was any question of its adequacy or (potential?) lack of ambiguity.

    "Fourthly and finally, it may be that Bohr's intuition was right — in that there is no reality below some "classical" "macroscopic" level. Then fundamental physical theory would remain fundamentally vague, until concepts like "macroscopic" could be made sharper than they are today." [p. 16]

    I'm not sure what to make of that. I wouldn't have expected Bell to use the word 'vague' in a loose way if in fact he meant, say, 'probabilistic', so I guess he must have meant it in some "philosophical" way instead, but isn't the point of the Motl Interpretation that we can drop all this (useless) "philosophising" (assuming I have it right here, that is)? And talking about concepts like "macroscopic" being made sharper would surely upset Luboš unless we deprecate the activity? But maybe I have that wrong too? Oh well ... :)

  18. Reading the whole section I got the impression that he knows (of course!) that QM is a perfectly fine and consistent description of nature , he simply doesn't like the way it is.

    BTW, couldn't the textbook just avoid all the confusion with the interpretations just by saying that our knowledge of a certain system is represented by a unit vector in the Hilbert space... When they instead write that a system itself is described by such a vector, they could easily bring people to think to an objective reality or knowledge.

  19. Sean Carroll is as confused as anyone can be regarding QM. He simply doesn’t get it. The opinions of 60 or 60,000 “experts” don’t matter either. You should know that.

    As I have said many times on this blog, it is confusing to use the word “interpretation” when discussing QM. The word is vague; it has a range of meanings and that’s not helpful. I prefer to think that those who actually do get it, including Motl, Bohr, Heisenberg and many others, have attained a beautiful and deep insight into how things really work. There is no mystery in QM; it is complete and perfectly elegant in itself.

  20. Lubos, just out of curiosity:

    1. When did you finally "get" QM?

    2. Was there some paper, person or text book that profoundly helped you realize that QM was right and there was nothing more to be said on the matter?

    I'm going to have to do a search on what you've written on QM in the past...

  21. Mainly (but not only) in respect of people's cognitive quibbles with QM:
    Aside from all kinds of conditioned-in causes/sources of a souped up motivation, the neurophysiology of nearly everyone clearly tends to give rise to an irresistible pull from the notion of priors, and a similar (likewise largely subliminally seductive) 'suction' from the notion of substructures, in this case notably theoretical such.

    It is by defintion 'inEPT' (;>) to not involve (at least do so to some extent) evolutionary & psychological type considerations when faced with infuriating (or just awfully irritating) failures of folk to fathom fundamental aspects of physics or other aspects of What Is going on.

  22. Spot on. There is nothing to interpret, but people apparently can't deal with it.

  23. Hi, Lubos,

    Do you have any comments about W. Zurek's derivation of the Born rule using "envariance"? Thank you for your time.

    Houston, TX USA

  24. Dear Kevin, it looks like circular reasoning. In fact, it looks worse. The assumptions are less nontrivial than what is proven.