Thursday, January 09, 2014

Hamiltonians, not dogmas, pick the right bases, prices, and rates

What big-government advocates and hidden-variable cranks have in common

This blog post will combine diverse topics – including the philosophy of science, economics, general quantum physics, quantum field theory, and string theory. It will try to answer the following question:
What is the most widespread general error that prevents people from acquiring the rational and/or correct understanding of problems?
My answer is that a vast majority of people constrain themselves by dogmas, assumptions, group think, and stereotypes – while they should try hard to impartially look for the answers to all the relevant questions.

Even more concretely, they think that the right answers to pretty much everything (and the relevant bases, right prices, and optimal rates, among similar pieces of information) are predetermined in some way; instead, the world chooses the optimal answers dynamically, according to the "Hamiltonian".

I am going to discuss the organization of the society as well as several topics in physics to make the point. Someone could protest that it's strange to correlate economics with physics. They're independent subjects, aren't they? Well, partially. They're independent to the extent that the answers to questions in both disciplines are not "guaranteed to perfectly agree" according to a dictionary that would be universally valid. And someone may understand physics and misunderstand economics (or the other way around).

On the other hand, there are lots of correlations. As I will try to argue, many people are making closely analogous mistakes when they are thinking about economics and when they thinking about physics.

The TBBT video above reminds you of the unmatched intelligence and wisdom of Sheldon Cooper. (For the sake of extreme, nearly artificial modesty, I will try to largely avoid speaking about myself.) In that episode, Dr Sheldon Cooper has demonstrated that he is not just a string theory guru; he is the best theorist (and practician!) of management and economics, too. Later in that episode, he even demonstrated his understanding why it's so important to suppress the labor unions.

You could ask: What is the mysterious power that makes Sheldon Cooper so much smarter than pretty much everyone else? Well, he gives you the answer. It's because he is thinking about everything as a physicist and physics, when understood properly, is the machinery producing the working knowledge of everything that is important in the Universe.

This is a point that the laymen usually fail to grasp completely. When my relatives try to visualize what it means for me or similarly oriented folks to be mathematicians, they think that it means that we should be good at adding 800+360 (canasta scores). I've had no successes in explaining that this is not really mathematics, except for "mathematics" understood as a subject for small schoolkids. It doesn't mean that I am bad at basic arithmetics; just the desire to be better in these exercises than yesterday is something that I will kindly leave to idiot savants working in the circus. It's not what mathematics or science are about.

And of course that if I try to explain that physics is really the fundamental explanation of all phenomena in the Universe and mathematics is even capable of saying whether it's reasonably realistic to expect to get a very large number of jokers assuming that the cards were shuffled perfectly (or any other question that may become controversial), I am being treated as a heretic hybridized with a lunatic.

We don't live in a scientific world. Nearly everyone believes that the essential things around us are governed by witches, supernatural phenomena, sinful CO2 emissions, and other scientifically indefensible things. Similar tension emerges when I try to suggest that the scientific reasoning should be used to decide which things are healthy and lots of other things; everyone "knows" that those things should be decided by self-anointed shamans from tabloid magazines. Soon or later, pretty much everyone resorts to some randomly chosen and usually lame "authorities", "consensus", and other kinds of group think proving that they have no clue even though they are arguing about something.

At any rate, Nature works differently. Science is really a unified whole and physics is the most robust and deepest pillar underlying all of sciences or the sharpest human perspective on everything we may ever perceive. Where does the power of physics come from? Well, physics – and I mean theoretical physics here – is increasingly successfully deriving everything from the first principles. It questions and tests every assumption, it tries to justify and strengthen or challenge, refute and replace every assumption that less fundamental branches of science and human activity treat as an indisputable axiom.

This is not meant to say that in practice, a physicist – or even your humble correspondent – will be better at treating any problem that depends on the brain. Not at all. But what it does mean is that whenever one faces a major dilemma that seems to probe the boundaries of a scientific discipline different from physics, one needs physics-like reasoning to quantify the domain of validity of the assumptions of the non-physics discipline and the validity itself.

Pretty much everything we may observe in the everyday life (and even not so everyday life) has been in principle understood by physics. It seems that all observations ever made by humans are explained by the Standard Model of particle physics, general relativity, and a few minor (and so far not quite "connected") additions such as a dark matter particle. These branches of our understanding of everything are merged in string theory which leaves some open questions but this uncertainty and incompleteness doesn't really imply serious problems for real-world questions.

Regulation and big-government advocates

The "unified" flaws of human reasoning that I want to discuss in the rest of this blog post have implications for the organization of the human society (and the economy) as well as for the foundations of physics. As the title already says,
many people just want to impose artificially decided discrete choices and particular values of certain parameters even though it's the laws of physics – dynamics, usually defined by a Hamiltonian – that actually does the right choices and the right choices are usually different from the people's guesses and preconceptions.
In the context of the human society, this flaw is represented mainly (but not only) by a widespread brain defect known as the left-wing ideology. The sufferers from this disease tend to think that the best society is one whose "parameters" (examples will be discussed) are adjusted to values that these leftwingers tend to guess. These values are always wrong and in most cases, they are extreme. These people often think that the right value of a quantity should be zero – simply because they are overlooking all the positive roles that a nonzero (or larger) value play.

There are numerous examples and one could talk about them for hours. The simplest huge class are prices. Of course that these people fail to understand that the optimum prices and wages – those that properly allocate the resources, products, human labor, and skills – are determined by the free markets, by the balance between the supply and demand. Because of various "social causes", these people prefer to dictate the prices and wages differently. Soon or later, this is guaranteed to lead to imbalances and suboptimal utilization of the resources. People are less happy than they could be.

But it's not just about prices and wages. The leftwingers dream about regulating pretty much everything. They would like to determine the interest rates for any maturity date – the whole curve. They would like to determine the unemployment rate – usually zero or close to zero, by which they prove their complete misunderstanding of the positive role that the unemployment plays in allowing the economy to realize its potential (unemployment is a motivation; the unemployed workers represent a bath that allows the companies to quickly expand or fill holes and avoid interruption of production and other problems, and other advantages).

Similarly, leftwingers usually prefer a particular value of "income inequality", usually close to zero, because they more or less completely overlook the positive (and I would say existential) role that the concentration of the capital has played, is still playing, and will always play in the progress of the human society. A recent particularly pathological incarnation of these egalitarian misconceptions are efforts by many central bankers to redistribute wealth via inflation that they absurdly consider a good thing, efforts of deluded politicians to maximally "stimulate" the economies (by increasing the public debt that they don't care about), to reduce all interest rates, and so on.

Some "more sophisticated" leftwingers understand something about the crucial role of the price system but they try to "improve it". For example, they love to determine the right profit margins of individual companies. But the profit margins are ideally dictated dynamically as well. An industry that may experience large fluctuations in the profit needs a higher profit margin. An industry that may need to make large investments for its optimum development may also need a higher profit margin, and so on. The economy typically "learns" these things by itself. The leftwingers are unaware of most of these "details" – reasons why there are pressures in both directions and what they're good for – which is why they feel so bold and why their recommendations about the "right profit margin" are almost always wrong.

Similar "sophisticated" leftwingers realize that competition is good for the economy, the low price and high quality of the products. So they want to "encourage" this good thing, e.g. by breaking larger companies into smaller ones. But this shows another misunderstanding of the same kind. It's often a very good idea for companies to remain large. They may save lots of money for secretaries, for the development of many things that suddenly need to be done only once and not twice (or many times), and larger companies are also more resilient because certain kinds of noise may largely average out. Much of the progress in the economy since the 19th century has been about the concentration of the capital and the consolidation and unification of the means of production. Karl Marx, Vladimir Lenin, and their übercrackpot followers arguably hate the capitalism of big corporation even more than they hate the early simple forms of capitalism but that changes nothing about the fact that capitalism, and especially its optimized edition which includes lots of large corporations, is the result of decades of optimization of the production and services. This process involving consolidation has many advantages over the everlasting fragmented coexistence of thousands of smaller companies. Too big companies may start to resemble a socialist country which is bad but when it happens, it gradually becomes possible for the competition to grow. It makes no sense to try to speed up this process; it makes no sense to try to slow this process down, either. Companies do what they can to optimize the business.

Prices, wages, relative interest rates, the unemployment rate, profit margins, sizes of the companies and their number were my examples of the quantities that leftwingers would like to adjust according to their arbitrary guesses (often extreme ones, namely zero, which result from completely naive and idiotic ideas about the "ideal world") but the real world's Hamiltonian is actually making a better job – in a big majority of the cases and statistically – and these people are universally incapable of seeing how stupid their reasoning about the world is.

But leftwingers are too easy a target – every sensible person in the world agrees that they're deluded imbeciles – so let's add another group easy target. Some of you will agree that two groups of easy targets are less easy.

Bases treated as "preferred" in quantum mechanics

Now I will switch to the foundations of quantum mechanics which is seemingly a totally different topic but I hope that I will convince you that the principle is pretty much the same.

We are literally drowning in the ocean of anti-quantum zealots. If you follow physics topics on the Internet, you encounter them every day. For example, the most popular article on the server ironically called is a short essay by an idiot called Johann Cruz who screams that the quantum entanglement can't possibly exist and who asks the readers to please him by telling him that it's been just a lie and there's actually no reason to believe that quantum entanglement exists. The comment thread contains something like 100 comments and a majority of them captures the voices of similar anti-quantum idiots.

A major sentiment that drives various people towards the hidden-variable theories and similar delusions is closely analogous to the dreams about the "predetermined regulation" that is associated with the aforementioned left-wing brain defects. These people just want to allow a "certain kind of questions" and the answers to these questions are supposed to be objective. Quantitatively speaking, they want to hire themselves as gods who pick the right observables, the beables, and declare all other observables as contextual or unreal or f*ckable or some other unflattering nouns or adjectives.

But it's completely wrong in quantum mechanics to segregate observables in this way. The actual quantum mechanics allows any Hermitian operator on the Hilbert space (one with finite matrix entries, a subtlety that will be discussed later) to be considered a good observable. Each observable has its eigenstates and its eigenvalues (the spectrum) and the general state vectors in the Hilbert space may be written as linear combinations of the eigenstates. The particles' positions or the particles' velocities are no better than other observables like the angular momentum or the Hamiltonian. In fact, the eigenstates of the angular momentum and the Hamiltonian – and similar generators of symmetries – are almost always superior.

Hamiltonian (near-)eigenstates form the most "practical" basis

We sometimes want to say that a "physical system is really found in a particular state \(\ket\psi\)" so that we could consider this sentence "objectively true" just like propositions in classical physics. For sufficiently localized physical systems, it's the energy eigenstates that are "somewhat more practical" than their general complex linear superpositions. For example, we often say that the hydrogen atom is sitting at the \(1s\) ground state or the first excited \(2s\) state (they're energy eigenstates – and when degenerate, we often want to pick eigenstates of a few symmetry generators that commute with the Hamiltonian as well; the angular momentum is the most typical example). Despite the secret wishes of the Bohmian crackpots, we usually can't find a hydrogen atom with the electron at a sharply defined location or one boasting a well-defined velocity.

Why is it that saying that the atom is in the \(1s\) state is so natural and uncontroversial while saying that the location of the electron is \((0.3,-0.2,0.4)\) in Bohr radii is not? It's because of two reasons. One of them is that the energy eigenstates are stationary; the other one is that objects tend to dissipate energy and increasingly resemble low-lying energy eigenstates and their complex linear combinations.

First, the energy eigenstates have a simple dependence on time:\[

\ket{\psi(t)} = \ket{\psi(0)} \exp(Et/i\hbar)

\] Only the phase is changing (uniformly rotating around) and the overall phase doesn't affect any observable properties of the physical system. So if the system is found in an energy eigenstate at one moment of time, it will be in the same state forever (as long as the relevant Hamiltonian remains the same and isn't modified by interactions with previously neglected other subsystems i.e. objects).

On the contrary, a position eigenstate or other generic state is evolving frantically. In the case of the hydrogen atom, the position changes about \(10^{15}\) times per second (the Bohr frequency). You can't really say that the atom is in that state because the wave function in the position representation oscillates a quadrillion times before you complete the sentence. Because the energy spectrum isn't equally spaced, the evolution of the general wave function isn't periodic. On the other hand, the electron may stay in the \(1s\) energy eigenstate for macroscopic periods of time.

Aside from "stationarity", the second keyword I mentioned as a reason was "dissipation". In the vacuum, a hydrogen atom – and pretty much any system – chaotically interacts with the fields around and it sometimes emits particles and energy. Its energy per degree of freedom – also known as the temperature – is decreasing. At the end, it's inevitably low (the surrounding vacuum has cooled the hydrogen atom) and we find the object in one of the low-lying energy eigenstates. That's why these states are so important.

This is true for all objects in an environment whose temperature may be approximated as the absolute zero. If the temperature is higher, we may want to be interested in "typical pure microstates at some temperature" and/or the mixed thermal states (thermal density matrices). But to a large extent, these finite-temperature analyses are analogous to the analyses of the ground state and low-lying states.

The energy eigenstates may "look" very complicated and in different contexts, they have a very different character, different degeneracy, different symmetries, and different possible low-energy excitations (different spectrum and organization of the nearby low-lying states). This complexity and diversity is produced by the different relevant approximations to the string theory's "omnipotent" Hamiltonian in different situations; think about the ground states and low-lying states of a string, an atom, a molecule, a conductor, a superconductor, a crystal, a Fermi liquid, a quantum computer, and so on. This complexity and diversity of the low-lying energy eigenstates is something completely neglected by the "physics-wise leftists" who would like to predetermine the straitjacket of "beables" or otherwise "preferred observables" that all objects in the world are "obliged to worship".

If some operator (and its eigenstates) is preferred, it's the Hamiltonian and the Hamiltonian is hard. The Hamiltonian determines which properties of a physical system behave "somewhat classically". We may ask whether the atom is in the \(1s\) or \(2s\) state and treat the answer "almost classically". But this would be totally impossible with the location of the electron inside the atom for which the quantum mechanical aspects never cease to be important.

Sometimes the Hamiltonian eigenstates are not "perfect" and we want to consider some approximate position eigenstates of large systems, for example, as the "real states" that a physical system picks in an almost classical sense. But that's only because the uncertainty principle becomes almost immaterial for large enough states. In the intermediate situations, we often consider bases which are eigenstates of some "internal part of the Hamiltonian" (the relative degrees of freedom are ideally encoded by listing the eigenstates of the internal parts of the energy) and "center-of-mass location" etc. (which optimizes the analysis of the system for interactions with large macroscopic external bodies). In some sense, this is nothing else than the Born-Oppenheimer approximation. The fast degrees of freedom (electrons' motion in the molecule, for example) are treated quantum mechanically while the slow ones (locations of the nuclei in a molecule) may be treated classically for a while and their quantum nature (which has more limited consequences if the systems are large and slow) may be incorporated at the very end.

Why decoherence tends to pick bases where objects are "localized"

"Schrödinger's cat" is the famous example (one that Schrödinger stole from Einstein) showing that even macroscopic objects may exist in complex linear superpositions of the states we are familiar with – e.g. \(\ket{\rm dead}\) and \(\ket{\rm alive}\). But why is it only the states "dead" and "alive" that define the preferred basis that can be more or less interpreted classically?

The fathers of quantum mechanics vaguely knew the answer but the modern complete answer was only formulated in the 1980s. It's the decoherence; see e.g. this Harvard lecture. What is it?

Well, the density matrix – the "operator of probabilities" in some limited sense – may be shown to be quickly diagonalizing in the basis "dead" and "alive". This diagonalization occurs because of the cat's (object's) interactions with the environment and because of the tracing over the degrees of freedom of the environment. See the lecture above for some details how that works.

Now, why is it the "common-sense" bases like "dead" and "alive" in which the density matrix acquires the approximately diagonal form? It seems that in the preferred basis, the cat's organs have well-defined positions. Previously, I would say that it's the energy eigenstates and not position eigenstates that tend to behave "more classically". So why is decoherence picking this basis of something like "position eigenstates"?

The answer is all about the Hamiltonian again. The position eigenstates "win" in the decoherence battle because of the locality of the Hamiltonian. What do I mean by that? At the fundamental level (it's made manifest in quantum field theory), the Hamiltonian is a position integral of its density:\[

H = \int \dd^3 x\, \rho(x,y,z)

\] Note that this form of the Hamiltonian is in no way "guaranteed". The Hamiltonian could include lots of terms that depend on fields at several (or infinitely many) points of the spacetime, e.g. \[

c\cdot \vec E({\rm Boston})\cdot \vec E({\rm Miami}).

\] And some approximate or effective Hamiltonians indeed contain such bilocal, multilocal, or nonlocal terms. But at some level (and it's exact in QFT), the Hamiltonian is local i.e. an integral. And it has consequences for decoherence.

If you allow me to caricature the Hamiltonian a little bit, we may replace the integral by the "sum over places" and write the density as \(\rho=\rho_{\rm meat}\cdot a^\dagger_{\rm radiation}\). The Hamiltonian is able to produce radiation at every point of the three-dimensional space depending on the density of cat meat at the very same point. (I will be silently assuming that the wavelength of the radiation is shorter than the size of the regions to which we have separated the space so that the states of the radiation quanta in different regions may be orthogonal to each other.)

Because of this "point by point" structure of the Hamiltonian, the radiation field is constantly measuring the density of meat at each place. When we trace over the degrees of freedom of this radiation field (environment), we annihilate the off-diagonal elements between the eigenstates with different values of \(\rho_{\rm meat}(x,y,z)\), and this is true for every point \((x,y,z)\). Consequently, it's the reason why the preferred basis looks like a collection of vectors which are eigenstates of \(\rho_{\rm meat}(x,y,z)\). From the density of meat at each point, we may reconstruct the shape of the cat and sharply determine whether the animal is alive or dead.

If you review the arguments above, the "localization" of organs in the preferred basis boiled down to a property of the Hamiltonian again. It would be wrong to impose the "preference for localized eigenstates of the cat" as a dogma. Instead, the right answer – and all details of decoherence – follow from a careful analysis of the Hamiltonian. The preference for these bases is a derived, emergent feature of physics, not an assumption. And for some Hamiltonians in some situations, the answers may be different than the naive, classically thinking "Bohmists" expect.

One string's Hilbert space: which Fock space is relevant and why some divergences are harmless

I have argued that the right prices, interest rates, unemployment rates, bases preferred by decoherence, and many related things are determined dynamically i.e. by the Hamiltonian – in other words, by the free markets. Predetermined answers invented by humans are more or less guaranteed to be wrong. Leftists are more or less guaranteed to be wrong about the economy and hidden-variable enthusiasts are guaranteed to be wrong about quantum phenomena. The latter example arguably admits a more rigorous proof than the former but the character of the errors leading to the errors is similar in both situations.

But there are even "more systemic" issues that are determined by the Hamiltonian. My last example will be "the identity of the Hilbert space" itself. This comment may be ambiguous; what I mean is e.g. that the right Fock space in the infinite-dimensional harmonic oscillator. I will discuss a simple example of a free open string. But the description of its fluctuations is nothing else than some scalars (plus fermions, in the superstring case) in 1+1 dimensions. So the moral lessons apply not only to the single string's Hilbert space but to hundreds of situations in any quantum field theory.

Fine. What is the Hamiltonian of the open string? It's something like\[

H = \int_0^\pi \dd\sigma \, (p^2+x^{\prime 2})

\] The Hamiltonian is interpreted e.g. as the spacetime \(P^-\) in the light-cone gauge, it's linked to \(m^2\) of the corresponding particle. The fields \(p^i(\sigma)\) and \(x^i(\sigma)\) are functions of the string spatial coordinate \(\sigma\) and they carry an extra spacetime transverse index \(i=1,2,\dots , D-2\).

The Hamiltonian may be seen to be nothing else than one for an infinite-dimensional harmonic oscillator – a general fact about any free (non-interacting) quantum field theory. If we expand \(x^i\) to the Fourier series\[

x^i(\sigma) = \sum_n \cos(n\sigma) x_n^i

\] and similarly for \(p^i\) which are indeed the canonical momenta with the usual commutation relations, we will see that the Hamiltonian in terms of these Fourier modes will be, up to numbers like \(2,\pi\) that I neglect here,\[

H = \sum_n (p_n^2 + n^2\cdot x_n^2)

\] The Einstein summation over the repeated index \(i\) is implicitly contained in the concise formula above. The factor \(n^2\) appears because of the prime (sigma-derivative) attached to \(x\) in the formula for \(H\) at the beginning; yes, this primed \(x\) was squared. The last form for \(H\) is a collection of infinitely many oscillators. The mass is \(m=1\) or \(m=1/2\) – we will say \(1\) because we will be interested in the \(n\)-dependence only. (I am also ignoring the difference between the indices \(+n\) and \(-n\) and doubling of degrees of freedom associated with it. Those details change nothing about the scaling with \(n\) which is the point of all the concerns here.) The spring constant \(k=m\omega^2\) is equal to \(n^2\); it is the coefficient in front of \(x_n^2\). It follows that the frequency of the oscillator \(x_n\)-\(p_n\) scales like \(n\). That's also why the zero-point energies \(\hbar\omega/2\) lead you to sum \(n\) over all integers. That's why we encounter the sum of integers \(1+2+3+4+\dots = -1/12\). But I don't want to discuss these regularization issues here.

Instead, I want to discuss the width of the Gaussians in the ground state. The Hamiltonian for one of the harmonic oscillators \(p_n^2+n^2x_n^2\) may be written as \(n(p_n^2/n+nx_n^2)\) which has the advantage that the coefficients in front of \(p_n^2\) and \(x_n^2\) inside the parenthesis are inverse to one another. That's why we may immediately use them to see that the width of the Gaussian is \(x_{0n}\sim 1/\sqrt{n}\) and the width in the momentum basis is the inverse number, \(p_{0n}\sim \sqrt{n}\).

How large the string (see the animation of a typical vibrating string) is? Well, let's ask how far a random point is from the center of mass. The answer will involve \(\sum x_{0n}\) from the Fourier expansion. But \(\sum_n 1/\sqrt{n}\) is divergent (as a power law). However, many of the terms may have the minus sign and most of them may cancel – it is not clear whether the typical result will be divergent. A more controllable quantification of the divergence is obtained if we try to calculate \(\int \dd\sigma \,x(\sigma)^2\) in a string state with the center of mass at the origin. In that case, we will be led to \(\sum_n (1/n)\) without any minus signs in the sum which is still logarithmically divergent.

A beginner could be annoyed by this divergence in string theory (and all quantum field theories). Shouldn't have string theory eliminated all divergences? Well, this divergence is totally harmless because string theory doesn't really allow you to construct probes that could measure \(x(\sigma)\) of a string. Everything is made out of strings and they're large and imply a limited resolution due to the uncertainty principle. The energy may be measured more easily and it is indeed well-defined for these states.

We may even say that \(\int \dd\sigma \,x(\sigma)^2\) isn't even an operator on the Hilbert space because its matrix entries in between well-behaved finite-energy elements of the Fock space aren't finite! Even though these position functions \(x(\sigma)\) were used as the classical starting point that we later quantized, they become ill-defined (and are dropped from) the final product – quantum string theory! What needs to be well-defined are other quantities related to the energy and they are OK. This is a general story about many derivations: lots of "subtleties" are often added on top of the "motivating" physical ideas so that these physical ideas may ultimately be viewed as mere approximations or heuristic simplifications. They're no longer fully well-defined concepts or quantities in the exact theory.

One may also say that the "divergent part" of this \(\int \dd\sigma(x^2)\) is coming from the large values of \(n\) which are high-frequency oscillations along the string. Those will quickly average out whenever you use any finite-frequency probe. So even though the string seems to occupy an infinite volume of space if you send all UV cutoffs to infinity, such a divergence will have no undesirable observable consequences because there's no reason why an observable such as \(x(\sigma)\) of a string – and similarly the value of any bosonic field taken strictly at one point in any quantum field theory – should be finite. It's more important that we have a Hilbert space and we may express the Hamiltonian as a matrix relatively to a basis of this Hilbert space. And yes, we can. Some divergent expectation values of \(x(\sigma)^2\) in a state can't stop us from getting these physically vital answers – and nothing else matters.

But what is the Hilbert space? It's the Fock space formed by the ground state \(\ket 0\) and its excitations by finitely many oscillator's creation operators \(\alpha_{-n}^i\). I want you to think about the form of these allowed states of the Fock space in the position basis. Let's express these states as functions of \(x_n^i\). Well, they will be polynomials multiplying the basic Gaussian\[

\prod_n \exp(n x_n^2)

\] For a one-dimensional harmonic oscillator, every function \(\psi(x)\) may be rewritten as a combination of the energy eigenstates (finite excitations of the vacuum state). But that's no longer the case for the infinite-dimensional harmonic oscillator. The simple reason is that for a "generic" functional, we may need "infinite excitations" of the ground state. For example, if you imagine a state whose wave function goes like\[

\prod_n \exp(x_n^2)

\] without the extra factor \(n\) in the exponent (a dramatic enough change of the \(n\)-dependence of the exponent is the only feature that matters for our discussion), you may still imagine that there is a normalization constant for which this state is normalized. However, this state is a squeezed state relatively to the previous one and the squeezing of infinitely many coordinates will produce a "big representation" of contributing states that are infinitely excited (infinitely many oscillators are inserted – more precisely, the total \(n\) of the creation operators in the dominant contributions will diverge).

These "bizarre" features of the state with the "wrong" exponents may be explained by one of its transparent properties: the expectation value of the Hamiltonian \(H\) in this state diverges. And unlike the divergent expectation value of \(x^2\), this is a problem. When we discuss a Hilbert space, we want to be restricted to finite-energy excitations and states whose high-frequency modes oscillate too wildly – too differently from the "right, low-energy way" that is dictated by the Hamiltonian – are just eliminated from the Hilbert space even though you could envision "functionals of the coordinates or momenta" that represent these states. They're just not relevant for physics because the energy isn't finite. Again, it's the Hamiltonian that makes the selection.

If we include interactions (in string theory or a quantum field theory), the relevant Hilbert space gets deformed once again, in ways that can't be fully described by the right width of the Gaussians for \(x_n\). In fact, the wave functionals must be "close enough" to a wave functional that isn't really Gaussian at all (although it's approximately Gaussian if the coupling is weak).

Once again, I have argued that even the question "which states are sufficiently well-behaved" to be included in the Hilbert space has an answer that existentially depends on the Hamiltonian and its opinions. Even these questions about "which discontinuities are tolerable" for the wave function(al)s can't be answered in an aprioristic, kinematical way; the right answer always depends on the Hamiltonian, on the dynamics.

Final words

I hope that you have at least partially appreciated the similarity between the exemplifications of the logical fallacies in quantum physics, string theory, and economics. Most people in the world ultimately resort to some of their beliefs, assumptions that are not questioned, or choices that were picked by their soulmates. These beliefs are defended by group think, consensus, bullying, or other logically invalid methods. But the truth is that virtually all these things are calculable from the first principles and dynamically determined by the Hamiltonian. In many if not most cases, the proper, physics-based analysis results in answers that differ from the answered preconceived by most groups, including the most self-confident ones.

And that's the memo.

Off-topic: Soccer fans in my hometown were surely pleased by IFFHS that posted a toplist of the world's most successful soccer teams in 2013. Viktoria Pilsen ended as the 13th team (winners were 1-2-3 Bayern Munich, Real Madrid, Chelsea London) in the world, ahead of the Manchester United, Manchester City (yes, we didn't even have to beat them!), Arsenal, and ahead of all Italian clubs, among others. The second Czech team, Slovan Liberec, was 43rd and the third one, the current leader of our national league Sparta Prague, was 132nd. Amusing. I have no idea about their methodology (it probably incorporated the victories from the national leagues despite their qualities and other things that made Pilsen "look" so heroic) but this ranking looks really really good, especially because the budgets of Pilsen are about 100 times lower than the budgets of teams around us in the table. ;-) If all the towns above us were as small as Pilsen and if the achievement could be shared among all citizens of the cities, there would only be 12 x 170,000 = 2 millions (0.03% of the world population) above "us".


  1. In case you're in too good a mood today, here's an article that's diametrically opposed to your take on science.

    It's kind of pathetic really. Science has gradually been removing all the toys and baubles these people used to play with, and now they're reacting with some serious Angst. You have to smile at comments like:

    "But when scientists use this locker-room braggadocio to belittle the
    human viewpoint, to belittle human life and values and virtues and
    civilization and moral, spiritual, and religious discoveries, which is
    all we human beings possess or ever will, they have outrun their own
    empiricism. They are abusing their cultural standing. Science has become
    an international bully."

    I'd like to know what on earth he means by "spiritual and religious discoveries." What are they? What is the evidence for them? Here's another good one:

    "Too many have forgotten their obligation to approach with due respect the scholarly, artistic, religious, humanistic
    work that has always been mankind’s main spiritual support. Scientists
    are (on average) no more likely to understand this work than the man in
    the street is to understand quantum physics. But science used to know
    enough to approach cautiously and admire from outside, and to build its
    own work on a deep belief in human dignity. No longer."

    It would be interesting to hear him explain what "humanism" means, or to describe the experimental evidence for the existence of "spirits." I personally have to agree that I am "not likely to understand" things such as why Christians insist on the Trinity or why Moslems claim that believers in the Trinity will burn in hell for quadrillions and quintillions of years, just for starters, why the Hussites insisted on Communion in both kinds or why the Popes disagreed so vehemently that they launched a series of wars that tore central Europe apart for decades, why the iconoclasts insisted that we will go to hell for displaying pictures and the iconodules insisted that we will go to hell if we don't, etc., etc. However, since I don't accept the central premises of any of these religions, as far as I'm concerned the questions are irrelevant. The same goes for the rest of Gelernter's "spiritual truths."

  2. Representing society as an operator on Hilbert space, with optimal parameter values then following the implied distributions on particular axes, is very neat as the impossibility of writing down the Hamiltonian becomes a proxy for the imbecility of the Left. I believe a more empirical view, one possibly accessible to more average minds and actual quantification, is to forget the Hamiltonian and consider constructions of measures on said Hilbert space. The free market then becomes a random walk driven by the choices of billions of minds, and diktats from on high represent barriers (boundary conditions) restricting access to regions very likely containing optimal values. Martingale limits then represent social states and walks admit catastrophic paths (reaching non-differentiable points on the manifold) like the one we are seemingly on.

  3. I am reminded of the story of when John Von Neumann encountered a Keynesian economist and remarked of his economic models "I'd have thought him a contemporary of Newton."

  4. "obligation to approach with due respect
    the scholarly, artistic, religious, humanistic work"
    "The sanctity of life is what we must affirm"
    We live in a Golden Age. David Gelernter whines about all the yellow.
    2:20 onward.

    Current sunspot AR1944 is loaded and aimed to belch an X7 solar flare plus coronal mass ejection right down our throats. The continental electricity grid roman candles, switching stations melt, transformers explode, and ten foot high voltage inverters suckling 500,000 volts DC hydro from Canada flash into plasma. Civilization would be down for 5-7 years, or forever. New York starves within 60 days. Quote Shakespeare to the sanctity of human life come to smash your skull and affirmatively redistribute your resources during the first week.

    Weapons and ammo are insufficient. Have cleaning kits, too.

  5. God smote Sodom and Gomorrah with fire and brimstone, flashing Lot's wife Margarita into a pillar of salt (hold the lime). God is a Liberal. A like-minded Conservative would have vended genital lubricants spiked with chemical castration, doing designated voluntary good to others by doing well for himself.

  6. Wow, who needs TV when such nice TRF posts are available ... yummi :-))) !

    Enjoying that now ...

  7. This post is apparently inspired with recent Tegmark's notion that our universe can be described as a Hamiltonian and density matrix. He then suggests some ways to quantify those conditions and what values they might take in different regimes.

    The secret of geniality is in hiding of your sources...;-)

  8. Hm, it seems that the superficial constraints left wingers want to impose on the economic system act as explicit symmetry breaking terms of the Hamiltonian that describes tha natural dynamics of the system ... ;-)

  9. Interesting article, the writer seems paranoid and angry. However, remember that religion and spirituality are not the same thing. Religion is an artifact of man, through the expression of which man seeks to understand spirituality. But pride, dogma, wrath and greed often cloud the judgement of the religious man, just like any other man.

  10. Why do the Schrödinger's cat examples always use {dead,alive} as basis of the Hilbert space, can we not agree to use {sleeping,playing} ...?

    Come on folks, I know that theoretical physicists basically like kitties ... ;-)

  11. The insight that the evolution of a physical system may be described by a Hamiltonian is due to Hamilton, not Tegmark.

    The description of the Universe in terms of a density matrix is due to John von Neumann, not Tegmark.

    I haven't read any Tegmark's paper for 5 years and looking at the abstract, the paper you linked to has clearly nothing whatsoever to do with the topic of this blog post. Your brain is more mushed than mushed potatoes in the KFC.

  12. I will try to humanize it next time in this way, Dilaton! Yup, I like cats - even in real life, like Líza. ;-)

  13. @Lubos--
    If I were (note the subjunctive, Lucretius) a troll, or even feeling trollish, I would point out that Lee Smolin has written arxiv papers on gauge invariance in economic models (he is a real chameleon--or, more correctly, Equus perimeterus asinus). Also, one of your fav. students James Wetherell has written,
    "The Physics of Wall Street"... (enough prodding--I don't want to elicit that George Carlin quote...:)

    I wish I had taken that QM or QFT course that you
    taught in 2007...mine was taught by a visiting Japanese professor who only spoke Japanese and braket. Your neutrino notes were appreciated as well---lucky students. Maybe Coursera or EdX or Udacity can get you to teach an online String Theory course.
    About splitting up businesses. I also abhor govt oversight and interference in economics, but what
    the big investment banks are getting away with is
    similar to what Mafia families do---carve up territory, collude, flaunt laws, have government workers either frightened or otherwise in their pockets, and know that they either will not be punished or else given paltry fines (for them) because they are "too big to fail". They also have placed sleepers in the govt (whether Dems or Reps) --Obama's financial advisors, the Fed directors, the rating agencies, etc. No bank should be too big to fail, and if
    anyone (even right wingers) are curious, they should read Michael Lewis' "Liars' Poker", or his more recent "The Big Short". In addition to being shocking, both (particularly the first) are extremely funny.

    I think that you posted this a while ago about govt bureaucracies, but it is so well done...

  14. kashyap vasavadaJan 9, 2014, 11:14:00 PM

    Hi Lubos:It will take me sometime to digest the excellent and deep article on QM. In the meantime a question: Is non local Hamiltonian ruled out in QFT by Lorentz invariance? Have you seen any useful non local Hamiltonian in NR QM?

  15. alejandro riveroJan 9, 2014, 11:41:00 PM

    How is it that the econonomical part of your post has not equations? Why is it different to physics? If you are entertaining your audience into economy, you could look for some of the classical papers around in the net. For optimisation, Kuhn-Tucker article "Nonlinear programming" is avalaible somewhere. I have also found the translation of Kantorovich "Mathematical Methods of Organizing and Planning Production“, with his famous conception of "shadow prices". Also K.J. Arrow "Extension of the Basic Theorems of Classical Welfare Economics".

    Your way of presenting economy seems more as Natural History, as if it were some branch of evolution theory as settled in the XIXth century by Darwin, without any notion of modern biology. I think it deserves a more mathematical treatment.

  16. May I recommend that you acquire this excellent product Dilaton: I now rely solely on mine for the most important decisions. Should I buy this motorcycle? Should I tell my girlfriend that she is getting a bit fat? Just slide open the door of the Schrodinger's Cat Executive Decision Maker and you will know the answer.

  17. kashyap vasavadaJan 10, 2014, 1:55:00 AM

    I withdraw the second question!! There must be dozens e.g dipole-dipole , interaction between two electrons in different shells, two atoms in a molecule etc. But it must be that the violation of Lorentz invariance would not be much serious. Sorry

  18. This time I agree with you Gordon. I think that a good physics derived mathematical model for an economy needs a type of chaos theory: too many variables of great sensitivity that can swing solutions rapidly. This is due to the human psychology factor which does not exist in a Hamiltonian analogy but has to be included in an economic model.

    I also see that if people were angels free markets would work perfectly to sustain the societies and no intervention would be necessary on a regular basis. After all our organic bodies very seldom need intervention to share with all cells, according to their needs. Unfortunately, as you point out with the banks, people are not angels and look out for number one, and associations of people look out for the association. And it is not only the banks and specific self interests, imo. The whole stock exchange system which is now world wide is an enormous casino, where huge amounts of money are gained and lost daily. People not being angels we all know what would happen in casinos if there were no supervision and regulations.

    Observing spontaneously evolving human societies/nations through history one can see that the stable attractor is the feudal system, as far as statistics goes.. In my opinion unchecked open markets will lead to feudalism any society doing the experiment.

  19. I have been reading your blog frequently for a few months now and have noticed that you seem to attract the craziest of people.. :D Having checked out a lot of articles on blogs either written by you or discussing you, I have to ask: what gives you so much stamina and patience to constantly explain yourself to stubborn and incessant idiots? Do you believe that if you try hard enough they will see the light? Has this ever happened in the past?

    I salute your commitment to combating misinformation and academic villainy. :D

  20. Hi Lubos,
    thanks for this excellent post. I fully agree with your analysis. Let me ask for your view of the following aspects.
    Could it be that the real world economic "Hamiltonian" admits unstable run-away dynamics for certain initial conditions? For example, is antitrust regulation avoiding monopolies a legitimate concern at some level? If so, how do we determine the minimum necessary regulation to avoid run-away conditions without destroying the free market in the process?
    Thanks, Tobias

  21. I think the first principle one should adopt in life is: "Thou shalt have no gods before thee." This gives one more room to manoeuvre. :)

    I very much like the tenor of your piece, Luboš, and although to my mind free-market capitalism is the best candidate by far—indeed the only candidate—for the godhead of wealth generation, I don't think it's all-powerful on its own. We need a system of laws to enable it to operate. As I see it these laws are essentially against corruption of the market whereby some competitors squeeze out others by means having nothing to do with free-market competition.

    Some of these laws against corruption are in place already, prior to any consideration of the market. For example I am not free to adopt the corrupt practice of shooting my competitor down the road because he has just started making better widgets and selling them cheaper than I can. Actually, strictly speaking, I am free to do that; it's just that it carries a risk of costs I might not be able to bear, so I need to factor those into my business model. (In cases like this of course a simple back-of-the-envelope calculation usually suffices to rule this business tactic out. For most of us anyway.)

    Other corrupt practices are more subtle. For example, I think there is a very strong tendency for larger corporations to get in bed with regulators and unduly increase the regulatory burden so as to squeeze out the smaller competition whose overhead is thereby disproportionately increased.

    OK, so the problem here is corruption and not economics as such. So deal with the corruption. Fine. But this is where it all starts to get murky for me. For example, it is not obvious to me that, in general, imposing some kind of economic penalty on the miscreant corporation wouldn't be as efficient as banging up its officers in eliminating the incentive for corruption.

    I am neither for nor against big corporations. Good luck to them. But I am wary of the power and influence they can wield and thus the greater scope they might have to distort and corrupt the operation of the free market.

    As much as I sympathise with and loudly cheer your desire to chuck the filthy leftard bathwater out I would like to keep the baby. Again, it isn't obvious to me that he isn't headed for the drain too if corporations get too big, however one might define size for that purpose.

    Maybe I need to read your piece again more carefully but right now I think Gordon and Anna make very good points.

  22. This mumbo jumbo from the physicist who actually thinks Oswald killed JFK. Apparently the reason for this belief has something to do with eigenstates.

  23. Sorry, but I can’t find much to agree with here.

    Perhaps our ideas of an “angel” is like are very different, but to me it seems that in a world inhabited by “angels” the free market would not work at all. Angels know nothing about maximizing profit of utility, competition etc. I would rather say that angels live pretty boring sort of lives under communism as imagined by Marx.

    The reason why need free markets is precisely because (in my opinion very fortunately) we are not angels. I can imagine a more boring world to live in than one inhabited by angels, the thought alone makes suicide look appealing, except that angels can’t even commit suicide.

    I consider also you belief in “regulation” as touching but naïve. Let me just point out that one of the most highly regulated company on Wall Street ever was… Bernie Madoff’s firm…

    Now all the regulators are claiming that all they needed was more money and more resources. There are plenty of reasons to doubt this. Unfortunately, I don’t think many readers of this blog understand Russian but for those who do here is an interesting documentary which gives some of the reasons:

  24. Did you intend to phrase that as implying one should worship oneself before any deity? Because I'm not sure that's all that much better.

  25. Caligula would have agreed with John.

  26. No. Respect and admiration I understand. Worship I don't. (It seems odd that any being would demand it — I don't understand that at all.) No, Werdna, the last thing anyone should worship is oneself.

    The same goes for any ideology, but especially when it concerns something as ugly as economics. It was this latter I had in mind here.

    Dominus tecum. :)

  27. By angels I mean people who are not out for number one unethically, only ethically. ( angels might be a culture related analogy)

    I am sorry to say that my observation says that most people given the chance will take shortcuts in ethics, thus no angels.

    I agree that free markets are the best solution for a society, I disagree on unregulated by law ones.

    If you look at human history, once money was invented and the barter system went out societies grew larger than clans, Free markets operated unregulated. The result is feudalism of some type.

    The people with arms to start with collared the markets, so to speak, set up their castles and power structures,. It ends up with 10% of the population around the power lords and the rest serfs. Over and over again. It is even a good description of the Soviet Union and of China now.

    It is a problem that has to be solved by the modern society, how to have the benefits of the free market system without ending in a feudal structure. I think it should be checks and balances by law, and at this point international law.

  28. History is not physics, but they have some basic principles in common. One of them is that in general it is not a good idea to ignore everything that is known and invent everything yourself from scratch. In fact, in very exceptional cases this approach may work in physics, but essentially never in history.

    Your approach reminds me of Jean Jacques Rousseau. Rousseau was also not a historian, but with about as much justification as you, he decided that originally men lived under some kind of communism until:

    ‘The first man who, having fenced in a piece of land, said "This is mine," and found people naïve enough to believe him, that man was the true founder of civil society. From how many crimes, wars, and murders, from how many horrors and misfortunes might not any one have saved mankind, by pulling up the stakes, or filling up the ditch, and crying to his fellows: Beware of listening to this impostor; you are undone if you once forget that the fruits of the earth belong to us all, and the earth itself to nobody.’

    I have no time and energy for a discourse on history, but briefly: there has never been a society that barter on a “barter” system. The idea that society started with a “free market” and then, when someone discovered that he could use a mace or a sword to “cut corners” ( or “corner markets”) it switched to “feudalism” is “original” in exactly the same sense that some of the ideas on physics that from time to time prop up on this forum and annoy Lubos are “original”. In fact, would you believe that Western “feudalism” dates only to about 9 th century and has very few analogues in other cultures (the main exception being Japan) while the use of recognizable money goes back to something like 3000 BC (silver coins about 600 BC).

    There is no evidence at all that before that time “free markets” reigned supreme.

    I am (a little) curious how exactly you think that “regulation” is going to stop people “cutting corners”? “Regulation”, of course, consists above all of creating more “corners” to cut, so logic and common sense suggest that with more corners to cut, more corners are going to be cut. Perhaps you imagine (I don’t actually think you do) that draconian punishments will do the trick, but then the Russian film I linked to, should disabuse you of this idea.

    “I think it should be checks and balances by law, and at this point international law.”

    “Should be” and in “we should all be beautiful, healthy, wise and happy”? Do you see the world moving in the direction you are suggesting or rather the opposite one?

    I am sorry to be so discouraging but I have already long ago lost interest in discussing how the world “should be” and am now only interested in how the world is. And the way it is is exactly as described here:

    Mikado: I'm really very sorry for you all,but it's an unjust world, and virtue is triumphant only in theatrical performances.
    See how the Fates their gifts allot,
    For A is happy — B is not.
    Yet B is worthy, I dare say,
    Of more prosperity than A!

    Ko-Ko, Pooh-Bah & Pitti-Sing:
    Is B more worthy?

    I should say
    He's worth a great deal more than A.


    Yet A is happy!
    Oh, so happy!
    Laughing, Ha! ha!
    Chaffing, Ha! ha!
    Nectar quaffing, Ha! ha! ha!
    Ever joyous, ever gay,
    Happy, undeserving A!
    Ever joyous, ever gay,
    Happy, undeserving A!

    Ko-Ko, Pooh-Bah & Pitti-Sing:

    If I were Fortune — which I'm not —
    B should enjoy A's happy lot,
    And A should die in miserie —That is, assuming I am B.

    Mikado & Katisha:
    But should A perish?

    Ko-Ko, Pooh-Bah & Pitti-Sing:
    That should be
    (Of course, assuming I am B).


    B should be happy!
    Oh, so happy!
    Laughing, Ha! ha!
    Chaffing, Ha! ha!
    Nectar quaffing, Ha! ha! ha!
    But condemned to die is he,
    Wretched meritorious B!
    But condemned to die is he,
    Wretched meritorious B!

  29. Perhaps I should add, although it should not really be necessary, that (unlike the more crazy “libertarians” who sometimes visit this blog) I am not against “regulation”. However, what market economies need is a small amount of “good regulation” and not lots of “bad regulation”. Unfortunately “good regulation” like “good technology” or “good science” is not something you can just demand or achieve simply by passing laws. Good regulation, that is relatively free of human errors and relatively unaffected by weaknesses of human nature, judgement etc, etc, can only be discovered by careful research and experimenting - a process that is actually quite close to what is done in science. This is a subject in which there is quite a lot of research being done, but very little of it is known to the general public, which prefers to think in terms of scapegoats and “evil characters”, which are really just the more successful versions of themselves (like “A” and “B” in “See how the Fates their gifts allot” from “The Mikado”))

  30. Thank you for pointing out the article by David Gelernter, which unlike you, I found deep and fascinating. I read it as a serious discussion by a serious person about a serious problem (the nature of consciousness) rather than a religious screed. Nowhere do I detect that Gelernter is asking the reader to (in your words) ‘accept the central premises of any of these religions’. Furthermore, if you read the entire article, you will see that Gelernter argues

    “Science needs reasoned argument and constant skepticism and open-mindedness. But our leading universities have dedicated themselves to stamping them out—at least in all political areas. We routinely provide superb technical educations in science, mathematics, and technology to brilliant undergraduates and doctoral students. But if those same students have been taught since kindergarten that you are not permitted to question the doctrine of man-made global warming, or the line that men and women are interchangeable, or the multiculturalist idea that all cultures and nations are equally good (except for Western nations and cultures, which are worse), how will they ever become reasonable, skeptical scientists? “

    Not knowing any better, one might have guessed those words were written by the host of this blog rather than by Gelernter.