From QM to QFT to ST
Tomorrow, the Physics Stack Exchange should experience a silent completion of a Holy Trinity. Users are given a golden "tag" badge when the answers by a single user to questions with a particular tag reach the score 1,000.
I have just learned it has to be from at least 200 questions now. A third "tag" is waiting to join the elite club; it had the score 998 and 195 questions today. After the daily update, it should break through both thresholds, unless I am overlooking some other condition in which case I apologize for the wrong rumors in advance (update: it did happen).
For the sake of simplicity, the "tag" badges are being collected by the same user. Just to be sure, the prominent "tags" are:
quantum-mechanicsThe moment to applaud would be now! Troglodytes...
Quantum mechanics (QM) has arguably represented the most important conceptual revolution in the 20th century physics (although relativity was novel and important, too). Quantum field theory (QFT) allowed us to construct quantum theories that were compatible with Einstein's special relativity and reach the "theory of nearly everything" (TONE), using Lisa Randall's terminology, namely the Standard Model. String theory (ST) is the last step (at least so far) that allows us to add the newest insights and ingredients so that gravity and general relativity (GR) may be reconciled and some inadequacies of the whole QFT framework may be fixed.
There is a sense in which quantum field theory may be viewed as a minor, incremental addition to the formalism of quantum mechanics. Why?
QFTs obey all the postulates of QM, at least formally. There is a linear (positively definite) Hilbert space, observables are represented by linear Hermitian operators, their eigenvalues determine possible values of these observables that may result from a single measurement, and the squared absolute values of the complex amplitudes \(|c_n|^2\) quantify the probabilities of various outcomes that may be measured "statistically" if the experiment with the same initial conditions is repeated many times. The Hamiltonian generates the evolution in time and the evolution over an interval of time is given by a unitary evolution operator.
But what the "other common observables" are in QFTs is special. In non-relativistic quantum mechanics, we think of observables as \(x\), \(p\), perhaps with some indices, and functions of them. This is a special feature of a subset of (non-relativistic) quantum mechanical models, not another postulate of QM. In QFTs, the number of particles may change so it would be meaningless to describe the states in terms of a fixed number of particles' coordinates or momenta. Instead, the observables may be constructed as functionals of field variables \(\phi_i(x,y,z,t)\) with various additional indices, and so on.
So QFTs form a subset of QM theories that is special in one sense: they allow the overall theories to respect the Lorentz transformations, the basic symmetry principle behind Einstein's 1905 special relativity. This reconciliation has several implications. Particles are inevitably supplemented with antiparticles (particles of antimatter) which are even distinct from the original particles in many cases, at least in cases when the particles carry some (unbounded or otherwise non-equivalent-to-minus-themselves) charges. It must be possible to create particle-antiparticle pairs. It's meaningful to study the spectrum of the theory, possible values of \(m^2\), and so on.
QFTs are special cases of QM theories that respect some symmetries and, to respect these symmetries, they are typically constructed from some particular basic observables, namely the field operators. But there actually are some "generalizations" of QM we must accept when we switch from non-relativistic QM models to relativistic QFTs. The former never produce divergences in meaningful quantities. The wave functions are always finite and normalizable and so are the matrix elements, and so on. In QFTs, divergences in intermediate results are omnipresent. The final predictions or experimentally measurable quantities must be (and, in well-defined QFTs, are) finite, of course. The path from a seemingly finite formal starting point through the divergent intermediate results to the final, finite results is the process involving regularization and renormalization.
It's really just the final predictions for the experiments that QFTs are "obliged" to produce, and they must agree with the empirical data, so all other steps that you do may be considered as "more or less clever tricks" that should a priori compete with all other conceivable methods to invent predictions for all experiments. The previous sentence has one purpose, Milton Friedman's F-twist, if you wish. It doesn't matter a single bit if someone finds the starting or intermediate points of the calculations "dirty" or "counterintuitive". If there's a well-defined machinery that produces results agreeing with the experiments, it's great and such an agreement justifies all the starting and intermediate points, too. In fact, the bolder the starting points of an ultimately successful journey were, the more non-trivial and stronger the confirmation was for the trustworthiness of these assumptions. And be sure that the Standard Model does work. People have learned to like the renormalization techniques etc. and found many justifications of it. But most importantly, the procedure works and produces the right predictions. So if you don't like renormalization, it is your psychological problem, not a problem of QFTs.
By QFTs, we mean theories obeying the Lorentz symmetry (at least locally) as well as postulates of QM that allow you to define some local fields \(\phi_a(x,y,z,t)\) that commute or anticommute at spacelike separation. Most typically, we construct such fields (including the composite ones like the stress-energy tensor) from some "elementary fields" whose dynamics is determined by a classical Lagrangian that we later quantize. But in principle, there may exist QFTs that don't have any classical limit – that cannot be obtained by quantization of a classical theory – and we indeed have some examples where this might be the case although we have some other examples where people believed that the Lagrangians were impossible but they were later found. But even in those bootstrap-like cases, there are some fields like the stress-energy tensor that should be commuting at spacelike separations, and so on.
There also exist classical theories that cannot be quantized to yield consistent QFTs, e.g. gauge theories with anomalies. If you removed the leptons but kept the quarks in the Standard Model, the classical field theory would be OK but the QFT would be inconsistent.
Contrived features of QFTs
QFT works as the explanation of all the known non-gravitational physical phenomena we know as of today. But this framework also has some features that look problematic or contrived or oversimplified or otherwise unsatisfactory. All of them are linked to the fact that QFTs dogmatically demand that everything must be ultimately constructed out of some point-like building blocks, the elementary particles.
Even a priori, this is a contrived assumption. We know that the atoms are not point-like. Electrons orbit the nuclei; atoms have an internal structure. But the electrons and quarks inside the protons and neutrons (which are inside the nuclei) are assumed to be structureless. They are point-like. They have no smaller particles inside. Well, the profile of some functions describing the "real world electrons" may be complicated as photons and particle-antiparticle pairs are being glued to the bare electrons. But we may still imagine that there is a bare point-like electron inside. If you imagine that elementary particles are composed of smaller point-like particles like preons, you may "delay" the moment when the problem becomes obvious but you don't really solve the conceptual trouble because you may still ask "why are the preons structureless". Moreover, the preon theories don't work well and there can't be any more "substructure" at distances shorter than the Planck length because geometry at these ultrashort distances doesn't really exist, at least not a geometry obeying some of the usual rules.
The strictly point-like character of the elementary building blocks automatically implies some vices of QFTs that are no inconsistencies (except for cases when non-renormalizability seems unavoidable) but that still suggest that QFT as a framework fails to be the final framework for physics. Of course, I will finally get to the problems with quantum gravity but let me start elsewhere.
The point-like character of the particles means that different elementary particle species must be made out of "different stuff". Hundreds of atoms, the smallest bricks of elements (and millions of molecules, the smallest bricks of compounds) are made of the same smaller particles, e.g. electrons, protons, and neutrons. The diversity is a result of many possible arrangements of these three elementary particles into bound states. On the other hand, a QFT like the Standard Model has dozens of elementary particles – the electron, the muon, the photon, and so on – and their diversity cannot be explained by a different arrangement of anything because two points' internal affairs just can't be any different. A point is a point. If you want to distinguish two different point-like particles, you must attach special labels and treat them as pieces of different materials. That's not an inconsistency but it surely suggests that QFT fails to see "inside" these objects. Their being different should be due to "something being different inside their bodies" and by declaring their body to be as small as points, we have given up the desire to "look inside". Bush 43 and Gore don't differ just by different labels (or passports); something is different about their inner structure, and the same thing should be true for the muon and the charm quark.
So there is a seemingly uncontrollable diversity of QFTs where we can choose the spectrum of possible particles – how many elementary particles we start with and what charges they have under the forces mediated by other particles – and this "smells" like we are playing with some man-made LEGO stuff rather than discovering how things "must" work or objectively "do" work in Nature. The rules of the game can't tell us how many elementary particles or their families there should be, and so on.
Another problem of the point-like character are the UV divergences. The point-like particles running in the quantum loops of Feynman diagrams may carry arbitrarily high momenta or, equivalently, the points where they interact (split and join) with other particles, the vertices of the Feynman diagrams, may be arbitrarily close to each other in the real spacetime. This limit of "many nearby events" or "high loop momenta" leads to the ultraviolet (UV), short-distance divergences. In renormalizable theories, the infinite parts of these divergent quantities may be subtracted. But the finite part is still generically undetermined and the values of the parameters must be extracted from the experiments.
So even if the infinities are cured, there are always some finite undetermined continuous parameters. We strictly have many families of QFTs and each family contains uncountably many QFT models because all those parameters may be fine-tuned to any values in an allowed interval. This loss of "complete predictivity" has been attributed to the divergences in the intermediate results whose finite parts may be isolated in many different ways. So this aesthetic imperfection is due to the point-like character of the particles, too.
Now, I have said that there are many QFTs. It would be really bizarre if one of them were "completely right" because that would mean that it's been (randomly?) picked from many other, totally disconnected QFTs. Why should there be many disconnected candidate theories at all? Below, I will argue that the disconnected character of QFTs is de facto due to the "excessively sharp", point-like nature of the building blocks, too.
The flip side of the same coin is that we often get just one classical Lagrangian for a quantum field theory so it looks like one corner of the parameter space is "more classical" than another. That's arguably unnatural because we want a fully quantum theory, not one that can be "objectively closer to a classical theory" or "objectively further from one".
And we are getting to quantum gravity. A QFT built out of the Einstein-Hilbert action of GR is inevitably non-renormalizable, so the number of continuous parameters you must measure before you can make predictions about the generic processes isn't finite. In fact, it is infinite. A theory forcing you to do an infinite amount of work and learn an infinite amount of extra stuff (from the experiments) before you may predict other experiments isn't terribly predictive.
The divergences in quantized GR are the "quantitative" part of some conceptual tension between QFT and GR. In GR, the spacetime geometry is dynamical and even the general noise of QM may clearly be a driver of the fluctuations of the geometry. We expect "quantum foam" to morally describe what's happening with the spacetime at very short distances. However, QFTs seem to assume some fixed spacetime properties, at least a fixed spacetime topology, to start with. This is apparently needed to define the vanishing commutators of fields that are space-like-separated, and other things.
So it's the defining features of QFTs that really prevent us from confirming that there is anything brutally dynamic about the spacetime at short distances. The intrinsic DNA of QFTs contradicts the expected moral picture of the quantum foam. Moreover, the curved spacetime in GR has some dimension which is a geometric property, too. If we say \(D=4\), it clearly means that we have added yet another dogmatic yet arbitrary assumption, one that wasn't derived. Each such assumption indicates that the theory isn't quite at the "bottom of the explanatory ladder". It isn't really a powerful theory but the empirical data we are trying to build upon; if we are constrained in the QFT framework, we are not learning anything about the structure of the spacetime beyond the directly measured experimental facts.
String theory solves all these things
As Edward Witten clarified in a talk, strings may literally be thought of as regulated, blown up, beautified generalizations of the point-like particles that make the Feynman diagrams thicker and therefore smoother. Because the history of such strings in the spacetime, the world sheet, has no preferred points, they (what would be the Feynman vertices in QFTs) can't be "too close" to each other, so UV divergences disappear. ST allows you to prove that UV divergences are absent from the beginning in many other ways, e.g. by pointing out that each torus that is "too thick" may also be viewed as a torus that is "too thin" (would-be UV divergences are interpreted as IR divergences that you have already counted and that may either cancel or admit a totally consistent interpretation), among related insights.
So the building blocks have an internal structure; the Renormalization Group philosophy behind QFTs that seemingly allowed you to probe arbitrarily short distances inevitably breaks above the string scale. But the RG doesn't break down in a sick, generic way; it breaks down in a very smart way. The breaking resembles the Velvet Revolution rather than the animalistic Syrian or Ukrainian civil war (in this metaphor, the latter would keep all the problems and create new ones in the short-distance behavior of QFTs). This twist removes UV divergences including the problems with non-renormalizability of gravity and such an internal (stringy) structure of the basic building blocks also allows you to show that all particles of the Standard Model are "made out of the same stuff" (pieces of strings) that is just differently organized (differently vibrating loops of the string stuff). All interactions have the same microscopic origin as well (splitting and joining of strings). In perturbative string theory, one may show that all the UV divergences go away, no other divergences appear as a replacement, and gravity is automatically predicted to be one of the forces that inevitably follow from a mode of the string (along with some less constrained gauge fields and Dirac or Klein-Gordon particles interacting with them). So interacting (splitting and joining) strings propagating in a pre-existing flat (or another fixed) spacetime are automatically ready to get "condensed" and the condensates' influence on other strings you add is indistinguishable from the influence of a curved spacetime. Moreover, the equations obeyed by this curvature are nothing else than Einstein's equations of GR coupled to matter (at least that's the long-distance approximation).
Now, once you know that the building blocks could also be higher-dimensional, you may investigate all possible generalizations. Membranes. Blobs. And \(p\)-branes for any \(p\). You will find out that the "constructive definition" of a new theory that would be fully analogous to perturbative string theory only works for strings because the world volumes for the higher-dimensional objects inevitably contain "world volume gravity" which is a generically ill QFT itself. So such a QFT with gravity makes all the problems you wanted to solve in the spacetime – like non-renormalizable divergences of GR – reappear in the world volume.
So strings are special, at least if you want to construct a theory in the spacetime by a straightforward analysis of some world sheets or world volumes of higher-dimensional objects embedded into the spacetime. Other objects reproduce the problems of naively quantized gravity again.
You could suggest that there are higher-dimensional branes whose dynamics is more bootstrapy – isn't as straightforwardly derived from classical world volume Lagrangians as it is in strings' world sheets. So you may think about a whole plethora of new hypothetical theories generalizing ST. However, you will find out that ST indeed contains higher-dimensional objects of any dimensions with "bootstrapy", i.e. not too naive, constructive, or straightforward, internal dynamics. The mutual relationships of them may be calculated from strings and by other methods based on the knowledge of string theory and you will realize that there are many constraints. Once you learn about the existence of the consistency constraints for these extra objects and the way how ST "miraculous yet so safely" manages to obey all of them "just right", you should get convinced that probably all conceivable consistent theories with some consistently interacting higher-dimensional models are linked to string theory just like those we have already found.
This is a very general theme. String theory doesn't say that "everything is unique" or "everything only has one solution". In many cases, it has many solutions. There are five 10-dimensional limits of string theory with spacetime supersymmetry and one 11-dimensional one – known as M-theory. The set of semirealistic stabilized four-dimensional vacua has many elements - the number has been estimated as "a googol to the fifth" or so. But ST simply constrains many things that were arbitrary before ST while leaving several or many solutions to other constraints. The stringy constraints are very stringent and selective in some cases but they still manage to allow models that agree with all the qualitative features of the Universe as we know it. On the other hand, the stringy constraints allow many diverse solutions which seems necessary to connect many ideas and describe the relatively messy Standard Model as a solution of the completely unified equations with "one type of stuff" that underlies everything.
String theory turns out to be unified – all the supersymmetric (and spontaneously SUSY-breaking) vacua are connected. They are demonstrably different limits of a space of solutions to some underlying equations. The connectedness is due to a generalization of the strings' ability to change the topology of short-distance (stringy) bridges. We only know how to write these equations explicitly in individual "patches" that cover the manifold of solutions to ST. Sometimes we only know perturbative expansions for the theories governing the patches. But whenever we try to make the conditions in such a theory extreme, we find out that the theory doesn't break down but it simply morphs into the equations for another patch that completely smoothly continues the physical laws of the first one. This is a hugely non-trivial observation that indicates that we're dealing with a "damn real and unique big structure".
All the solutions that seemed disconnected are really connected within one string/M-theory.
I also said that QFTs allow you to say whether some point in the parameter space is "objectively nearly classical". String theory is a quantum theory that has many classical limits in all directions. So if you try to increase the value of a parameter to make the physics "hugely quantum", you will ultimately get into some other limit where another weakly coupled description becomes OK. So we typically see that "extremely, really shockingly quantum" regime of something is one that admits "another mundane, nearly classical description". The vacua are connected by transitions and dualities of many kinds. So string theory isn't a jungle of hypothetical "patches of dragons"; it is a tightly connected network of intimately known limiting theories that just demonstrably manage to fit together seamlessly.
For many important corners of the stringy configuration space, we may get several equivalent descriptions. This is another indication that we are dealing with an important, large, physically relevant mathematical structure. It's like getting many snapshots of the would-be new (American) continent from several explorers. They may photograph the Liberty Island from different directions but you may check that these snapshots agree with each other i.e. with the assumption that they're snapshots of the same 3D object, thus strengthening the idea that they really took pictures of a real island (instead of believing that you are just dealing with several independent crooks who photoshopped a liberty-island-like fantasy).
ST makes many things dynamical – including the spacetime dimension, the spacetime topology, the number of light particle species, and tons of other things. Their being dynamical means that the answer to the question "what are the properties of these things" are no longer dogmas that have to be inserted; they are something we can calculate or at least "constrain" from a deeper set of conditions. We are looking inside the things (objects and properties of Nature) that looked like indivisible bodies or irreducible axioms in QFT. That's what the progress in theoretical physics looks like.
String theory has passed so many internal consistency checks and tests that compare it with so many features of the real-world particle physics and it has unified so many ideas that are needed to study physics of QFT and physics beyond QFT that I have no "real doubt" that it's the right theory underlying Nature. For me, it's a hardcore conspiracy theory similar to the belief in the staged moonlanding to think that string theory is not the right theory. Some people who are not really familiar with all the arguments and the internal interconnections within the string-theoretical ideas may disagree but they're analogous to the people who don't really understand the evidence supporting the claim that the people have landed on the Moon. The situations are completely analogous – just the understanding of the validity of ST requires one to learn much more than the belief in the real moonlanding.
String theory has also led to some radically new ways to describe the known physics – not only the "ordinary" dualities such as S,T,U-dualities but also Matrix theory and holography, especially the AdS/CFT-correspondence. Again, these surprises were not only nice but they are also pieces of circumstantial evidence that we are getting intimately familiar with some physical questions we couldn't have attacked at all just a few decades ago. Now we are becoming the masters of the new continent, using the metaphor involving the explorers in the Americas, as we have acquired many new tools to penetrate deep into the New World. We are no longer taking just photographs or videos of America from several directions; we have initiated the fracking, too.
We haven't answered all questions that have resulted, are resulting, and will result from the replacement of QFT by ST. Lots of progress or "just a little bit of progress" may appear in the future. But we already know that it's a highly constrained and probably unique corporation of ideas that Nature and mathematics together designed to go beyond QFT; the task for scientists is to try to follow in the footsteps of these two babes. QFT is a theory of nearly everything (TONE), as we said, and it works damn too well so the right theory going further clearly can't be too different from QFT. The right next step must be, in some sense, similarly incremental as the step from non-relativistic QM to QFT. On the other hand, we must make a step because there are some clear limitations of the QFT framework. And ST has apparently done and is still making exactly the right kind of steps, whenever the rightness may already be evaluated in some way. It is therefore displaying the perfect balance between originality and conservatism, the maximum number of shocking surprises that are not downright contradictions or stupidities. It is giving us the right number of constraints that are teaching us something but that don't exclude or contradict important structures or theories (vacua) we have learned over the decades. Everyone who still tries to avoid ST in the "beyond QFT" research is quickly falling one one of the two sides, the "too conservative side" or the "too progressive/terrorist side", of the delicate stringy rope that bridges the valley between Mt TONE and Mt TOE and that good researchers must carefully keep on walking on. Many people, like Mr Shmoit in both forms, fall on both sides of the rope simultaneously and intellectually castrate themselves by doing so.
According to a cynic, string theory is just an incremental step beyond the framework of quantum field theory. But this step seems to be enough to unify all the good ideas in physics as we know them in 2014.
And that's the memo.