Tuesday, May 15, 2012

Does hard work guarantee discoveries and answers?

The short answer is No.

In his newest article,
SUSY: A Matter Of Prior Beliefs,
Tommaso Dorigo of CMS argues that many people at CERN have worked hard so they deserve to cause a paradigm shift in physics – in his case, he believes that phenomenologists should stop the research of supersymmetry in order to appreciate the contributions of the experimenters. Tommaso's opinions are a combination of utter irrationality, dishonesty, self-brainwashing, and victimism.


The first year at the LHC has significantly increased lower limits on masses of all kinds of hypothesized new particles. We suddenly know that if a certain new particle exists, it must be heavier than a threshold that is significantly higher than it was a year ago. For exotic particles such as new gauge bosons, massive gravitons moving in extra dimensions, tiny black holes, and many other things, the new limits are as high as several TeVs.

The search of supersymmetry has been the single most important class of searches at the LHC because SUSY seeems to be the most motivated and the most likely new physics that experiments may find. However, the limits on SUSY are the least constraining ones. Among candidates for new physical phenomena, SUSY is still the most viable one, having the greatest chance to be discovered soon, e.g. in 2012. For example, the stop quark may still be as light as 300 GeV. As the experimenters will try to raise the limit to 900 GeV or so in 2012, they will probe an interval that corresponds to tripling of the lower limit on the mass. That's quite a change and there's a significant chance that the squark will be found in this interval. No other type of new physics will make a comparable shift on the log scale for the energy. If some limits on new black holes are raised from 4 TeV to 4.5 TeV, it's not hard to see that the chance that the objects will be discovered in this relatively narrow interval, measured by the ratio of the end points, is rather low. Not much will be changing about the particles whose lower limits on their masses are already high today.

What I want to say is that it is a sign of complete dishonesty if someone picks SUSY as the scapegoat and suggests that SUSY should be particularly punished by the "empty" results from the LHC (except for the Higgs boson that had to be found). In fact, the LHC has refused to discover all forms of new physics so far and light superpartners are elements of a very small set of light particles that the LHC may still discover. SUSY should be the least punished candidate theory, not the most punished one. If you admit that there should still be phenomenologists who try to work on models of new physics, it's actually obvious that the percentage of those who work on SUSY should increase because the LHC has excluded other possibilities up to much higher energies.

On Thursday, I reviewed and extended Phil Gibbs' calculation of the change of the probability that SUSY or low-energy SUSY exists in Nature, the change that is induced by the exclusion plots from the LHC papers. In an example, we assumed that the LHC has eliminated 2/3 of the parameter space. This percentage is calculated according to the prior distribution (or "measure") and takes all previously believed arguments that prefer lighter masses etc. into account. The figure 2/3 is probably an overestimate and I think that it is a huge one.

But even if the figure were 2/3, the impact on the big question "low-energy SUSY Yes/No" is tiny, pretty much negligible. You know, if one wants to believe that low-energy SUSY is hiding in the remaining third of the parameter space, she just needs to believe that an event whose probability was 33% has occurred at CERN. Now, events whose probabilities are "just" one third are occurring all the time. Claiming that such events are extraordinary signals that should change our mind is mathematically equivalent to believing that one-sigma bumps in graphs are very important. In fact, the probability 66% corresponds to less-than-one-sigma "certainty" that the answer should be No: you need 68% for a one-sigma argument!

The decrease of the probability that low-energy SUSY is right is by a factor of three if your prior probability was extremely low. But if it were higher, close to 60%, the relative decrease is much smaller, just by a dozen of percent or two. I have reviewed these calculations. If your prior probability that SUSY is right is much closer to 100%, the probability is almost unaffected by the exclusion of 2/3 of the space. But even if you start with the odds as believed by a SUSY skeptic, all the LHC exclusion papers have only added a one-sigma bump, imagine a deficit, that may be used as a very weak evidence against SUSY. At the same time, we have new insights whose confidence level is often above 4 sigma and that strengthen the case for SUSY – for example, the recent identification of the 130 GeV gamma-ray line in the Fermi data that would nicely agree with the explanation that it came from the pair annihilation of the lightest superpartner, from the SUSY-inspired dark matter. This modest paper pretty clearly overcompensates the anti-new-physics arguments from all the LHC papers that have been published so far.

Now, Dorigo is pretty explicit in expressing his desperate feelings that his work at the LHC was useless because it hasn't radically changed people's minds. I understand that it must be pretty depressive to share your credit with 3,000 other people plus 3,000 additional people in your friendly competing team, especially if even the combined excellent, nearly flawless hard work of these 6,000 people isn't making any qualitative impact on the thinking of the phenomenologists and theorists. It isn't making a big impact on the way how particle physicists do research and how they use their brains. Of course, the 125 GeV Higgs boson is an exception. It's being incorporated into a growing collection of papers.

If the LHC discovered another new particle or force, the impact would be huge. But the impact of negative evidence is much weaker. The exclusions are already affecting the kind of models that phenomenologists are studying; some of the models or points in their parameter spaces are either safely excluded or highly disfavored and of course, physicists are abandoning them, either resolutely or gradually. But when it comes to bigger questions, such as low-energy SUSY Yes/No, the LHC simply hasn't brought us enough evidence to change our mind about this question. This doesn't mean that the running of the collider hasn't been amazing or that the people have done anything less than a spectacular amount of quality work. They should be praised and I am praising them now, too. But that's something else than the question how big an impact the experiment has made on a big open question in physics, e.g. the question whether Nature uses low-energy supersymmetry to solve or improve the hierarchy problem. A simple calculation is enough to show that the LHC has made almost no impact on this question.

I think it's a matter of common sense: the LHC has only excluded what it has excluded. If it hasn't found new particles such as superpartners in the first 5 inverse femtobarns of the 7 TeV data, it doesn't imply that the LHC won't find a new particle in the first 15/fb of the 8 TeV data. The 2012 dataset will be independent and much larger than the 2011 dataset. It can easily find 3-sigma if not 5-sigma excesses even in plots that didn't even show 2-sigma excesses after the 2011 run. It clearly can happen: the LHC will go totally beyond the questions it was able to probe in 2011 so it may bring new answers.

If we could use the 2011 data to become sure that the LHC won't discover supersymmetry (or, less likely, other types of new physics) in 2012 or 2014 or 2015 (there is a break in 2013), we could just stop the LHC right now. But such a conclusion would be hasty or irrational so no sensible person is making it. Only Tommaso Dorigo tries to make it. It's surprising why he continues to work for CERN if he's so sure that they won't discover anything.

Concerning hard work that hasn't answered some big questions that the hardly working people wanted to be answered, we have lots of similar examples. Hundreds of string phenomenologists have tried to find the right vacuum of string theory describing our Universe for something like 30 years so far. We still don't know which one it is. Should we use some sort of compassion and affirmative action and pick a vacuum that will be the right one, in order to reward these string phenomenologists for the excellent hard work they have done? You know it sounds as a ludicrous joke, don't you? One may work hard and one may have IQ above 145, as almost all string theorists do, but it still doesn't guarantee that a big question will be answered in 30 years. String theorists have found many amazing things in the recent decades – arguably much more lasting and valuable insights, many of which they couldn't even predict, than almost all other scientists in the world – but they simply haven't answered some questions that they wanted to be answered from the beginning.

Of course, I could offer you some "old", no longer emotional examples from the history of science.

The case of the LHC experimenters is totally analogous. They're doing excellent work but whether this work actually brings us an answer to a big question – e.g. whether SUSY exists at low energies in our Universe – depends on the actual quantification of the evidence, a calculation involving the information contained in the results. The strength of evidence isn't measured by man-hours of work of string theorists, LHC experimenters, or anyone else. Would you believe that someone who considers himself a scientist could find such an obvious point controversial? Tommaso Dorigo unfortunately does find it controversial. I will discuss it at the end of this blog entry.

But let me start with some fun.

Tommaso Dorigo offers us three everyday life examples that are meant to be analogous to the exclusion of 2/3 of the parameter space (or much less). He recommends everyone to prematurely give up in all similar everyday situations. The first one is about cakes:
In some cultures a popular game played in special events is to hide a small coin or jewel in a big cake; everybody then gets a slice, and the person who finds the precious treasure can keep it. Now imagine you play such a game, and you start eating your slice bit by bit, to be sure you are not gulping down the treat with the cake. You keep finding nothing, and your dish is soon close to empty; only the tip of your slice remains to be checked. You therefore now grow extremely excited: surely you're going to find it in the next bit!
In the cake called the SUSY parameter space, we have eliminated (much less than) 2/3 of it. After I eat 2/3 of my cake and find no coin in it, it's still plausible that the coin is in the remaining third. If my prior belief that the coin is in my piece of cake was very low, the chances have dropped by a factor of three. If my prior belief was closer to 100%, the chances haven't changed significantly and I remain almost certain that the coin has to be in the remaining third. The situations are totally analogous, indeed. I think that no serious participant of the cake game would stop looking for the coin after he has eaten 2/3 of his piece. This would simply be a totally sloppy, wasteful attitude to the game. The question whether the coin is in the rest remains almost as open as it was to start with.

Another example, trains.
You arrive at a deserted train station in the evening. You know that there's one train exactly every hour to your destination; however, you do not know the minutes at which trains pass. You also seem to remember that at some time late in the evening trains stop circulating. You sit and wait, and after 58 minutes have passed your train has not come yet. You then rise from the bench and pick up your suitcase, certain that the train is about to arrive.
First of all, the right number should have been 40 minutes or less, not 58 minutes, because it's not true that the LHC has eliminated 58/60 of the parameter space. How rational people react? If I haven't seen the train for 40 minutes and there's only a one-hour window, I may become more nervous but I will surely not give up. Never, never, never give up. I would actually wait at least for 75 minutes, to include the Academic 15 minutes and allow the driver to be a scholar who may be 15 minutes late.

The evidence that the train won't come at all is extremely shaky if not negligible if we have only failed to see it for 40 minutes. It is, once again, equivalent to less-than-one-sigma evidence (excess or deficit) signalling that the train could refuse to come. Of course that I won't be quite decided after 40 minutes. I won't be decided even after Tommaso's 58 minutes. In fact, in my life, I have jumped to a train that was already starting within the next 30 seconds several times. In fact, I managed to do the same with an airplane, too. If the question is as important as a possibly wasted air ticket or the discovery or exclusion of SUSY, be sure that I won't make premature conclusions. I will fight to the last minute.

When 99.7% of the window when the train may arrive is wasted or excluded, we still have just 3-sigma evidence that the train will probably not come. Even this percentage makes it legitimate to remain calm: many physicists dismiss evidence that is as weak as 3 sigma so they may consistently ignore the exclusion of 99.7% of a parameter space, too. So of course that phenomenologists won't stop all the research of SUSY if 2/3 – and not even when it is 58/60 – of the parameter space is excluded. In the same way, most of us won't give up a train if we still have 20 minutes or 2 minutes to get there.
In a science-fiction story (I believe it is Ray Bradbury's "The Martian Chronicles", but I could be mistaken) a man decides to seek the help of a private detective. The detective explains that his client can be confident the case will be solved: he failed to solve the previous 150 ones, so it's extremely improbable that he'll fail on this one, too.
Again, the right number is that he failed in the two previous 2 cases. But to make the analogy with SUSY accurate, we must say that this is the best detective in the world. No one else can find it with a greater chance to succeed. Once again, of course that we will pick the detective to have a chance to find the truth. We may also try another, less promising detective to increase the chance. The detective may fail again but we are sure we won't find the truth if we don't hire this detective or any similarly good one. The argument that we can't find the culprit just because the best detective has had an imperfect record is simply wrong. 33% isn't an impressively low probability in any sense.
In the three above examples, there is an obvious flaw in the reasoning of the protagonist: a wrong a priori assumption. The failure to account for the unknown probability that reality is according to one's wish is a childish mistake that we sometimes fall in even as adults.
There is no mistake in my reasoning. If 2/3 of an "opportunity window" disappears, we may gradually become more anxious, whether we think about cakes, trains, detectives, or supersymmetric models. But the decrease of the likelihood estimated by us that is caused by the elimination of 2/3 of the "opportunity window" is so tiny that it doesn't even classify as "evidence" in the physical terminology. "Evidence" is reserved for arguments that have at least 95% or 99.7% confidence level; 66% confidence level is just too low to be even called "evidence". There are lots of evidence in physics that may be caused in this way and shift our opinion in either direction but the elimination of 2/3 of an "opportunity window" simply can't belong among them.

Let me skip some boring and utterly stupid anti-SUSY rants that Tommaso offers in the following paragraphs and continue with the last three paragraphs that expose Tommaso's irrational, emotional, affirmative-action-like beliefs in their nakedness.
I believe Science progresses more rapidly if scientists keep their minds open to the widest range of possibilities. Well, let me restate that: I believe Science does not progress at all if scientists fail to do so!
Scientists work on many possibilities. The most boring one is that the Standard Model continues up to some huge energy scales. It's plausible but it makes the light Higgs mass mysterious, as in the hierarchy problem. Maybe it's not a problem because the Higgs mass is low for anthropic or other reasons. It's plausible. But even if it's right, this scenario is boring enough – everything has been calculated about it. We can't discover too many new things about this scenario which is why people don't write hundreds of papers how the Standard Model works up to huge scales. There's almost nothing new to write about here.

The Standard Model up to huge scales actually has many bugs: fine-tuning, instability of the Higgs potential, problems to account for dark matter, baryogenesis, and many other things. That's why people investigate other possibilities. There are many of them but the supersymmetric possibilities represent a huge fraction of the models because they're the most well-motivated ones (or at least among them). The LHC hasn't changed anything at all about the claim that SUSY is the most likely single kind of new physics beyond the Standard Model. In fact, as I pointed out at the beginning, it has made this proposition stronger. It has increased the relative importance of SUSY in the beyond-the-Standard-Model model building.

Dorigo continues:
I am therefore inclined to believe that choosing a point mass PDF for one's beliefs on the correctness of a unconfirmed theory is a wrong, anti-scientific attitude. I certainly acknowledge that SUSY is a beautiful idea, and I indeed would be happy if it were found some day (even better, if I myself found it! I am indeed searching for SUSY particles in my research time with the CMS experiment!); yet the failure to observe SUSY as we raise the energy of proton-proton collisions and the accumulated size of our datasets in ATLAS and CMS cannot be dismissed as "no information". It is important information!
It is important information but only for the detailed questions which corners of the SUSY parameter spaces are favored or disfavored. When it comes to the big Yes/No question about low-energy SUSY, the LHC's information is equivalent to a less-than-one-sigma bump. It is not important information for this question: it is almost no information at all. If we learn that the right point is in a third of the original space, we have learned less than two bits of information but this information only tells us about the "where" question and almost nothing about the "whether" question. Any other argument we have in these discussions is incomparably stronger.

By the way, I don't understand how one could deny that the ultimate right distribution is a delta-function. When we look for something, it may be at many places now so the distribution reflecting our ignorance is fuzzy but if the right answer is somewhere and the search continues, of course that the distribution will converge to a delta-function. That's what the Higgs mass has been doing, too. What exactly does Tommaso want to question about this tautology? He apparently wants to clump the right model with some wrong models (like if a collection of wrong climate models is proposed to describe the climate) and say: look, many friends of the right model are wrong so he must be wrong, too! ;-) The right model has to respect the will of the majority and admit that he is wrong. ;-) That's how I understand Tommaso and that's why I think he is a childish nutcase.

But the main paragraph I wanted to react to is this one:
Keeping oneselves anchored to a point-mass PDF that "SUSY is correct" equates to dismissing as garbage all the negative results of the LHC searches. I will say more: it equates to saying that it is useless to do experimental research, because SUSY might be hiding where we have no access with particle collisions or other experiments. Given that, and given that we must already be sure that SUSY is correct, why searching for it?
Converting this discussion into this emotional "you are hating us and consider us garbage" exchange is completely irrational. I am not dismissing anyone's work or expertise or whatever. I am just saying that a particular body of research has almost no relevance for a particular big question. I am not just saying that: I have rigorously proved that it is the case. If you consider any research that hasn't been able to determine whether low-energy SUSY exists in Nature to be garbage, well, then I must say that according to the terminology you have chosen, not me, the 2011 LHC run is garbage.

Before the LHC began the collisions, I have stated very explicitly that even if the LHC wouldn't see any SUSY, I will believe that it exists in Nature and it exists up to energies that are much lower – by orders of magnitude – than the Planck scale. One could have asked the question "how would we react in this situation" before the LHC began its journey and I have actually asked and answered this question many times. Of course that I will be convinced that SUSY up to energies much lower than the Planck scale is realized in Nature even if the LHC won't find any SUSY up to 2020. To change my mind, one would actually have to find significant evidence that could change our mind. In the absence of any experimental discoveries, it's much more likely that such game-changing evidence would come from some hypothetical progress or paradigm shift in the theory.

But if the theory remains what it is and the LHC will discover neither SUSY nor any other new physics, of course that it will continue to be the case that some SUSY, with some higher degree of fine-tuning than what we believed to be OK just years earlier, is the likeliest explanation. Unless some completely different ways to think will emerge in the theoretical research, I will also continue to believe in the naturalness arguments which imply that the Standard Model up to a huge energy scale is much less likely than SUSY with 50 TeV superpartners that is somewhat fine-tuned but still much less fine-tuned than the Standard Model valid up to arbitrarily high scales.

Let's repeat a sentence from the quote above:
I will say more: it equates to saying that it is useless to do experimental research, because SUSY might be hiding where we have no access with particle collisions or other experiments. Given that, and given that we must already be sure that SUSY is correct, why searching for it?
As I have said previously, in the absence of new game-changing evidence, I will indeed keep on thinking that SUSY is relevant at some point of the parameter space regardless of the LHC run up to 2020.

Why is the LHC searching things at all then? It is doing so because we don't just want some qualitative Yes/No questions – whose answers we kind of know because of deeper arguments anyway – to be answered. We want to know much more detailed things about physics at the LHC scale. It's obvious that the Standard Model valid up to 2020 is a possibility, the simplest one to describe in words and one that many people consider the nightmare scenario; I am personally not horrified at all.

But it is not the only possibility. The SUSY possibilities represent another large subclass. But individual elements of the subclass may be very different from each other, too. It is somewhat demagogic to pretend that the separation of the beyond-the-Standard-Model to supersymmetric or non-supersymmetric models is the most fundamental separation to be made. Some pairs of supersymmetric models or some pairs of non-supersymmetric models are conceptually much further from each other than some mixed pairs.

The LHC will move our experimental reach well beyond the reach in the past. But the shift will still be negligible relatively to the gap between the electroweak scale and the Planck scale. Even though the LHC is very expensive, is working extremely well, and the people working on it are hard-working, very competent folks in most cases, it simply doesn't have the power to decide about some of the most far-reaching questions about the Universe.
I am confident, though, that the attitude of those SUSY enthusiasts who choose the point-mass PDF is going to change if we continue excluding parameter space points at the LHC. Phenomenologists are pragmatic and smart people (someone funnily used the word "street-smart" in connection to one of them in the comments thread I mentioned above), so even the stubborn among them will soon choose some other point mass to anchor themselves and their careers to.
But such an alternative point mass to "anchor ourselves to" would first have to be discovered.

As far as we can say today, it doesn't exist and all such point masses that would predict new physics for the 2011 LHC run have been ruled out. If the discovery of the new point mass won't occur, and the non-discovery by the LHC isn't a discovery either, of course that nothing will change about the broad structure of the phenomenological research and of course that SUSY will remain the single largest subgroup of this research. Only the details will be adjusted to agree with all the newest data and exclusions. That's exactly how science should proceed, that's exactly what the actual evidence tells us. Attempts to selectively "interpret" or "abuse" the LHC as a supersymmetry killer are absolutely unjustifiable by rational arguments and by the evidence.

I am sure that Tommaso Dorigo would prefer to see a paradigm shift (many of us would), especially one in which he participated, but non-discoveries usually don't imply paradigm shifts. So the detailed hard work at the LHC, if it continues to be as "empty" as it has been so far, will only have an impact on the details of the phenomenological research, not the big picture. Some game-changing discoveries – whether they're of theoretical or experimental character – are needed to change the game, of course. Hard work doesn't guarantee such paradigm shifts whether someone finds this obvious fact cruel or not.

And that's the memo.

No comments:

Post a Comment