Friday, June 20, 2014

BICEP2 gets published in PRL

Discovery upheld, paper nearly unchanged

Lots of sourballs and jealous experimenters (and theorists) have been trying to sling mud on the March 2014 discovery of the primordial gravitational waves by BICEP2. Some of them have been suggesting that it wasn't even kosher to write or talk about the discovery before it gets through the "peer review".

Well, those people associating the process of "peer review" with supernatural abilities have yet another reason to shut their mouth because the work was just published in the appropriately prestigious Physical Review Letters:
Detection of B-Mode Polarization at Degree Angular Scales by BICEP2 by Ade et al. (BICEP2)
Not just the abstract above but the whole paper is available for free. We should have entered the era in which almost all cosmo/astro/particle/theoretical physics papers should be available for free due to a contract.

Unless you have memorized individual sentences in the original draft (see arXiv) really carefully, you won't really find a difference. The paper claims the discovery of these waves. I can't even safely say whether it lists more methods to be confident that the discovery is real or fewer methods to do so.

The abstract claims that the null hypothesis is excluded at "more than five sigma" confidence level (and, later in the abstract, "seven sigma" confidence level) using the first method, that the dust is 5-10 times smaller than the observed signal if various models of the dust available in the literature are being used, and that cross-correlation arguments and the right spectral index exclude the dust at "three sigma" or "one point seven sigma" even without any models. It also says that if all these things are ignored, it's plausible that some completely new model of the dust could change the conclusion or at least the confidence level. What a surprise. Any development in any part of science may change things.

To summarize, it's a robust collection of evidence that the signal is real even though the certainty isn't perfect.

If one compares the papers carefully, he may see that some parts of the previous draft were omitted. For example, the analysis based on the reverse-engineered PDF file from the Planck Collaboration is no longer included in the paper. The two anonymous referees may have had various reasons to force the BICEP2 Collaboration to omit these pieces.

What I find very important to emphasize is that the referees are not an infallible omnipresent Gods who are superior relatively to the BICEP2 folks in some metaphysical way. Chances are 50%-50% that every individual change that the referees forced upon the BICEP2 authors has made the paper better; or it has made it worse. The idea that the authors are submitting a highly imperfect paper and the referees – or the process of the "peer review" itself – guarantees that the resulting paper is perfect is a downright silly idea.

It's likely that the referees (or one of them) are someone who is close enough to the Planck Collaboration if not a member of it. At least, if I were the editor, I would be very likely to choose a referee who is close enough to Planck. We shouldn't forget about this likely proposition when we read e.g. the last sentence of the abstract of the published paper:
...with $$r=0$$ disfavored at $$7.0\sigma$$. Accounting for the contribution of foreground, dust will shift this value downward by an amount which will be better constrained with upcoming data sets.
The original draft was more specific in this sentence. It said that the confidence level was reduced to $$5.9\sigma$$ if the best foreground model is assumed. But you see that qualitatively, even this sentence is the same.

At the very end of the paper, we learn that these "upcoming data sets" should be from the 353 GHz Planck maps and from the (BICEP2-superseding) Keck Array. If a Planck person were the relevant referee, this "mandatory insertion" of the comment about the upcoming Planck data may be viewed as a self-advertisement. The BICEP2 folks will add a new-frequency, 100 GHz data to their existing data. The latter are (and this paper is) exclusively based on the frequency 150 GHz. And it seems totally plausible to me that the new data from the BICEP2-related team will actually teach us much more than anything that the Planck will be able to publish.

As far as I am concerned, the publication of the discovery in PRL isn't really a big event. What it means is that one external expert or a few external experts were asked to add their voices and communicate with the BICEP2 folks about their work. No clear problem with the discovery was found so the modifications were restricted to minor changes – the usual minor changes that are supposed to prove that the referees have been at work.

Of course that the question whether the changes were improvements or not is a subtle one. It has no clear answer. If the referees had been members of the BICEP2 from the beginning, their opinions would have already affected the draft – the draft would have been different by similar minor changes that the referees have imposed on the paper in the PRL. The referees aren't too different from "just two other potential co-authors who could have joined the research but he has not". The idea that an "extra person who looks at it" makes things vastly more accurate is heavily overrated, much like the supposed "independence" of the external reviewers. They're external and independent but they may still be wrong – they may be wrong about the same things as the authors and they may also be wrong in a different way or more wrong than anyone in the BICEP2 Collaboration.

If the uncertainty ultimately evaporates and it's clear that the BICEP2 discovery is legitimate, I think it will be important to look at their original draft, and not the edited (or partly censored?) version in the PRL, to quantify their contribution to physics. There are cases in which peer review is likely to bring more confusion to the business, and I think that the mandatory editing or censorship of papers from large enough and competent enough collaborations is always an example of that. Any softening of claims in such papers is potentially due to jealousy of the folks who were not lucky to participate in the discovery.

There are reasons to think that the peer review increases the average quality of the published papers in a wide array of situations. However, we shouldn't forget about the other side of the coin. Peer review involves some kind of censorship and if correct claims are being censored, it's clearly not helping the progress in science. Some claims in published papers may be wrong but it may often be better to allow the authors to publish their results and views in an uncensored form.

1. Lubos, really. How convenient you do not quote the ONLY sentence which matters: "However, these models are not sufficiently constrained by external public
data to exclude the possibility of dust emission bright enough to
explain the entire excess signal."

End of story, sorry.

2. It's complete bullshit. This sentence is the only one that doesn't matter. It only says the tautology - something that is true in all of science - that it's always possible that some completely new model explains the whole signal in terms of something completely different than everyone has considered so far.

3. I'm an astrophysicist who researches dust (in a different context), I wish dust got this much careful attention all the time. It does not, very often, for a lot of discoveries that get a lot of press, the simplest models of dust are assumed.

So it was kind of perplexing and amusing and disturbing to watch this significant attention that went along the lines of "what if the dust models are more complex !!! BICEP2 has to be wrong !!!"

The strange desire for BICEP2 to be wrong prompted people to spontaneously consider new, exotic models of dust whereas previously vanilla models of dust were fine.

And honestly I'm sure that the vanilla model of dust assumed by BICEP2 is wrong ... but wrong by a factor of 10? Unlikely. Maybe a factor of 1.25 or 1.50.

4. It will be amusing if the dust ends up lower than the BICEP2 estimate, though not many people will be laughing.

I see this not as a bias against discovery, but as a bias against discovery by little people. If Planck had made this discovery I doubt that as many people would be militantly opposed. But, maybe that's what you meant?

I personally think it's better for science if (experimental) discoveries are distributed evenly or nearly evenly long the distribution of the logarithm of the cost of the experiment, i.e. 20 million dollar experiments discovering as much as 2 billion dollar experiments. I meet some people who argue that everything in science should be done within large collaborations, I'm pretty sure that would be a sociological toxin that would eventually eliminate discovery.

5. I completely agree, I should have said it, too. Bias against discovery by cheap enough experiments.

6. Sometimes it seems to me that the dominance of such dimwits as the one above, is a typical symptome of society at our times:

Most people have neither the capability nor the motivation to learn physics in depth at any technical level, they just want to lean back and be continuously entertained without putting in some effort from their part.

And for such lazy average laypeople, revolutions that are overthrowing everything again and agian, heated discussion and debates, flame wars bout scientific topics, etc offer simply more action and "fun" to watch than sitting down with a serious textbook, learn correctly what the scientific method is and how it works, etc ...

I guess this is the reason why tons of lazy dimwits and knownothings are loudly crying for revolutions in physics, and they blindly worship any troll that is capable of making a lot of noise and stirr up things for them to watch :-/

7. And not to forget that even though it is supposed to be some kind of an introduction to QFT, worldsheets and other fun remarks about string theory pop up regularly out of nothing throughout the whole book, LOL .. :-D.

8. Exactly, Dilaton! They're dreaming about revolutions in physics simply because such a dreamed-about total revolution could prove that they were right from the beginning when they decided not to learn any science whatsoever - now, all the science was proven to be wrong so they haven't wasted any time and energy, unlike others! ;-)

9. Lubos you say:

"What's your problem with all these obvious facts?"

My problem is the general long term implications if such mentality prevails. If we say that the "average person" cannot understand modern physics then some people may say that there is no point in communicating the achievements of theoretical physics to the general public because they are not appreciated anyway. The consequences of such alienation of science and physics in particular will be disastrous taking into account the number of young people that are inspired to become scientists exactly due to this outreach of science to the public (in the form of popular books, science press, blogs etc).

10. There's point of communicating science because someone - usually the more intelligent listeners and/or those with some backgrounds - will get something and the knowledge of science within the mankind increases.

Your assumption that it's a necessary condition for science communication to be meaningful that average/most people will "get it" is absolutely unsubstantiated and self-evidently incorrect.

11. What "end of story" ?

If you think that setence means the end of any story, you certainly have misread it, so you should go back and reread and think about it more carefully to see that (independent from its relevance) it does not put an end on any storry.

Sorry to all the sourballs :-)

12. Anyone who wants to write code can now just sit down and do it -- girls don't need special programs to get them interested. If you want to write apps for Apple ther is an entire platform and clear documentation on how to use it, tons of tutorials, Ditto for all sorts of programming languages out there and free code editors, etc., etc.. If the girls ain't coding it's their own lack of interest and/or talent for such activities. This is 2014, not 1970. I predict that poor boys in Indonesia are going to write a lot more code (once they are exposed to tech) than rich girls in Silicon Valley who are swiming in a sea of tech.

13. So this might be a case where the journal peer-review process has done more harm than good?

Peer-review should be public and open to all people who are knowledgeable enough to judge the paper at hand, and not just to 2-3 by an editor selected people ...

Of course average peolple who have no clue should NOT take part, just to say ... ;-)

14. "... complex numbers, something you learn in the second year in secondary school."

Johan, I'm intrigued. At what age would that be, and what country are you are talking about? Deutschland?

I was schooled in the 1950/60s in England. In those days (and even now, if I'm not mistaken) our secondary* schools (starting at age 11) introduced complex numbers only at (so-called) A Level, when one was in the final two years (sixth and seventh), i.e. between the ages of 16 and 18. [Private (fee-paying) schools** had more flexibility, and brighter boys could advance more quickly.]

At that point secondary schooling ended. For the more academic, typically this was then immediately followed by a 3-year undergraduate course at University, with graduation at age 21.

Anyway, I think quite a few are perfectly capable of learning about complex numbers at an earlier age than is the standard in England. Even so, I was given to understand that our educational standards (in my day at least) were pretty good by international standards. So your seemingly much more advanced system (age 12? for complex numbers) does surprise me somewhat. Hence my questions. However, I am quite prepared to be surprised but my guess is that our two definitions of secondary, as in 'secondary school', differ in terms of age at intake.

I'd be grateful if you could clarify things here for me.

* There were various types, some less academic than others that didn't teach to A Level, but I won't bore you with the details.

** These, of course, are known in England as 'public schools'. :) But the reasonable reasons for this seemingly perverse nomenclature lie in history, long before the state decided to involve itself in education.

15. Matt in his blog just wrote that it might take a few years to have a definitive answer. Do we really have to wait that long or he is extremely cautious and exaggerates as usual?

16. Eelco HoogendoornJun 20, 2014, 3:18:00 PM

Yup. My father told me to go play football. My mother had no idea what I was doing. Yet I obsessively hacked away at my stupid qbasic programs from the first time I found out about it anyway. It brought me nothing in terms of social status, especially not with respect to the opposite sex, of course. Yet I loved it anyway, and I have no regrets 'wasting away' many a summer holiday this way. Every good coder shares a similar history. Only one of whom I know is a female, but she has lost interest in programming in adulthood.

On the one hand, one may think of such initiatives by google as a harmless thing, but on the other hand, any attempt to push people in an ideologically preconceived mould is probably going to be detrimental to net happiness. I think women in STEM is a great thing; those that really belong there really do have a very positive impact on the otherwise horrid social dynamics of an all-male workplace. But what is not such a great thing is burnt out women of average potential that have been propped up by this kind of positive discrimination, who find out once they have to carry their own weight in the real world, that they have no real passion for the subject, and quit their job after 3 months.
Meh. The good news is that the effect of such a campaign will be negligible anyway.

17. Dear Giotis,

the improved BICEP "2.5", Keck Array, has completed taking the data both at 150 GHz (like BICEP2) and 100 GHz (new).

https://en.wikipedia.org/wiki/Keck_Array

It could be released anytime soon. This year and in 2015, BICEP3 is taking data. All these things will dramatically improve the diversity of views how to look at the potential B-modes. The credibility of the current BICEP2 claims may decrease but more likely, it may increase substantially.

I am afraid that if someone is "really unconvinced" by BICEP2 now, he will probably remain unconvinced for many years because he will simply deny all the next experiments just like he denies the BICEP2 discovery right now. Indeed, it may take a long time for Planck to catch up or a similar space-based probe to be sent and collect the data that may satisfy these people.

It's in the eyes of the beholder. But if someone considers the findings by BICEP2 to be about equally likely to show something and to show nothing, I don't see a difference why he couldn't deny pretty much any other experiment, too.

Cheers
LM

18. I sometimes think that the clueless are in fact the ones in charge.
e.g. - http://www.math.umb.edu/~sp/papers.html
As they say -'read the whole thing'.
It's quite incredible.

19. Steven Pinker at your former employer Harvard says that the technical name for people who think little boys and girls are the same is childless.

20. The Earth spins on its axis (and not just in theory - Foucault pendulum). BICEP2 is at a spin pole. Is there no threatened doyen who can leak a non-symmetric affine connection (Christoffel connection. "Spin density" quadratic terms appearing in the Lagrangian) to elegantly muddy the waters more? For starters, was there a monitored Foucault pendulum at the BICEP2 site? NO!

Since a majority of theory does a accept cosmic background radiation vorticity, nobody need seek to observe it. Were there such a somebody, he would be thereby proven unqualified to perform the experiment for being theoretically wrong at the onset.

21. Hi John, you are absolutely right to be suspicious. I learned complex numbers in the 1970s at the age of 15 (fourth and not second year of secondary school), in Flanders (I'm deliberately not saying Belgium, because standards are much lower in the French speaking parts of Belgium - they really are). In the fourth grade math tests (Trends in International Mathematics and Science Study) Flanders usually scores relatively high. My memory may be failing, but I distinctly remember what fun it was to solve cubic equations with complex roots. It was about the same time I read Feynman's QED, and although I'm one of his most ardent admirers, I find his analogy to this day more confusing than simply explaining what a complex number is..

22. kashyap vasavadaJun 20, 2014, 7:44:00 PM

There is lot of unfortunate truth in what you are saying for our capitalist society. Electron tunneling in a flash drive is not easier to explain than QED. So most of the support from general public and politicians comes mainly from current or future (hoped for) applications and not from them being really anxious to understand the universe. On the other hand some laymen type approximate explanation would not really hurt science

23. Hey man, Lawrence Krauss also wrote a paper on it: http://arxiv.org/abs/1403.5166

24. I think QED the book does require a modification of Feynman's 65 remarks. Something that starts with it will take me twenty years to explain this to you....

25. andrew oh-willekeJun 21, 2014, 1:46:00 AM

The main function of peer review in scientific journals and academic scholarship generally, is to screen out crackpot submissions that less expert reviewers might not catch. The role of a peer reviewer in providing editorial suggestions is secondary.

26. "One can't even explain the actual general postulates of quantum mechanics to an average physics PhD..."

Do you have a reference &/or some keywords or something regarding this comment? Would I find what you're referring to in Von Neumann's QM book, or Parthsarathy's "Quantum Stochastic Calculus", or something like that?

27. If you want to painlessly learn coding in various computer languages for free, go to codeacademy. Soon, though, if things keep going the way they are in academia, there will be equal numbers of men and women coding, whether they are any good at it or not, because companies will be forced to have equality of numbers in employees, in directors etc. Merit based hiring practices are being marginalized.
If you so much as suggest that men and women have differences in brain structure, or differences in ability doing various tasks, prepare to be crucified by the pc inquisition.
It doesn't matter that it is bloody obvious to anyone with a brain---ask Lawrence Summers.

28. In the U.S.: I don't know at what point they teach complex numbers nowadays, but when I went to high school (graduated 1962) they were taught in Algebra II, which in those days was in either the sophomore or junior year (10th or 11th grade, age 15 or 16). For most of those who took the course, it was in the 11th grade.

29. Good to see this. I think they have done a good job of considering all systematics entering the experiment and my hunch is that the result will hold in the future.

Out of topic: Dear Lubos can you have a look at this question in physics.stackexchange http://physics.stackexchange.com/questions/119921/are-there-massless-bosons-at-scales-above-electroweak-scale ?

I have hand waved an answer but the questioner is persistent. The basic question is at the end "why no zero mass gauge bosons have appeared at LHC since the electroweak scale ( called symmetry breaking scale in this http://iopscience.iop.org/1742-6596/18/1/002/pdf/1742-6596_18_1_002.pdf ) is at around 100 GeV. I am out of my depth, but it is important that a good answer comes up on google searches in the future.

30. I doubt that, crackpot submissions are trivial to catch.

31. I came there and wrote that the question seems confused to me. By basic rules of gauge symmetry breaking, the gauge boson associated with the broken (generators of the) symmetry are *massive*. They cannot be massless at the same moment.

At most, they are approximately massless at energy scales much higher than their mass - where the mass may be neglected relatively to the energy, of course. And they may be massless at high temperatures, but in that case, the symmetry is also fully restored.

At any rate, having a broken symmetry combined with *massless* gauge bosons is a contradiction.

32. The 2-3 journal reviewers, if the editor has taken a bad choice, may wrongly reject good papers and adopt nonsense in the worst case.

Appropriate (international academic) peer review should be public and open to the whole relevant expert community instead of just being done by t most 3 hand picked people choosen by a journal editor, who's decisions may not always be perfect either ...

Dont you see the terrible small sample bias (known in statistics too for example) tradidional journal peer-reviewing suffers from? In addition to this statistical errors, there may be systematic errors, due to personal bias of the reviewers and editors ...

33. The data exists. Why is it not public yet? Why should the released files created with it, not accurately reflect it?
There seems a lot of effort to discredit with little evidence that this result is wrong. Probably like Lubos says jealous competition

34. thanks , Lubos.

You are not so active there now and it is a pity, but it is true that the site has very many elementary and confused questions.

John will overtake you in points !!

35. it has happened in the climate related field, "peer review" has become "club review".

36. The difference is now over 5,000 points in my favor - maybe I will care about it when it shrinks to 1,000 and I will escape from him a little bit. ;-)

But the leadership in this score hasn't really brought me anything so it's likely that I won't worry about losing it LOL.

37. Madness.. You're showing the worse of you, missing the most basic principles of communication, clearly not understanding the concept of person... You're damaging your own reputation

38. exactly my point.

39. Actually I'd say your argument is against Lubos, as it offers an alternate _cultural_ explanation for the gap, and he is arguing that it is not cultural.

On other hand, arguments paralell to yours imply that a way to narrow the gap could be to ask for 50% quota in occupations were women are majority. Small shop sellers, supermarket cashiers, office cleaning etc.

40. The plan of Google... is it to atract most women, or an small elite? Because perhaps they are just competing for a niche of high IQ persons who actually is distributed in other areas, and they do not worry at all about arguments about median skills.

41. Concept of person? LOL. I have no idea what you're talking about. Maybe you need some medical help.

42. Dear Tom, it's fine, you hate modern science, and I consider people hating modern science to be jealous inferior morally crippled animals and I emphasize this point because I find it important to emphasize the important truths. I hope that we can survive these differences.

43. Agree 100%.

44. It's precisely what I say all the time. I'm a cleaning woman, here at Princeton U. and I keep telling people how important it is to take care of dust properly, but they don't want to listen to me. It's so frustrating!

45. Haha! ✓ :)

But, courtesy of The Wayback Machine, voilà!

47. How odd. The link was good just two days ago.
Is it a nefarious conspiracy?
Is it the power of TRF?

Inquiring minds want to know!

48. Statements such as yours without reference are hard to argue against.

Example - In what way does the article state the gender gap is cultural? Rather is states it is biological.

The article argues for an emancipation of women from the claptrap of "equality in everything". Thus it argues for women's happiness. I see no hatred.

No one with sense cares about a non-cultural, natural gap and they certainly reject Marxist quotas in any sphere.

49. I found the talk of Zaldarriaga on the topic very entertaining: http://www.ustream.tv/recorded/47637248

50. Grace Hopper

And, oh yes, this

51. Hey Lumo, are you now on an answering spree there, LOL ?

Damn it, now I have to read them even though I am rather minimizing my hanging out time there ... :-(0) ;-) :-)

52. The problem is the same you refer to: the real world is complex. First, even elementary distinctions such as "often does not" vs. "never does" and "rarely does" vs. " shouldn't" are fluidly confused. So "women don't work often in IT" quickly becomes "girls should be excluded from IT schools", if you're not careful. Second, not all IT jobs require a genius lifestyle programmer. Instead, a lack of IT skills leads to unnecessary exclusion from a lot of jobs. For instance a female classmate of mine started an IT company although she didn'y show any special interest in high school. If the university had a no-girls policy less wealth would have been generated.