Monday, February 09, 2015

If done right, temperature adjustments are great

Many skeptics' adjustment-phobia unmasks their anti-scientific credentials

Christopher Booker, whom I met in Nice a few years ago and whom I like, wrote the most read Earth-category article in the Telegraph over the last 3 days,
The fiddling with temperature data is the biggest science scandal ever
The title summarizes the main point of this article that presents many examples suggesting that the temperature data from the weather stations have been repeatedly retroactively adjusted.

I tend to agree that these adjustments are likely to have made the warming trend look higher – and the case for global warming more robust – than the most accurate data would manage. However, I am not quite certain about the size and relevance of this effect and I feel very uncomfortable about many climate skeptics' knee-jerk emotional reaction showing that they hate the very idea of an adjustment.

For this reason, I also agree with Steven Mosher, an independent climate skeptic (with experience in graduate studies in literature and management in 3D graphics companies) who has also co-authored a book about Climategate. In a guest blog post,
Guest post : Skeptics demand adjustments,
he is presented as a co-author of the Berkeley Earth temperature record (BEST). He criticizes the climate skeptics who want to interpret every adjustment as something criminal.

Instead, he explains that adjustments are used because they're good things when they are done right.

Social security may send you more accurate payments adjusted for inflation, adaptive optics improves some telescopes as well as Ronald Reagan's Star Wars, eyeglasses adjust some vision disorders, and similarly adjustments may be applied to remove as accurately as we can the spurious effect of various biases that affect the weather stations – those arising from location changes, the time-of-observation bias, a detectable change of the instrument, and others.

The LBT telescope uses adaptive optics to make "adjustments" and achieves a sharper image than the Hubble Space Telescope. Almost all the distortion caused by the atmosphere is basically removed.

Adjustments are a good thing and the instinctive criticism of all adjustments as a matter of principle is simply not right. I agree with Mosher: these "principled" critics of all adjustments are surely throwing the baby out with the bath water. And by the way, I do agree with the description of those who get crazy whenever somebody mentions the word "adjustment" as anti-science nut jobs, and yes, I do think that a large number of such people exists among the WUWT regular readers (but probably among most laymen in the world, too, and maybe they are a majority among the TRF readers as well, sorry).

Check that the red, green, and blue curves agree almost exactly.

But one essential argument appears at the very end of Mosher's text: Whether or not you apply one of the specified major adjustments or two or none of them doesn't visibly affect the resulting graph at all. It changes nothing about the qualitative picture. So not only is the "principled" criticism of all adjustments conceptually misguided: Even if this criticism were justifiable, the conclusion that "the result of an analysis that used some adjustments is compromised" would still be incorrect because many adjustments simply don't matter for many questions!

I am not saying that the local temperature data are unaffected by such adjustments – they surely are (and I guess that most examples mentioned by Chris Booker plus similar examples probably really mean that the "historical chronicles" give us a more accurate picture than the adjusted temperature records). But when we talk about the global mean temperature, the effect of all these adjustments seems negligible.

It is hard to be too confident about the temperature record from 1850, to mention an example. I am not sure whether it's almost correct or almost completely wrong. But the agreement between the different datasets that treat the homogenization and other issues differently – NASA's GISS, NOAA's NCDC, the Japanese global dataset, BEST, and Hadley Centre + Met Office's HadCRUT4 – leads me to believe that all the methods that these surface weather station scientists apply to the raw data are basically right.

If there's something wrong about the weather record, it must affect pretty much "most" of the raw data from the weather stations. Those raw data are used in pretty much all the datasets mentioned in the previous paragraph – which could explain the high degree of their agreement (even if all of them are very inaccurate). But the global averages seem to be rather robust with respect to the omission of a substantial percentage of the weather stations etc. so I tend to believe that the graph of the surface global mean temperature isn't mainly determined by several "bad apples".

In particular, we have the problem of the urban heat islands. While it is obvious that the urban heat islands do add a temperature change comparable to 1 °C which may depend on time (we know this effect from the Prague-Klementinum station that has been recording the data for more than 200 years; and this downtown Prague weather station is the warmest one in all of Czechia today), I have never believed that this issue was treated very badly or neglected by the folks who evaluated the surface temperature record. For this reason, while I found Anthony Watts et al. project mapping the weather station near asphalt to be amusing, I've never believed that this was really deciding about the "big questions" of the climate debate. My near-certainty that the bulk of the 20th century global warming shown by the surface weather record is not due to the asphalt was growing over the years.

The temperature record resulting from the weather stations differs from the graphs generated by the satellites – not by too much but substantially enough for the satellites to completely deny the claim that 2014 had a chance to have been the warmest year, and so on. But the differences between the satellite and weather-station-based temperature records probably mean that these two methodologies measure "substantially different quantities" (quantities with inequivalent definitions) that shouldn't be assumed to behave equally, and that's why they don't behave equally. The disagreement between the two methodologies isn't necessarily due to any "big error" on either side.

There's been some personal tension between Mosher and Watts – who have been friends in the past – and I won't amplify it by mentioning any details. But while it's true that one must be careful not to apply adjustments incorrectly or selectively in order to push the data in a fixed direction, one must also honestly recognize that many adjustments make the underlying science (or technology) more accurate – and many of these adjustments don't matter much, anyway.

The skeptics who contradict the last two principles are approximately as blinded and as biased as the climate alarmists who have really managed to spuriously increase warming trends by a selective application of would-be clever "adjustments" to the datasets. Well, while they're equally blinded, they don't want us to waste trillions of dollars, unlike the alarmists! ;-)

From most aspects, your humble correspondent is no lukewarmer at all. I think it's fair to say that the recent global mean temperature change is zero for all practical purposes, the percentage of the variability caused by (largely unpredictable) natural drivers is so high that the fact that the human contribution is nonzero may be ignored, and the threat of a significant global disruption coming from the climate change over the next 100 years is basically non-existent, and the proponents of the climate hysteria who have benefited from this hysteria should be treated exactly as all other fraudsters who have stolen tens of billions of dollars and who want much more to come.

But if you talk about the relationship to some basic procedures used in the scientific method and technology, not just in the climate science – e.g. to "adjustments" – I am surely a "lukewarmer", together with Steven Mosher, because the label of the full-fledged skeptics has been hijacked by anti-science nut jobs, indeed.

A reply to a blog post

A blogger named Steven Goddard whose actual name is Tony Heller posted a reaction to this blog post of mine. Here is my answer:
Dear Steve, the satellite record measures a different quantity than the weather-station-based global mean temperature, so there is no good reason to think that their graphs and trends should agree. In other words, the disagreement between these two different types of datasets doesn’t imply that there is a mistake in either of them.

Aside from this flawed argument, you haven’t offered *any* other argument that would imply that the adjusted graphs are wrong and the graphs preferred by you are right. It’s just your emotions and prejudices and the rational content is zero.

I hope it is OK if I think that you are just one of millions of people – on both sides – with equally irrational and biased attitudes to all these questions, so these three paragraphs represent all the time I will dedicate to your blog post.


  1. I agree, many adjustments are quite useful. I point this out to people all the time. I actually agree with a couple of the adjustments to the temperature record, like the time of day and equipment change adjustments.

    BUT then we hit the homogenization process as used by NOAA/GISS. Those adjustments are substantial and growing (honestly, the latest version looks like an exponential curve).

    In the case of NOAA/GISS the problem stems from the fact that in adjusting out the station moves, they inadvertently put back most of the UHI adjustment. This is because one of the main reasons for a station move is remove UHI contamination. So when the homogenization routine realigns the break point, it artificially lowers past temperatures and can create a warming trend where one wouldn't have existed otherwise.

  2. Lubos - I hate "chiming in" messages; I suspect them of being based on nothing substantial or real, little more than a desire for approval. But I can't resist making an exception this time. I have always thought the people who make blanket condemnation of temperature adjustments were yahoos.

    Aside from that: Are you aware of Ross McKitrick's argument that UHI is geographically more extensive than the establishment believes? If so, what do you think of it? He argues that it's significant even in small communities, and their growth over time creates a warm bias.

  3. Look at the continental USA.

    I agree on your overall premise. I know that the USA is only some small percentage of the planet, and changes to its data do not substantially alter the global temperature trend.

    Still if the USA has cooled over the last 100 years, it would not make AGW easier to swallow for the American public. Due to temperature adjustments over the past 15 years the USA has gone from a region undergoing cooling to one heating up. A link with some graphs and quotes from Hansen, etc.

    If one is to sell AGW to the US consumer, then a cooling USA would not help. Good intentions or not on the adjustments, the thing smells pretty bad.

  4. There is really no way to quantify the effect of urban heat islands it is surely non-zero. The satellite measurements are much more meaningful.
    By contrast, the local (coastal) measurements of sea level changes are much more meaningful than satellite measurements, which, just as in the temperature case, are actually measuring a different quantity. Water really does seek its own level and that makes it possible to do meaningful averages of the data from the two thousand or so tidal gauges.
    The increase in CO2 has had no measurable influence on our sea levels.
    I am convinced that no attempt to reduce CO2, no matter how costly, can have an sea-level effect greater than two inches by the year 2100.
    I have picked sea level forecasts because that is where the GW alarmists are in their greatest detachment from reality.

  5. Watts got started because it was claimed that all sites in the US were in parks. So he organized a project and disproved that. Needless to say, that left folks in the US with a lot of skepticism of official temperature records. Also, US data does appear to have been consistently adjusted in a way that substantially increases warming. I have no idea if it is correct, but it suspicious.

    The best criticism I have read recently is that the individual corrections are not documented. If that's true, that criticism is valid. All of the data, including the corrections and the reasons for them should be open to all. Such a criticism is not anti-science.

    Unfortunately, the climate (sorry) surrounding the discussions has been poisoned by reasonable distrust by skeptics, and fear and loathing by the establishment. And, of course, a lot of the skeptics making noise have no idea about how to do measurements or adjustments in the first place.

    Your comment about magnitudes is important in evaluating the overall effect and meaning.

  6. Lubos, go over to Tony Heller's site where he shows time and again that the "adjustments" are always not only to warm the present but also to cool the past. They are making cooling adjustments now to data that is over 30 years old! There is no scientific justification for that. Not only this but most of the US continental data is not real, it has been made up! A site will not send data or stop sending data, but that does not stop them reporting it. Oh no, they just make up the data as they need it. Really scientific that is. Also they do things like not have any data for large portions of the world's land masses (such as Africa) and so they just make it up and lo and behold the made up data always has a warming trend.
    Everyone has every right to be angry about this. There is no scientific justification for what is being done.

  7. Fine, but you are automatically assuming that the older answer had to be the more accurate one. The main "argument" in this belief of yours is pure prejudice.

    The U.S. could have warmed or cooled a little. Neither answer is "more troubling" than the other one and if the Americans were at least slightly rational, this modification couldn't affect their opinion whether the adjustments were done right or not. Well, some slightly greater cooling would actually be a threat.

    Without any specific U.S. data, it seems more likely to me that the U.S. 48 saw a warming trend in the last 100 years or 40 years because the world average almost certainly did.

    As I and others calculated, about 30% of weather stations saw the negative sign in the recent 80 years or so. So the warming is more likely for a random place than cooling. Because the U.S. 48 is 2% of the surface of Earth which is a small percentage but still a big area relatively to the smallest places where the weather differs from others, it's actually much more likely than 70% (because much of the local noise averaged out) that the average temperature of this area saw an increasing trend (via linear regression) over the last 80 years or so.

    So if one believes that the global average did what it did, it is reasonable to think that the U.S. average probably saw the same sign of the change.

    But it's very clear - even from your comment - that your reasoning has nothing to do with a legitimate measurements or logical arguments. You just don't want a certain sign of the slope of the U.S. 48 temperatures, so you decide that the result that gets this undesirable sign has to be fraudulent. This is simply not a valid argument, it only shows your prejudice.

  8. Hi, sorry but even if some correction is undocumented, it doesn't really prove that it was incorrect or fraudulent.

    There exists an obvious "omnipresent" problem that could have made the temperature trend look spuriously cooling - or that could incorrectly lower the warming trend - and it's the following:

    A station (measuring T1(t)) is often moved from a place that is already affected by the urban heat to a nearby place (where we measure T2(t)) without the urban heat island effect.

    Now, because the stations are so close, one could identify T1 and T2 and simply splice the pieces of the graph from T1(t) and T2(t). But that's incorrect because the urban heat island already made T1 warmer than "it should have been". So if we want to measure the temperature of the place consistently, we must adjust T2 to the estimated temperature we would get at T1, and this adjustment really means a discontinuous positive (warming) jump added to the temperature graph corresponding to T1-T2, to the size of the urban heat island.

    If this correction were omitted, and it was often omitted, one was getting an incorrect "cooling signal" which was caused by the transfer of the station from a warmer (UHI) place to a cooler one. It is undoubtedly correct to apply the adjustment, the positive jump by T1-T2 times theta(time), the estimated urban heat island that was measured at the old UHI-spoiled station.

  9. Hi, good to learn that Steve Goddard isn't the real name. I follow that blog regularly.

    But the idea that the hyperlink to the main page of the page - which is what you gave me - contains evidence supporting your claim - supports your claims is as demagogic as the claim that the IPCC report contains evidence of a dangerous climate change. In both cases, it simply doesn't.

    Stations were moved to cooler (UHI-stripped) places in the past, and that's why corrections are applied even "retroactively". See my reply to mesocyclone below. Corrections of this type indeed *raise* the calculated warming trend, and the new, corrected value is the right one.

  10. Dear Smoking Frog, good to hear about some synergy because, as you can see, the yahoos dominate in the comment section! ;-)

    UHI is relevant at a significant percentage of the places but it is not true that every town which has been or is effect by UHI has to throw away all its temperature data. Instead, it may use them when the corresponding adjustments are made to optimally estimate what the temperature would be doing at the place if the urbanization didn't occur on that place!

    As I clarify in the reply to mesocyclone and elsewhere, indeed, it's been typical that stations were moved to cooler places - away from urbanization - and that would have added a spurious cooling jump of the temperature down, and this methodology-caused downward jump has to be subtracted which means that the post-adjustment warming trend will look higher (more warming-like). But the adjustment is right and the post-adjustment graph is a more accurate representation of the underlying dynamics. Because this (transfer of weather stations to cooler places in a town etc.) was happening at many places, the same adjustments affected many places which means that even the averages for countries and the whole land had to get these legitimate "warming increasing" adjustments if they were previously forgotten.

    I do want to believe that Dick understands *this* but I am not sure about anything.

  11. Dear Gene, is there no way to quantify UHI? Why don't you just measure it by thermometers inside and outside the urban heat island?

  12. I think you are wrong about the Urban Heat Islands, primarily because the IPCC reports essentially wave them away by citing a Phil Jones paper about temperatures in China, supposedly demonstrating that urban heat islands are not a significant issue.

  13. The fact that it was written (also) by Phil Jones doesn't prove that it is wrong.

    I believe that the statement is correct, too. For the global mean temperature, the percentage of the stations for which the (changing, otherwise irrelevant) UHI mattered sufficiently is so small that they won't change anything qualitative about the global mean temperature's trend.

  14. I get what you're saying

    Imagine it as three potential station locations, one just outside the UHI impacted area of a city, another even more distant, and another even farther away.

    For the sake of simplicity, each potential location has nearly identical temperatures. As the city spreads engulfing one location after another, the station is moved. Each time the move results in break point and a return to a largely uncontaminated state.

    From what I can see, it doesn't matter if you treat the moves as break points in a single record or as multiple records...unless you remove UHI for every single station move, it will LOOK like temperatures are increasing even though they aren't. Yes, I'm aware this is going to be the biggest problem with multiple moves. But from what I can see, most of what we can do to stitch the record together is going to just inadvertently keep sticking in the dominant PHYSICAL station maintenance correction.

    I suppose a simple way of dealing with each of these methods to see if it is indeed a to try multiple, fictitious datasets with the conditions of this see if BEST, Hadley, GISS and other methods exaggerate fictitious temperature trends. If they do, there's almost certainly going to be a problem in the real-world data. Me, I'm a bit too lazy for that. I bet SOMEONE could do it though.

  15. Lubos: like most sceptics, I agree that some adjustments to actual recorded temperatures are essential in order to standardise the record over time. However I'm not happy with the widespread use of the term 'urban heat island' as if there was a large number of on-land climate stations so remote from the man-made built environment that they are in a state of nature. Almost by definition, an on-land weather station has to have some sort of structure built round it, access for vehicles, nowadays probably a power supply. All of these installations may affect the local microclimate. They can indeed be adjusted for if the immediate environment of the station is controlled and does not change with time. But then there is the slightly further out environment. What effect does new construction at distances of, say, 10s to 100s of metres from a station site have? Has this even been studied? I have the impression that over the last few decades more and more of the remaining on-land weather stations are sited at airports, rather than on greenfield sites. Airports in general have a habit of expanding over time. How can the effect of such expansion be confidently adjusted for?
    When world-wide thermometer measurements got going in the late 19th century the population of the world was perhaps 1.5 milliards (thousand million). There were few paved roads, no airports, and few motorised vehicles. Nowadays we have perhaps 4 times the number of people. Virtually all those people live in some kind of built shelter, as they did in 1900. But now they require 4 times as large a built environment just to supply their housing needs. More of them use motorised transport than was the case in 1900, and that tranpsort moves faster so requires more road space with a more solid surface. If they have a job they work in a building constructed for that purpose, with heating and/or air conditioning becoming standard. All of these developments primarily affect the local microclimate. But they will also have some effect further away. How can we measure such an effect? Even greenfield sites must be assumed to be affected by the built environment up to at least a few km away, unless we can demonstrate otherwise. So the first question is: How many on-land climate stations are so remote from the built environment that we can ignore any possible effect of the built environment on temperature measurements?

    And that's just the built environment! There is also the matter of the much larger area affected by non-built land-use changes.

  16. Dear br, by the statement that the UHI affects a very small percentage of the land surface, I mean not only the counting in squared miles but also the percentage of actual weather stations.

    Over 80% or 90% of the weather stations are so far away from any significant UHI - they are rural or sparsely populated towns - that the effect is simply negligible.

    If the urbanization had some mysterious effect that influences even villages that are 10 km or 100 km away from the town, well, then they do and it will be reflected in the rural stations, too. This will be a real effect and it's correct if this real effect is shown in the temperature records. But this effect wouldn't be "urban heat islands". Note the word "island" which, by definition, means isolated regions covering a small percentage of the area.

    Of course that the thermometers can't attribute where the measured temperature differences come from. They can't tell us it's due to the solar variation, PDO, CO2, or faraway asphalt.

  17. Lubos, you mixed it up.

    Mosher did not call people who question adjustments anti-science nut jobs.

    He just called the people at WUWT 'anti-science nut jobs' as in saying even people with such low intelligence could agree to not call Skepticalscience 'SS'.

  18. Dear Shub, I haven't mentioned the funny exchange whether the acronym of Skeptical Science is "SS" at all - if I have to say it now, then indeed, the right acronym of the website is "SS" :-) - so your additional story contains zero evidence that I have mixed something up.

    Most posters on WUWT, and not only there, question adjustments, and from the viewpoint of high enough standards, it makes sense to call them anti-science nut jobs.

  19. I am referring to this:

    "I do agree with the description of those who get crazy whenever somebody mentions the word "adjustment" as anti-science nut jobs ..."

    Who did you agree with?

    The evidence is here:

    You are right that there are lots of people who instinctively react to adjustments and declare the whole temperature thing to be nonsense.

  20. But lubos: many stations weren't moved to cooler places; the cities, and airports they were located in grew up around them...rural airfields with a vegetative cover in many places became asphalt covered heat sinks.
    recording stations were located in the wash of air conditioner condensing units...parking lots and the like.
    Without documentation on each station, and its change in its site characteristics over the years, along with the documentation for adjustments, how can anyone draw any conclusions about the appropriateness of the adjustments?

  21. Dear David, what can one do with such obviously meaningless questions such as yours?

    Obviously, if you don't give me any specific information or data about some abstract adjustment you seem to talk about, I can't draw any conclusions about it or construct any proof of anything.

    But neither can you, and that's really my main point.

    There are lots of very good reasons why very many adjustments were made, and why the magnitude of the adjustment was what it was. You just can't assume that something was *wrong* just from the information that there were adjustments.

  22. Lubos, I left a comment here a while back. It seems Disqus has eaten it up! Could you please look?

  23. Sorry, you have posted a virtually identical comment before. Why are you repeating yourself?

    I am confident that I agree with Steven Mosher here about the desirability of adjustments, about the stupidity of rejecting them as a matter of principle, about the fact that many people like that are found on "mass" climate skeptic activists blogs, and I even agree with him that it's infantile to be repeatedly using silly acronyms such as SS because it's really bad when such (playful) observations become a pillar of someone's argumentation.

  24. My posting 'record'?

    I must not have posted more than a dozen comments here, over the years.

    Quite an opportune moment to be using 'posting record' as an excuse to censor comment, don't you think? Just when you were shown to have made a small and silly mistake.

  25. Dear David, fine, but I am not even sure whether you correctly attribute all this work.

    This is done by meteorologists, right? I have doubts that meteorologists inheriting the old weather station data get grants for one temperature graph or another.

    It's the climatologists who get grants for papers "interpreting" these things.

  26. nah...Gavin Schmidt at nasa massages data sets, as did Hansen, and Michael Mann asserted that his work product was his personal property, and therefor was not subject to FOIA requests....
    The fine fellows at Hadley also massage their data sets...I haven't been able to find their algorithms used for making their adjustments.
    Phil Jones is notorious for losing his original data.
    I agree with you that many concerns voiced by the "skeptic" community are baseless, but I also feel that the climate science community isn't even a little transparent, at all.

  27. Some years ago I downloaded temperature data sets (raw data) that covered many land based stations in Norway, Sweden, Iceland, Denmark and Finland form about 1890 – 2005.

    I plotted curves for the individual country's and also combined them all I one graph with no fiddling or adjustments.

    None of the graphs had warming trend's close to the official graphs at the time which looked more like copies of the world warming trend issued by the IPPC.

    The only stations I could find that had a significant warming trend were located in expanding cities.

    The raw data sets are no longer available for download. Wonder why….

  28. This should cheer you up, Obama called the Paris killings that the Kosher Hypermarket "random shootings."

  29. Would you clarify it more for me. Do you see yourself as anti-immigration for economic reasons, or white for cultural purity or white power or Vanilla Nazi. Or what.

  30. I'm simply saying that the natural state of the normally maintained temperature record is going to look like this

    Move the station to avoid UHI and the day/night temperatures break low. Paint it every few years and the day temperatures break low each time.

    Its very difficult, especially in a warming world, to know what was a real warming segment and what needs to be adjusted out. So I'm all for adjustments. I just don't think we're going to get it right. We're stuck with waiting for the satellite data to accumulate so we can get a better idea.

  31. Shannon, I am a businessman myself and I take a great joy that I have created jobs for hundreds who feed thousand mouths. And also mentored high level people one of whom became a millionaire, and created good value for my customers.

    However, I have also seen the mindless destructive greed. People kill for the green, not just the petty criminal. That is the original reason for religion to temper some people's lust and selfishness.

    I think any immigrant should come on real need base and should be able to integrate well with the host, it is to the benefit of both parties. In some respect I agree with John because many individuals official and non officials push this problem for pure self interest.

  32. A self-explanatory example of inevitable data adjustment is the correction of the tides and sound velocity profile in depth data during processing of bathymetric surveys. You can not do anything if these corrections are not applied correctly. In fact most of the problems with the data of this type, especially when compared with previous studies, come precisely from the tides (and its vertical reference and/or geodesic issues), and of course not always are harmless errors or simple confusion (e.g. a few inches up or down can pose many cubic meters of material to dredge in benefit/detriment of one of the parties).

    So I find it hard to believe that informed people may accuse `the adjustments´ by themselves, but more likely to malicious biases on those adjustments, which I suppose is the critique of Lindzen and others.

    Also I do not think that's the biggest problem with data from ground stations, rather are the poor coverage and resolution which makes them vulnerable to criticism.

  33. No disagreement. The criticism about documentation is not to show that the data is wrong The point is that it is better to have the information available for review, especially in such a contentious field.

  34. Perhaps I am confused but five or six decades ago, when I was actually publishing in scientific journals, we called these things systematic errors. You surely must make adjustments whenever you understand what is going on and you must be crystal clear when you do not.
    The difference between good and bad adjustments is just the difference between science and fantasy.
    I completely agree that ground station measurements are often questionable and are far too sparse to be taken as representing global changes.

  35. If, like my twin brother, you live in Manhattan in mid-summer, you can go a few miles in any direction and find the temperature about three or four kelvins cooler.
    I’m sure this is an exceptional case and that, globally, your opinion that UHI influence is quite small is quite true.
    The main discrepancy between terrestrial and satellite measurements is just that they are measuring different things.