When someone starts to shout “but its in the peer-reviewed literature” as an argument-ender to me, I usually respond that peer review is not the finish line, meaning that the science of some particular point is settled. It is merely the starting point, where now a proposition is in the public domain and can be checked and verified and replicated and criticized and potentially disproved or modified.
The CRU scandal should, in my mind, be taken exactly the same way. Unlike what more fire-breathing skeptics have been saying, this is not the final nail in the coffin of catastrophic man-made global warming theory. It is merely a starting point, a chance to finally move government funded data and computer code into the public domain where it has always belonged, and start tearing it down or confirming it.
To this end, I would like to share a post from year ago, showing the kind of contortions that skeptics have been going through for years to demonstrate that there appear to be problems in key data models — contortions and questions that could have been answered in hours rather than years if the climate scientists hadn’t been so afraid of scrutiny and kept their inner workings secret. This post is from July, 2007. It is not one of my most core complaints with global warming alarmists, as I think the Earth has indeed warmed over the last 150 years, though perhaps by less than the current metrics say. But I think some folks are confused why simple averages of global temperatures can be subject to hijinx. The answer is that the averages are not simple:
A few posts back, I showed how nearly 85% of the reported warming in the US over the last century is actually due to adjustments and added fudge-factors by scientists rather than actual measured higher temperatures. I want to discuss some further analysis Steve McIntyre has done on these adjustments, but first I want to offer a brief analogy.
Let’s say you had two compasses to help you find north, but the compasses are reading incorrectly. After some investigation, you find that one of the compasses is located next to a strong magnet, which you have good reason to believe is strongly biasing that compass’s readings. In response, would you
- Average the results of the two compasses and use this mean to guide you, or
- Ignore the output of the poorly sited compass and rely solely on the other unbiased compass?
Most of us would quite rationally choose #2. However, Steve McIntyre shows us a situation involving two temperature stations in the USHCN network in which government researchers apparently have gone with solution #1. Here is the situation:
He compares the USHCN station at the Grand Canyon (which appears to be a good rural setting) with the Tucson USHCN station I documented here, located in a parking lot in the center of a rapidly growing million person city. Unsurprisingly, the Tucson data shows lots of warming and the Grand Canyon data shows none. So how might you correct Tucson and the Grand Canyon data, assuming they should be seeing about the same amount of warming? Would you
average them, effectively adjusting the two temperature readings
towards each other, or would you assume the Grand Canyon data is cleaner
with fewer biases and adjust Tucson only? Is there anyone who would not choose the second option, as with the compasses?
The GISS data set, created by the Goddard Center of NASA, takes the USHCN data set and somehow uses nearby stations to correct for anomalous stations. I say somehow, because, incredibly, these government scientists, whose research is funded by taxpayers and is being used to make major policy decisions, refuse to release their algorithms or methodology details publicly. They keep it all secret! Their adjustments are a big black box that none of us are allowed to look into (and remember, these adjustments account for the vast majority of reported warming in the last century).
We can, however, reverse engineer some of these adjustments, and McIntyre does. What he finds is that the GISS appears to be averaging the good and bad compass, rather than throwing out or adjusting only the biased reading. You can see this below. First, here are the USHCN data for these two stations with only the Time of Observation adjustment made (more on what these adjustments are in this article).
As I said above, no real surprise – little warming out in undeveloped nature, lots of warming in a large and rapidly growing modern city. Now, here is the same data after the GISS has adjusted it:
You can see that Tucson has been adjusted down a degree or two, but Grand Canyon has been adjusted up a degree or two (with the earlier mid-century spike adjusted down). OK, so it makes sense that Tucson has been adjusted down, though there is a very good argument to be made that it should be been adjusted down more, say by at least 3 degrees**. But why does the Grand Canyon need to be adjusted up by about a degree and a half? What is biasing it colder by 1.5 degrees, which is a lot? The answer: Nothing. The explanation: Obviously, the GISS is doing some sort of averaging, which is bringing the Grand Canyon and Tucson from each end closer to a mean.
This is clearly wrong, like averaging the two compasses. You don’t average a measurement known to be of good quality with one known to be biased. The Grand Canyon should be held about the same, and Tucson adjusted down even more toward it, or else thrown out. Lets look at two cases. In one, we will use the GISS approach to combine these two stations– this adds 1.5 degrees to GC and subtracts 1.5 degrees from Tucson. In the second, we will take an approach that applies all the adjustment to just the biases (Tucson station) — this would add 0 degrees to GC and subtract 3 degrees from Tucson. The first approach, used by the GISS, results in a mean warming in these two stations that is 1.5 degrees higher than the more logical second approach. No wonder the GISS produces the highest historical global warming estimates of any source! Steve McIntyre has much more.
** I got to three degrees by applying all of the adjustments for GC and Tucson to Tucson. Here is another way to get to about this amount. We know from studies that urban heat islands can add 8-10 degrees to nighttime urban temperatures over surrounding undeveloped land. Assuming no daytime effect, which is conservative, we might conclude that 8-10 degrees at night adds about 3 degrees to the entire 24-hour average.
Postscript: Steve McIntyre comments (bold added):
These adjustments are supposed to adjust for station moves – the procedure is described in Karl and Williams 1988 [check], but, like so many climate recipes, is a complicated statistical procedure that is not based on statistical procedures known off the island. (That’s not to say that the procedures are necessarily wrong, just that the properties of the procedure are not known to statistical civilization.) When I see this particular outcome of the Karl methodology, my mpression is that, net of the pea moving under the thimble, the Grand Canyon values are being blended up and the Tucson values are being blended down. So that while the methodology purports to adjust for station moves, I’m not convinced that the methodology can successfully estimate ex post the impact of numerous station moves and my guess is that it ends up constructing a kind of blended average.
LOL. McIntyre, by the way, is the same gentleman who helped call foul on the Mann hockey stick for bad statistical procedure.
The giss methodology for adjusting the raw data is published on the NASA web site (hanson 1998 and 2001). If that’s not enough they also have the code. The raw data is also available. Check it out. It’s a great resource. You can grab the raw data and drop any stations you don’t trust if you don’t like the NASA method.
And ChiefIO has shown just how awful the GISS code really is.
I see you are aware of the code. So what is this talk about “these government scientists, whose research is funded by taxpayers and is being used to make major policy decisions, refuse to release their algorithms or methodology details publicly?”
That’s not just misleading, it’s plain wrong.
I think it’s sad that this site seems to want to keep people ignorant and angry when we should be engaging people in science. Climate science is an area where there are gigs of raw data that even a layman can access and investigate. Instead of linking to it we are telling people that it doesn’t exist. It does! Let’s dig in!
j:
I guess in your zeal to dis, you missed this:
The code was not released until later that year, after a great deal of teeth-pulling. As Stan points out, even with its release, it’s taking some very dedicated detective work to even begin to figure out what the heck it does.
http://www.climate-skeptic.com/2007/09/a-good-first-st.html
Sorry, but maybe you ought to do your drive-bys a little more slowly.
??? The code was released in 2005! This is 2009! You don’t think it’s a bit disingenuous to say in 2009 that they “refuse to release their algorithms or methodology details publicly”???
Again, this site just wants to keep people ignorant and angry instead of trying to enlighten.
j:
Try adding some more question marks.
The code was released in Sept 2007. The post above was originally posted in July 2007. People took a stab at the code after it was released and no one got it to run until Chiefio got parts of it to run a few months ago. He thinks he’s found something odd, but everyone else has been kept busy trying to understand why climate scientists can’t plot lake sediment proxies right-side-up. And now, of course, we have emails.
All in due time.
I’m not sure why you can’t seem to get these dates straight in your head.
Do we have a Global Warming?
Just simply see pictures at
http://www.cf.ac.uk/maths/subsites/zhigljavskyaa/climatechange/
It looks like you are right. Regardless, do you really think it is open and honest in 2009 to put up a post with the words “refuse to release their algorithms or methodology details publicly” when the code is available not just to academics, but on their webpage, and the methodology was described in a paper published in the 1990’s.
Here’s a better question. How many of this sites readers knew that the data, the methodology, and the code is freely available? If they did, I’m sure they didn’t learn it from this site.
I was directed here by someone who suggested it would provide an objective skeptical view. I hope that this isn’t considered to be one of the more objective skeptic sites.
I’m going to look elsewhere. Perhaps one of the readers can suggest a blog that outlines the non-consensus viewpoint without trying to obfuscate the facts?
j:
I suppose “This post is from July, 2007” is an example of “hidden in plain sight”.
And if you have trouble reading “This post is from July, 2007”, I could just imagine the fun you’d have reading the code, which appears to be a combination of fortran, pig latin, gibberish, and Golic Vulcan.
As far as an “objective skeptic site”, give me an example of an “objective warmer site” so I can gauge what you mean.
Of course, rather than seeking some unachievable ideal of “objective”, which appears to get you all tied up in your shorts, maybe you can just try reading some “skeptic sites” and challenging yourself to apply your own objectivity.
j,
could you please explain to us the adjustments that GISS uses??
In the same vein, could you please explain to us the adjustments that NOAA, where GISS gets their data, make??
You might also show us whether NOAA actually has the RAW data so we can do comparisons.
Yes, it has been a long time and it is ludicrous that no one will show us what is being done.
I can only claim minor programming skills. What I have looked at is so bad that none of the companies I have worked for since 1973 would have retained people who made such messes for others to clean up.
J.
The bedrock of science is the ability to replicate processes. Please identify a single individual who has been able to successfully replicate the GISS process, or even their “adjustments”.
Non-replicatable processes may be common in “climate science” but not in actual science.
“It looks like you are right. Regardless, …” translates, I suppose, as “I don’t intend to let your facts undermine my prejudices”.
See Al Gore’s response to Climategate here
http://www.youtube.com/watch?v=5ymxLA5oRYI
But it leads us to the ‘correct’ Political conclusions…so who cares if it’s factually accurate or physically correct.
As if that were the point at all!
Hey, what happened to the Tucson U of Az data prior to 1890? If someone already mentioned that, excuse me. Looks like they just removed it because it didn’t “fit”, doesn’t it?
“j” is the perfect example of a faux-intelligent warmist: Trying so hard to appear literate on the data and absolutely demolished by the fact-based “John M”s of the world. “j” is delusional and foolish and armed with only an out-sized ego regarding his own knowledge. Those of us who dwell in the world of truth have long known AGW was bullshit and are enjoying the hell out of not only its’ deconstruction but also the astounding twists and turns the “j”s and algores of the world have been forced into. Conservatism, anti-leftism and truth will ALWAYS win in the face of the “j”s of the world and their deceitful ways…
sherlock: according to wikipedia (insert wiki disclaimer here) the U. of Arizona was founded in 1885 and held its first classes in 1891, so it seems to me not at all unreasonable that there would be no data from a U of A site prior to 1890.
I wonder if the extremist who opined that all who do not believe in global warming should be tried like the Nazi war Criminals were @ Nuhrmberg after WW2, is now having post inquisitional regrets? Good thing no one was burned @ the Stake.
What’s funny is sometimes the AGW guys (unintentionally) tell you just how weak their “science” is:
FRUIT OF THE POISON TREE, TARTS FROM THE POISON FRUIT
http://naturalfake.wordpress.com/2009/12/03/fruit-of-the-poison-tree-tarts-from-the-poison-fruit/
You say, “I think the Earth has indeed warmed over the last 150 years,”
Would you mind PROVING this?
The Urban Heat Island adjustment data appears to be fundamentally flawed.
It appears to be applied on the basis that the current measurement is closest to the true value – basically “We can see the thermometer today, and it isn’t lying.”
But the primary goal is to determine the average gridcell temperature – not the average city temperature. Since the gridcells are far larger than cities, you should be subtracting out any effect a city might be having from the -current- value, not adjusting the pre-city (or smaller-city) data.
The same Phil Jones that is so central to the Climategate emails is the one that performed the seminal UHI adjustments.
Uriel, The available data strongly suggest some warming over the past 150 years. The trend is fairly clear, and at least reasonably reliable (albeit far from absolutely certain, as some pretend). But the trend isn’t unprecedented, nor dramatically different from other know periods of change. So the case for it’s being some radical crisis is pretty much unsubstantiated hype, at least at this point. The key point is causality. The evidence indicts CO2, but without any firm evidence it’s the major factor, or even a very significant one. On top of all this is the apparent likelihood of proposed ‘remedies’ being ineffective and so woefully inefficient one can’t help but wonder if they aren’t really just ‘Trojan Horses’ for ulterior agendas…..
All-in-all, it’s pretty clear that “Something is rotten in the state of Denmark”….so to speak.
I wouldn’t trust anything with Hanson’s name on it. It has the same taint as the CRU data.
I remember, the TOB adjustment thread on ClimateAudit; it was about 3 years ago that someone found that both NOAA and GISS began to “cool” the 1930s, but warm the 1990s relative to earlier reconstructions. Someone at NOAA emailed back Steve and pointed him to a paper which explained the “time of observations” adjustment. It was only coincidental that the 1930s cooled and the 1990s warmed, according to NOAA. Of course, this adjustment made the 1990s the warmest decade of the 20th Century.
Much of what is done at this 2 institutions is gridcell extrapolation. The “averaging” is spurious at best. And, at GISS it occurs at different time intervals. One GISS verion might adjust a gridcell down compared to an earlier version. The method of extrapolation is unpublished, and changes from year to year. When NOAA or GISS publish a news bulletin saying that such and such a location had the warmest Feb-June in 150 years they are refrencing an average temp derived from multiple gridcells that have constantly been adjusted, tweaked, and massaged.
The public is ignorant of these methods. Perhaps it is time to educate them.
ADiff,
I think the available data only proves that, on average, most of the globe has warmed (at least since 1920). The question is to what degree. The people at GISS, Hadley, and NOAA are comparing apples and oranges.
Since 1960, over 2/3rds of the reporting stations worldwide have closed. Most of the remaining reporting stations are located at major airfields. The remainder are NOAA CO-OP stations, which Anthony Watts has demonstrated utterly fail to meet the minimum standards established by NOAA. Quite a few of these stations have been moved in the last 30 years, and thier moves were undocumented by NOAA. One could also easily argue that since 1960, suburban and urban sprawl greatly accounted for much of the increase.
To illustrate my point, just subtract one adjustment (ie the TOBS) from the station data, and you reduce almost 30% of the alleged AGW component of the recent temp increases. The UAH/RSS sat data provides a much better account of lower tropespheric warming than does raw station data. Of course, the results are not quite so melodramatic. Of course, the sat data only goes back to 1979. And unfortunately, NOAA no longers uses satellite coverage to get thier SST ocean data. They purged this data last June. They said the sat data introduced a cold bias into thier reconstructions. Imagine that!. So instead of getting 100% ocean coverage of the mid-latitudes, they will use a few dozen ocean buoys, which of course they can adjust upwards.
I also find it ironic that GISS, a NASA agency, forgoes using data from unmanned satellites and instead uses unreliable thermometer data.
More or less roughly from early IPCC assessments….
*
* * * * *
* * * *
* * * *
* * * *
* * * *
* * *
at least approximately….which is to say: not just to what degree, but also why & in what manner – relative to the past…. The MWP & LIA suggest general falls and rises are not unprecedented nor dependent on some recently devised (human) ‘deus ex machina’ …. Which all precedes (as opposed to catastrohopists who leap to ‘C’ before even ‘A’ has been demonstrated, let alone ‘B’) determination of effective causality, analysis of likely impact (if any) of proposed ‘remedies’ and (finally!) some kind of rational analysis of expected costs accounting (and here’s where the alarmists are most wildly off-base) for the adaptive capacity of normal human activity…you know, the kind that doesn’t require committees, NGOs, UNGOs, central governments or any of that dreck… The alarmists appear to believe that unless there’s some central authority tasked with the authority to undertake the task (and funded appropriately) ordinary folks don’t have the sense or ability to even come in out of the rain on their own, much less manage to deal with normal historic climate change! What I want to know is how on Earth did humanity, to say nothing of “the Earth” (you know, those Polar Bears and all) manage to survive the MWP and the LIA without central governments, let alone without some global governance authority and the UN? Pray tell!
Well, that formatting didn’t work too well! Try to imagine the very early Mann curves showing late increases paralleling earlier increases falling off to the LIA….
🙂
In my opinion you should discard entirely stations that are located in large cities.
I looked at trends in the NordKlim dataset
http://www.george-barwood.pwp.blueyonder.co.uk/GlobalWarming/NordklimTrends.html
It has annotations for where the stations are. You can easily see the large winter warming in the cities.
Nice post, I haven’t come here as often lately because of climategate.
The CRU scandal should, in my mind, be taken exactly the same way. Unlike what more fire-breathing skeptics have been saying, this is not the final nail in the coffin of catastrophic man-made global warming theory. It is merely a starting point, a chance to finally move government funded data and computer code into the public domain where it has always belonged, and start tearing it down or confirming it.
I completely agree with this, it is only a tiny starting point.
The climate alarmists are distracting you. What counts is whether climate changes due to greenhouse gases, not whether the climate changes. Please work on what matters! okay? thanks, Ray Hull