Hockey Stick: RIP

I have posted many times on the numerous problems with the historic temperature reconstructions that were used in Mann’s now-famous "hockey stick."   I don’t have any problems with scientists trying to recreate history from fragmentary evidence, but I do have a problem when they overestimate the certainty of their findings or enter the analysis trying to reach a particular outcome.   Just as an archaeologist must admit there is only so much that can be inferred from a single Roman coin found in the dirt, we must accept the limit to how good trees are as thermometers.  The problem with tree rings (the primary source for Mann’s hockey stick) is that they vary in width for any number of reasons, only one of which is temperature.

One of the issues scientists are facing with tree ring analyses is called "divergence."  Basically, when tree rings are measured, they have "data" in the form of rings and ring widths going back as much as 1000 years (if you pick the right tree!)  This data must be scaled — a ring width variation of .02mm must be scaled in some way so that it translates to a temperature variation.  What scientists do is take the last few decades of tree rings, for which we have simultaneous surface temperature recordings, and scale the two data sets against each other.  Then they can use this scale when going backwards to convert ring widths to temperatures.

But a funny thing happened on the way to the Nobel Prize ceremony.  It turns out that if you go back to the same trees 10 years later and gather updated samples, the ring widths, based on the scaling factors derived previously, do not match well with what we know current temperatures to be. 

The initial reaction from Mann and his peers was to try to save their analysis by arguing that there was some other modern anthropogenic effect that was throwing off the scaling for current temperatures (though no one could name what such an effect might be).  Upon further reflection, though, scientists are starting to wonder whether tree rings have much predictive power at all.  Even Keith Briffa, the man brought into the fourth IPCC to try to save the hockey stick after Mann was discredited, has recently expressed concerns:

There exists very large potential for over-calibration in multiple regressions and in spatial reconstructions, due to numerous chronology predictors (lag variables or networks of chronologies – even when using PC regression techniques). Frequently, the much vaunted ‘verification’ of tree-ring regression equations is of limited rigour, and tells us virtually nothing about the validity of long-timescale climate estimates or those that represent extrapolations beyond the range of calibrated variability.

Using smoothed data from multiple source regions, it is all too easy to calibrate large scale (NH) temperature trends, perhaps by chance alone.

But this is what really got me the other day.  Steve McIntyre (who else) has a post that analyzes each of the tree ring series in the latest Mann hockey stick.  Apparently, each series has a calibration period, where the scaling is set, and a verification period, an additional period for which we have measured temperature data to verify the scaling.  A couple of points were obvious as he stepped through each series:

  1. Each series individually has terrible predictive ability.  Each were able to be scaled, but each has so much noise in them that in many cases, standard T-tests can’t even be run and when they are, confidence intervals are huge.  For example, the series NOAMER PC1 (the series McIntyre showed years ago dominates the hockey stick) predicts that the mean temperature value in the verification period should be between -1C and -16C.  For a mean temperature, this is an unbelievably wide range.  To give one a sense of scale, that is a 27F range, which is roughly equivalent to the difference in average annual temperatures between Phoenix and Minneapolis!  A temperature forecast with error bars that could encompass both Phoenix and Minneapolis is not very useful.
  2. Even with the huge confidence intervals above, the series above does not verify!  (the verification value is -.19).  In fact, only one out of numerous data series individually verifies, and even this one was manually fudged to make it work.

Steve McIntyre is a very careful and fair person, so he allows that even if none of the series individually verify or have much predictive power, they might when combined.  I am not a statistician, so I will leave that to him to think about, but I know my response — if all of the series are of low value individually, their value is not going to increase when combined.  They may accidentally in mass hit some verification value, but we should accept that as an accident, not as some sort of true signal emerging from the data. 

  • morganovich

    there seems to be evidence that O2 isotopes in tree rings are also worthless as a temperature proxy.

    http://westinstenv.org/sosf/2008/06/13/oxygen-isotopes-in-tree-rings-a-poor-yardstick-of-climate-change/

    seems leaves keep a constant temperature (self regulated) in the same way that a warm blooded animal does. amazing that numerous predictions were made about past temperature and no one ever even bothered to check the basic assumption that leaf temperature varies…

  • dearieme

    I wonder when the “tipping point” was – when inept scientists become conscious crooks.

  • Kriek Jooste

    How to make a Hockey Stick in the comfort of your own home:

    1. Get a bunch of possible proxies for temperature, a nice mix of them. Mostly tree rings. Make sure most of them are from the Northern Hemisphere, since we’re making a Northern Hemisphere graph. If you look at each individually you’ll see they’re often wildly different from each other, especially tree rings. This doesn’t matter. It also doesn’t matter if they don’t all cover the whole amount of time, just splice them together wherever you fancy.
    2. Average all this data together. If you’ve got enough data it should now just be noise, so averaging should give you an almost solid line of almost constant temperature. See, no need to worry about that Medieval warm period, you can claim it was a small regional anomaly. Don’t worry about what value this average level of temperature is exactly, we’ll get to that.
    3. Now get lots of temperature records from surface stations in the Northern Hemisphere, but you’ll find that the only collection vast enough and spanning enough time is that in the US. Don’t worry about their location, since most of them will be in urban areas now, where they weren’t in densely developed urban areas before.
    4. Pick the average temperature from the start of your series for surface stations, set that as the end temperature for your series of proxies for the 1000 years before that. Splice the two together.
    5. Keep quiet about the Southern Hemisphere.

    See, now you have a hockey stick, mostly constant temperature for centuries and a sudden rise towards the end.

  • Leon Brozyna

    So, let me see if I’ve got this right. They study recent tree rings to match them up to recent temperature data, treat the tree rings as a static thing like rock sediment and assign values to them to represent temperature. Then, a few years later, they go back and these very same tree rings from a living, dynamic, ever-changing tree have *gasp* changed?! Bet they’ll be looking for a grant to figure out how much pressure new growth/rings generate on older growth/rings, causing changes in the old values.

  • KODjr

    As you said, tree rings vary in width for a number of reasons. Off the top of my head, there are at least five: temperature, moisture, nutrients, disease, and predation.

    The thing is, for the first three, at least, a surplus can produce the same result as a deficit. Too much heat stunts a tree’s growth, just as too little heat does. Too much moisture rots a tree’s rootlets, which impairs growth just as surely as too little moisture does. An oversupply of nutrients “burns” the roots, and results in the same diminished growth that comes from insufficient nutrients.

    As for the last two, any disease (primarily fungi) or predation (whether by insects, deer, or giraffes) will impair the tree’s ability to grow.

    Given all these factors, I don’t see how the tree-ring readers can responsibly report anything other than “This particular year was a good/average/bad year for this particular tree.”

  • Luis Dias

    RIP it is.

    Unless of course we are talking about the media.

  • dreamin

    “Just as an archaeologist must admit there is only so much that can be inferred from a single Roman coin found in the dirt, we must accept the limit to how good trees are as thermometers.”

    I agree 100%, and I would add that averaging 10 lousy estimates doesn’t necessarily give you a good estimate.

    It reminds me of Richard Feynman’s parable of the Chinese Emperor’s nose. A man wants to know the length of the Emperor’s nose, but nobody has ever seen the Emperor’s face. So he asks 1000 villagers to guess and then averages all their guesses.

    Of course this is silly on its face, but if you look carefully at the IPCC reports, it becomes clear that the IPCC engages in just this sort of reasoning.

  • Scientist

    It amuses me to see how often denialists say that Mann et al has been discredited or disproved. Obviously, you wouldn’t need to say this repeatedly if it was actually true. Mann et al’s conclusions have been independently confirmed in numerous subsequent papers.

  • What INDEPENDANT paper is that?

  • Scientist

    Google Scholar lists 704 citations to Mann, Bradley and Hughes (1998). Have a look through. You’ll find numerous independent studies confirming their results.

    And why not learn how to spell? Basic errors like that hardly enhance your credibility.

  • og

    “Obviously, you wouldn’t need to say this repeatedly if it was actually true”

    Yep, obviously. Because repeating lies about clearly nonexistent AGW over and over again, is speaking truth to power, and trying to alert the world to the junk science being foisted onto them is “denialist”

    And why not learn how to do math? Basic errors (like assuming you can model meaningful temperature data on less than 500 years of data (the age of the thermometer)) hardly enhance your credibility.

  • His 704 citations certaintly shows his AGW delusion.

    He apparently does not realize that a number of them actually contradict the HS. paper.But never mind.

    LOL

  • Scientist

    You clearly do not understand that there are many more papers independently confirming the results that there are which question them.

    And you still can’t spell, or write like an adult.

  • og

    Yep. Because Google is such an important research tool. And totally unbiased, too. Once again: With less than 100 years of accurate temperature data, about a planet 4.75 billion years old, you can predict global warming how? (no, you don’t have any direct observations beyond about a hundred years, anything before that is “extrapolation”(read:”Guess”))

    “And you still can’t spell, or write like an adult.”

    Ah. The classic ad hominem attack, last resort of the failure to win a debate.

  • Scientist

    You think that unless we have observed the planet since its formation, we can’t understand what factors affect the temperature of its atmosphere? How incredibly stupid. It’s very very simple to predict global warming, if the concentration of one of the gases which plays a major role in the energy balance of the atmosphere is going up. The age of the planet is utterly irrelevant, but you got it wrong in any case, by about 200 million years.

  • og

    Ooh, 200 million years.

    I am utterly stupid. That’s why I’m just a neanderthal and you’re a “scientist”. What are your credentials, anyway?

    You cannot predict something based on statistically insignificant data, period. Anyone who thinks they can, is an abject moron. Period.

  • og

    Oh, and continue to attack me personally, if you like. Ad hominem (surely, as a “scientist”, you understand some latin) amuses me so.

  • Scientist

    So how can we understand anything at all about the Earth or the universe, in your opinion?