Lipstick on a Pig

Apparently, Michael Mann is yet again attempting a repackaging of his hockey stick work.  The question is, has he re-worked his methodologies to overcome the many statistical issues third parties have had with his work, or is this more like AirTran changing its name from ValuJet to escape association in people’s mind with its 1996 plane crash?

Well, Steve McIntyre is on the case, and from first glance, the new Mann work seems to be the same old mish-mash of cherry-picked proxies, bizarre statistical methods, and manual tweaking of key proxies to make them look the way Mann wants them to look.  One thing I had never done was look at all the component proxies of the temperature reconstructions all in one place.  At the link above, Steve has all the longer ones in a animated GIF.  It is really striking how a) almost none of them have a hockey stick shape and b) even the few that do have HS shapes typically show the warming trend beginning in 1800, not in the late 19th century CO2 period. 

If you would like to eyeball all 1209 of the proxies Mann begins with (before he starts cherry picking), they are linked here.  I really encourage you to click through to one of the five animations, just to get  a feel for it.  As someone who has done a lot of data analysis, it is just staggering that he can get a hockey stick out of these and claim that it is in some way statistically significant.  It is roughly equivalent to watching every one of your baseball team’s games, seeing them lose each one, and then being told that they have the best record in the league.  It makes no sense.

The cherry-picking is just staggering, though you have to read the McIntyre articles as a sort of 2-3 year serial to really get the feel of it.  However, this post gives one a feel of how Mann puts a thin statistical-sounding veneer to cover his cherry-picking, but at the end of the day, he has basically invented a process that takes about a thousand proxy series and kicks out all but the 484 that will generate a hockey stick.

Update:  William Briggs finds other problems with Mann’s new analysis:

The various black lines are the actual data! The red-line is a 10-year running mean smoother! I will call the black data the real data, and I will call the smoothed data the fictional data. Mann used a “low pass filter” different than the running mean to produce his fictional data, but a smoother is a smoother and what I’m about to say changes not one whit depending on what smoother you use.

Now I’m going to tell you the great truth of time series analysis. Ready? Unless the data is measured with error, you never, ever, for no reason, under no threat, SMOOTH the series! And if for some bizarre reason you do smooth it, you absolutely on pain of death do NOT use the smoothed series as input for other analyses! If the data is measured with error, you might attempt to model it (which means smooth it) in an attempt to estimate the measurement error, but even in these rare cases you have to have an outside (the learned word is “exogenous”) estimate of that error, that is, one not based on your current data.

If, in a moment of insanity, you do smooth time series data and you do use it as input to other analyses, you dramatically increase the probability of fooling yourself! This is because smoothing induces spurious signals—signals that look real to other analytical methods. No matter what you will be too certain of your final results! Mann et al. first dramatically smoothed their series, then analyzed them separately. Regardless of whether their thesis is true—whether there really is a dramatic increase in temperature lately—it is guaranteed that they are now too certain of their conclusion.

and further:

The corollary to this truth is the data in a time series analysis is the data. This tautology is there to make you think. The data is the data! The data is not some model of it. The real, actual data is the real, actual data. There is no secret, hidden “underlying process” that you can tease out with some statistical method, and which will show you the “genuine data”. We already know the data and there it is. We do not smooth it to tell us what it “really is” because we already know what it “really is.”

Update:  I presume it is obvious, but the commenter "mcIntyre" has no relation that I know of to the "mcintyre" quoted and referred to in the post.  As a reminder of my comment policy, 1) I don’t ban or delete anything other than outright spam and 2) I strongly encourage everyone who agrees with me to remain measured and civil in your tone — everyone else is welcome to make as big of an ass out of him or herself as they wish.

By the way, to the commenter named "mcintyre,"  I have never ever seen the other McIntyre (quoted in this post) argue that CO2 does not act as a greenhouse gas.  He spends most of his time arguing that the statistical methods used in certain historic temperature reconstructions (e.g. Mann’s hockey stick but also 20th century instrument rollup’s like the GISS global temperature anamoly) are flawed.  I have read his blog for 3 years now and can honestly say I don’t know what his position on the magnitude of future anthropogenic warming is.  Mr. McIntyre is apparenlty not alone — Ian Jolliffe holds the opinion that the reputation of climate science is being hurt by the statistical sloppiness in certain corners of dendro-climatology.

  • Mesa Econoguy

    Now I’m going to tell you the great truth of time series analysis. Ready? Unless the data is measured with error, you never, ever, for no reason, under no threat, SMOOTH the series!

    Stochastic data (such as temperature, or equities prices) is not meant to be smoothed, at least not to glean useful information.

    By my smoothed series (5 yr. moving average), I’m up 15% this year.

    Mann is a joke, fraud, and amateur statistician.

  • McIntyre

    If the noisy concerns of internet bloggers were really of any consequence, Mann’s paper could not possibly have survived the peer review necessary for being published in PNAS.

    even the few that do have HS shapes typically show the warming trend beginning in 1800, not in the late 19th century CO2 period – have a look at this, and explain if you could why we should not have expected any CO2 related warming until the late 19th century.

  • B.D.

    have a look at this, and explain if you could why we should not have expected any CO2 related warming until the late 19th century.

    Because, scientist 2-year-old who can’t post under his own name, the rise from the mid-1700s to the late 1800s was well within the “natural” range that goes back to 1000 AD. It is only around 1900 where CO2 levels increase dramatically outside of the natural range.

  • McIntyre

    So, B.D. two year old who seems to think I am someone he doesn’t like. You think that a change in CO2 doesn’t have any effect on temperatures until that change reaches a certain threshold, and that threshold is defined by what natural variation has been over the preceding 750 years? I don’t believe the atmosphere knows it’s supposed to behave like that.

  • dearieme

    Mr Briggs’ statement corresponds exactly with what I was taught on my postgrad stats course. Moreover, we did exercises to show that it was true.

  • Bobby Lane

    To the poster named McIntyre:

    Read this and see if you still think that CO2 matters one wit.

    http://wattsupwiththat.wordpress.com/2008/09/04/even-doubling-or-tripling-the-amount-of-co2-will-have-little-impact-on-temps/#more-2769

    Like the above article and the Skeptic himself say, C02 is nothing compared to the other problems. CO2 is a front, it is a media-friendly gimmick. Even if AGW is 100% real and we are the actual cause of all the warming far in excess of what might be termed as natural causes, CO2 would still be as unimportant as it actually is now. Thus, it matters not when CO2 began to rise outside of normal means except for historical and mathematical footnotes. You are arguing over the equivalent of ascertaining the exact age of Bugs Bunny.

  • McIntyre

    Bobby Lane – blog posts do not trump the scientific understanding of CO2 developed over many decades. The post you quote is pure nonsense. If you choose to take your prompts from random blogs instead of from scientific journals, that’s up to you, but unfortunately you are likely to embarrass yourself badly if you do so.

  • B.D.

    McIntyre,

    There are lots of problems with your CO2 time series compared to Mann’s proxies:

    Why does your CO2 time series show higher CO2 in 1200 vs. 1000, but Mann’s proxies show higher temperatures in 1000 vs. 1200?

    Why do Mann’s proxies show a temperature minimum in the late 1500s, but your CO2 time series shows that CO2 had a minimum at 1600?

    Why does your CO2 time series show CO2 higher than ever previously in 1900, but Mann’s proxies and the instrumental record show temperatures lower than most of the period up to 1400?

    It looks much more like CO2 lags temperature than leads it, and that CO2 concentrations do not correlate well with temperatures.

  • McIntyre

    The ice core CO2 record is from Law Dome. Law Dome is in the Antarctic. Mann et al’s temperature reconstruction is for the northern hemisphere. The Antarctic is not in the northern hemisphere. In the middle ages changes in CO2 concentrations were too small to cause significant climate change. These simple facts answer your questions. It really does’t look like CO2 follows temperature in the current epoch.

  • B.D.

    So CO2 behaves differently now than in the past? I don’t believe CO2 knows it’s supposed to behave like that.

  • McIntyre

    It behaves the same now as always. Why childishly misread what I wrote? Do you consider such an approach some kind of good debating technique, or something?

  • B.D.

    McIntyre,

    You chose to use Law Dome to illustrate when temperatures should begin rising. I don’t care if it is on Mars. If it is not relevant, than why did you use it?

    Your latest plots show CO2 trending slightly upward while temperature trends downward from 1000 to 1500, a steep CO2 drop but no change in the temperature trend until about 1900, and then both temperature and CO2 shoot up from there. So how did CO2 know it should correlate with a temperature drop before 1900 and a temperature rise after? It must have some pretty magical properties.

  • B.D.

    P.S. McIntyre,

    From 1910 to 1950, CO2 concentrations rose by about the same amount that they dropped from 1560 to 1600. The temperature rose 0.7 degrees from 1910 to 1950, and dropped less than 0.4 degrees from 1560 to 1600. Again, I don’t believe that CO2 knows it’s supposed to behave that way.

  • McIntyre

    You seem to be struggling with the idea that other things besides CO2 affect the climate. What do you expect, an exact correlation with temperature? Please, try not to debate idiotic points. The rather simple point that you can’t seem to observe from the graphs is that CO2 has driven temperatures up over the last 200 years. Before 200 years ago, changes in CO2 concentration were not large enough to significantly alter the climate. Now what is your problem with this simple, basic science?

  • B.D.

    And you seem to be struggling with the idea that the plots you are putting forth are anything other than crap. Even the IPCC first assessment acknowledged a much more variable climate from 1000 to 1900 than your boring (and debunked) hockey stick shows. There really can be no meaningful discussion about how CO2 is affecting temperatures that justifies trillions of dollars in economic impact until we have a realistic grasp on historical climate and your ilk seems to be struggling with that idea. The instrument record is not long enough and the proxy reconstructions are a joke. They can’t be correlated to local temperatures, but they can be to global temperatures because supposedly temperature anomalies are correlated over large distances? Well then the thermometer in my backyard says the globe is cooling.

  • McIntyre

    Ah yes, the IPCC in 1990 accidentally published the ‘true’ 1000-1900 temperatures, and they’ve been trying to cover it up ever since. That must be it.

    What is your source for your claim of ‘trillions of dollars in economic impact’?

  • B.D.

    What is your source that the IPCC has been trying to cover up the true temperatures since the FAR?

  • McIntyre

    Oh dear. My source? Did you not realise I was taking the mickey out of you?

  • B.D.

    Yes, Einstein. I realized it. You are really dumber than you think. You have not answered a single response to the crap you spew. Instead you just spew new crap. What kind of idiotic debate tactic is that, or do you consider that approach to be some kind of good debating technique?

  • McIntyre

    Your parroting makes you look autistic. I have responded to all your questions. If you don’t like the responses, that’s your problem.

  • JP

    Thus far no comments concerning Mann’s use of sediment proxies, or his inclusion of unpublished ice core data from Lonnie Thompson(Bona-Churchhill series). The Finns warned against using thier lake sediment as temp proxies because of the problems of cherry picking thier series. They said human interraction with the lake enviorment (farming, using the lakes as catchments, etc…).The Socotra Island speleothem proxy series, like Thompson’s Bona-Churchill series is unpublished. Also, Mann failed to use Ababneh updated Graybill series from Sheeps Mountain (she failed to replicated Mann’s results with the same proxies), and instead Mann used his orginal Bristlecone series.

    They question remains: Can Mann get a HS without using the Greybill Series, and only use non-MBH9X proxies?

  • McIntyre

    I’m Scientist and I am dumb.

  • B.D.

    That was very immature of me. Apologies.