This discussion, including the comments, over at Climate Audit, really is amazing. Just when you think all the procedural errors that could be mined from the Mann hockey stick have been pulled to the surface, another gem emerges.
Here is how I understand it (please correct me if I am wrong in the comments): Michael Mann uses a variety of proxies to reconstruct history (he actually pre-screens them to only use the ones that will give him the answer he wants, but that is another problem that has been detailed in other posts). To be able to tell temperature with these proxies (since their original measurements are things like mm of tree ring width, not degrees) they must be scaled based on periods in which the thermometer-measured surface temperature record overlaps the proxy record.
Apparently, when making these calibrations, he used the surface temperature record from 1850-1995, but also did other runs with sub-periods of this, such as 1850-1949 and 1896-1995. OK so far. Well, McIntyre believes he has found that when running these correlations, the sign of the correlation factor for a single proxy actually changes.
What does this mean? Well, lets assume proxy 1 is tree ring width from a particular tree, and a calibration based on 1850-1995 has such-and-such ring width data correlated at x per degree. This means that an increase in ring width of X implies a temperature increase of one. But, when calibrating on one of the other periods, the exact same proxy has a calibration of -Y. This means that an increase in the ring width of Y yields a temperature DECREASE of one.
I had a professor of physics back in undergrad who used to just drive me crazy with his insistence on good error estimations in the lab (which he was right to emphasize, just proving I was not meant for the lab). He used to say that if your error range crossed zero, in other words, if your range of possible answers included both positive and negative numbers, then you really did not understand a process. You don’t understand a relationship, he would say, if you don’t even know the sign. Well, Mann has gotten over this little problem, I guess, because he is perfectly able to have the same physical process have exactly opposite relationships with temperature depending on what 50 year period he is working with.
OK, so Steve caught him with one bad proxy. Heck, he has over a thousand others. But now McIntyre is reporting in the comments he has found 308 such cases, where Mann has correlations that change signs like this. Wow.
Postscript: By the way, one of the most fundamental rules of regression analysis is that when you throw a variable into the regression, you should have some theoretical reason for doing so. This is because every single variable you add, no matter how spurious, is going to improve the fit of a regression (trust me on this, it’s in the math).
In the case of proxy regressions, it is simply unacceptable to rely on the regression for the sign. You rely on physics for the sign, not the regression. If you don’t even know the sign of the relationship between your proxy and temperature, then you don’t understand the proxy well enough physically to justify even calling it a proxy.
This is a big, big deal in financial modelling. I can’t tell you how often it is emphasized in financial modelling to make sure you have a working theory as to how and why a variable should affect a regression, and then when you get the result, you need to test it against your original theory. And if they are too far apart, you need to doubt the computer result. Because in financial modelling, if you get too much confidence in regressions against spurius data, you can go bankrupt (in climate, it instead seems to lead to fame, large grants, and hanging out with vice-presidents).
Update: Oops, I missed the first post on this at Climate Audit, which discusses the issues in my postscript in more depth. This is a good example, and it is not surprising they revert to a financial example as I did, as financial modelers have the greatest immediate incentives not to fool themselves.
We (the authors of this paper) have identified a weather station whose temperature readings predict daily changes in the value of a specific set of stocks with a correlation of r=-0.87. For $50.00, we will provide the list of stocks to any interested reader. That way, you can buy the stocks every morning when the weather station posts a drop in temperature, and sell when the temperature goes up. Obviously, your potential profits here are enormous. But you may wonder: how did we find this correlation? The figure of -.87 was arrived at by separately computing the correlation between the readings of the weather station in Adak Island, Alaska, with each of the 3315 financial instruments available for the New York Stock Exchange (through the Mathematica function FinancialData) over the 10 days that the market was open between November 18th and December 3rd, 2008. We then averaged the correlation values of the stocks whose correlation exceeded a high threshold of our choosing, thus yielding the figure of -.87. Should you pay us for this investment strategy? Probably not: Of the 3,315 stocks assessed, some were sure to be correlated with the Adak Island temperature measurements simply by chance – and if we select just those (as our selection process would do), there was no doubt we would find a high average correlation. Thus, the final measure (the average correlation of a subset of stocks) was not independent of the selection criteria (how stocks were chosen): this, in essence, is the non-independence error. The fact that random noise in previous stock fluctuations aligned with the temperature readings is no reason to suspect that future fluctuations can be predicted by the same measure, and one would be wise to keep one’s money far away from us, or any other such investment advisor
Update #2: I guess I have to issue a correction. I have argued that climate scientists tend to be unique in trying to avoid criticism by labeling critics as “un-scientific”. In retrospect, it does not appear climate scientists are unique:
The iconoclastic tone have attracted coverage on many blogs, including that of Newsweek. Those attacked say they have not had the chance to argue their case in the normal academic channels. “I first heard about this when I got a call from a journalist,” comments neuroscientist Tania Singer of the University of Zurich, Switzerland, whose papers on empathy are listed as examples of bad analytical practice. “I was shocked — this is not the way that scientific discourse should take place.”
The CA posting was poorly spelled out. It’s not clear what he is really indicting Mann for. That a correlation has different values (to include negative and positive) in different subsets of data is not surprising. Especially for noisy systems.
The scientific justification for Mann massaging the data to get his correlations is that the results support the conclusion he made before he started his work.
I’m actually OPEN to the idea that Mann has a mistake, but I have not heard exactly what it is. Is McI saying that 1200 proxies should have the same correlation (even same correlation sign) in two different periods? Or was there some usage of the subdivided periods for validation or screening, where having two different correlations is the issue? Why is only sign flipping the issue? A sign flip from 0.1 to -0.1 might be less of a big deal than 0.1 going to 0.5?
I just don’t even GET what is being complained about. Just in terms of a clear articulation. Not even disagreeing yet. And clearly the blog writer, here, couldn’t follow it either. Lots of ooga chaka and other snark so thick you can cut it with a jigsaw.
I think the issue is that for a given proxy, selecting different periods over which to compare changes the sign of the resulting factor (warming vs cooling or vise versa). This is an indication that the periods are more important than the proxy itself. Or even worse, that any proxy could say anything you wanted if you were willing to play with the period over which you compare them.
In any case, the whole idea of selecting and manipulating proxies without review in un-scientific, un-proffessional, and total hogwash. Whether or not this current critisism of the proxies is correct or not is beside the point. No one should ever be forced to guess as to EXACTLY how the proxies were developed and used. It should be explained clearly before the results are EVER taken seriously or reported in ANY way.
I had the same experience years ago in horserace modeling (which may be considered a subset of financial modeling). I realzied immediately that you can’t rely soley on correlations of past data to make successful betting predictions. Once you’ve got what you think is a reasonable model, you’ve got to test it against INDEPENDENT data not included in creating your model.
I just don’t have a fucking clue about anything. Fuck this shit, yo!
Wow. Good explanation of the technical discussion on CA.
TCO: I’m not sure why you say Meyer himself doesn’t understand this. I’m very weak in math, and I understand it. Try rereading this critical paragraph:
“What does this mean? Well, lets assume proxy 1 is tree ring width from a particular tree, and a calibration based on 1850-1995 has such-and-such ring width data correlated at x per degree. This means that an increase in ring width of X implies a temperature increase of one. But, when calibrating on one of the other periods, the exact same proxy has a calibration of -Y. This means that an increase in the ring width of Y yields a temperature DECREASE of one.”
In other words, Mann assigns a completely different temperature value to a given measurement of a proxy for different time periods of his study. And the difference is not nominal — in one case it is +1 in and in another, -1. The value is alleged to be designed to arbitrarily predict the desired outcome.
This is called cheating.
@TCO
I don’t believe the problem is that CA is suggesting that the different proxies have the same correlation or the same sign, but instead that the correlation factor sign shouldn’t change over two adjacent (or perhaps any) periods of time. The idea is that if the use of the proxy requires two separate correlations factors on both sides of 0, then the phenomenon which relates the proxy to the dependent variable isn’t really well understood (this is illustrated by the finance analogy). If this were a problem in a handful of the proxies used Mann might have been able to explain it away but if it is true for nearly a quarter of the ~1200 proxies it should at least be explained especially in light of the fact that proxies selected for this modeling are specifically chosen because of the degree of correlation.
That said, I don’t know that there might not be some reasonable answer here. I suspect there isn’t but if I were trying to think of all possible explanations I might suggest the following:
1) When were the dates of the sign change for these proxies? If they were all at a similar time or there were several groups of changes its possible that there was some process(es) (the dawning of the industrial age, population explosion, more efficient courting techniques by cows) by which whatever function Mann was using for correlation spit out a sign change. If it were a large scale process then its possible that a good chunk of proxies might have an almost disjointed cutoff.
2) Some of the functions Mann is using for proxies (are rightly) discontinuous functions. Maybe they’re piecewise or expansions which are having factors turned off. This would be hard to tell w/o the data.
3) Maybe (this is really stretching) the switched sign changes are a product of something in his code that required it. We’re really verging onto throwing whatever onto the wall w/this stuff but its possible considering the number of proxies used and depending on the nature of his program.
That said, I’m not going to pretend like this all shouldn’t be explained already (to be fair I haven’t read Mann’s literature to see if it has) and that it doesn’t raise a number of eyebrows.
It seems quite likely to me that any particular tree species has an optimum temperature, and other things being equal, it grows best at that temperature and worse at lower and higher temperatures. That is, the actual response curve is an inverted U. If you calculate correlations for a period when the temperatures were mostly below optimum, the sign is positive. If you calculate correlations for a period when the temperatures were mostly above optimum, the sign is negative.
But no matter how well you understand such a function, you can’t invert it and use it as a thermometer – it gives two results for each input.
i was under the impression that the overall confidence in tree models for temperature estimation had been diminished significantly last year when researchers discovered that tree leaves actually regulate their temperature and tend to stay within a very narrow range (like body temperature). it had always just been assumed that tree temperature varied with environment and therefore was a good proxy, but this may well not be true at all.
anyone have any detailed knowledge on this?
morganovich: where did you get this idea from? Tree rings have nothing to do with leaf temperatures, and are well known to correlate strongly with temperature. No-one ‘just assumes’.
Let’s be careful . . .
Warren referenced a McIntyre post that discussed Mann 2008. As I understand Mann’s activities, Mann 2008 did not use tree ring data, but Warren’s post features comments on tree rings. So I wonder if Warren is misunderstanding McIntyre’s concerns. Certainly, when Mann earlier used Tree Ring Data, there were a multitude of problems with what he did. Also, Mann 2008 features some mind-numbing mistakes that have been discussed elsewhere. However, I am not sure that Warren understands the McIntyre comments that he referenced.
How to be sure that change in tree ring width need to be
due to temperature change, more CO2 in the air feeds the
trees, look what usualy happen to man that eat to much,
he get fatter.
Inquirer, what Mann 08 says is that you don’t need the Bristlecone Pine study he used in MBH to validate the “Hockey Stick”. What McIntyre shows in his recent blogs about Mann08 is that the good doctor did not remove his tree ring studies, he just found additional proxies that agreed with the BCP study. He could then take the BCP study out and say, “Look! My earlier paper is vindicated! Take that M & M!” The BCP study, as well as other dendroclimatological studies, are still included in his proxy list for the paper.
But the other studies he found and used to make this “validation” have serious questions about their own validity. One even has the authors of the study say in their paper that it is not a valid proxy for modern temperature as the site was contaminated during particular periods with documented evidence. Mann uses this study without throwing out the contaminated data, as the contamination causes a superior match to the temperature record that he is using. He references 1209 proxies, although he only used about 30% of them in his reconstruction. Prominent in these are the Briffa BCP studies, Tornetrask, and the Finnish study I mention above. All of these studies have questionable interpretations as has been examined by Steve McIntyre over at CA, as well as by other investigators at other blogs. Jeff Id at the Air Vent has performed some great statistical analysis on Mann 08, showing that, as long as you require the GISS/HadCRUT temp record appear at the end of whatever random data sequences you generate, the good doctor’s method will generate a hockey stick every time.
hunter-
the study was done by a couple of U penn scientists and published in the june 12 2008 volume of “nature”.
they examined 39 species of trees in differing climates and discovered that they had individually constant temperatures and that those temperatures were also very similar across species and climate.
“These results also have implications for scientists studying past climate change by measuring the ratios of different isotopes of oxygen (which have different numbers of neutrons) in tree-ring cellulose. The amount of a particular isotope present in the cellulose is influenced by the temperature of the leaves, and scientists had assumed that leaf temperature was the same as the ambient temperature. The new study has shown this isn’t the case.”
so the point here is that if tree leaves (which provide oxygen transfer to the trunk) have a homeostatically balanced temperature as opposed to simply reflecting ambient temperature (as had been assumed. astoundingly, no one ever actually checked leaf temperature variation before) then oxygen isotope ratios may be poor climate proxies.
this may also go some way toward explaining why temperatures inferred from tree rings fail to line up well with those observed directly in recent periods.
so my question is:
are we sure trees make good proxies at all?
also:
loehle has done some interesting work on the divergence issues of tree ring proxies and actual temperature records:
Abstract:
Tree rings provide a primary data source for reconstructing past climates,
particularly over the past 1,000 years. However, divergence has been observed in
twentieth century reconstructions. Divergence occurs when trees show a positive
response to warming in the calibration period but a lesser or even negative response
in recent decades. The mathematical implications of divergence for reconstructing
climate are explored in this study. Divergence results either because of some unique
environmental factor in recent decades, because trees reach an asymptotic maximum
growth rate at some temperature, or because higher temperatures reduce tree
growth. If trees show a nonlinear growth response, the result is to potentially truncate
any historical temperatures higher than those in the calibration period, as well as
to reduce the mean and range of reconstructed values compared to actual. This
produces the divergence effect. This creates a cold bias in the reconstructed record
and makes it impossible to make any statements about how warm recent decades
are compared to historical periods. Some suggestions are made to overcome these
problems.
Are we sure if trees make good proxies? Yes, we are. When trees are the best thing we have and we predetermine that they must be good enough to prove our beliefs, then the trees make excellent proxies. If we didn’t have the trees, we would use tea leaves or something. or maybe some sort of divining rod.
Rick,
Who is “we” ? You just state your opinion without evidence or argument… what is the point of posting a comment like that?
morganovich – got a page number for that Nature paper? I can’t see it in the contents of the June 12 issue.
http://www.nature.com/nature/journal/v454/n7203/full/nature07031.html
abstract:
The oxygen isotope ratio (delta18O) of cellulose is thought to provide a record of ambient temperature and relative humidity during periods of carbon assimilation1, 2. Here we introduce a method to resolve tree-canopy leaf temperature with the use of delta18O of cellulose in 39 tree species. We show a remarkably constant leaf temperature of 21.4 plusminus 2.2 °C across 50° of latitude, from subtropical to boreal biomes. This means that when carbon assimilation is maximal, the physiological and morphological properties of tree branches serve to raise leaf temperature above air temperature to a much greater extent in more northern latitudes. A main assumption underlying the use of delta18O to reconstruct climate history is that the temperature and relative humidity of an actively photosynthesizing leaf are the same as those of the surrounding air3, 4. Our data are contrary to that assumption and show that plant physiological ecology must be considered when reconstructing climate through isotope analysis. Furthermore, our results may explain why climate has only a modest effect on leaf economic traits5 in general.
this is the piece to which i was referring:
Nature 454, 511-514 (24 July 2008) | doi:10.1038/nature07031; Received 10 March 2008; Accepted 28 April 2008; Published online 11 June 2008
Subtropical to boreal convergence of tree-leaf temperatures
abstract:
The oxygen isotope ratio (delta18O) of cellulose is thought to provide a record of ambient temperature and relative humidity during periods of carbon assimilation1, 2. Here we introduce a method to resolve tree-canopy leaf temperature with the use of delta18O of cellulose in 39 tree species. We show a remarkably constant leaf temperature of 21.4 plusminus 2.2 °C across 50° of latitude, from subtropical to boreal biomes. This means that when carbon assimilation is maximal, the physiological and morphological properties of tree branches serve to raise leaf temperature above air temperature to a much greater extent in more northern latitudes. A main assumption underlying the use of delta18O to reconstruct climate history is that the temperature and relative humidity of an actively photosynthesizing leaf are the same as those of the surrounding air3, 4. Our data are contrary to that assumption and show that plant physiological ecology must be considered when reconstructing climate through isotope analysis. Furthermore, our results may explain why climate has only a modest effect on leaf economic traits5 in general.
The article specifically says that only isotope ratio studies might be affected. It does not say that suddenly trees in their entirety are not to be used as proxies.
but if trees fail to respond to temperature in the way that had been assumed, this casts some doubt on the assumptions that temperatures are the predominant factor in determining tree ring growth. temperature homoestasis would make it quite unlikely. this would seem to explain and bolster the “divergence” discussed in the loehle piece i mentioned above.
all in all, there seem to be a large number of problems and inconsistencies in using tree rings as a proxy.
the simple fact is that if you can get a flip in sign in the response of a system to a stimulus from one period to the next, the stimulus you are considering is not the driving variable.
if isotopes are ineffective and ring width shows frequent divergence, how you you arrive at tree proxies being “well known to correlate strongly with temperature.”
there seems to be significant evidence that they don’t and shouldn’t.
Who assumes that temperatures are the predominant factor in determining tree ring growth? Not anyone involved in studying trees or climate, that’s for sure.
When I saw the first releases and articles about this study my curiosity set in. My initial reaction was to write a blog entry so I decided to investigate. The first thought was to look at the related states of the PDO and the AMO. I did a crude and rough comparison of the PDO and AMO states. http://penoflight.com/climatebuzz/PDOAMOG1.jpg
Then compared the combined states to the drought tendencies for those combinations. Drought tendencies. http://penoflight.com/climatebuzz/PDOAMO1.jpg
By that method I could not blame what they were saying on the PDO / AMO. I checked the precipitation records for the US from 1976 to 2006 and there was a slight increase in precipitation nationally. Hence, if there study areas had drought they were anomalous to the national trend for the period. Despite the divergence from the PDO / AMO drought tendencies I think it is most likely wrong to blame ‘Global Warming’.
Not knowing the precise areas of study (yet) it is not possible to investigate further. Some have mentioned land-use, urbanization, etc. That is a possibility, depending on the exact areas of study the level of impact would change. If I come to any solid conclusions I will be blogging them.
My family own some forrests. The trees grow well in wet years and less so in dry years. To use trees as proxies would have to include a factor for the precipitation. Obviously, that is not possible, and hence it puzzles me that people still defend tree rings as proxies for temperature.
“vindsavfuktning, kallvindar med mögelproblem” – you really should read some of the basic literature. It’s painfully obvious that you don’t have the remotest understanding of how climate signals are extracted from tree ring data.
neither, from the sound of it, do you hunter.
Oh yes?
yes.
there are 3 basic methodologies used:
ring width, ring density, and chemical composition.
which one do you think works? and how do you explain the divergence over time periods?
“Which one works” is too vague to be an answerable question. Climate studies with trees are rather more complicated than you seem to think.
how are they more complicated than measuring ring width, density, or composition?
no amount of dazzling and baffling statistical analysis will make a poor proxy into a good measure of temperature, so one of these needs to be shown to be a good and consistent proxy.
you have yet to put up any substantive evidence of any kind.
when you do, i would be happy to discuss it.
Do you understand that there is a large difference between ‘measuring’ and ‘interpreting’?
of course, but if the thing being measured (and used as a proxy) is not, in fact, a good proxy, no amount of interpretation will make the results meaningful.
So apparently you believe that all species of trees grow equally well regardless of what the temperature is.
that is a straw man argument.
i believe that a large number of factors (of which temperature is likely one) affect tree growth and that temperature as a single variable cannot be meaningfully inferred from such a multi-input outcome as ring width or density.
you appear interested merely in wasting time as opposed to providing any actual information, so at this point, i’ll leave you to it…
“i believe that a large number of factors (of which temperature is likely one) affect tree growth” – yeah, no shit Sherlock!
“and that temperature as a single variable cannot be meaningfully inferred from such a multi-input outcome as ring width or density” – and there you lose the plot. I learnt about tree rings and climate in primary school. Where were you educated?
You can learn more here, if you want to:
http://www.ace.mmu.ac.uk/Resources/gcc/3-3-3.html
“i believe that a large number of factors (of which temperature is likely one) affect tree growth” – only likely? So you do believe that it is possible that temperature has no effect at all on tree growth.
“temperature as a single variable cannot be meaningfully inferred from such a multi-input outcome as ring width or density” – there are thousands of scientific papers published over many decades which show that you are wrong. If you’ve got persuasive evidence that they’ve all made some grand and basic error, then I’ll look forward to seeing your journal paper. Science or Nature should lap it up.
i say likely because it is very possibly not a linear relationship.
are these the same papers where the actual sign of the response changes in many cases if you alter the reference period (divergence)?
if your entire contribution to a discussion is contrariness, straw men, and appeal to authority, why do you bother?
i also say likely because i do not believe the variable has been (and quite possible even can be) sufficiently isolated to draw meaningful conclusions. it is certainly reasonable to suspect an influence, but to assume one a priori is not good science.
You’re extremely careless with your language. Don’t imply that there may not be a relationship, if you only mean to say that the relationship is not linear.
Denying that temperature affects tree rings is like denying that obesity is caused by overeating. Questioning how accurate tree ring reconstructions can be is sensible; claiming that tree ring reconstructions are fiction is crazy. Why would anyone take your views seriously when they are as ridiculous as that?
actually, i am quite precise. until something is proved, you cannot say “definite”. (and even then new info can prove you wrong)
you are conspicuously lacking in any evidence that temperature can be meaningfully and consistently inferred from tree rings. it stretches credibility that temperature is a sufficiently isolatable variable that a broad response like ring growth can determine it. how would you separate out rainfall? pest infestations? CO2 level? sunshine? can you show that temperature is more important to growth than these?
the burden of proof falls upon the one who proposes a theory. (especially when other have found clear holes in it
As a regular and frequent browser of Climate Change websites, one learns to spot certain characteristics both of data and contributors. The ad hominem attack is solely confined to the tactical responses of the AGW supporter. If I was morganovich, I’d switch to another thread, as hunter is clearly never going to listen to reason, nor respond to logic. He BELIEVES.
timbrom – good at spotting ad hominems aren’t you, you cocksucker? Can you also spot arguments from incredulity? Hint: look at morganovich’s most recent post.