“The whole point of the RC post is that we do not know about solar cycles to determine how much effect they have (did either of you actually read the post?).”

For fuck sake Waldo, did YOU read that paper you picked in the other thread? A very large part of it was dealing with how the forcings we know of, LIKE SOLAR CYCLES, effect the Earth’s temperature. How about you go back and read a PEER REVIEWED journal (you know the things you give so much more credibility than a stupid blog like realclimate), and try to actually learn something this time. If that paper doesn’t explain certain topics well enough for you, look up the citations used in the sections where you get confused (ok, you might as well pull the entire reference listing), then see if you can learn something again. If you can’t repeat until you do, or until you admit it to yourself that you will never understand.

Firstly, great to see a reasoned post which appears on face value to be from a real scientist.

I am leaning somewhat toward the skeptic side of this debate not because I am a scientist (I am an engineer), nor becasue I have read much of the literature, but because on face value the AGW predictions have failed, and far from “its hot out here”, if sea temperature is anything to go by then it is perhaps just slightly cooler.

Walsoroooo bemoans the lack of science in here (perhaps rightly), but the intention of this blog is not to be a discussion between “scientists”, but a forum for discussion between interested people. It is fair for him to pleade with the bloggers about what is obvious, but what seems increasingly obvious to me, and so far, all other engineers I meet, is that the King has got fewer and fewer clothes on every week. I think the bored shepherd boy has cried wolf twice already too.

A scientific theory which is all about prediction clearly needs to be able to predict to some degree, and not just make excuses for why it keeps failing. Netdr quite rightly uses the “racing forms” analogy. If it is invalid then it should be pretty easy for a real scientist to outline the predictions that have been validated, and for these to then be open for discussion.

Mann et al at “realclimate”, on January 6, 2005, stated that McIntyre and McKittrick were mistaken, and cited “Jolliffe” as an authority on PCA, stating that
non-centered PCA was a valid procedure.

Tamino later posted a series on “PCA” on his blog, “Open Mind”. Part 4 was posted on March 6, 2008. In this posting he asserted that
non centerd PCA was perfectly okay, stating,
” You shouldn’t just take my word for it, but you should take the word of Ian Jolliffe, one of the world’s foremost experts on PCA…”

Finally there’s a reply from Ian Jolliffe himself, who in effect said McIntyre and McKittrick’s criticism was correct, and Mann’s hockey stick
was based on dubious statistics.

From the January 6, 2005 “realclimate” post,

http://www.realclimate.org/index.php/archives/2005/01/on-yet-another-false-claim-by-mcintyre-and-mckitrick/

“6 Jan 2005
On Yet Another False Claim by McIntyre and McKitrick”…

“McIntyre and McKitrick (MM), in one of their many false claims regarding the Mann et al (MBH98) temperature reconstruction, assert that the “Hockey Stick” shape of the reconstruction is an artifact of the “non-centered” Principal Components Analysis (PCA) convention used by MBH98 in representing the North American International Tree Ring Data Bank (ITRDB) data series. “…

“Contrary to MM’s assertions, the use of non-centered PCA is well-established in the statistical literature, and in some cases is shown to give superior results to standard, centered PCA. See for example page 3 (middle paragraph) of this review. For specific applications of non-centered PCA to climate data, consider this presentation provided by statistical climatologist Ian Jolliffe who specializes in applications of PCA in the atmospheric sciences, having written a widely used text book on PCA. In his presentation, Jollife explains that non-centered PCA is appropriate when the reference means are chosen to have some a priori meaningful interpretation for the problem at hand.” ….

On March 6, 2008, “Open Mind” defended Mann’s hockey stick.

http://tamino.wordpress.com/2008/03/06/pca-part-4-non-centered-hockey-sticks/

“First let’s dispense with the last claim, that non-centered PCA isn’t right. This point was hammered by Wegman, who was recently quoted in reader comments thus:

“The controversy of Mann’s methods lies in that the proxies are centered on the mean of the period 1902-1995, rather than on the whole time period. This mean is, thus, actually decentered low, which will cause it to exhibit a larger variance, giving it preference for being selected as the first principal component. The net effect of this decentering using the proxy data in MBH and MBH99 is to produce a “hockey stick” shape. Centering the mean is a critical factor in using the principal component methodology properly. It is not clear that Mann and associates realized the error in their methodology at the time of publication.”

Just plain wrong. Centering is the usual custom, but other choices are still valid; we can perfectly well define PCs based on variation from any “origin” rather than from the average. It fact it has distinct advantages IF the origin has particular relevance to the issue at hand. You shouldn’t just take my word for it, but you *should* take the word of Ian Jolliffe, one of the world’s foremost experts on PCA, author of a seminal book on the subject. He takes an interesting look at the centering issue in this presentation.”

And finally we get a reply from Jolliffe himself:

http://tamino.wordpress.com/2008/08/10/open-thread-5-2/#comment-21873

“Ian Jolliffe // September 8, 2008 at 9:36 am

Apologies if this is not the correct place to make these comments. I am a complete newcomer to this largely anonymous mode of communication. I’d be grateful if my comments could be displayed wherever it is appropriate for them to appear.

It has recently come to my notice that on the following website, related to this one, my views have been misrepresented, and I would therefore like to correct any wrong impression that has been given.
http://tamino.wordpress.com/2008/03/06/pca-part-4-non-centered-hockey-sticks/

An apology from the person who wrote the page would be nice.

In reacting to Wegman’s criticism of ‘decentred’ PCA, the author says that Wegman is ‘just plain wrong’ and goes on to say ‘You shouldn’t just take my word for it, but you *should* take the word of Ian Jolliffe, one of the world’s foremost experts on PCA, author of a seminal book on the subject. He takes an interesting look at the centering issue in this presentation.’ It is flattering to be recognised as a world expert, and I’d like to think that the final sentence is true, though only ‘toy’ examples were given. However there is a strong implication that I have endorsed ‘decentred PCA’. This is ‘just plain wrong’.

The link to the presentation fails, as I changed my affiliation 18 months ago, and the website where the talk lived was closed down. The talk, although no longer very recent – it was given at 9IMSC in 2004 – is still accessible as talk 6 at http://www.secamlocal.ex.ac.uk/people/staff/itj201/RecentTalks.html
It certainly does not endorse decentred PCA. Indeed I had not understood what MBH had done until a few months ago. Furthermore, the talk is distinctly cool about anything other than the usual column-centred version of PCA. It gives situations where uncentred or doubly-centred versions might conceivably be of use, but especially for uncentred analyses, these are fairly restricted special cases. It is said that for all these different centrings ‘it’s less clear what we are optimising and how to interpret the results’.
I can’t claim to have read more than a tiny fraction of the vast amount written on the controversy surrounding decentred PCA (life is too short), but from what I’ve seen, this quote is entirely appropriate for that technique. There are an awful lot of red herrings, and a fair amount of bluster, out there in the discussion I’ve seen, but my main concern is that I don’t know how to interpret the results when such a strange centring is used? Does anyone? What are you optimising? A peculiar mixture of means and variances? An argument I’ve seen is that the standard PCA and decentred PCA are simply different ways of describing/decomposing the data, so decentring is OK. But equally, if both are OK, why be perverse and choose the technique whose results are hard to interpret? Of course, given that the data appear to be non-stationary, it’s arguable whether you should be using any type of PCA.
I am by no means a climate change denier. My strong impressive is that the evidence rests on much much more than the hockey stick. It therefore seems crazy that the MBH hockey stick has been given such prominence and that a group of influential climate scientists have doggedly defended a piece of dubious statistics. Misrepresenting the views of an independent scientist does little for their case either. It gives ammunition to those who wish to discredit climate change research more generally. It is possible that there are good reasons for decentred PCA to be the technique of choice for some types of analyses and that it has some virtues that I have so far failed to grasp, but I remain sceptical. ”

Ian Jolliffe

**************************

To summarize, Mann created the “hockey stick” using non-centered Principal Component Analysis. McIntyre and McKittrick, and later Wegman, stated
that Mann’s use of non-centered Principal component Analysis was flawed, and that the algorithm in effect was “searching for hockey sticks”.

Mann et al at “realclimate”, on January 6, 2005, stated that McIntyre and McKittrick were mistaken, and cited “Jolliffe” as an authority on PCA, stating that
non-centered PCA was a valid procedure.

Tamino later posted a series on “PCA” on his blog, “Open Mind”. Part 4 was posted on March 6, 2008. In this posting he asserted that
non centerd PCA was perfectly okay, stating,
” You shouldn’t just take my word for it, but you should take the word of Ian Jolliffe, one of the world’s foremost experts on PCA…”

Finally there’s a reply from Ian Jolliffe himself, who in effect said McIntyre and McKittrick’s criticism was correct, and Mann’s hockey stick
was based on dubious statistics.

From the January 6, 2005 “realclimate” post,

http://www.realclimate.org/index.php/archives/2005/01/on-yet-another-false-claim-by-mcintyre-and-mckitrick/

“6 Jan 2005
On Yet Another False Claim by McIntyre and McKitrick”…

“McIntyre and McKitrick (MM), in one of their many false claims regarding the Mann et al (MBH98) temperature reconstruction, assert that the “Hockey Stick” shape of the reconstruction is an artifact of the “non-centered” Principal Components Analysis (PCA) convention used by MBH98 in representing the North American International Tree Ring Data Bank (ITRDB) data series. “…

“Contrary to MM’s assertions, the use of non-centered PCA is well-established in the statistical literature, and in some cases is shown to give superior results to standard, centered PCA. See for example page 3 (middle paragraph) of this review. For specific applications of non-centered PCA to climate data, consider this presentation provided by statistical climatologist Ian Jolliffe who specializes in applications of PCA in the atmospheric sciences, having written a widely used text book on PCA. In his presentation, Jollife explains that non-centered PCA is appropriate when the reference means are chosen to have some a priori meaningful interpretation for the problem at hand.” ….

On March 6, 2008, “Open Mind” defended Mann’s hockey stick.

http://tamino.wordpress.com/2008/03/06/pca-part-4-non-centered-hockey-sticks/

“First let’s dispense with the last claim, that non-centered PCA isn’t right. This point was hammered by Wegman, who was recently quoted in reader comments thus:

“The controversy of Mann’s methods lies in that the proxies are centered on the mean of the period 1902-1995, rather than on the whole time period. This mean is, thus, actually decentered low, which will cause it to exhibit a larger variance, giving it preference for being selected as the first principal component. The net effect of this decentering using the proxy data in MBH and MBH99 is to produce a “hockey stick” shape. Centering the mean is a critical factor in using the principal component methodology properly. It is not clear that Mann and associates realized the error in their methodology at the time of publication.”

Just plain wrong. Centering is the usual custom, but other choices are still valid; we can perfectly well define PCs based on variation from any “origin” rather than from the average. It fact it has distinct advantages IF the origin has particular relevance to the issue at hand. You shouldn’t just take my word for it, but you *should* take the word of Ian Jolliffe, one of the world’s foremost experts on PCA, author of a seminal book on the subject. He takes an interesting look at the centering issue in this presentation.”

And finally we get a reply from Jolliffe himself:

http://tamino.wordpress.com/2008/08/10/open-thread-5-2/#comment-21873

“Ian Jolliffe // September 8, 2008 at 9:36 am

Apologies if this is not the correct place to make these comments. I am a complete newcomer to this largely anonymous mode of communication. I’d be grateful if my comments could be displayed wherever it is appropriate for them to appear.

It has recently come to my notice that on the following website, related to this one, my views have been misrepresented, and I would therefore like to correct any wrong impression that has been given.
http://tamino.wordpress.com/2008/03/06/pca-part-4-non-centered-hockey-sticks/

An apology from the person who wrote the page would be nice.

In reacting to Wegman’s criticism of ‘decentred’ PCA, the author says that Wegman is ‘just plain wrong’ and goes on to say ‘You shouldn’t just take my word for it, but you *should* take the word of Ian Jolliffe, one of the world’s foremost experts on PCA, author of a seminal book on the subject. He takes an interesting look at the centering issue in this presentation.’ It is flattering to be recognised as a world expert, and I’d like to think that the final sentence is true, though only ‘toy’ examples were given. However there is a strong implication that I have endorsed ‘decentred PCA’. This is ‘just plain wrong’.

The link to the presentation fails, as I changed my affiliation 18 months ago, and the website where the talk lived was closed down. The talk, although no longer very recent – it was given at 9IMSC in 2004 – is still accessible as talk 6 at http://www.secamlocal.ex.ac.uk/people/staff/itj201/RecentTalks.html
It certainly does not endorse decentred PCA. Indeed I had not understood what MBH had done until a few months ago. Furthermore, the talk is distinctly cool about anything other than the usual column-centred version of PCA. It gives situations where uncentred or doubly-centred versions might conceivably be of use, but especially for uncentred analyses, these are fairly restricted special cases. It is said that for all these different centrings ‘it’s less clear what we are optimising and how to interpret the results’.
I can’t claim to have read more than a tiny fraction of the vast amount written on the controversy surrounding decentred PCA (life is too short), but from what I’ve seen, this quote is entirely appropriate for that technique. There are an awful lot of red herrings, and a fair amount of bluster, out there in the discussion I’ve seen, but my main concern is that I don’t know how to interpret the results when such a strange centring is used? Does anyone? What are you optimising? A peculiar mixture of means and variances? An argument I’ve seen is that the standard PCA and decentred PCA are simply different ways of describing/decomposing the data, so decentring is OK. But equally, if both are OK, why be perverse and choose the technique whose results are hard to interpret? Of course, given that the data appear to be non-stationary, it’s arguable whether you should be using any type of PCA.
I am by no means a climate change denier. My strong impressive is that the evidence rests on much much more than the hockey stick. It therefore seems crazy that the MBH hockey stick has been given such prominence and that a group of influential climate scientists have doggedly defended a piece of dubious statistics. Misrepresenting the views of an independent scientist does little for their case either. It gives ammunition to those who wish to discredit climate change research more generally. It is possible that there are good reasons for decentred PCA to be the technique of choice for some types of analyses and that it has some virtues that I have so far failed to grasp, but I remain sceptical. ”

Ian Jolliffe

So straght from the horse’s mouth, Mann’s use of non centered PCA was crap-

**************************

What I find fascinating is the mentality of the denialist camp. It’s obvious that netdr did not read or digest the post, and yet Mr. M…I mean, netdr’s first reaction is to begin looking for propagandistic statements such as “that is exactly what the CO2 mafia does.” Actually, this is exactly what someone like Limbaugh or Hannity does. Again netdr, you should just begin each post with “Oh yeah, well” and then continue from there. And then there’s Doc Navy who ostensibly has some scientific background who also apparently did not digest the content of the post and who immediately leaps on the idea that Gavin “shut himself down.”

As I’ve posted to Wally any number of times, there is no real science here, simply the adolescent need to declare victory over the bad guys. And this evaporates once we actually look at the what the bad guy scientists say rather than read some highly inflammatory cross-posting from C3 or some other blog-source.

Oh folks, how can you ignore this when it is so obvious?

Sorry it won’t wash.

“Update (Jul 24):Thus, my contention that solar doesn’t explain much of the variability in this record isn’t valid. It could explain anything from 5 to 40% (with unknown error bars). Sorry for any confusion.”

****************************************************************
The solar cycles could explain up to 40 % of the warming.

Your hide remains firmly nailed to the shed Fred !

“The cooling portion of a climate cycle that included the LIA ended roughly around 1800, give or take a few years. Now, what happens at the end of a cycle swing? Obviously, it goes back the other direction. Therefor, calling the warming that occurred NATURALLY after the LIA a “recovery” is just fine”

Obviously you think you know of all the supposed cycles you need to add together to understand the climate system perfectly, and you think they all add up to warming after 1800. So go on then, list all the cycles, their periods and their amplitudes. Show that they all add up to global warming. And explain why CO2 is not a greenhouse gas in your understanding.

Alternatively, admit that you said something spectacularly stupid, based on appalling ignorance and grotesque misunderstandings. And then stop bullshitting about subjects you know nothing of.

“I’m surprised that Warren hasn’t banned you”

Ha ha! You think he reads these comments? Have you ever seen even a single response to anything that anyone has said? He’s considerably more stupid even than you, and doesn’t have the intellect to understand the comments, let alone respond.

1) The whole point of the RC post is that we do not know about solar cycles to determine how much effect they have (did either of you actually read the post?).

2) The point of my post was to give Doc and hunter an actual expert on the subject. But most importantly in regard to your above comments…

Ready?

3) Gavin is talking only about one particular study as an example of a claim which overstates its conclusions. He is not talking about his entire post, simply a single study that he commented upon in the body of his post. You did see that, didn’t you? No? Oh, well here, let me help you.
This is para. #8

“Some contrarian commentators have recently fallen into the habit of mass mailing any new solar-related abstracts and implying that the existence of solar forcing in the past negates any possible recent anthropogenic impact on climate.”

In other words, some people are just too ready to declare solar activity the fabled forcing. Gavin says whoa nelly!

“Since these studies do not have any implication for the radiative impact of CO2, and don’t change the fact that there has been no effective change in any solar indices since about 1950, it is hard to see a substantial basis for this (implied) argument.”

Since these studies no not change the nature of CO2, and since there has been no change in solar activity since 1950, it is hard to see that there is a correlation between solar forcings and climate change.

Now here is where Gavin starts talking about the study he will later correct himself on:

“For instance, there has been a lot of recent attention paid to Mangini et al. (2005) where a solar link to a new Alpine speleothem record was claimed. However, a quick analysis (right) indicates that the explained variance in the record (smoothed over 25 years) correlated to the 14C-production function (a slightly cleaner solar proxy than the resdiual atmospheric 14C (Muscheler et al, 2005 – see comment/link below)) is only about 5%. Hardly a definitive refutation of anthropogenic greenhouse gas forcing.”

That’s it. In other words, there is nothing in Mangini et al to suggest that the study of solar forcings refutes any findings about CO2. Gavin estimates that the “production function” is only 5%. This is the only thing he is corrects – this 5% number, that’s it.

Later, because he is an honest commentator (unlike some people we know), Gavin comes back to point out that he made a mistake when talking about the variability of this particular study and writes “my contention that solar doesn’t explain much of the variability in this record isn’t valid. It could explain anything from 5 to 40% (with unknown error bars).” So his original point – that we don’t know enough yet about solar forcings – is still absolutely intact, with a de facto example to boot.

It’s pretty sad, folks, when a guy with my level of scientific training has to point something like this out. You should just begin each of your posts with “Oh yeah, well” before you begin posting about “climate mafias” and the like.

Sorry, netdr, no hides on the shed yet, my friend. Are you sure you’re actually an engineer? Why does your moniker link back to CS – a question you continually ignore.

Incredible !

You didn’t even read what you posted. Simply incredible.

Doc nailed your hide to the shed Fred !

The most significant part of your whole quote can be found at the bottom of the page.

“Update… Thus, my contention that solar doesn’t explain much of the variability in this record isn’t valid.”

He just shut himself down.

Doc