Comments on: Regression Abuse

By: Adamson

Adamson — Sun, 29 Nov 2009 01:30:57 +0000

Hi Anthony

Thanks, good job.

But it would be helpful to step through this part in more detail, for the rest of us:

“Measured temperatures are going up, so his regression locked onto this proxy and … flipped the sign. In effect, it reversed the proxy. As far as his models are concerned, this proxy is averaged in with values of the opposite sign, like this:”

Thanks in advance

By: Ralph Seeley

Ralph Seeley — Sat, 28 Nov 2009 16:16:34 +0000

The error of the blind use of statistics was demonstrated very forcibly to me when I was working with Sir Maurice Kendall (joint author of the Advanced Theory of Statistics in the 1970s). We were using Regression Analysis to forecast turning points for the UK economy. He had found a new regressor variable called “Carbon” that proved to be a superb fit for the historical data. We presumed this new variable to be some type of energy composite and were eager to learn more. It wasn’t. It was the outline shape of his pet black cat (called “Carbon”) that had chosen to lie on a sheet of graph paper on the great man’s lap. He had simply drawn round it and read off the results.

By: allen/asheville

allen/asheville — Tue, 03 Nov 2009 18:51:42 +0000

i’m retired from forestry, 29 yrs.USDA, & though i don’t actually have a forestry degree, i think i’m pretty close.i haven’t read the comments so some of this may be repetitive, but i haven’t been a firm believer in models or how they’re derived as i worked closely w/ foresters. 1st though not always, they use other people’s data.NOT THEIR OWN.so they don’t seem to investigate how robust the data is or was. then of course, nowadays, any clown can BUY a nice program to do stuff,& no background needed.hey, best fit should do it.

Trouble in my mind enters in by noting ‘FRESHNESS’ of data. regression gives us HISTORICAL information, so projections, such as often are made simply take the line & project it out. sorry, i’m not a believer

By: Fred3

Fred3 — Tue, 03 Nov 2009 00:44:24 +0000

This is the best summary of the controversy over Mann’s use of Tiljander data.

By: Roger

Roger — Sun, 01 Nov 2009 15:31:43 +0000

As a meteorologist this is embarassing. Obviously, the statistical analyses are suspect. Rather than paint all of us in the same light I want to point out that the most useful class I talk as an undergraduate and graduate student in meteorology was meteorological statistics and one of the key points taught in that class was that you have to know there is a physical relationship between variables before you can claim “significance”. Sadly that rule seems to be overlooked by these climate scientists. The other thing I learned in that class was that we just barely scratched the surface of statistical analysis so asking for help from a statistician was always a good thing.

By: hunter

hunter — Sat, 31 Oct 2009 22:32:18 +0000

Mann’s problem is that his definition of ‘good results’ is ‘hockey stick’.

By: Annabelle

Annabelle — Sat, 31 Oct 2009 15:44:35 +0000

Thanks very much, a very clear explanation. I’ve been struggling to understand the Tiljander issue as I get lost in the technicalities.

By: markm

markm — Fri, 30 Oct 2009 23:48:12 +0000

Mann reminds me of programmers who debug without fully understanding the errors in the software, but just change things until all the tests pass. It’s a dangerous practice even with something as simple as controlling the temperature of a waffle iron–there’s the possibility that software that worked on the test unit will blow up in the next unit. (That’s a real world example; I didn’t get the chance to correct my predecessor’s mistakes, because by the time I knew what had to be done, we’d lost the customer.)

But another way of saying “change things until all the tests pass” is “until you get the respected results”. Mann fiddles with the software until it massages the data into the hockey stick he expects, and thinks that proves it is correct.

By: stan

stan — Fri, 30 Oct 2009 14:30:37 +0000

Michael Mann is the perfect case study for the adage — “a little bit of knowledge is a dangerous thing.”

Leaving him unsupervised with statistics is more dangerous than an unsupervised child with a loaded gun — more people get hurt.

By: SB7

SB7 — Fri, 30 Oct 2009 03:51:58 +0000

Except for your use of “data-mining” as a bit of a smear in the postscript, I’m right with you. Data mining is just a discipline; its techniques can be used or misused like any other. In fact, the number one rule my data mining prof had for us — and he drummed it in as thoroughly as your econ prof did — is that “domain knowledge comes first.” All the machine learning techniques in the world don’t mean anything unless you understand the underlying system. His second commandment was, as Alan mentions, “always validate on withheld data.” It blows me away that people don’t bother to do that either.