(cross posted from Coyote Blog)

For a while, I have criticized the practice both in climate and economics of using computer models to increase our apparent certainty about natural phenomenon.   We take shaky assumptions and guesstimates of certain constants and natural variables and plug them into computer models that produce projections with triple-decimal precision.   We then treat the output with a reverence that does not match the quality of the inputs.

I have had trouble explaining this sort of knowledge laundering and finding precisely the right words to explain it.  But this week I have been presented with an excellent example from climate science, courtesy of Roger Pielke, Sr.  This is an excerpt from a recent study trying to figure out if a high climate sensitivity to CO2 can be reconciled with the lack of ocean warming over the last 10 years (bold added).

“Observations of the sea water temperature show that the upper ocean has not warmed since 2003. This is remarkable as it is expected the ocean would store that the lion’s share of the extra heat retained by the Earth due to the increased concentrations of greenhouse gases. The observation that the upper 700 meter of the world ocean have not warmed for the last eight years gives rise to two fundamental questions:

1. What is the probability that the upper ocean does not warm for eight years as greenhouse gas concentrations continue to rise?
2. As the heat has not been not stored in the upper ocean over the last eight years, where did it go instead?

These question cannot be answered using observations alone, as the available time series are too short and the data not accurate enough. We therefore used climate model output generated in the ESSENCE project, a collaboration of KNMI and Utrecht University that generated 17 simulations of the climate with the ECHAM5/MPI-OM model to sample the natural variability of the climate system. When compared to the available observations, the model describes the ocean temperature rise and variability well.”

Pielke goes on to deconstruct the study, but just compare the two bolded statements.  First, that there is not sufficiently extensive and accurate observational data to test a hypothesis.  BUT, then we will create a model, and this model is validated against this same observational data.  Then the model is used to draw all kinds of conclusions about the problem being studied.

This is the clearest, simplest example of certainty laundering I have ever seen.  If there is not sufficient data to draw conclusions about how a system operates, then how can there be enough data to validate a computer model which, in code, just embodies a series of hypotheses about how a system operates?

A model is no different than a hypothesis embodied in code.   If I have a hypothesis that the average width of neckties in this year’s Armani collection drives stock market prices, creating a computer program that predicts stock market prices falling as ties get thinner does nothing to increase my certainty of this hypothesis  (though it may be enough to get me media attention).  The model is merely a software implementation of my original hypothesis.  In fact, the model likely has to embody even more unproven assumptions than my hypothesis, because in addition to assuming a causal relationship, it also has to be programmed with specific values for this correlation.

This is not just a climate problem.  The White House studies on the effects of the stimulus were absolutely identical.  They had a hypothesis that government deficit spending would increase total economic activity.  After they spent the money, how did they claim success?  Did they measure changes to economic activity through observational data?  No, they had a model that was programmed with the hypothesis that government spending increased job creation, ran the model, and pulled a number out that said, surprise, the stimulus created millions of jobs (despite falling employment).  And the press reported it like it was a real number.

Postscript: I did not get into this in the original article, but the other mistake the study seems to make is to validate the model on a variable that is irrelevant to its conclusions.   In this case, the study seems to validate the model by saying it correctly simulates past upper ocean heat content numbers (you remember, the ones that are too few and too inaccurate to validate a hypothesis).  But the point of the paper seems to be to understand if what might be excess heat (if we believe the high sensitivity number for CO2) is going into the deep ocean or back into space.   But I am sure I can come up with a number of combinations of assumptions to match the historic ocean heat content numbers.  The point is finding the right one, and to do that requires validation against observations for deep ocean heat and radiation to space.

Of the 193 countries covered in the study, the researchers were able to use actual, reported data for only 33. To produce the estimates for the other 160 countries, and to project the figures backwards to 1995, the researchers created a sophisticated statistical model. [1]What’s wrong with a model? Well, 1) the credibility of the numbers that emerge from these models must depend on the quality of “real” (that is, actual measured or reported) data, as well as how well these data can be extrapolated to the “modeled” setting ( e.g. it would be bad if the real data is primarily from rich countries, and it is “modeled” for the vastly different poor countries – oops, wait, that’s exactly the situation in this and most other “modeling” exercises) and 2) the number of people who actually understand these statistical techniques well enough to judge whether a certain model has produced a good estimate or a bunch of garbage is very, very small.

Without enough usable data on stillbirths, the researchers look for indicators with a close logical and causal relationship with stillbirths. In this case they chose neonatal mortality as the main predictive indicator. Uh oh. The numbers for neonatal mortality are also based on a model (where the main predictor is mortality of children under the age of 5) rather than actual data.

So that makes the stillbirth estimates numbers based on a model…which is in turn…based on a model.

This sound familiar to anyone?   The only reason it is not a good analog to climate is that the article did not say that they used mortality data from 1200 kilometers away to estimate a country’s historic numbers.

Smart, numerically facile people who glibly say they support the science of anthropogenic global warming would be appalled if they actually looked at it in any depth.   While gender studies grads and journalism majors seem consistently impressed with the IPCC, physicists, economics, geologists, and others more used to a level of statistical rigor generally turn from believers to skeptics once they dig into the details.  I did.

• Enforce good data and code archiving practices.  Our goal should be that no FOIA is necessary to get the information needed to replicate a published study
• Create an openness to scrutiny and replication which human nature resists, but generally exists in most non-climate sciences.

I worry that over the last few months, with the Virginia FOIA inquiry and the recent investigations of Michael Mann, skeptic’s focus has shifted to trying to take out their frustration with and disdain for Michael Mann in the form of getting him rung up on charges.   I fear the urge to mount Mann’s head in their trophy case is distracting folks from what the real goals here should be.

I know those in academia like to pretend they are not, but professors at state schools or who are doing research with government money are just as much government employees as anyone in the DMV or post office.  And as such, their attempts to evade scrutiny or hide information irritate the hell out of me.  But I would happily give the whole Jones/Mann/Briffa et all Climategate gang a blanket pardon in exchange for some better ground rules in climate science going forward.

Skeptics are rightly frustrated with the politicization of science and the awful personal attacks skeptics get when alarmists try to avoid debate on the science.  But the correct response here is to take the high ground, NOT to up the stakes in the politicization game by bringing academics we think to be incorrect up on charges.  I am warning all of you, this is a bad, bad precedent.

Postscript: I now your response already — there are good and valid legal reasons for charging Mann, here are the statutes he broke, etc.  I don’t disagree.  But here is my point — the precedent we set here will not be remembered as an academic brought down for malfeasance.  It will be remembered as an academic brought down by folks who disagreed with his scientific findings.  You may think that unfair, but that is the way the media works.  The media is not on the skeptic side, and even if it were neutral, it is always biased to the more sensational story line.

I would have thought the same standard exists in scientific research, ie one has an ethical obligation to reveal data or experiments that do not confirm one’s underlying hypothesis or may potentially cast some doubt on the results.  After all, we are after truth, right?

Two posts this week shed some interesting light on this issue  vis a vis dendro-climatology.  I hesitate to pile on much on the tree ring studies at this point, as they have about as much integrity right now as the study of alchemy.  If we are going to get some real knowledge out of this data, someone is going to have to tear the entire field down to bedrock and start over (as was eventually done when alchemy became chemistry).  But I do think both of these posts raise useful issues that go beyond just Mann, Briffa, and tree rings.

In the first, Steve McIntyre looks at one of the Climategate emails from Raymond Bradley where Bradley is almost proudly declaring that MBH98 had purposely withheld data that would have made their results look far less certain.  He taunts skeptics for not yet figuring out the game, an ethical position roughly equivalent to Bernie Madoff taunting investors for being too dumb to figure out he was duping them with a Ponzi scheme.

In the second, Judith Curry takes a look at the Briffa “hide the decline” trick.  There is a lot of confusion about just what this trick was.  In short, the expected behavior of tree ring results in the late 20th century diverged from actual measured temperatures.  In short, the tree rings showed temperatures falling since about 1950 when they have in fact risen.   Since there is substantial disagreement on whether tree rings really do act as reliable proxies for temperatures, this is an important fact since if tree rings are failing to follow temperatures for the last half century, there could easily be similar failures in the past.  Briffa and the IPCC removed the post-1950 tree ring data from key charts presented to the public, and used the graphical trick of overlaying gauge temperature records to imply that the proxies continued to go up.

Given the heat around this topic, Curry tries to step back and look at the issue dispassionately.  Unlike many, she does not assign motivations to people when these are not known, but she does conclude:

There is no question that the diagrams and accompanying text in the IPCC TAR, AR4 and WMO 1999 are misleading.  I was misled.  Upon considering the material presented in these reports, it did not occur to me that recent paleo data was not consistent with the historical record.  The one statement in AR4 (put in after McIntyre’s insistence as a reviewer) that mentions the divergence problem is weak tea.

It is obvious that there has been deletion of adverse data in figures shown IPCC AR3 and AR4, and the 1999 WMO document.  Not only is this misleading, but it is dishonest (I agree with Muller on this one).  The authors defend themselves by stating that there has been no attempt to hide the divergence problem in the literature, and that the relevant paper was referenced.  I infer then that there is something in the IPCC process or the authors’ interpretation of the IPCC process  (i.e. don’t dilute the message) that corrupted the scientists into deleting the adverse data in these diagrams.

The best analogy I can find for this behavior is prosecutorial abuse.  When prosecutors commit abuses (e.g. failure to share exculpatory evidence), it is often because they are just sure the defendent is guilty.  They can convince themselves that even though they are breaking the law, they are serving the law in a larger sense because they are making sure guilty people go to jail.  Of course, this is exactly how innocent people rot in jail for years, because prosecutors are not supposed to be the ultimate aribiter of guilt and innosence.  In the same way, I am sure Briffa et al felt that by cutting ethical corners, they were serving a larger purpose because they were just sure they were right.  Excupatory evidence might just confuse the jury and lead, in their mind, to a miscarriage of justice.   As Michael Mann wrote (as quoted by Curry)

Otherwise, the skeptics have an field day casting doubt on our ability to understand the factors that influence these estimates and, thus, can undermine faith in the paleoestimates. I don’t think that doubt is scientifically justified, and I’d hate to be the one to have to give it fodder!

Given that global warming is “unequivocal” [1], to quote the 2007 IPCC report [2], the null hypothesis should now be reversed, thereby placing the burden of proof on showing that there is no human influence [3]. Such a null hypothesis is trickier because one has to hypothesize something specific, such as “precipitation has increased by 5%” and then prove that it hasn’t. Because of large natural variability, the first approach results in an outcome suggesting that it is appropriate to conclude that there is no increase in precipitation by human influences, although the correct interpretation is that there is simply not enough evidence (not a long enough time series). However, the second approach also concludes that one cannot say there is not a 5% increase in precipitation. Given that global warming is happening and is pervasive, the first approach should no longer be used. As a whole the community is making too many type II errors [4].

Are you kidding me — if already every damn event in the tails of the normal distribution is taken by the core climate community as a proof of their hypothesis, how is there even room for type II errors?  Next up — “Our beautiful, seasonal weather — proof of global warming?”

Remember that the IPCC’s conclusion of human-caused warming was based mainly on computer modelling.  The IPCC defenders will not admit this immediately, but press them hard enough on side arguments and it comes down to the models.

The summary of their argument is this:  for the period after 1950, they claim their computer models cannot explain warming patterns without including a large effect from anthropogenic CO2.  Since almost all the warming in the latter half of the century really occurred between 1978 and 1998, the IPCC core argument boils down to “we are unable to attribute the global temperature increase in these 20 years to natural factors, so it must have been caused by man-made CO2.”  See my video here for a deeper discussion.

This seems to be a fairly thin reed.  After all, it may just be that after only a decade or two of serious study, we still do not understand climate variability very well, natural or not.  It is a particularly odd conclusion when one discovers that the models ignore a number of factors (like the PDO, ENSO, etc) that affect temperatures on a decadal scale.

We therefore have a hypothesis that is not based on observational data, and where those who hold the hypothesis claim that observational data should no longer be used to test their hypothesis.    He is hilarious when he says that reversing the null hypothesis would make it trickier for his critics.  It would make it freaking impossible, as he very well knows.  This is an unbelievingly disingenuous suggestion.  There are invisible aliens in my closet Dr. Trenberth — prove me wrong.  It is always hard to prove a negative, and impossible in the complex climate system.  There are simply too many variables in flux to nail down cause and effect in any kind of definitive way, at least at our level of understanding  (we have studied economics much longer and we still have wild disagreements about cause and effect in macroeconomics).

He continues:

So we frequently hear that “while this event is consistent with what we expect from climate change, no single event can be attributed to human induced global warming”. Such murky statements should be abolished. On the contrary, the odds have changed to make certain kinds of events more likely. For precipitation, the pervasive increase in water vapor changes precipitation events with no doubt whatsoever. Yes, all events! Even if temperatures or sea surface temperatures are below normal, they are still higher than they would have been, and so too is the atmospheric water vapor amount and thus the moisture available for storms. Granted, the climate deals with averages. However, those averages are made up of specific events of all shapes and sizes now operating in a different environment. It is not a well posed question to ask “Is it caused by global warming?” Or “Is it caused by natural variability?” Because it is always both.

At some level, this is useless.   The climate system is horrendously complex.  I am sure everything affects everything.  So to say that it affects the probability is a true but unhelpful statement.   The concern is that warming will affect the rate of these events, or the severity of these events, in a substantial and noticeable way.

It is worth considering whether the odds of the particular event have changed sufficiently that one can make the alternative statement “It is unlikely that this event would have occurred without global warming.” For instance, this probably applies to the extremes that occurred in the summer of 2010: the floods in Pakistan, India, and China and the drought, heat waves and wild fires in Russia.

Wow, suddenly skepticism, and even outright harsh criticism, of peer-reviewed work is OK, as long as it is not in climate I suppose.

On Thursday, Dec. 2, Rosie Redfield sat down to read a new paper called “A Bacterium That Can Grow by Using Arsenic Instead of Phosphorus.” Despite its innocuous title, the paper had great ambitions. Every living thing that scientists have ever studied uses phosphorus to build the backbone of its DNA. In the new paper, NASA-funded scientists described a microbe that could use arsenic instead. If the authors of the paper were right, we would have to expand our….

As soon Redfield started to read the paper, she was shocked. “I was outraged at how bad the science was,” she told me.

Redfield blogged a scathing attack on Saturday. Over the weekend, a few other scientists took to the Internet as well. Was this merely a case of a few isolated cranks? To find out, I reached out to a dozen experts on Monday. Almost unanimously, they think the NASA scientists have failed to make their case. “It would be really cool if such a bug existed,” said San Diego State University’s Forest Rohwer, a microbiologist who looks for new species of bacteria and viruses in coral reefs. But, he added, “none of the arguments are very convincing on their own.” That was about as positive as the critics could get. “This paper should not have been published,” said Shelley Copley of the University of Colorado.

The article goes on to describe many potential failures in the methodology.  None of this should be surprising — I have written for years that peer-review is by no means proof against bad science or incorrect findings.  It is more of an  extended editorial process.  The real test of published science comes later, when the broader community attempts to replicate results.

The problem in climate science has been that its proponents want to claim that having research performed by a small group of scientists that is peer-reviewed by the same small group is sufficient to making the results “settled science.”  Once published, they argue, no one (certainly not laymen on blogs) has the right to criticize it, and the researchers don’t (as revealed in the Climategate emails) have any obligations to release their data or code to allow replication.   This is just fresh proof that this position is nuts.

The broken climate science process is especially troubling given the budgetary and reputational incentives to come out with the most dramatic possible results, something NASA’s James Hansen has been accused of doing by many climate skeptics.  To this end, consider this from the bacteria brouhaha.  First, we see the same resistance to criticism, trying to deflect any critiques outside of peer-reviewed journals

“Any discourse will have to be peer-reviewed in the same manner as our paper was, and go through a vetting process so that all discussion is properly moderated,” wrote Felisa Wolfe-Simon of the NASA Astrobiology Institute. “The items you are presenting do not represent the proper way to engage in a scientific discourse and we will not respond in this manner.”

WTF?  How, then, did we ever have scientific process before peer-reviewed journals appeared on the scene?

But Jonathan Eisen of UC-Davis doesn’t let the scientists off so easily. “If they say they will not address the responses except in journals, that is absurd,” he said. “They carried out science by press release and press conference. Whether they were right or not in their claims, they are now hypocritical if they say that the only response should be in the scientific literature.”

Wow, that could be verbatim from a climate skeptic in the climate debate.

And finally, this on incentives and scientific process:

Some scientists are left wondering why NASA made such a big deal over a paper with so many flaws. “I suspect that NASA may be so desperate for a positive story that they didn’t look for any serious advice from DNA or even microbiology people,” says John Rothof UC-Davis.

Mr. James Fallows
National Correspondent, The Atlantic

Dear Mr. Fallows:

This afternoon on National Public Radio you proclaimed that “there is essentially no disagreement whatsoever” among economists that more stimulus spending is necessary today [emphasis in the original].

You are misinformed.

Last year, hundreds of economists signed a petition, circulated by the Cato Institute, whose key clause reads “it is a triumph of hope over experience to believe that more government spending will help the U.S. today.”  Among the economists who signed this petition in opposition to ‘stimulus’ spending are three Nobel laureates in economics (Edward Prescott, Vernon Smith, and my colleague James Buchanan).  Others signers include Chicago’s Eugene Fama and Sam Peltzman, Harvard’s Jeffrey Miron, Texas A&M’s Thomas Saving, Cornell’s Rick Geddes and Dean Lillard, University of Virginia’s Lee Coppock and Kenneth Elzinga, Duke’s Michael Munger and Edward Tower, University of Rochester’s Mark Bils and Ronald Schmidt, Rutger’s Michael Bordo and Leo Troy, University of Southern California’s John Matsusaka and Kevin Murphy, and one of the world’s preeminent scholars of money and banking, Carnegie-Mellon’s Allan Meltzer.

Perhaps these economists and the many others who’ve signed this petition (including myself) – and who continue to speak out against what we believe to be the folly of ‘stimulus’ – are mistaken.  But for you to announce publicly that there is “no disagreement whatsoever” among economists that more stimulus spending is desirable is so wildly inaccurate that it borders on being irresponsible.

Sincerely,
Donald J. Boudreaux
Professor of Economics
George Mason University
Fairfax, VA 22030

I compared macroeconomic and climate models in my Forbes column here.

The typical response by politicians, of course, is to try to get more money from taxpayers.  California has a ballot initiative this November proposing to raise vehicle licensing fees to all its citizens in order to fund state parks.  Unfortunately, this kind of funds earmarking by ballot initiative is already threatening to bankrupt California.  One problem with this approach is that it demolishes accountability — once an unelected state agency gets dedicated funds the legislature can’t touch, there is nothing that taxpayers can do if these funds are not spent wisely short of another time-consuming ballot initiative to revoke them.

In the case of state parks, the accountability problem is even worse, as the initiatives replace park user fees, which at least enforce accountability in that users can stop visiting if park services are not up to expectations, with a no-strings-attached bureaucratic windfall.  Compounding the problem, in many states the parks organizations report to rubber-stamp boards rather than the governor or any elected official, so taxpayers have absolutely no path to enforce accountability.

Ironically, however, one genuine sin committed by too many economists is the sin of public hubris — of posing as seers who can divine the details of the future economy, of fooling themselves and the public that economists possess greater knowledge than they really do.

In their papers and books (and now blogs), Keynesian economists model the economy with simple symbols, such as “C” for consumption spending, “I” for investment spending and “k” for the economy’s stock of capital goods such as diesel engines, steel mills and industrial chemicals.

Governments and private researchers gather data on consumer spending, on investment spending and on the market value of all the stuff called “k.” Economists plug these data into computer-based mathematical models filled with “C’s” and “I’s” and “k’s” and other symbols from alphabets both Latin and Greek. These models then spit out precise predictions.

Voila! exclaims the economist slathered in hubris. “See my multivariable model and my precise-to-several-decimal places predictions! I’m a scientist!”

In fact, he’s an alchemist. He is misled — by the intricacy of the equations on his computer screen and by the apparent concreteness of the data that he shoves into those equations — into thinking that he’s doing science. He is misled into thinking that these leaden, aggregated data from the past can be transformed into golden truths about the future.

University of Arizona will host one of eight regional climate-science centers to help the federal government study the potential effects of climate change on natural resources and the environment.

The Southwest Climate Center will bring together scientists from six universities to study a range of issues and offer guidance to federal resource managers through the Interior Department, which will oversee the regional center.

Jonathan Overpeck, co-director of the UA Institute of the Environment, will be the center’s principal investigator….

Interior Secretary Ken Salazar on Wednesday announced the selection of UA to host the center, which will receive an initial five-year, $3.1 million grant.

The University of Arizona climate department has distinguished itself in the past by running this fine temperature monitoring station, located between buildings in the middle of an asphalt parking lot:

More important to the selection than the UA’s staff actual ability to, you know, monitor the climate is likely Jonathan Overpeck’s impeccable credentials in the alarmist community.  Overpeck was a coordinating lead author of the IPCC AR4 paleoclimate chapter, and thus had a leading role in promoting the hockey stick and attempting to make the Medieval Warm Period go away.

The powers that be that give out large grants certainly weren’t going to give the center to Arizona State, which had the temerity to actually have skeptic Robert Balling on staff (with Sherwood Idso as an adjunct professor) .  If ASU wants any real climate cash, they likely will need to find a way to get rid of Balling under some pretext.

We can see that employing skeptics is very bad for business.  After all, Exxon gave the ASU climate department $49,500, compared to  62 times this amount to UA from alarmists in Washington.  Of course, we all know that the Exxon money was far more corrupting.  ASU likely perverted science entirely for 49K, but UA would never do so for 3.1 million.