Finem Respice takes on a number of the insider responses of the “nothing to see here, move along” type.  My favorite:

“The language used by scientists in the emails in question is indicative of scientists under a great deal of political pressure from the outside.”

The heart bleeds with an anguish and despair so palpable, so imbued with the darkly iridescent and sickly sweet venom of suffering that it is plainly visible out to 50 meters as a colorful aura, brightly fluorescing through the ribcage, glowing like a beacon of sorrow to any empath with the skill level of a Freshman at Vassar who thinks she remembers once reading a book on shamanism.

As a group one rarely sees scientists (or, indeed, any vocational group other than politicians) so deeply in love with the by-hook-or-by-crook of politics, the grand import of jetting off to Nice for the next climate meeting and the limelight that accompanies all these world-saving goings on as those few, those lucky few exposed in the CRU emails. (Just throw in a bit of expense scandal and you might as well be in the House of Commons- oh, wait a second….) It is all but impossible not to come away with a sense of what is plainly a naked lust for naked ambition simply oozing out of those texts. I am utterly devoid of sympathy for any such that later claim to have been forced to compromise their composure, their decorum or their data because of the unfortunate realities of politics.

When I think back to all the thousands of words I have written on positive feedback on this site, only to have it all said in two lines of a post at the same site:

On Positive Feedback

Name three positive feedback systems in nature. Get back to me on that when you’re done.

I might have said added “long-term stable” between name and three, but its pretty close.

The issue of non-publication agreements has come up before as an excuse for EAU-CRU not to release FOIA’d data.

The U-turn by the university follows a week of controversy after the emergence of hundreds of leaked emails, “stolen” by hackers and published online, triggered claims that the academics had massaged statistics.

In a statement welcomed by climate change sceptics, the university said it would make all the data accessible as soon as possible, once its Climatic Research Unit (CRU) had negotiated its release from a range of non-publication agreements.

This excuse had a certain credibility problem even before the email release, as it was something like the fifth or sixth excuse, used only when the others failed.  Now we see from the emails it was more of a “strategy” for avoiding scrutiny rather than a real concern.

These aren’t corporations arguing over iPod code.  These are universities and public institutions.  Where was the outrage from East Anglia?  Where was the guy saying “What do you mean non-publication agreement – this is scientific information gathered at public expense affecting enormous public policy decisions — of course its going to be freaking published.”  Instead, EAU’s response to the request for non-publication seems to be “good, that gives us another excuse not to release anything so that skeptic’s can pick us apart.”

Excellent.  Hard to excerpt because it is all so spot on.  I will give two examples:

2. “Climate change denier”. A first cousin to the first annoyance. Again, thirty years ago, “climate change denier” would have meant someone who denied that the Medieval Warm Period ever happened. Or that the Little Ice Age ever happened. What a kook fringe thing to believe that would have been! And now, those of us who still believe in natural climate change are called “climate change deniers”?? ARGHH….

6. A lack of common sense. Common sense can be misleading, of course. But when there is considerable uncertainty, sometimes it is helpful to go ahead and use a little anyway. Example: It is well known that the net effect of clouds is to cool the Earth in response to radiant heating by the sun. But when it comes to global warming, all climate models do just the opposite…change clouds in ways that amplify radiative warming. While this is theoretically possible, it is critical to future projections of global warming that the reasons why models do this be thoroughly understood. Don’t believe it just because group think within the climate modeling community has decided it should be so.

A lot of folks have asked me why I wasn’t more energized initially about the climate emails.  One reasons is that they don’t really reveal much that I and others who did not go along blindly with the orthodoxy knew for years.  The emails are kind of like getting a written confession from OJ Simpson to Nicole Brown’s murder-  newsworthy, but it wouldn’t be confirming anything I didn’t already know.

Here is a good example.  I wrote this November 10, 2007:

By the way, here is a little lesson about the integrity of climate science.  See that light blue line [in this graph from the IPCC Fourth Assessment]?  Here, let’s highlight it:

For some reason, the study’s author cut the data off around 1950.  Is that where his proxy ended?  No, in fact he had decades of proxy data left.  However, his proxy data turned sharply downwards in 1950.  Since this did not tell the story he wanted to tell, he hid the offending data by cutting off the line, choosing to conceal the problem rather than have an open scientific discussion about it.

The study’s author?  Keith Briffa, who the IPCC named to lead this section of their Fourth Assessment.

The recent data release apparently includes this chart of Briffa’s proxy data discussed above (from Steve McIntyre via Anthony Watts):

For the very first time, the Climategate Letters “archived” the deleted portion of the Briffa MXD reconstruction of “Hide the Decline” fame – see here. Gavin Schmidt claimed that the decline had been “hidden in plain sight” (see here. ). This isn’t true.

The post-1960 data was deleted from the archived version of this reconstruction at NOAA here and not shown in the corresponding figure in Briffa et al 2001. Nor was the decline shown in the IPCC 2001 graph, one that Mann, Jones, Briffa, Folland and Karl were working in the two weeks prior to the “trick” email (or for that matter in the IPCC 2007 graph, an issue that I’ll return to.)

A retrieval script follows.

For now, here is a graphic showing the deleted data in red.

Figure 1. Two versions of Briffa MXD reconstruction, showing archived and climategate versions. The relevant IPCC 2001 graph, shown below, clearly does not show the decline in the Briffa MXD reconstruction.

Contrary to Gavin Schmidt’s claim that the decline is “hidden in plain sight”, the inconvenient data has simply been deleted.

I think his hockey stick looks a little flacid.  Fortunately someone invented Tiljander sediments, a form of data Viagra (as long as one flips it upside down) and such games were no longer necessary.

Pay no attention to those guys behind the curtain.   Its all settled science.  There is total consensus among tens of thousands of scientists.  Which is why these guys were so hesitant to let anyone else view the details of their work, or allow anyone to criticize them in print.  The social network of the climate emails, via Warwick Hughes.

From Lou Glazner via Anthony Watts

1. The scientists colluded in efforts to thwart Freedom of Information Act requests (across continents no less). They reference deleting data, hiding source code from requests, manipulating data to make it more annoying to use, and attempting to deny requests from people recognized as contributors to specific internet sites. Big brother really is watching you. He’s just not very good at securing his web site.

2. These scientists publicly diminished opposing arguments for lack of being published in peer-reviewed scientific journals. In the background they discussed black-balling journals that did publish opposing views, and preventing opposing views from being published in journals they controlled. They even mention changing the rules midstream in arenas they control to ensure opposing views would not see the light of day. They discuss amongst themselves which scientists can be trusted and who should be excluded from having data because they may not be “predictable”.

3. The scientists expressed concern privately over a lack of increase in global temperatures in the last decade, and the fact that they could not explain this. Publicly they discounted it as simple natural variations. In one instance, data was [apparently] manipulated to hide a decline in temperatures when graphed. Other discussions included ways to discount historic warming trends that inconveniently did not occur during increases in atmospheric CO2.

4. The emails show examples of top scientists working to create public relations messaging with favorable news outlets. It shows them identifying and cataloging, by name and association, people with opposing views. These people are then disparaged in a coordinated fashion via favorable online communities.

What the emails/files don’t do is completely destroy the possibility that global climate change is real. They don’t preclude many studies from being accurate, on either side of the discussion. And they should not be seen as discrediting all science.

Free speech and transparency are meaningless unless they apply to the critics of the government as well as to its supporters.  Here is the heart of the scandal:

Wei-Chyung and Tom,
…
1. Think I’ve managed to persuade UEA to ignore all further FOIA requests if the people have anything to do with Climate Audit.
2. Had an email from David Jones of BMRC, Melbourne. [EMAIL NOT FOUND IN CRU EMAILS – Willis] He said they are ignoring anybody who has dealings with CA, as there are threads on it about Australian sites.
3. CA is in dispute with IPCC (Susan Solomon and Martin Manning) about the availability of the responses to reviewer’s at the various stages of the AR4 drafts. They are most interested here re Ch 6 on paleo.
Cheers
Phil [Phil Jones, head of the University of East Anglia CRU]

Climate Audit is a site dedicated to science.  You will seldom see any political polemic there, even in the comments.  Unlike sites like RealClimate, dissenting voices are not edited out of the comments.  And it is a site that has dedicated itself (thus the title) to backchecking, verifying, and attempting to replicate various climate studies, particularly historic temperature records and paleo-climatology work.

Essentially, Phil Jones is saying that, as a government official, he is going to ignore legal information requests under FOIA from certain groups that seek to hold him accountable — information requests, I might add, that should not have even been necessary in the first place by any reasonable rules of scientific openness.

Update: You really have to read the whole long post above.  It is a story of the CRU trying to stay one step ahead of those who want to make them accountable, as most scientists are to replication efforts.   One approach they took that really requires chutzpah – in response to the constant CRU stonewalling, more and more people began pinging them with FOIA’s.  Remember, this deluge of FOIA’s was only necessary due to the CRU’s constant stonewalling.  Their next tactic – use the volume of FOIA’s as evidence that it is somehow a concerted attack and therefore can be ignored:

I’ve saved all three threads as they now stand. No time to read all the comments, but I did note in “Fortress Met Office” that someone has provided a link to a website that helps you to submit FOI requests to UK public institutions, and subsequently someone has made a further FOI request to Met Office and someone else made one to DEFRA. If it turns into an organised campaign designed more to inconvenience us than to obtain useful information, then we may be able to decline all related requests without spending ages on considering them. Worth looking out for evidence of such an organised campaign

Update #2: I am curious how many other scientific fields support this attitude towards defending and replication one’s work

Dear Tom,
…
One of the problems is that I’m caught in a real Catch-22 situation. At present, I’m damned and publicly vilified because I refused to provide McIntyre with the data he requested. But had I acceded to McIntyre’s initial request for climate model data, I’m convinced (based on the past experiences of Mike Mann, Phil, and Gavin) that I would have spent years of my scientific career dealing with demands for further explanations,
additional data, Fortran code, etc. (Phil has been complying with FOIA requests from McIntyre and his cronies for over two years). And if I ever denied a single request for further information, McIntyre would have rubbed his hands gleefully and written: “You see – he’s guilty as charged!” on his website.

You and I have spent over a decade of our scientific careers on the MSU issue, Tom. During much of that time, we’ve had to do science in “reactive mode”, responding to the latest outrageous claims and inept science by John Christy, David Douglass, or S. Fred Singer. For the remainder of my scientific career, I’d like to dictate my own research agenda. I don’t want that agenda driven by the constant need to respond to Christy, Douglass, and Singer. And I certainly don’t want to spend years of my life interacting
with the likes of Steven McIntyre.

I hope LLNL management will provide me with their full support. If they do not, I’m fully prepared to seek employment elsewhere.

With best regards,
Ben [Ben Santer]

Anthony Watts has a statement from the American Meteorological Society about the Hadley CRU emails.  The basic message is “nothing to see here, move along:”

For climate change research, the body of research in the literature is very large and the dependence on any one set of research results to the comprehensive understanding of the climate system is very, very small. Even if some of the charges of improper behavior in this particular case turn out to be true — which is not yet clearly the case — the impact on the science of climate change would be very limited.

I sent this letter to their executive director:

I really believe your group’s statement is missing the point.  Your group’s position seems to be that this episode can be safely ignored as it taints only a small volume of a large mass of research.  But a large body of research is not useful if that body systematically excludes any work critical of the orthodoxy.  In fact, you should re-reread this paragraph from your own letter:

The beauty of science is that it depends on independent verification and replication as part of the process of confirming research results.  This process, which is tied intrinsically to the procedures leading to publication of research results in the peer-reviewed literature, allows the scientific community to confirm some results while rejecting others.

The Hadley emails describe a large conspiracy to corrupt exactly this process.  We see scientists conspiring to keep secret results and working papers that would have allowed their worked to be checked, verified, and replicated.  We see researchers working to prevent publication of any research that might falsify their work.

Sure, we see a few examples of researchers fudging their work.  But the issue is not necessarily the specific cases of fudging — in fact, I don’t care if its sloppiness or chicanery — from a science standpoint it almost doesn’t matter.  The problem here is the creation of a system and a culture where such fudging and sloppiness can occur without any of the independent scrutiny you seem to laud as part of the scientific process.  How much more exists?  We’ll never know, until organizations like yours stop enabling this climate Omerta and start demanding true scientific openness.

A reader wrote me that the comments found in the Hadley CRU program code are possibly far more damning than the emails, and in fact this appears to be the case given these excerpts at Anthony Watts’ site.

In the past, I have written that as an experienced modeller, I am extremely suspicious when anyone’s models very closely match history.  This is a common modelers trick – use various plugs and fudge factors and special algorithms to force the model to match history better (when it is used to “back-cast”) and people will likely trust the model more when you use it to forecast.   For a variety of reasons, I have been suspicious this was the case with climate models, but never could prove it.  One example from the link above

Looking back over history, it appears the model is never off by more than 0.4C in any month, and never goes more than about 10 months before re-intersecting the “actual” line.  Does it bother anyone else that this level of precision is several times higher than the model has when run forward?  Almost immediately, the model is more than 0.4C off, and goes years without intercepting reality.

Now we are closer, with programming code comments in the various climate programs that say things like this (from the code that apparently does some of the tree ring histories)

. FOIA\documents\osborn-tree6\mann\oldprog\maps12.proFOIA\documents\osborn-tree6\mann\oldprog\maps15.proFOIA\documents\osborn-tree6\mann\oldprog\maps24.pro ; Plots 24 yearly maps of calibrated (PCR-infilled or not) MXD reconstructions
; of growing season temperatures. Uses "corrected" MXD - but shouldn't usually
; plot past 1960 because these will be artificially adjusted to look closer to
; the real temperatures.

or this

• FOIA\documents\osborn-tree6\briffa_sep98_d.pro;mknormal,yyy,timey,refperiod=[1881,1940]
;
; Apply a VERY ARTIFICAL correction for decline!!
;
yrloc=[1400,findgen(19)*5.+1904]
valadj=[0.,0.,0.,0.,0.,-0.1,-0.25,-0.3,0.,-0.1,0.3,0.8,1.2,1.7,2.5,2.6,2.6,$
2.6,2.6,2.6]*0.75 ; fudge factor
(...)
;
; APPLY ARTIFICIAL CORRECTION
;
yearlyadj=interpol(valadj,yrloc,x)
densall=densall+yearlyadj

The link above has 30+ similar examples.  The real insight will be when folks like Steve McIntyre and his readers start digging into the code and replicating it — then we will see what it actually does and what biases or plugs or overrides are embedded.  Stay tuned.

I am disappointed to see folks like Lord Monkton calling for scientists to go to jail over what has been discovered in the Hadley CRU emails.  No one is going to jail, at least based on what we know so far.  Laws were broken, but of the type that perhaps people lose their jobs but not their freedom.  And demanding that people go to jail just paints skeptics as opportunistic, over-the-top and vindictive.   We sound like the looniest of the alarmists when we say stuff like this.

This is not to say that the emails (as well as the source code, which Steve McIntyre and his readers are starting to dig into) don’t give us useful insights about the climate science process.  And what they really point to for me is the danger of a monoculture.

For years, with the media’s active participation, criticism of the mainstream scientific position on global warming has been painted as somehow outside the bounds of reasonable discourse.  Skeptics are called “deniers,” with the intent to equate them with those who deny the Holocaust.  At every turn, global warming activists with the help of the media, have tried to make it uncomfortable, even impossible, to criticize the science of catastrophic man-made global warming.  In the extreme, this has degenerated into outright threats.

NASA’s James Hansen has called for trials of climate skeptics in 2008 for “high crimes against humanity.” Environmentalist Robert F. Kennedy Jr. lashed out at skeptics of 2007 declaring “This is treason. And we need to start treating them as traitors” In 2009, RFK, Jr. also called coal companies “criminal enterprises” and declared CEO’s ‘should be in jail… for all of eternity.”

In June 2009, former Clinton Administration official Joe Romm defended a comment on his Climate Progress website warning skeptics would be strangled in their beds. “An entire generation will soon be ready to strangle you and your kind while you sleep in your beds,” stated the remarks, which Romm defended by calling them “not a threat, but a prediction.”

In 2006, the eco-magazine Grist called for Nuremberg-Style trials for skeptics. In 2008, Canadian environmentalist David Suzuki called for government leaders skeptical of global warming to be thrown “into jail.” In 2007, The Weather Channel’s climate expert called for withholding certification of skeptical meteorologists.

The examples go on ad infinitum.  Several folks have emailed me and asked why I have not joined the feeding frenzy over the “climategate.”  In part, this is because I don’t think there is anything in the emails that is a whole lot worse than what many of the actors have been saying publicly.  The media has played along not only because many of its members were sympathetic to the message, but because the catastrophe played well into the “if it bleeds, it leads” culture.  Even when the media was not “picking a winner” in the science, it supported the catastrophist message in its editorial decisions, choosing to cover (for example) ad nauseum a 30-year low in Arctic sea ice but failing to even mention a 30-year high in Antarctic sea ice which occurred on nearly the same day (more here).  Ditto hurricanes, tornadoes, floods droughts, etc — only events and records in one particular tail of the normal distribution were covered.  Even when they worked to be fair,  the media were frequently criticized by alarmists for  allowing even a mention of the skeptic position in an article otherwise generally supporting the orthodoxy.  The term “false balance” was coined.

The result was a group who were effectively exempt from criticism — and knew it.

The most amazing thing to watch has been the absolute scorn and obstructionism piled on Steve McIntyre and his readers and partners.  I  have read Steve’s work for years, and find it to be incredibly fair and deeply analytical.  I took as one of my early roles at my climate site the explanation to laymen of exactly what McIntyre was talking about in his posts.  He often challenged the climate orthodoxy – which in most scientific disciplines is highly valued, but in climate science is a crime.  In the emails we even see scientists within the monoculture raising the exact same issues that they have blasted McIntyre for — apparently it is OK to raise such issues as long as 1) you are an insider and 2) such concerns are suppressed in any public document.

Perhaps the single most abusive part of the monoculture has been its misrepresentation of peer review.  Peer review was never meant as a sort of good housekeeping seal of approval on scientific work.  It is not a guarantee of correctness.  It is really an extension of the editorial process — bringing scientists from relevant fields to vet whether work is really new and different and worthy of publication, to make sure the actual article communicates the work and its findings clearly, and to probe for obvious errors or logical fallacies.

Climate scientists have tried to portray peer review as the end of the process–  ie, once one of their works shows up in a peer-reviewed journal, the question addressed is “settled.”  But his is never how science has worked.  Publication in a peer-reviewed journal is the beginning, not the end.  Once published, scientists attempt alternatively to tear it down or replicate its conclusions.  Only work that has survived years of such torture testing starts to become “settled.”

The emails help to shed light on some aspects of peer review that skeptics have suspected for years.  It is increasingly clear that climate scientists in the monoculture have been using peer review to enforce the orthodoxy.  Peer review panels are stacked with members of the club, and authors who challenge the orthodoxy are shut out of publication, while authors within the monoculture use peer review as a shield against future criticism.  We see in the emails members of the monoculture actually working to force editors who have the temerity to publish work critical of the orthodoxy out of their jobs.  We are now learning that when alarmist scientists claim that there is little peer-reviewed science on the skeptic’s side, this is like the Catholic Church enforcing a banned books list and then claiming that everything in print supports the Church’s position.

History teaches us that whenever we allow a monoculture – whether is be totalitarian one-party rule or enforcing a single state religion, corruption follows.  Without scrutiny of their actions, actors in such monocultures have few checks and little accountability.  Worse, those at the center of such monocultures can become convinced of their own righteousness, such that any action they take in support of the orthodoxy is by definition ethically justified.

This, I think, is exactly what we see at work in the Hadley CRU emails.

Except when its not:

I tried hard to balance the needs of the science and the IPCC , which were not always the same

–Keith Briffa, who was in charge of the IPCC section that includes temperature reconstructions such as the Hockey Stick.

Despite loads of public scorn heaped on Steve McIntyre and Ross McKittrick for their criticisms of the Mann hockey stick, it turns out in private folks like the Hadley Center’s John Mitchell, the review editor for the relevant chapter of the last IPCC report, shared many concerns identical to those of M&M. The email at the link is pretty amazing – it is practically an outline of the section of my skeptic presentation dealing with the hockey stick.  But not a whiff of this uncertainty was ever made public or was included in the IPCC report.

Mitchell and the Hadley Center have tried every trick in the book to avoid FOIA of anything that would publicly reveal his true concerns about Mann’s study.  When we understand the incentives that are driving him to suppress his own scientific views, and to publicly ridicule those who share his private concerns, we will understand better what is broken in the climate science process.