The gist of my comments were that the trend in the data didn’t make any sense, and upon checking with the data owner, it turns out much of the trend is due to changes in the data collection process.  I stick by that conclusion, though not some of the other suppositions in those posts.

I was excited to see Dr. Mills response (thanks to reader Charlie Allen for the heads up).  I will quote much of it, but to make sure I can’t be accused of cherry-picking, here is his whole post here.  I would comment there, but alas, unlike this site, Dr. Mills chooses not to allow comments.

So here we go:

Two blog entries [1-online | PDF] [2-online | PDF] [Accessed June 18, 2009] mischaracterize analysis in a new report entitled Global Climate Change Impacts in the United States. The blogger (a self-admitted “amateur”) created a straw man argument by asserting that the chart was presented as evidence of global climate change and was not verified with the primary source. The blog’s errors have been propagated to other web sites without further fact checking or due diligence. (The use of profanity in the title of the first entry is additionally unprofessional.)

Uh, oh, the dreaded “amateur.”  Mea Culpa.  I am a trained physicist and engineer.  I don’t remember many colleges handing out “climate” degrees in 1984, so I try not to overstate my knowledge.  As to using “bullsh*t” in the title, the initial post was “I am calling bullsh*t on this chart.”  Sorry, I don’t feel bad about that given the original post was a response to a post on a political blog.

The underlying database—created by the U.S. Department of Energy’s Energy Information Administration—contains approximately 930 grid-disruption events taking place between 1992 and 2008, affecting 135 million electric customers.

As noted in the caption to the figure on page 58 of our report (shown above)—which was masked in the blogger’s critique—

First, I am happy to admit errors where I make them (I wonder if that is why I am still an “amateur”).   It was wrong of me to post the chart without the caption. My only defense was that I copied the chart from, and was responding to its use on, Kevin Drum’s site and he too omitted the caption. I really was not trying to hide what was there.   I am on the road and don’t have the original but here it is from Dr. Mills’ post.

Anyway, to continue…

As noted in the caption to the figure on page 58 of our report (shown above)—which was masked in the blogger’s critique—we expressly state a quite different finding than that imputed by the blogger, noting with care that we do not attribute these events to anthropogenic climate change, but do consider the grid vulnerable to extreme weather today and increasingly so as climate change progresses, i.e.:

“Although the figure does not demonstrate a cause-effect relationship between climate change and grid disruption, it does suggest that weather and climate extremes often have important effects on grid disruptions.”

The associated text in the report states the following, citing a major peer-reviewed federal study on the energy sector’s vulnerability to climate change:

“The electricity grid is also vulnerable to climate change effects, from temperature changes to severe weather events.”

To Dr. Mills’ point that I misinterpreted him — if all he wanted to say was that the electrical grid could be disturbed by weather or was vulnerable to climate change, fine.  I mean, duh.  If there are more tornadoes knocking about, more electrical lines will come down.  But if that was Dr. Mills ONLY point, then why did he write (emphasis added):

The number of incidents caused by extreme weather has increased tenfold since 1992.  The portion of all events that are caused by weather-related phenomena has more than tripled from about 20 percent in the early 1990s to about 65 percent in recent years.  The weather-related events are more severe…

He is saying flat out that the grid IS being disturbed 10x more often and more severely by weather.  It doesn’t even say “reported” incidents or “may have” — it is quite definitive.  So which one of us is trying to create a straw man?   It is these statements that I previously claimed the data did not support, and I stand by my analysis on that.

And it’s not like there is some conspiracy of skeptics to mis-interpret Mr. Mills.  Kevin Drum, a  huge cheerleader for catastrophic AGW, said about this chart:

So here’s your chart of the day: a 15-year history of electrical grid problems caused by increasingly extreme weather.

I will skip the next bit, wherein it appears that Dr. Mills is agreeing with my point that aging and increased capacity utilization on the grid could potentially increase weather-related grid outages without any actual change in the weather  (just from the grid being more sensitive or vulnerable)

OK, so next is where Mr. Mills weighs in on the key issue of the data set being a poor proxy, given the fact that most of the increase in the chart are due to better reporting rather than changes in the underlying phenomenon:

The potential for sampling bias was in fact identified early-on within the author team and—contrary to the blogger’s accusation—contact was in fact made with the person responsible for the data collection project at the US Energy Information Administration on June 10, 2008 (and with the same individual the blogger claims to have spoken to). At that time the material was discussed for an hour with the EIA official, who affirmed the relative growth was in weather-related events and that it could not be construed as an artifact of data collection changes, etc. That, and other points in this response, were re-affirmed through a follow up discussion in June 2009.

In fact, the analysis understates the scale of weather-related events in at least three ways:

• EIA noted that there are probably a higher proportion of weather events missing from their time series than non-weather ones (due to minimum threshold impacts required for inclusion, and under-reporting in thunderstorm-prone regions of the heartland).
• There was at least one change in EIA’s methodology that would have over-stated the growth in non-weather events, i.e., they added cyber attacks and islanding in 2001, which are both “non-weather-related”.
• Many of the events are described in ways that could be weather-related (e.g. “transmission interruption”) but not enough information is provided. We code such events as non-weather-related.

Dr. Mills does not like me using the “BS” word, so I will just say this is the purest caca. I want a single disinterested scientist to defend what Dr. Mills is saying. Remember:

• Prior to 1998, just about all the data is missing. There were pushes in 2001 and 2008 to try to fix under reporting.  Far from denying this, Dr. Mills reports the same facts.  So no matter how much dancing he does, much of the trend here is driven by increased reporting, not the underlying phenomenon.  Again, the underlying phenomenon may exist, but it certainly is not a 10x increase as reported in the caption.
• The fact that a higher proportion of the missing data is weather-related just underlines the point that the historic weather-related outage data is a nearly meaningless source of trend data for weather-related outages.
• His bullet points are written as if the totals matter, but the point of the chart was never totals.  I never said he was overstating weather related outages today.   The numbers in 2008 may still be (and probably are) understated.  And I have no idea even if 50 or 80 is high or low,  so absolute values have no meaning to me anyway.  The chart was designed to portray a trend — remember that first line of the caption “The number of incidents caused by extreme weather has increased tenfold since 1992. ” — not a point on absolute values.   What matters is therefore not how much is missing, but how much is missing in the early years as compared to the later years.
• In my original post I wrote, as Dr. Mills does, that the EIA data owner thinks there is a weather trend in the data if you really had quality data.  Fine.  But it is way, way less of a trend than shown in this chart.  And besides, when did the standards of “peer reviewed science” stoop to include estimates of government data analysts as to what the trend in the data would be if the data weren’t corrupted so badly.   (Also, the data analyst was only familiar with the data back to 1998 — the chart started in 1992.
• Dr. Mills was aware that the data had huge gaps before publication.  Where was the disclosure?  I didn’t see any disclosure.  I wonder if there was such disclosure in the peer reviewed study that used this data (my understanding is that there must have been one, because the rules of this report is that everything had to come from peer-reviewed sources).
• I don’t think any reasonable person could use this data set in a serious study knowing what the authors knew.  But reasonable people can disagree, though I will say that I think there is no ethical way anyone could have talked to the EIA in detail about this data and then used the 1992-1997 data.

Onward:

Thanks to the efforts of EIA, after they took over the responsibility of running the Department of Energy (DOE) data-collection process around 1997, it became more effective. Efforts were made in subsequent years to increase the response rate and upgrade the reporting form.

Thanks, you just proved my point about the trend being driven by changes in reporting and data collection intensity.

To adjust for potential response-rate biases, we have separated weather- and non-weather-related trends into indices and found an upward trend only in the weather-related time series.

As confirmed by EIA, if there were a systematic bias one would expect it to be reflected in both data series (especially since any given reporting site would report both types of events).

As an additional precaution, we focused on trends in the number of events (rather than customers affected) to avoid fortuitous differences caused by the population density where events occur. This, however, has the effect of understating the weather impacts because of EIA definitions (see survey methodology notes below).

Well, its possible this is true, though unhappily, this analysis was not published in the original report and was not published in this post.   I presume this means he has a non-weather time series that is flat for this period.  Love to see it, but this is not how the EIA portrayed the data to me.  But it really doesn’t matter – I think the fact that there is more data missing in the early years than the later years is indisputable, and this one fact drives a false trend.

But here is what I think is really funny- — the above analysis does not matter, because he is assuming a reporting bias symmetry, but just  a few paragraphs earlier he stated that there was actually an asymmetry.  Let me quote him again:

EIA noted that there are probably a higher proportion of weather events missing from their time series than non-weather ones (due to minimum threshold impacts required for inclusion, and under-reporting in thunderstorm-prone regions of the heartland).

Look Dr. Mills, I don’t have an axe to grind here.  This is one chart out of bazillions making a minor point.  But the data set you are using is garbage, so why do you stand by it with such tenacity?  Can’t anyone just admit “you know, on thinking about it, there are way to many problems with this data set to declare a trend exists.  Hopefully the EIA has it cleaned up now and we can watch it going forward.”  But I guess only “amateurs” make that kind of statement.

The blogger also speculated that many of the “extreme temperature” events were during cold periods, stating “if this is proof of global warming, why is the damage from cold and ice increasing as fast as other severe weather causes?” The statement is erroneous.

This was pure supposition in my first reaction to the chart.  I later admitted that I was wrong.  Most of the “temperature” effects are higher temperature.  But I will admit it again here – that supposition was incorrect.  He has a nice monthly distribution of the data to prove his point.

I am ready to leave this behind, though I will admit that Dr. Mills response leaves me more rather than less worried about the quality of the science here.  But to summarize, everything is minor compared to this point:  The caption says “The number of incidents caused by extreme weather has increased tenfold since 1992.”  I don’t think anyone, knowing about the huge underreporting in early years, and better reporting in later years, thinks that statement is correct.  Dr. Mills should be willing to admit it was incorrect.

Update: In case I am not explaining the issue well, here is a conceptual drawing of what is going on:

Update #2: One other thing I meant to post.  I want to thank Dr. Mills — this is the first time in quite a while I have received a critique of one of my posts without a single ad hominem attack, question about my source of funding, hypothesized links to evil forces, etc.  Also I am sorry I wrote “Mr.” rather than “Dr.” Mills.  Correction has been made.

The first thing I would observe is that the decline seems exaggerated through some scaling and smoothing gains.    The raw data, from the Cyrosphere Today site   (note different units, a square mile = about 2.6 sq. km).

But the most interesting part is what is not mentioned, even once, in this section of the report:  The Earth has two poles.  And it turns out that the south pole has actually been gaining sea ice, such that the total combined sea ice extent of the entire globe is fairly stable (click for larger version).

Now, there are folks who are willing to posit a model that allows for global warming and this kind of divergence between the poles.  But the report does not even go there.  It demonstrates an inferiority complex I see in many places of the report, refusing to even hint that reality is messy in fear that it might cloud their story.

Steve McIntyre picks up my critique on the electrical grid disruption chart (here and here) and takes it further.  Apparently, this report (which I guess I should be calling the Climate Change Synthesis Report or CCSP) set rules for itself that all the work in the report had to be from peer-reviewed literature.  McIntyre makes a grab at the footnotes for this section of the report for any peer-reviewed basis, but comes up only with air.   He also references a hurricane chart in the report apparently compiled by the same person who compiled the grid outage report.  Roger Pielke rips up this hurricane report, and I have it on my list to address in a future post as well.

Increased extremes of summer dryness and winter wetness are projected for much of the globe, meaning a generally greater risk of droughts and floods. This has already been observed, and is projected to continue. In a warmer world, precipitation tends to be concentrated into heavier events, with longer dry periods in between.

Later in the report they make the same claims for the US only.  I can’t speak for the rest of the world, but I don’t know what data they are using.  This is from the National Climate Data Center, run by the same folks who wrote this report:

Maybe my Mark I eyeball is off, but it sure doesn’t look like any trend here, or that there we are currently at any particularly unprecedented levels today.  Of course, the main evidence they have of increasing extreme rainfall is in this chart — but of course this is “simulated” history, rather than actual, you know, observations.

A quick glance at this chart, and what do we see?  A line historically rising surprisingly in parallel with global temperature history, and then increasing in the future.

But let’s look at that chart legend carefully.  The green “historic” data is actually nothing of the sort – it is simulation!  The authors have created their own history.  This entire chart is the output of some computer model programmed to deliver the result that temperature drives heavy precipitation, and so it does.

But folks found it really hard to question Aristotle.  The world had gone all-in on Aristotle.  Even the Church had adopted Aristotle’s description of the universe as the one true and correct model.  So folks assumed the observations were wrong, or spent their time shoe-horning the observations into Aristotle’s theories, or just ignored the offending observations altogether.  The Enlightenment is a complex phenomenon, but for me the key first step was the willingness of people to start questioning traditional authorities (Aristotle and the church) in the light of new observations.

I am reminded of this story a bit when I read about “fingerprint” analyses for anthropogenic warming.  These analyses propose to identify certain events in current climate (or weather) that are somehow distinctive features of anthropogenic rather than natural warming.  From the GCCI:

The earliest fingerprint work focused on changes in surface and atmospheric temperature. Scientists then applied fingerprint methods to a whole range of climate variables, identifying human-caused climate signals in the heat content of the oceans, the height of the tropopause (the boundary between the troposphere and stratosphere, which has shifted upward by hundreds of feet in recent decades), the geographical patterns of precipitation, drought, surface pressure, and the runoff from major river basins.

Studies published after the appearance of the IPCC Fourth Assessment Report in 2007 have also found human fingerprints in the increased levels of atmospheric moisture (both close to the surface and over the full extent of the atmosphere), in the decline of Arctic sea ice extent, and in the patterns of changes in Arctic and Antarctic surface temperatures.

This is absolute caca.  Given the complexity of the climate system, it is outright hubris to say that things like the “geographical patterns of precipitation” can be linked to half-degree changes  in world average temperatures.  But it is a lie to say that it can be linked specifically to human-caused warming, vs. warming from other causes, as implied in this statement.   A better name for fingerprint analysis would be Rorschach analysis, because they tend to result in alarmist scientists reading their expectation to find anthropogenic causes into every single weather event.

But there is one fingerprint prediction that was among the first to be made and is still probably the most robust of this genre:  that warming from greenhouse gasses will be greatest in the upper troposphere above the tropics.  This is demonstrated by this graph on page 21 of the GCCI

This has always been a stumbling block, because satellites, the best measures we have on the troposphere, and weather balloons have never seen this heat bubble over the tropics.  Here is the UAH data for the mid-troposphere temperature — one can see absolutely no warming in a zone where the warming should, by the models, be high:

Angell in 2005 and Sterin in 2001 similarly found from Radiosonde records about 0.2C of warming since the early 1960s, below the global surface average warming when models say it should be well above.

But fortunately, the GCCI solves this conundrum:

For over a decade, one aspect of the climate change story seemed to show a significant difference between models and observations.

In the tropics, all models predicted that with a rise in greenhouse gases, the troposphere would be expected to warm more rapidly than the surface. Observations from weather balloons, satellites, and surface thermometers seemed to show the opposite behavior (more rapid warming of the surface than the troposphere). This issue was a stumbling block in our understanding of the causes of climate change. It is now largely resolved.   Research showed that there were large uncertainties in the satellite and weather balloon data. When uncertainties in models and observations are properly accounted for, newer observational data sets (with better treatment of known problems) are in agreement with climate model results.

What does this mean?  It means that if we throw in some correction factors that make observations match the theory, then the observations will match the theory.  This statement is a pure out and out wishful thinking.  The charts above predict a 2+ degree F warming in the troposphere from 1958-1999, or nearly 0.3C per decade.  No study has measured anything close to this  – Satellites show 0.0C per decade and radiosondes about 0.05C per decade.    The correction factors to make reality match the theory would have to be 10 times the measured anomaly.  Even if this were the case, the implied signal to noise ratio would be so low as to render the analysis meaningless.

Frankly, the statement by these folks that weather balloon data and satellites have large uncertainties is hilarious.  While this is probably true, these uncertainties and inherent biases are DWARFED by the biases, issues, uncertainties and outright errors in the surface temperature record.  Of course, the report uses this surface temperature record absolutely uncritically, ignoring a myriad of problems such as these and these.  Why the difference?  Because observations from the flawed surface temperature record better fit their theories and models.  Sometimes I think these guys should have put a picture of Ptolemy on their cover.

Page 13 of the GCCI report has this chart I have already seen circulating around the alarmist side of the web:

There are two problems here.

One, the compression of the X axis puts the lower and upper scenario lines right on top of each other.  This really causes the higher scenario  (which, at 900ppm, really represents a number higher than we are likely to see even in a do-nothing case) to visually dominate.

The other issue is that the Y-axis covers a very, very small range, such that small changes are magnified visually.  The scale runs from 0% of the atmosphere up to 0.09% of the atmosphere.  If one were to run the scale to cover a more reasonable range, he would get this  (with orange being the high emissions case and blue being the lower case):

Even this caps out at just 1% of the atmosphere.  If we were to look at the total composition of the atmosphere, we would get this:

Remember that the report leaps out of the starting gate by making the amazingly unequivocal statement:

1. Global warming is unequivocal and primarily human-induced. Global temperature has increased over the past 50 years. This observed increase is due primarily to human induced emissions of heat-trapping gases.

To make this statement, they must dispose of other possible causes, with variations in the sun being the most obvious.  Here is the chart they use on page 20:

Wow, this one is even shorter than the glacier chart.  I suppose they can argue that it is necessarily so, as they only have satellite data since 1978.  But there are other sources of data prior to 1978 they could have used**.

I will show the longer view of solar activity in a minute, but let’s take a minute to think about the report’s logic.  The chart tries to say that the lack of a trend in the rate of solar energy reaching Earth is not consistent with rising temperatures.  They are saying – See everyone, flat solar output, rising temperatures.  There can’t be a relationship.

Really?  Did any of these guys take basic thermodynamics?  Let’s consider a simple example from everyone’s home — a pot on a stove.  The stove is on low, and the water has reached an equilibrium temperature, well below boiling.  Now we turn the stove up — what happens?

In this chart, the red is the stove setting, and we see it go from low to high.  Prior to the change in stove setting, the water temperature in the pot, shown in blue, was stable.  After the change in burner setting, the water temperature begins to increase over time.

If we were to truncate this chart, so we only saw the far right side, as the climate report has done with the sun chart, we would see this:

Doesn’t this look just a little like the solar chart in the report?  The fact is that the chart from the report is entirely consistent both with a model where the sun is causing most of the warming and one where it is not.  The key is whether the level of the sun’s output from 1987 to present is a new, higher plateau that is driving temperature increases over time (like the higher burner setting) or whether the sun’s output recently is consistent with, and no higher than, its level over the last 100 years.  What we want to look for, in seeking the impact of the sun, is a step-change in output near when temperature increases of the last 50 years began.

Does such a step-change exist?  Yes.  One way to look at the sun’s output is to use sunspots as a proxy for output – the more spots in a given 11 year cycle, the greater the sun’s activity and likely output.  Here is what we see for this metric:

And here is the chart for total solar irradiance (sent to me, ironically, by someone trying to disprove the influence of the sun).

Clearly the sun’s activity and output experienced an upwards step-change around 1950.  The average monthly sunspots in the second half of the century were, for example, 50% higher than in the first half of the century.

The real question, of course, is whether these changes result in large or small rates of temperature increase.  And that is still open for debate, with issues like cloud formation thrown in for complexity.  But it is totally disingenuous, and counts on readers to be scientifically illiterate, to propose that the chart in the report “proves” that the sun is not driving temperature changes.

**By this logic, they should only have temperature data since 1978 for the same reason, though by one of those ironies I am starting to find frequent in this report, all the charts, including this one, use flawed surface temperature records rather than satellite data.  Why didn’t they use satellite data for the temperature as well as the solar output for this chart?  Probably because the satellite data does not include upward biases and thus shows less warming.  Having four or five major temperature indices to choose from, the team writing this paper chose the one that gives the highest modern warming number.

Here is the melting glacier chart from right up front on page 18, in the section on sea level rise (ironic, since if you really read the IPCC report closely, sea level rise comes mainly from thermal expansion of the oceans – glacier melting is offset in most models by increased snow in Antarctica**).

Wow, this looks scary.  Of course, it is clever chartsmanship, making it look like they have melted down to zero by choice of scale.   How large is this compared to the total area of glaciers?  We don’t know from the chart — percentages would have been more helpful.

Anyway, I could criticize these minor chartsmanship games throughout the paper, but on glaciers I want to focus on the selected time frame.  What, one might ask, were glaciers doing before 1960?  Well, if we accept the logic of the caption that losses are driven by temperature, then I guess it must have been flat.  But why didn’t they show that?  Wouldn’t that be a powerful chart, showing flat glacier size with this falloff to the right?

Well, as you may have guessed, the truncated time frame on the left side of this chart is not flat.  I can’t find evidence that Meier et al looted back further than 1960, but others have, including Oerlemans as published in Science in 2005.  (The far right hand side really should be truncated by 5-10 years, as they are missing a lot of datapoints in the last 5 years, making the results odd and unreliable).

OK, this is length rather than volume, but they should be closely related.  The conclusion is that glaciers have been receding since the early 19th century, LONG before any build-up of CO2, and coincident with a series of cold decades in the last 18th century  (think Valley Forge and Napoleon in Russia).

I hope you can see why it is unbelievably disingenuous to truncate the whole period from 1800-1960 and call this trend a) recent and b) due to man-made global warming.  If it is indeed due to man-made global warming since 1960, then there must have been some other natural effect shrinking glaciers since 1825 that fortuitously shut off at the precise moment anthropogenic warming took over.  Paging William of Occam, call your office please.

Similarly, sea levels have been rising steadily for hundreds, even thousands of years, and current sea level increases are not far off their average pace for the last 200 years.

** The climate models show warming of the waters around Antarctica, creating more precipitation over the climate.  This precipitation falls and remains as snow or ice, and is unlikely to melt even at very high numbers for global warming as Antarctica is so freaking cold to begin with.

Yesterday I called into question the interpretation of this chart from the GCCI report where the report used electrical grid outages as a proxy for severe weather frequency:

I hypothesized:

This chart screams one thing at me:  Basis change.  Somehow, the basis for the data is changing in the period.  Either reporting has been increased, or definitions have changed, or there is something about the grid that makes it more sensitive to weather, or whatever  (this is a problem in tornado charts, as improving detection technologies seem to create an upward incidence trend in smaller tornadoes where one probably does not exist).   But there is NO WAY the weather is changing this fast, and readers should treat this whole report as a pile of garbage if it is written by people who uncritically accept this chart.

I had contacted John Makins of the EIA who owns this data set yesterday, but I was too late to catch him in the office.  He was nice enough to call me today.

He said that there may be an underlying upward trend out there (particularly in thunderstorms) but that most of the increase in this chart is from improvements in data gathering.  In 1997, the EIA (and Makins himself) took over the compilation of this data, which had previously been haphazard, and made a big push to get all utilities to report as required.  They made a second change and push for reporting in 2001, and again in 2007/2008.  He told me that most of this slope is due to better reporting, and not necessarily any underlying trend.   In fact, he said there still is some under-reporting by smaller utilities he wants to improve so that the graph will likely go higher in the future.

Further, it is important to understand the nature of this data.  The vast majority of weather disturbances are not reported to the EIA.  If the disturbance or outage remains local with no impact on any of the national grids, then it does not need to be reported.  Because of this definitional issue, reported incidents can also change over time due to the nature of the national grid.  For example, as usage of the national grid changes or gets closer to capacity, local disturbances might cascade to national issues where they would not have done so 20 years ago.  Or vice versa – better grid management technologies might keep problems local that would have cascaded regionally or nationally before.  Either of these would drive trends in this data that have no relation to underlying weather patterns.

At the end of the day, this disturbance data is not a good proxy for severe weather.  And I am left wondering at this whole “peer-reviewed science” thing, where errors like this pass into publication of major reports — an error that an amateur like myself can identify with one phone call to the guy listed by this data set on the web site.  Forget peer review, this isn’t even good basic editorial control  (apparently no one who compiled the report called Makins, and he was surprised today at the number of calls he was suddenly getting).

Postscript: Makins was kind enough to suggest some other data bases that might show what he believes to be a real increase in thunderstorm disturbances of electrical distribution grids.  He suggested that a number of state PUC’s keep such data, including the California PUC under their reliability section.  I will check those out, though it is hard to infer global climate trends from one state.