Although the IPCC calendar webpage doesn’t link to session documents of the 34th session (Kampala Nov 2011), David Holland has alertly located the documents – see here.

IPCC rejected a proposal for anonymous peer review – see document here (page 12 on).

We haven’t discussed this topic previously (in an IPCC context). On reflection, the adoption of a form of anonymous peer review by IPCC seems to me to be a very good idea and might somewhat mitigate some problems. There is no doubt in my mind that review responses are strongly conditioned by who is making the suggestion. The experiences of Ross and myself are vivid examples.

In the IPCC’s Special report on Carbon Dioxide Capture and Storage of WG III of the Fourth Assessment Report, reviews were anonymous. The review document described the results of anonymous review as being entirely positive:

The TSU prepared a list of Review comments with numbers. The TSU had a conversion table linking the numbers to the Reviewer’s names. During the treatment at the lead author meetings and the formulation of the authors’ responses the authors only saw the numbers. The authors knew that in case they would need to consult an Expert Reviewer for getting some clarifications about his comments, the anonymity could be lifted and the coordinates of the Expert Reviewer would have been made available to the authors. In practice it turned out they did not need to use this provision. The anonymity was continued until finalization of the final draft report.

The Reviewers and authors have been informed beforehand about this procedure. The number of comments was normal compared to other special reports. No Reviewer used improper or inappropriate language. The WG III co chairs and TSU held an enquiry among the authors and Review Editors. They considered the anonymity an improvement, because it made them concentrate fully on the content of the matter, disregarding the persons and their background, which was more time efficient.

The review document summarized the advantages of anonymous review as follows:

• Authors will concentrate on the content of the matter, excluding (subconscious) biases.
• There is positive experience in WG III AR4 – also the Task Force on the Greenhouse Gas Inventory Program (TFI) has practiced anonymous reviews with a positive judgment of the authors and Review Editors.
• Authors cannot be criticized anymore of ignoring comments of specific individuals or representatives of scientific schools or interest groups, as happened in the past.
• It remains possible for authors to contact Expert Reviewers if there is a need for clarification.

These seem pretty convincing arguments. Here are the arguments against:

• The Task Group on Procedures was installed in order to consider the recommendations of the InterAcademy Council (IAC). The IAC did not recommend anonymous Reviews, so there is no compelling reason to address this.
• There is a risk that Exper Reviewers could take advantage of their anonymity by burdening authors with unprofessional or inappropriate comments.
• Measures against biases are already sufficiently taken by having Review Editors and by having a authors working as a group.
• Transparency is crucial to the IPCC process. There would be an imbalance in transparency when authors are known by name and Reviewers are not.
• IPCC needs a consistent approach with regard to its Expert Reviews. Changing the approach to require anonymous Review comments would imply that there is a problem with the named Reviewer approach, which is not the case.
• Named Expert Review is more efficient as it allows writing teams to liaise with Reviewers when there is a need for clarification.

In my opinion, none of these reasons stands up.

The Task Group on Procedures was installed in order to consider the recommendations of the InterAcademy Council (IAC). The IAC did not recommend anonymous Reviews, so there is no compelling reason to address this.

The first argument starkly shows the hypocrisy and opportunism of IPCC. IAC did not recommend (or even consider) Jones-Stocker enhanced confidentiality. This was not not mentioned in the briefing documents for the Jones-Stocker amendment. Worse, they represented the changes as addressing issues raised by the IAC.

There is a risk that Exper Reviewers could take advantage of their anonymity by burdening authors with unprofessional or inappropriate comments.

The IPCC’s own experience with the Carbon Dioxide Task Group was reported to be the opposite. To the extent that accountability was an issue, the reviewer names could be removed from the version given to authors for comment, but restored in the final publication of review comments, thereby ensuring accountability.

Measures against biases are already sufficiently taken by having Review Editors and by having authors working as a group.

Neither of these measures had the slightest deterrent to AR4 author responses. In practice, authors seem to have divided up responsibilities in their chapter and to have been busy handling their own sections without worrying too much about how, for example, Briffa handled review comments in his section.

Transparency is crucial to the IPCC process. There would be an imbalance in transparency when authors are known by name and Reviewers are not.

I agree that transparency is “crucial” to the IPCC process. As discussed elsewhere, IPCC has opposed transparency in favour of confidentiality, with the situation getting worse with the furtive adoption of the Jones-Stocker amendment. In addition, it would be easy enough to add back the reviewer name when the review comments were published. The present system is designed not for transparency, but to enable authors to decide how to respond, depending on who the reviewer was.

IPCC needs a consistent approach with regard to its Expert Reviews. Changing the approach to require anonymous Review comments would imply that there is a problem with the named Reviewer approach, which is not the case.

This is perhaps the stupidest argument – even by IPCC standards. Once again, the pretence of infallibility. There are problems with the named reviewer approach. I can understand an argument that, after considering a balance of problems, an institution might choose one method rather than another. But worrying about the impact on infallibility is not a valid reason.

Named Expert Review is more efficient as it allows writing teams to liaise with Reviewers when there is a need for clarification.

Again, this is a fatuous argument. Their own experience with the Carbon Dioxide Capture Task Group permitted authors to locate reviewers for follow-up if necessary. In addition, there is little evidence from the Climategate emails that AR4 reviewers bothered to do this. Briffa, for example, didn’t try to clarify things with me or Ross.

Phil Jones’ written answers to the Muir Russell panel shed interesting light on the insularity of IPCC authors, who see nothing odd about a system in which reviewers do not see either author responses to their review comments or the comments of other reviewers until long after the release of the final document. Jones’ comments were made in connection with questions from Boulton about Jones’ threat to keep McKitrick and Michaels 2004 out of IPCC and McKtrick’s allegation that Jones and other chapter 2 authors, having grudgingly agreed to refer to McKtrick and Michaels 2004, had “fabricated” IPCC’s editorial comment that its results had no statistical significance. This topic is revisited in AR5 (First Draft) where IPCC (with surprising candour) admitted that there was no “explicit” basis for the disparaging claim in AR4.

Jones was interviewed by the Muir Russell panel (actually just Boulton and Peter Clarke) on April 9, 2010.

On April 15, Boulton sent follow-up questions to Jones about his handling of McKitrick and Michaels 2004, an issue that had attracted notoriety as a result of IPCC Coordinating Lead Author Jones’ email saying:

The other paper by MM [McKitrick and Michaels] is just garbage. [...] I can’t see either of these papers being in the next IPCC report. Kevin [Trenberth] and I will keep them out somehow — even if we have to redefine what the peer-review literature is!

Despite Muir Russell’s promises at his opening press conference and despite requests
from the Parliamentary Science and Technology Committee, for the most part, the Muir Russell panel failed to create a useful record of questions and answers. Boulton’s questions about McKitrick and Michaels 2004 are a rare exception.

In apparent compliance with Jones’ 2004 threat, the chapter of which Jones and Trenberth were Coordinating Lead Authors contained no mention of McKitrick and Michaels 2004 in the two drafts sent to reviewers (First Order Draft and Second Order Draft). In the final AR4 report, IPCC grudgingly mentioned McKitrick and Michaels 2004 (and the related de Laat and Maurellis 2006), adding the adverse editorial assertion that they ceased to have “statistical significance” when atmospheric circulation was considered – a comment which, according to McKitrick’s strongly worded submission to Muir Russell, was both untrue and which had no support in the peer reviewed literature that IPCC was supposed to draw on.

Boulton’s follow-up questions on April 15, 2010 asked Jones about the basis for the IPCC claim that the McKitrick and Michaels 2004 had no “statistical significance” as follows:

When the final IPCC FAR was published in May 2007, it included a new paragraph in Chapter 3, on page 244, that referred to the McKitrick and Michaels (2004) and De Laat and Maurellis (2006) papers, and that had not been included in either of the drafts shown to reviewers. It is assumed that this was either written by you, or in consultation with Trenberth, but in any case, the two of you, as Coordinating Lead Authors, bear responsibility for its inclusion. It reads:

“McKitrick and Michaels (2004) and De Laat and Maurellis (2006) attempted to demonstrate that geographical patterns of warming trends over land are strongly correlated with geographical patterns of industrial and socioeconomic development, implying that urbanisation and related land surface changes have caused much of the observed warming. However, the locations of greatest socioeconomic development are also those that have been most warmed by atmospheric circulation changes (Sections 3.2.2.7 and 3.6.4), which exhibit largescale coherence. Hence, the correlation of warming with industrial and socioeconomic development ceases to be statistically significant (highlighting added). In addition, observed warming has been, and transient greenhouse-induced warming is expected to be, greater over land than over the oceans (Chapter 10), owing to the smaller thermal capacity of the land”.

Q1: What is the justification for what appears as an ad hoc conclusion not based on published research that summarily dismisses an argument that is based on peer-reviewed research?

Q2: Why were these conclusions not shown to or discussed with expert reviewers during the IPCC Report preparation?

Q3: The references to sections 3.2.2.7 and 3.6.4 of the IPCC Report are misleading since neither section presents evidence that warming due to atmospheric
circulation changes occurs in the regions of greatest socioeconomic
development. Neither section even mentions industrialization, socioeconomic
development, urbanization or any related term. How can they therefore be
used to justify the stance of the above quotation?

Q4: No justification is given for the claim of statistical insignificance, which has a precise meaning. Do you have a p value that justifies this statement, and if not,
what does it mean?

I will not discuss all the answers in this note and urge interested readers to consult the original here.

Let me start this quick review with the fourth question of Issue 2, since “statistical significance” is a technical term and IPCC made very specific assertions on this point.

As McKitrick had alleged, Jones did not have a p-value that justified his claim, but argued that “there is no need to calculate a p value for a statement that is based on the laws of physics”.

The pattern of atmospheric-circulation-related warming appears similar to the geographical distribution of socioeconomic development. Such similarity makes it impossible to use purely statistical methods to ascribe patterns of warming trends to patterns of socioeconomic development. It remains possible, however, to ascribe patterns of warming trends to atmospheric circulation because its influence is in accord with the laws of physics and can be detected in day-to-day weather variations, on which timescales socioeconomic trends are infinitesimal. As stated, it is essential to extract the known and understood influences first and then look at the residuals. There is no need to calculate a p value for a statement that is based on the laws of physics.

The latter statement surely raises epistemological issues on which Phil Jones hardly stands as an authority. (For example, I am unaware of any publications by Jones in epistemological literature.) But even if Jones were a qualified epistemological authority (which he isn’t), the statement in question was not “based on the laws of physics”. The IPCC statement in question was “the correlation of warming with industrial and socioeconomic development ceases to be statistically significant” – this statement may be based on an argument from physics, but it is a statistical statement and was not itself based on the laws of physics. Boulton should not have accepted such flannel.

In response to the related first question, Jones denied that the IPCC assertion about the lack of statistical significance was “ad hoc”, but, instead of evidence, merely presented a whinging complaint about too many academic papers:

The fact that MM2004 is in the peer-review literature does not mean it is good science. There are examples of poor science across all areas of science in the peer-review literature. Occasionally scientists submit comments on poor or incorrect papers, but this sadly is something of a rarity. With the plethora of journals it is becoming harder and harder to read and respond to all the literature. One could make a full time job of publishing criticisms of poor or incorrect papers.

Any competent inquiry would have seen through Jones’ flannel, but not Muir Russell. Although Boulton asked supplementary questions, none addressed these issues. Nor did the final report contain anything but flannel on the topic.

By the way, there’s an interesting development on this front in IPCC AR5 (First Draft) which admitted (with surprising candour) that AR4 “provided no explicit evidence” for the claim that the McKitrick and Michaels 2004 (and de Laat and Maurellis 2006) results had no statistical significance as follows (more on this on another occasion):

McKitrick and Michaels (2004) and de Laat and Maurellis (2006) analysed surface air temperature trend fields and assessed potential for biases in terms of national socioeconomic and geographical indicators. Both studies concluded that urbanisation and related land surface changes have caused much of the observed warming. According to the AR4, the correlation of warming with industrial and socioeconomic development ceases to be statistically significant if one takes into account the fact that the locations of greatest socioeconomic development are also those that have been most warmed by atmospheric circulation changes. AR4 provided no explicit evidence for this overall assessment result.

Returning to Boulton’s (sensible) second question – a question that remains relevant with IPCC’s recent efforts to extinguish public discussion of draft documents in real time:

Why were these conclusions not shown to or discussed with expert reviewers during the IPCC Report preparation?

Why indeed?

Jones explained to Boulton (and one can sense an almost Gavinesque sigh) that IPCC reviewers don’t get to see author responses to their comments at each stage (First Draft or Second Draft), that they only saw the author responses long after publication of the final report, that this was an IPCC system and that reviewers “were aware” of and accepted this system as a condition of submitting review comments. Jones:

The comment/response files for each stage were not released after each review, but only released together when the final report was published in May 2007. You seem to be under the impression that expert reviewers saw responses to their
comments at each stage. This has never been the case in any IPCC Report. This was an IPCC decision and all reviewers were aware of this when they made their reviews.

A system in which reviewers do not see author responses or review comments from other reviewers until after (and some months after) the final report surely warrants some introspection even if IPCC reviewers have acquiesced in such a system in the past. Nor does such a system appear to me to comply with (reasonable) public expectations that IPCC procedures be “open” and “transparent” – as IPCC officials like to proclaim.

In addition, I would be surprised if this sort of system is compliant with, for example, US federal standards for “influential scientific assessments”, a failure which may well create problems down the road for US agencies seeking to rely on IPCC reports without triggering fresh public comment and peer review.

Unfortunately, IPCC seems far more concerned about secrecy than in requiring its contributors to archive data. I received another request to remove discussion of IPCC draft reports. On this issue, David Appell and I are in full agreement – see David Appell’s collection of ZOD chapters here. (Jan 30 Update - see below.)

IPCC’s most recent request was as follows:

From: IPCC WGI TSU On Behalf Of Pauline Midgley
Sent: January-24-12 8:54 AM
To: Stephen McIntyre
Cc: ‘IPCC WGI TSU’
Subject: WGI AR5 FOD

Dear Mr McIntyre,

In a recent thread on the blog that you host, Climate Audit, you quote text and a figure directly from the WGI AR5 First Order Draft. We would remind you that each page of this document is clearly marked “Do not cite, quote or distribute”. Therefore, we kindly request you to remove this text and figure from your blog and refrain from such actions, which do not respect the terms of the IPCC review process.

Scientific comments and criticisms on the WGI AR5 FOD are encouraged and welcomed from experts in the topics being assessed. In order for the authors of the chapters to take into account, and respond to, these comments in drafting the Second Order Draft, they must be made through the appropriate channel. This requires registration as an expert reviewer and uploading the review comments on the WGI AR5 FOD before 10 February 2012.

Please refer to the WGI web site for more information about the WGI AR5 FOD review: https://fod.ipcc.unibe.ch/registration.

All review comments and the author responses will be published on an IPCC web site as soon as possible following the completion of the WGI AR5. For more information about the AR5 review in general, please see the IPCC web site: http://www.ipcc.ch/pdf/ar5/review_of_wg_contributions.pdf.

As mentioned in our email to you of 16 December 2011, in order to have access to the Chapters and to submit review comments for consideration by the authors, all prospective expert reviewers of the WGI AR5 FOD are required to agree to the terms of the review, which specify that all materials provided for the review, including the chapter drafts, are considered confidential and shall not be cited, quoted or distributed. This is the standard IPCC practice in the preparation of its reports.

Thank you for your cooperation.

Yours sincerely,

IPCC WG1 TSU

I responded today as follows:

Dear Ms Midgley,

This is to acknowledge your email of January 24, 2012 in which you “kindly requested” that I remove a discussion of IPCC from Climate Audit. In order for me to properly consider your request, I would appreciate it if you would clarify the legal basis, if any, of this request.

For many years, IPCC policies have stated that the review process should be “open” and “transparent” and my comments were very much in that spirit.

The recent review of IPCC policies and procedures by the InterAcademy Council did not contain any recommendations that the review process be less open or less transparent. I realize that Thomas Stocker, following suggestions of Phil Jones, sought changes to IPCC policies to authorize confidentiality, rather than openness, and that the minutes of the IPCC plenary session in Abu Dhabi state that the following language was approved:

IPCC considers its draft reports, prior to acceptance, to be pre-decisional, provided in confidence to reviewers, and not for public distribution, quotation or citation.

However, this change was deceptively included in a package described as “addressing” IAC recommendations, even though this language had nothing to do with IAC recommendations, but was designed to implement changes sought by Phil Jones and Thomas Stocker long before the IAC review. (See discussion at Climate Audit entitled Stocker’s Earmarks http://climateaudit.org/2012/01/12/stockers-earmarks/ .) Because IPCC officials seem to have misled the IPCC plenary session on the purpose of this language, it seems to me that you lack any moral authority to insist that reviewers comply with your request.

Nor am I aware of any legal authority or case law under Canadian or international law that entitles you to require me to remove the discussion at Climate Audit. Although I registered as a reviewer of the First Draft, I have not downloaded any documents from IPCC in that capacity and did not agree to any confidentiality terms in order to download documents. Nor do I intend to agree to any confidentiality terms as a condition of downloading.

Nor, as I understand matters, does IPCC’s adoption of the resolution saying that “IPCC considers its draft reports, prior to acceptance, to be pre-decisional, provided in confidence to reviewers, and not for public distribution, quotation or citation” create an obligation under Canadian or international law that requires me to comply with your request to remove discussion of IPCC drafts from Climate Audit.

It is my understanding that your email only asked politely that I remove the discussion and did not constitute a formal legal demand that I do so. Unless I am obligated under either Canadian or international law to remove the discussion from Climate Audit, I would prefer not to remove the discussion.

If I am incorrect in my interpretation of your email and it is your view that I am obligated under either Canadian or international law to comply with your request, I would appreciate it if you would explain the basis of your legal theory. I will promptly consider your explanation. Please do not consider this email as an unconditional refusal of your request, but an invitation to you to explain the legal basis of your request under Canadian or international law so that I can take the matter up with my own legal counsel.

While I understand that you are not obliged to take notice of articles posted at Climate Audit, I am very familiar with issues involved in proxy reconstructions. My hope is that the discussion at Climate Audit will contribute to a better understanding of proxy reconstructions, even by IPCC contributors. As I understand it, there are no IPCC rules that prohibit IPCC contributors from taking note of articles at Climate Audit nor any authority for you to discourage IPCC contributors from doing so, should they be so inclined.

Regards,
Stephen McIntyre

Jan 30 Update:
IPCC sent me the following response:

Dear Mr McIntyre

Thank you for your email of 26 January addressed to Dr. Midgley. As has been standard practice and is stated in the Procedures of IPCC, to which we have to adhere in our work for the WGI contribution to AR5, IPCC draft reports that are made available for expert review are done so under the conditions marked on every page: “Do Not Cite, Quote or Distribute”.

In order to be sure that the authors see, consider and respond to your valuable comments on these drafts, they must be submitted through the mechanism provided at the WGI web site. This site will be used by all expert reviewers, over 1500 of whom have duly registered.

Thank you for your attention and your interest in IPCC WGI AR5.

Yours sincerely,

IPCC WGI TSU

As happens far too often in climate science, rather than answering my question, they simply re-iterated their original demand.

An excellent article at Bishop Hill here describing a clean sweep for Don Keiller in court (with David Holland as a “Mackenzie friend”) against the University of East Anglia and its solicitors. Decision is here.

The article reports on Keiller’s appeal to the First-Tier Tribunal (Case No. EA/2011/0152) in the General Regulatory Chamber – Information Rights. The appeal related to the second part of Keiller’s original FOI – instructions sent by CRU to Georgia Tech on their use of CRUTEM. CRU had argued that they didn’t have the information because Jones had deleted the email and they did not have access to the server in police possession. All arguments by the university were dismissed, with the judge being more than somewhat acid in some of his comments.

The University of East Anglia’s argument was hampered by their failure to present direct evidence from Phil Jones. (Assertions by Jones were presented by what Don Keiller described as “third-hand hearsay” – a conversation between Jones and David Palmer, passed on to Jonathan COlam-French, passed on to the UEA solicitor. Keiller and Montford observe:

It appears that UEA were keen that Jones should not appear on the witness stand, where he would be required to give evidence under oath. In fact it is noteworthy that, despite all the official “investigations”, Jones has never been required to answer questions under oath or provide a signed declaration.

Earlier this year, I reported on the refusal of Raphael Neukom, an associate of IPCC confidentiality advocate and WG1 Co-Chair Thomas Stocker at the University of Bern, to archive data used in a then recent multiproxy study, Neukom et al 2011 (Clim Dyn). In his refusal letter, Neukom stated that

Most of the non-publicly available records were provided to us for use within the PAGES LOTRED-SA initiative only

Neukom’s website lists a set of multiproxy data, many of the series said to be “available on request for LOTRED-SA contributors”.

Despite Neukom’s “excuse” that the data had been provided to him for use “within the PAGES LOTRED-SA initiative only“, Neukom et al 2011 has been prominently used in the IPCC AR5 draft assessment report. The Neukom reconstruction is used to compare the performance of models to paleoclimate reconstruction in South America in FOD Figure 5.9 as shown below:

Figure 1. Neukom et al 2011 south South American temperature reconstruction (heavy black). Colored lines show results from various climate models.

Unless you are a climate scientist, you would probably not describe the paleoclimate reconstruction as cohering particularly well with the various models, but that’s a story for another day.

In the running text of the draft AR5, Neukom et al is cited on several occasions, including as authority for the observation that SH summer temperatures in the Medieval Warm Period (you know, the one that is supposedly regionally restricted to Greenland and a few counties in England) were “mostly warmer than the 20th century climatology”:

Progress has been made in the SH since AR4, where new tree ring records from the Andes, northern and southern Patagonia, Tierra del Fuego, New Zealand and Tasmania (Boninsegna et al., 2009; Cook et al., 2006; Villalba et al., 2009), ice cores, lake sediments and documentary evidence from southern South America (Neukom et al., 2011; Prieto and García Herrera, 2009; Tierney et al., 2010a; Vimeux et al., 2009; von Gunten et al., 2009) and terrestrial and shallow marine geological records from eastern Antarctica (Verleyen et al., 2011) allow a better understanding of past temperature variations (Neukom et al., 2011). A multi-proxy reconstruction for southern South American (Neukom et al., 2011) finds austral summer temperatures between 900 CE and 1350 CE that are mostly warmer than the 20th century climatology (though associated with large uncertainties), with a sharp transition after 1350 CE to colder conditions that last until approximately 1700 CE.

Citation of Neukom et al 2011 by IPCC clearly takes its use outside the realm of LOTRED-SA associates, but Neukom has thus far taken no steps to ensure that proxy data of Neukom et al 2011 (now used by IPCC) is available to anyone outside his circle of cronies.

Neukom (together with Joelle Gergis) has also published a recent survey of Southern Hemisphere proxies Neukom and Gergis 2011, one of the objectives of which was to report on the “availability” of SH proxies. Unfortunately, with the sort of blindness all too familiar from Climategate, Neukom and Gergis conflate availability of data to cronies with public availability – a situation that I am presently testing with a long-unarchived Eric Steig data set.

Neukom and Gergis 2011 is one of a number of recent multiproxy articles appearing more or less on the eve of AR5, as they themselves note:

Given the importance of global circulation features like ENSO, IOD, SAM and the IPO, a concerted effort is now underway to consolidate existing high-resolution palaeoclimate records from these regions in time for the IPCC fifth assessment report (AR5) (Gergis et al., 2011; Neukom et al., 2010, 2011).

They provide a list of 174 proxies (expanded from the list on Neukom’s website) that supposedly meet the following criteria:

- extend prior to 1900
- are calendar dated or have at least 70 age estimates in the 20th century
- extend beyond 1970 to allow sufficient overlap with instrumental records
- are accessible through public data bases or upon request from the original authors

Series said to be available “upon request from the original owners” are, for the most part, the same series that had previously been described as only “available on request for LOTRED-SA contributors”. Neukom and Gergis report that there were only 14 records extending back to 1000 AD that met the above criteria. Somewhat surprisingly, although IPCC had claimed “progress since AR4″ in the development of SH proxy data, only one of the 14 series appears to have been developed since AR4 (with most of the series being developed prior to AR3.)

Many of these records are listed by Neukom and Gergis as only “available upon request”. The most intriguing such example is Eric Steig’s Siple Dome dD (and d18O) series – see excerpt below – which Neukom obtained as a “personal communications”.

The Siple Dome core was drilled in 1993-4: the data was publicly funded. Even by Lonnie Thompson standards, this is a long time for the data to remain both unpublished and unarchived, particularly given the scarcity of long SH proxies. One cannot help but think that the data set would have been promptly published if it had HS shape and, ergo, my prediction is that, if and when, the data ever is made “available”, it will not have a Hockey Stick shape.

As a test of Neukom and Gergis’ statement that the data sets are “available upon request”, I submitted a request for this data to Eric Steig (cc Neukom) a couple of days ago, thus far, without acknowledgement from either.

Dear Dr Steig,
Neukom and Gergis 2011 stated that the datasets listed in their tables were accessible “upon request from the original authors”. Among their series are Siple Dome delD (1000-1993) and d18O (1654-1994) series said to have been sent to Neukom as a “pers comm.” In 2010. Could you please provide me with a copy of the series that you provided to Dr Neukom.

Thank you for your attention,
Stephen McIntyre

My early line on the over/under for getting the data is the publication date of AR5.

Six more of the 14 long series were obtained by Neukom and Gergis from crony contact (rather than public archives) including: Law Dome (Curran); ocean sediment series 106KL (Rein); four South American tree ring series: Central cluster 3A/CAN Composite 20; Central Cluster 3C/CAN Composite 25 and Central Cluster 3B/CAN Composite 23 (all said at Neukom’s website were said to be “unpublished” and to be only “available on request for LOTRED-SA contributors” and Lenca was said to be “Not public”.) The Aculeo temperature reconstruction from pigments is only available in a smoothed version of the regression product – the underlying data has not been archived. In the case of the South American tree ring series, there are public archives of measurement data that appear to overlap the versions used here. The existence of different versions makes the need for careful archiving of versions as used all the more important.

Despite IPCC’s puff about “progress”, a more objective assessment of actual progress in the field would be to inform readers from other fields that there has actually been negligible progress since AR4 in the development of new long proxies.

Tree rings: the Lenca series ends in 1987; the three long Central Cluster/CAN Composite series are Fitzroya cupressoides (FICU) series that end between 1990 and 1995. One of these series is almost certainly a version of the Rio Alerce FICU series used in the AR3 reconstructions Jones et al 1998 and Mann et al 1998-99. At most, the Neukom and Gergis versions appear to be re-processing of data collected in the early 1990s.

Ice cores: the “14″ series of Neukom and Gergis appear to include five ice core series, none of which are new since AR4 (and again, most are older than that.) Two series from the 1993 Siple Dome core appear to be included (Na- Mayewski et al 2004; and dD – Steig and White, unpublished and unarchived); a Law Dome series (presumably related to the series illustrated in Jones and Mann 2004); a series from Berkner Island (Mulvaney et al 2002); and Quelccaya, Peru (Thompson 1983, updated 2003). The Law Dome series was definitely available for AR4. As an AR4 reviewer, I had asked that it be included in the illustration of actual SH proxies. IPCC correspondence about the decision not to show this proxy (which has an elevated MWP) is in the Climategate dossier. CRU’s Tim Osborn (also Lead Author in AR5), knowing that the proxy showed an elevated MWP (and that I knew this and that I knew that they knew that I knew…), proposed that they cover themselves by mentioning the proxy in the running text but not show it in the diagram, a device readily agreed to by IPCC authors reluctant to show a SH series with an elevated MWP.

Coral: the discontinuous Palmyra series of Cobb et al (2003). See 2006 CA discussion here.

Sediments: ocean sediment 106KL was already available in Rein (2004), not Rein (2007) as shown by Neukom and Gergis. The Aculeo pigment reflectance series of von Gunten et al 2009 is new.

As I presently understand the matter, only one of the 14 data sets is “new” since AR4: the Aculeo pigment reflectance series. And while the development of novel proxies is a good idea, until the pigment reflectance methodology is replicated in other lakes, little weight should be placed on this result (particularly without the underlying data being available for analysis.)

In my opinion, a candid assessment of progress in the field by IPCC would clearly state that (1) Neukom and Gergis’ list shows that there has been essentially no development since AR4 of SH proxies that permit comparison of the medieval and modern periods; and (2) that much (far too much) of the limited data still remains unarchived. Unfortunately, WG1 Co-Chair Stocker seems more concerned with ensuring confidentiality of IPCC drafts and internal correspondence.

In December, the WG1 TSU of the IPCC sent me a formal notice asking me to remove Climate Audit discussion of the IPCC Zero Draft. In this notice, they stated:

It has come to our attention that several Chapters of the Zero Order Draft (ZOD) of WGI AR5 are being cited, quoted and discussed on the blog that you host, Climate Audit, despite the fact that each of these chapters is clearly marked “Do not cite, quote or distribute”. We would respectfully request that you remove the relevant parts with discussions of the ZOD from your blog and, furthermore, that this does not happen with the FOD.

I’ve been mulling over how to respond. I was not a reviewer of the Zero Draft and had not made any personal agreements with IPCC as a condition of receipt. I had registered as a First Draft reviewer but have not downloaded any documents in this capacity as yet.

In preparing a response, I’d been wondering what authority, if any, was possessed by WG1 or its TSU that entitled it to require or request removal of this discussion from Climate Audit. I’d looked at IPCC Policies and Procedures in connection with previous CRU requests. The procedures used in AR4 (see here) had said that the “review process should be objective, open and transparent” and did not contain any language that specifically granted authority to the TSU of a Working Group to prohibit discussion in public of its draft reports. If anything, the overriding objectives of openness and transparency would seem to support such discussion – a process that seems entirely healthy to me and one that would actually enhance the IPCC.

It turns out that Phil Jones and Thomas Stocker, Co-Chair of AR5 WG1, both agreed with my interpretation of IPCC rules on this point i.e. that the Working Groups lacked specific IPCC authority to insist on confidentiality of their drafts, and that they had, behind the scenes, taken steps to change IPCC rules to authorize Working Groups to do so. Jones’ initial contacts with Stocker on this matter are documented in Climategate 2 and arose from Jones’ reading of Climate Audit posts advocating openness and transparency by IPCC – efforts that both Jones and Stocker opposed.

I only became aware of their actions recently as a result of an IPCC cease-and-desist letter to Galloping Camel, which had posted an excellent collection of WG1 and WG2 sources. To my considerable surprise, the IPCC letter to Galloping Camel contained a quotation from IPCC Policies and Procedures here (bolded below) that contained an endorsement of confidentiality that was absent in the AR4 polices. They wrote:

The IPCC Procedures in Article 4.2 of the Principles Governing IPCC Work state that “The IPCC considers its draft reports, prior to acceptance, to be pre-decisional, provided in confidence to reviewers, and not for public distribution, quotation or citation.” (http://www.ipcc.ch/pdf/ipcc-principles/ipcc-principles-appendix-a-final.pdf). We therefore request the immediate removal of the ZOD chapters from your website.

This language was definitely not in the AR4 version. Indeed the document linked in the Galloping Camel letter was time stamped January 10, 2012(!). When had the new language been introduced? And by what authority? Tracing the language led to a remarkable story.

The language was almost singlehandedly introduced by Stocker (after being involved by Phil Jones.) Complicating Stocker’s efforts to obtain official sanction for enhanced confidentiality was the lack of interest in this topic by the Interacademy Panel, which had been commissioned by IPCC to review its policies and procedures. Not only did its report not contain the recommendations sought by Stocker and Phil Jones, it re-iterated the importance of openness and transparency. Nor had the language sought by Stocker been recommended in any of the numerous documents on IPCC procedures up to the second week of April 2011, less than four weeks before Stocker’s language was adopted at the Abu Dhabi IPCC meeting in May 2011.

Despite these obstacles, Stocker emerged from the IPCC plenary with his objective. It’s a long story.

Prequel

On May 11, 2009, I reported my request for CRUTEM station data from the Met Office. In a comment, David Holland noted that the AR5 Working Group 1 TSU was in Switzerland and that Switzerland was in the process of adopting the Aarhus Convention on freedom of information.

Jones read this comment and became worried about the prospect of IPCC being subject to the Aarhus Convention. Jones immediately emailed Stocker (May 12 – 4778)

subject: Data access and IPCC
Dear Thomas,
I hope you are enjoying your new job! Apologies in advance for upsetting your morning! Below there is a link to Climate Audit and their new thread with another attempt to gain access to the CRU station temperature data. I wouldn’t normally bother about this – but will deal with the FOI requests when they come. Despite WMO Resolution 40, I’ve signed agreements not to pass on some parts of the CRU land station data to third parties.

If you click on the link below and then on comments, look at # 17. [here] This refers to a number of appeals a Brit has made to the Information Commissioner in the UK. You can see various UK Universities and MOHC listed. For UEA these relate to who changed what and why in Ch 6 of AR4. We are dealing with these, but I wanted to alert you to few sentences about Switzerland, your University and AR5.

Having been through numerous of these as a result of AR4, I suspect thatsomeone will have a go at you at some point. What I think they might try later is the same issue:
Who changed what and why in various chapters of AR5?
and
When drafts of chapters come for AR5, we can’t review the chapter as we can’t get access to the data, or, the authors can’t refer to these papers as the data haven’t been made available for audit.

Neither of these is what I would call Environmental Information,as defined by the Aarhus Convention. You might want to check with the IPCC Bureau. I’ve been told that IPCC is above national FOI Acts. One way to cover yourself and all those working in AR5 would be to delete all emails at the end of the process. Hard to do, as not everybody will remember to do it.

I also suspect that as national measures to reduce emissions begin to affect people’s lives, we are all going to get more of this. We can cope with op-ed pieces, but these FOI requests take time, as the whole process of how we all work has to be explained to FOI-responsible people at each institution.
Keep up the good work with AR5!
Cheers
Phil

Jones also notified Peter Thorne of the Met Office that he had alerted Stocker to Holland’s comment.

The next day, Stocker (May 13 – 4378) replied, telling Jones (cc Pauline Midgeley) that allowing access to climate data under laws prescribing “open access to environmental data” would be a “perversion” (Stocker would have been an interesting witness in the EPA case, where an opposite position was taken by IPCC supporters):

subject: Re: Data access and IPCC
Dear Phil (cc to Pauline Midgley, Head TSU WGI)
Thank you very much for bringing this to my attention. I knew about this when the first requests were placed on John Mitchell and Keith Briffa and they informed us. What I did not know is that they have already placed their focus on Bern (# 17)!

At that time I argued that in principle there are two interests to balance: (i) FOI, and (ii) your own privacy when it comes to opening emails or other mail. Obviously, I am not in the position to judge which one obtains and in fact I think a court would be needed to establish
exactly that balance.

However, the Arhus Resolution, it seems to me, had another motivation: open access to environmental data associated with damage, spills, pollution; the latter word is mentioned twice – “climate” never. So to take this convention and turn it around appears to me like a perversion. One important point to consider is whether Arhus really applies to the IPCC activities. In no way are we involved in decision making. We assess and provide scientific information. The decision makers are elsewhere.

More than ever need we be aware of this separation! We will discuss this in the TSU but then, this should be brought to the level of the Secretariat, at least, since it affects the very basis of our assessment work.
Thanks again and best regards,
Thomas

A couple of weeks later, Jones wrote Santer and Wigley (May 22 – 2135)

They have found out that Switzerland has agreed to but not yet ratified some Environmental Information Regulations (Aarhus Convention), so are probably looking to have a go at the University of Bern and Thomas Stocker at somepoint. Never thought I would know so much about the Law!

Climate Audit Submission to EPA

On June 23, 2009, I posted up my submission to EPA. (Re-reading this submission, I thought that it has held up well.) The submission observed that neither EPA peer review procedures in the Endangerment Study nor IPCC peer review procedures complied with US federal standards for “highly influential scientific assessments or with EPA’s own procedures and policies. (Last year, the EPA Inspector General reported on part of this topic – EPA made the embarrassing argument that the Endangerment Study was not a “highly influential scientific assessment” and therefore did not have to comply with these federal standards. In their defence, EPA argued that it was IPCC that was the “highly influential scientific assessment” – an argument which made it all the more important for IPCC to exceed US standards if IPCC reports were to be used as authority for important policies.)

In my EPA submission, I documented various forms of non-compliance by IPCC, commenting at CA as follows”

Indeed, the various discussions that we’ve had over the past months over IPCC’s amorphous legal status – i.e. IPCC participants having dual status as government employees, with their IPCC affiliation being applied to yield a cone of darkness over activities which would be subject to FOI if they were “merely” government employees.

Evasion of transparency has been a long-running concern of this site and I’ve used this comment opportunity to place this and related concerns on the record.

The following day (June 24 – 2440), Jones sent a copy of my submission to Stocker and Midgeley. Jones did not suggest to Stocker that this was a carefully reasoned submission and that they should look closely at criticisms of IPCC transparency and reflect on whether they might be able to improve its processes so that they exceeded US standards:

subject: Re: Data access and IPCC
to: Thomas Stocker , wg1
Dear Thomas,
Attached is a document that you should only bother to look at it you have time to spare – stuck on a train or long flight. It is a submission by a skeptic to EPA in the USA. [http://climateaudit.org/2009/06/23/climate-audit-submission-to-epa/] I’m sending it only for background. I wouldn’t want this issue to be raised at the Venice meeting, but I think you’ll likely to become more aware these people as AR5 advances. I was in Boulder last week and I spoke to Susan. We agreed that the only way IPCC can work is the collegiate way it did with AR4.
These people know they are losing (or have lost) on the science. They are now going for the process. All you need to do is to make sure all in AR5 are aware of the process and that they adhere to it. We all did with AR4, but these people read much more into the IPCC procedures.
See you in Venice
Phil

Stocker’s acknowledgement (June 24 – 4899) said that he was working on the problem:

subject: Re: Data access and IPCC
to: Phil Jones

Thanks Phil. We have not be inactive here at the TSU. I have approached a number of colleagues with the problem and expect more indications in the next few weeks to come. I hope that I will be able to have a clear view on the way forward by the time we think of nominations and when we like to inform our potential LAs and CLAs. Thanks and best regards and ’till Venice,
Thomas

On July 15 (1526), Tim Osborn of CRU wrote to Stocker seeking support from IPCC against David Holland’s assertions that IPCC policies required them to be “open” and “transparent”.

On July 29 (Climategate 1- 991. 1248902393.txt), Jones wrote Peterson of NOAA saying that he had persuaded Stocker and the IPCC Secretariat to raise FOI issues with the IPCC Plenary in the next IPCC meeting (Bali):

I have got the IPCC Secretariat and Thomas to raise the FOI issues with the full IPCC Plenary, which meets in Bali in September or October. Thomas [Stocker] is fully aware of all the issues we’ve had here wrt Ch 6 last time, and others in the US have

Despite Jones’ optimism about Bali, the Bali IPCC Plenary session doesn’t appear to have done anything to implement Jones and Stocker’s secrecy ambitions.

The Interacademy Report
In March 2010, in response to Climategate, the IPCC commissioned a report from the Interacademy Council, the terms of reference of which specifically included a “review IPCC procedures for preparing reports”.

This was the forum that Stocker should have submitted his concerns about enhancing confidentiality in the Working Groups. Did he do so? At present, I don’t know. If he didn’t, he should have. If he did, the IAC disregarded his recommendations as, in the final IAC report, which appeared in late August 2010, there is nothing that comes close to a recommendation along the lines that Stocker sought.

On the contrary, the IAC Report re-iterated the position that processes and procedures be “as transparent as possible”:

it is essential that the processes and procedures used to produce assessment reports be as transparent as possible.

Transparency is an important principle for promoting trust by the public, the scientific community, and governments. Interviews and responses to the Committee’s questionnaire revealed a lack of transparency in several stages of the IPCC assessment process, including scoping and the selection of authors and reviewers, as well as in the selection of scientific and technical information considered in the chapters.

Nowhere did the IAC report recommend the sort of additional confidentiality sought by Stocker and Phil Jones.

Busan IPCC Meeting, October 2010

At the first IPCC meeting after the IAC report (Busan, October 2010) (see here), the IPCC stated that it “agreed to implement many of the recommendations immediately. On others, the Panel decided to form four Task Groups to undertake further work intersessionally, with a view to completing work on the IAC recommendations at the Panel’s next session.”

One of the four Task Groups was the Task Group on Procedures, the terms of reference of which were to examine the IAC recommendations and, for each of the issues, propose implementation, including amendments to the IPCC Policies and Procedures (Appendix A):

The Task Group should address the issues listed below as mentioned in the IAC recommendations (Chapters 2 and 3), IPCC responses at its 32nd Session and IPCCXXXII/Doc. 22. For each of the issues the Task Group should establish a timetable for action, consider resource implications and identify responsibilities for implementation. It should propose amendments to the Appendix A to the Principles Governing IPCC Work and relevant guidance documents if needed taking into account decisions made at IPCC-XXXII.

Geneva, February 2011

The IPCC Task Groups met in Geneva in February 2011 to review progress. The provisional report of the Task Group on Procedures is available here, included in an April 2011 document.

Sections 2-10 of this report discussed issues within their terms of reference i.e. arising from recommendations of the IAC panel. In addition, they included an “Addendum” discussing issues that did not arise from the IAC panel – an initiative that appears to have been opposed by some members, introduced as follows:

To some extent the Task Group also discussed some suggestions that were related to the IAC report recommendations but may be viewed as being not strictly within the mandate given by the 32nd session of the IPCC. The Task Group considered these suggestions useful for further discussion and includes them in this document under the Addendum “Issues for further discussion on Procedures” (section 11 of this document).. Please note that this addendum does not reflect any consensus by the Task Group.

The introduction to the Addendum (see page 10) re-iterated that the issues in the Addendum fell outside the mandate of the Task Group and did not reflect a consensus:

11. Addendum: Issues for further discussion on Procedures page 10
11.1 Introduction
The Task Group noted that a number of issues were raised by Task Group members that maybe viewed as being not strictly within the mandate given by the 32nd session of the IPCC. However, the Task Group considered these issues useful for further discussion as part of an effort to further improving clarity of the Procedures, and the transparency and quality of the assessment process.

Please note that this Addendum does not reflect any consensus from the Task Group discussions at their meeting in Geneva 1-4 February 2011. The reviewers of this document may wish to give their viewpoints on the issues and thoughts below.

Section 11.3 of the Addendum contained a comment on confidentiality that was the first small wedge for the Jones-Stocker secrecy language. It stated that saying that guidance “may be needed” on citation of draft reports and “other documentation” “without contradicting the needed transparency and openness”. (Watch as this language morphs.)

11.3 Review process (sections 4.1 and 4.2 of the Procedures)
…
Confidentiality
Clear guidance may be needed on what the rules are for citation/publication of draft reports and other documentation during drafting and review and how the draft report need to be kept confidential without contradicting the needed transparency and openness, while different versions of the draft should be accessible after the completion of the report.
….

On February 9, 2011, this draft (with this very vague and buried reference to confidentiality) was circulated to governments with a four week period for review (see here). Invitations to the forthcoming meeting in Abu Dhabi were sent on Feb 23 by IPCC to governments and NGOs, together with a provisional agenda, item 5 of which was “Review of IPCC Processes and Procedures”.

Review comments on the “Geneva Draft” report are here, presumably dating from early March as scheduled. They include comments from both governments and Thomas Stocker (whose input is described as authorized under “P-32″).

No government commented that the TSUs needed greater powers to restrict discussion of draft reports. However, Stocker complained (see page 145; repeated on page 163 for good measure) that the Task Group took a “rather strict” view of its mandate and “missed the opportunity” to tighten up secrecy provisions. Stocker’s comments in this document are the first appearance of the language later used in the letter to Galloping Camel.

(2) The TG Procedures took a rather strict view of its mandate and did not make recommendations on topics that were not raised by the IAC Review. This means that some necessary adjustments to the Principles and Procedures to address other important issues such as confidentiality were not properly developed. This is potentially a missed opportunity.

(3) The draft makes an exception for Topic 10 Guidance Notes, a topic not raised by the IAC Review. WGI proposes that another exception should be made for confidentiality, which is a topic of great importance that was also discussed by the TG during its meeting in Geneva in February 2011. It is mentioned in the Addendum under the review process but clear guidance on confidentiality is needed in a broader context. It is part of the basic way in which IPCC goes about its work and is a necessary requirement for authors to be able to have a free and frank exchange of views. Interim discussions and communications during the preparation and finalisation of the assessment are *pre-decisional* information. As such, these remain confidential and related documents are not public, nor should they be cited, quoted or distributed, as is standard IPCC practice to indicate this on all documents under review. The ability of the WGs to produce an independent and unbiased assessment would be threatened if
material that is in the nature of a draft and/or incomplete information to be further developed were to be released prematurely. It is increasingly clear that this needs to be specified in the Procedures, also showing that it does not contradict the current Principles of IPCC, which state that the assessment is carried out on a “comprehensive, objective, open and transparent basis”.

Therefore WGI proposes moving this topic into the first part of the TG report, between Topics 3 and 4. The preceding text in this comment can serve as the basis for the TG consideration and the recommendation would be as follows:
“Section 4 of the Procedures should be amended to discuss the notion of the confidentiality of drafts and other interim documentation. Suggested text could be the following: “Drafts of the reports, interim discussions and communications, and other documentation created during the drafting and review process are considered pre-decisional materials and as such are confidential. Drafts and other documentation may not be cited, quoted or distributed. “”

(4) The Addendum is not very helpful as presented because it is not clear to a reader who was not part of the discussions in February 2011 in Geneva whether all these suggestions have a similar status in terms of degree of support, depth of discussion, etc. They are in fact highly variable and most of these suggestions will need to be discussed properly by the TG first. [Thomas Stocker, Cochair,WG I]

Stocker’s Earmark

As of early April 2011, a month before the IPCC was due to meet, the language long sought by Stocker and Phil Jones had not been recommended in any of the documents. It appears for the first time in a Report of the Task Group on Procedures included as Document 12 in the agenda for the Abu Dhabi meeting scheduled to begin on May 10.

Uniquely among the documents for the Abu Dhabi meeting, the online version of this document bears a time-stamp of May 12 (after the meeting); other documents linked from the agenda are dated April or earlier. It asked for government comments by May 5. This document was accompanied by a IPCC-XXXIII/INF. 1 , Review of the IPCC Processes and Procedures – Comments from Governments and IPCC Office Holders on the initial draft recommendations prepared by the Task Groups”.

Despite earlier concerns on the part of some members of the Task Group that they comply with their mandate to respond to IAC recommendations (which is what it was represented as having done to the IPCC plenary), in their revised report, the Task Group distinguished two classes of decision:

The Task Group on Procedures has dealt with two categories of proposed decisions:
I. Direct responses to IAC recommendations,
II. Indirect responses as a result of the IAC recommendations following the above mentioned
‘addendum’.

The language sought by Stocker was in section 6 bis4, the Task Group stated:

6bis 4 Confidentiality of draft reports
This issue was raised by the WG I co-chairs. Given the upcoming finalization of two Special Reports The Task Group deemed this issue important for consideration.

6bis 4.1 Task Group consideration
The Task Group noted that clear guidance is needed on what the rules are for the confidentiality of draft reports and other documentation during drafting and review. On one hand, there is a need for transparency and openness of the assessment process. On the other hand, publicizing drafts have serious drawbacks. There is a risk that drafts contain errors or statements that are still unbalanced and that have to be corrected at a later stage. These could prematurely circulate in the public domain, creating confusion, and that would be a bad service of IPCC to society. Therefore, the Task Group believes that drafts should be kept confidential until acceptance of the full report.

All drafts of IPCC assessment reports (including the final draft) will be considered to be confidential material, not for public distribution quotation, or citation until acceptance by the Panel of the final IPCC report. The first order draft, second order draft and the final draft, the expert and government review comments, and the author responses to those comments on both drafts will be made available on the IPCC open website on a clearly visible place, within xx weeks after the acceptance of the report by the Panel.

As always, one has to watch the pea. Stocker’s language was not an “indirect response” to IAC recommendations. Nor did it have anything to do with IAC. Stocker and Jones had sought to beef up rights of the Working Groups to demand confidentiality prior to the IAC report. Representing enhanced confidentiality as addressing an IAC recommendation was, so to speak, a “trick” (TM- climate science.)

Although the revised report of the Task Force (dated circa April 11) was distributed for comment, the distribution was very late in the process, less than a month before the IPCC meeting, and only one government (Netherlands) commented (in passing) on the language introduced by Stocker. See XXXIII/INF. 4 – Comments received from Governments and IPCC Office Holders by 6/05/2011 on the proposals by the Tasks Groups.

The only entity to comment at length on the language was Stocker himself. Stocker observed (accurately) that, despite its stated mandate of openness and transparency, “confidentiality is part of the basic way in which IPCC goes about its work”:

We are pleased that many of our comments on the draft in March were implemented in this proposal and in particular that the important point about confidentiality is now treated explicitly in section 6. Whilst this is clearly related to the review process, guidance on confidentiality is needed in a broader context, given that requests occur for access to working papers, emails, etc. Confidentiality is part of the basic way in which IPCC goes about its work and is a necessary requirement for authors to be able to have a free and frank exchange of views. Interim discussions and communications during the preparation and finalisation of the assessment are “pre-decisional” information. As such, these remain confidential and related documents are not public, nor should they be cited, quoted or distributed. It is standard IPCC practice to indicate this on all documents under review. The ability of the WGs to produce an independent and unbiased assessment would be jeopardised if material that is in the nature of a draft or incomplete information to be further developed were to be released prematurely. It is increasingly clear that this concept needs to be specified in the IPCC Procedures, also showing that this is not a contradiction of the current Principles of IPCC, which state that the assessment is carried out on a “comprehensive, objective, open and transparent basis”.

Buoyed by his April success, Stocker proposed that the policy be expanded to include other documentation (a recommendation not adopted in the final resolution):

As well as discussing confidentiality of the report drafts, WGI proposes adding wording that discusses the confidentiality of other interim documentation, e.g., “Drafts of the reports, interim discussions and communications, and other documentation created during the drafting and review process are considered pre-decisional materials and as such are confidential. Drafts and other documentation may not be cited, quoted or distributed.”
…

Please find below our other comments for consideration in finalising the proposal:
1. Introduction
The distinction between category I and category II proposals may require more explanation by the TG Co-Chairs. Priority should be given to the key decisions that were well supported by a
consensus in the TG, and those that immediately affect the next stages of the AR5 process.
…

6bis4. Confidentiality of draft reports: for the reasons described in our introductory comments, we very much welcome this new section and proposed decision in 6bis4.2. We again stress that the first sentence of the proposed decision text needs some editing in order to include the key term “pre-decisional” and to capture the other kinds of material in addition to the draft reports.
…

Stocker’s comments are worth reading in full as they touch on other interesting topics as well.

The IPCC meeting was held from May 10-13 following the revised agenda here . During that meeting, Stocker’s language was approved at a Plenary session. See minutes here.

Although Stocker’s recommended language in no way “addressed” recommendations of the IAC Panel (and arguably was even antithetical to their reiterated support for openness and transparency) and although the Task Group had been well aware that these issues were outside the mandate set by the 32nd IPCC plenary, the resolution represented this (and other) recommendations as addressing recommendations of the IAC Panel:

The document presented here contains the decisions by the Panel based on consideration of the report of the IPCC Task Group on Procedures to the IPCC 33rd Session and building on the decisions of IPCC 32nd Session. The Task Group addressed the InterAcademy Council (IAC) recommendations as presented in the IAC report, chapter 2, “Evaluation of IPCC assessment process”…

8. Confidentiality of draft reports
The Panel noted that issues related to confidentiality of draft reports is important and that clear guidance is needed on what the rules for the confidentiality of draft reports during drafting and review.

At its 33rd Session, the Panel decided that the drafts of IPCC Reports and Technical Papers which have been submitted for formal expert and/or government review, the expert and government review comments, and the author responses to those comments will be made available on the IPCC website as soon as possible after the acceptance by the Panel and the finalization of the report.

IPCC considers its draft reports, prior to acceptance, to be pre-decisional, provided in confidence to reviewers, and not for public distribution, quotation or citation.

A long story indeed. But Stocker (and Phil Jones) had gone to a lot of trouble to obtain the language used in the Galloping Camel letter.

Climategate 2.0 emails shed remarkable light on the role of Nature news “reporter”, Olive Heffernan, in the development of a “legend” to place CRU data obstruction in a better light.

They show that Jones had candidly admitted to Heffernan that his real reason for refusing data was simply to obstruct potential critics – a position essentially unchanged from his notorious 2005 refusal on the grounds that “we have 25 years invested in this”. Instead of reporting Jones’ admission, Nature attributed Jones’ data obstruction to an imaginary “inundation” of data requests following an initial supply of data in 2002.

In today’s post, I’ll review the development of the legend and Heffernan’s role in the process.

The curtain rises for today’s post on Aug 7, 2009 about two weeks after the FOIA/Mole incidents discussed in the past few posts. Heffernan asked Jones (721) whether Jones had sent supposed confidential data to Webster, but refused to send it to me. And, if so, on what grounds.

… I have one other question: McIntyre claims that you sent data to Peter Webstre at Georgia Tech, but that you would not supply him with the same data. Is that true, and if so, what was the reasoning?
Best,
Olive

This was one of the important questions that Muir Russell negligently failed to examine.

Jones’ reply to Heffernan (Aug 9 – 3497) was a pastiche of disinformation and unresponsiveness, none of which should have been accepted by a responsible reporter, but which passed unchallenged by Heffernan.

Jones commenced:

Olive,
I did send some of the data to a person working with Peter Webster at Georgia Tech. The email wasn’t to PW [Webster], but he was in the CC list. I don’t know how McIntyre found out, but I thought this was a personal email.

As we’ve already seen, Jones knew precisely how I had found out as, a few weeks earlier, he had quoted (4531) in full a comment at Climate Audit by Peter Webster reporting that CRU had sent station data to Georgia Tech:

I also see from looking at Climate Audit that this request results from Peter saying on CA that he’s not had any difficulty getting data from CRU (see what he said below on June 24).

In that earlier email, Jones had also argued that he had provided data to Webster in a “personal” email, a claim that Palmer had rejected (1320). Despite Palmer’s explicit rejection of the email being “personal”, Jones stubbornly continued to take this line in his correspondence with Nature.

Jones then made the completely untrue claim that this was “one of the first times” that CRU station data had been sent to a third party:

This was one of the first times I’d sent some data to a fellow scientist who wasn’t at the Hadley Centre. As I said I have taken pity on African and Asian PhD students who wanted some temperature and precipitation data for their country.

In fact, CRU’s original collection of station data was financed by the US Department of Energy under a contract that obliged CRU to deliver station data to the US Department of Energy, which then placed the data online (where it remains online to this day). CRU itself had placed CRUTEM1 station data online in 1996 (which remained online until July 29-30, 2009). Jones had sent station data to Mann and Rutherford and even, in 2002, to me.

Jones then segued into characteristic self-pity, falsely claiming that he’d been “driven” into being “less helpful”:

The email has only gotten me grief, so this is another reason for being much less helpful to people emailing CRU. This goes against my nature, but I’ve been driven to it. You’d better not say this, otherwise McIntyre will request the emails where to prove I’ve been unhelpful! I should have just said to the GA person – use GHCN, like I do to everyone else.

The idea that Jones had “been driven” to being obstructive was a legend that was readily adopted by the climate establishment. But it was untrue. Somewhere along the way – perhaps in response to Mann’s encouragement, as Mosher and Fuller argue, or perhaps on his own – Jones chose , using his own free will, to withhold data from potential critics, – a decision notoriously explained in his Feb 2005 email to Warwick Hughes (shortly after the publication of McIntyre and McKitrick 2005) as follows:

We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it.

Jones’ next sentence in his email to Heffernan shows that no change from the attitude in his 2005 refusal to Warwick Hughes:

I also don’t see why I should help people I don’t want to work with and who spend most of their time critisising me.

Here Jones candidly admitted the real reason why he had sent data to Georgia Tech and to Mann and Rutherford, but had refused to send the data to Warwick Hughes, Willis Eschenbach or me. It had nothing to do with confidentiality agreements from the 1980s that had been mysteriously “lost” during an office move. It had everything to do with Jones’ belief that he had the right to send data to pals and to refuse data to critics.

Jones’ failure to appreciate that publication of scientific articles entailed an obligation to provide data to potential critics, as well as to pals is exemplified in his subsequent whinging complaint:

Years ago I did send much paleo data to McIntyre but have also had nothing but criticism on his blog ever since. As I said, this criticism on blog sites is not the way to do science. If they want to engage, they have to converse in civil tones, and if people don’t want to work with them, they have to respect that and live with it.

There are multiple problems with this paragraph. Yes, Jones had sent me some paleo data, but it was untrue that he had “nothing but criticism” at Climate Audit ever since. On a number of occasions, I had complimented Jones for being somewhat responsive to requests.

A more serious issue is Jones’ view that merely providing data that had been used in an article relied upon by IPCC was a special favor that placed me under an obligation to refrain from criticism of the article. It was my view then (and remains my view) that archiving of supporting data was an obligation of the author, rather than an act of grace.

Nor do I accept Jones’ premise that asking for data constitutes a request to “work with” the originators of the data. It is no such thing. A request for data is simply a request for data. While much emphasis has been placed on FOI, FOI is really a last resort. An obligation to produce supporting scientific data arises on a number of grounds: journal policies; policies of funding agencies; policies of the institution of employment (including FOI); and, most notably, as an obligation of the scientific method.

Jones was criticized not because offence had been taken about Jones not wanting to “work with” the person requesting the data, but because Jones’ failure to provide data (particularly in the context of the repugnant explanation to Warwick Hughes) was not in compliance with any or all of the above obligations to provide data, a problem exacerbated by fabricated and untrue excuses by CRU and the University of East Anglia to FOI requests.

In his email exchange with Heffernan, Jones did not mention the supposed confidentiality agreements that he had long ago planned to “hide behind” – a strategy set out within a few weeks of the introduction of the FOI act in 2005 (see Climategate 1.0 email 490. 1107454306.txt).

Heffernan sent her planned quotes to Jones (Aug 10 – 3497) telling Jones to “shout asap if you see any problems” – a courtesy not extended to me, though I was also featured in the interview.

On Aug 12, Heffernan published an article and associated blog post that were instrumental in the creation of the FOI Inundation Myth, casting Jones as a heroic modern-day Noah.

Heffernan’s article and blog post conspicuously omitted Jones’ petulant desire not to be criticized. Instead, Heffernan attributed Jones’ refusal to an imaginary inundation of requests commencing in 2002:

Why won’t Jones give McIntyre the data?

Jones says that he tried to help when he first received data requests from McIntyre back in 2002, but says that he soon became inundated with requests that he could not fulfill, or that he did not have the time to respond to.

This claim was a total fabrication. After my (successful) 2002 request, I had no further contact with Jones until 2004, when I requested some unavailable data used in Jones et al 1998 (a legitimate, unobtrusive and successful request.) In the Climategate correspondence between Jones and Heffernan (thus far), Jones himself did not advance the “inundation” legend. Perhaps Jones himself introduced the legend in a presently unavailable email or in a telephone interview and the legend was uncritically accepted by Heffernan. It also seems possible that Heffernan herself contributed to the development of an explanation of Jones’ conduct that was more dignified than Jones’ petulance. Although there wasn’t a shred of evidence for Heffernan’s “inundation” story, it quickly became a widely accepted legend in the climate “community”.

Nature re-iterated Jones’ untrue claim that CRU had entered into confidentiality agreements that prohibited sending data to non-academics – a claim that East Anglia itself later abandoned – (see e.g. their response to my appeal here).

He [Jones] says that, in some cases, he simply couldn’t hand over entire data sets because of long-standing confidentiality agreements with other nations that restrict their use… Although these data are made available in a processed format that shows the global trend, access to the raw data is restricted to academics…

Nature also regurgitated Jones’ untrue claim that their ability to provide data was restricted by confidentiality agreements “mislaid” during an office move in the late 1980s – a claim also later abandoned by the University of East Anglia:

Jones says he can’t fulfil the requests because of confidentiality agreements signed in the 1990s with some nations, including Spain, Germany, Bahrain and Norway, that restrict the data to academic use. In some cases, says Jones, the agreements were made verbally, and in others the written records were mislaid during a move.

Nature falsely claimed that I was “especially aggrieved” that Jones had sent the data to Peter Webster and, in a remarkable conflation of cause-and-effect, fostered the legend that “as a result of this” [in the context, the July 2009 FOI requests], Jones had become “markedly less responsive”:

The dispute is likely to continue for some time. McIntyre is especially aggrieved that Peter Webster, a hurricane expert at the Georgia Institute of Technology in Atlanta, was recently provided with data that had been refused to him. Webster says his team was given the station data for a very specific request that will result in a joint publication with Jones. “Reasonable requests should be fulfilled because making data available advances science,” says Webster, “but it has to be an authentic request because otherwise you’d be swamped.”

Indeed, Jones says he has become “markedly less responsive to the public over the past few years as a result of this”.

More disinformation. I wasn’t “aggrieved” in the least that Jones had sent station data to Peter Webster. My sentiment was almost exactly the opposite. Jones’ action provided an opportunity to test CRU’s so-called confidentiality agreements.

Heffernan’s final sentence (which had been cleared with Jones) created a confusion between cause and effect that was quickly adopted by the climate community. In context, “this” (in the phrase “as a result of this”) could only be construed as the July 2009 FOI campaign seeking the mysterious confidentiality agreements. These requests were made in late July 2009 and were the result of Jones’ unresponsiveness “over the past few years”, rather than the cause of his prior unresponsiveness.

This non-sequitur appears to have originated with Heffernan rather than Jones, as the phrase occurred first in Heffernan’s email 3497 but not in Jones’ prior email interview (797).

In conclusion, the role of both Nature and Nature reporter Olive Heffernan is disquieting. Nature suppressed Jones’ candid admission that he simply didn’t see why he should provide data to people that wanted to criticize him and substituted a fabricated explanation more suitable to a hero of the revolution. Had Nature placed Jones’ actual answers on the record, the evolution of events might have been different. In this episode, Nature news reporting unfortunately seems more concerned about hagiography of its principal constituency (institutional scientists) than with accurate reporting.

Ross McKitrick, in his non-climate life, writes from time to time on particulate matter pollution in Ontario. The Toronto Globe and Mail ran a a story a few days ago about Toronto in 1912, showing the picture at left in its print edition. The amount of pollution looks like some present-day images of Chinese cities. It is also a challenge to those who think that pollution in Toronto (or similar cities) is at “unprecedented” levels.

In the late 19th century and early 20th century, Toronto’s primary form of home heating was coal. There was a substantial changeover to oil heating in the mid-20th century. The trans-Canada natural gas line reached Toronto in the 1950s and Toronto’s home heating is now mostly natural gas. Ontario’s history on this issue is probably similar to many North American (and perhaps European) jurisdictions.

I traced the image to a recent blog article here, which had other remarkable images. Out of the corner of my eye, I noticed that one of the commenters at the blog mentioned a story about his Aunt Molly ( I also have an Aunt Molly). He went on to mention the delivery of a coal stoker to his grandmother’s house on Cornish Road (where my grandmother McIntyre lived). (The commenter was one of my cousins.)

Even in the 1950s when I was a boy, there was still so much particulate matter in the air that your collars would get black after a day at school. “Ring around the collar” was the slogan of a popular detergent of the day. The blog showed the following interesting comparison of the amount of grime on Toronto buildings in the 1950s and 1960s versus today.

Confederation Life Building, Toronto. left -then; right – now.

A couple of other pictures that interested me from another blog thread.

This thread also has pictures of winter skating in Toronto in the early 20th century on the Don River and Toronto Harbour – activities that are presently as inconceivable as winter skating on the Thames.

In today’s post, I continue my re-appraisal of various untrue statements made by the University of East Anglia in order to avoid disclosure of CRUTEM station data. I do not consider motives at this time. Also see preceding posts here, here.

In East Anglia’s response to July 2009 FOI requests for alleged confidentiality agreements (here) , CRU stated that, since the 1980s, they had entered into confidentiality agrements that prohibited them from providing station data to third parties, but were unable to “locate” any such agreements noting that they had “moved offices several times during the 1980s”:

Since the early 1980s, some NMSs, other organizations and individual scientists have given or sold us (see Hulme, 1994, for a summary of European data collection efforts) additional data for inclusion in the gridded datasets, often on the understanding that the data are only used for academic purposes with the full permission of the NMSs, organizations and scientists and the original station data are not passed onto third parties. Below we list the agreements that we still hold. We know that there were others, but cannot locate them, possibly as we’ve moved offices several times during the 1980s. Some date back at least 20 years. Additional agreements are unwritten and relate to partnerships we’ve made with scientists around the world and visitors to the CRU over this period. In some of the examples given, it can be clearly seen that our requests for data from NMSs have always stated that we would not make the data available to third parties. We included such statements as standard from the 1980s, as that is what many NMSs requested.

In East Anglia’s submission opposing appeals by Jonathan Jones and Don Keiller (e.g. here), they made the even stronger claim that national meteorological services(NMSs) “invariably” released information only under licences that prohibited transfer to third parties:

It was invariably the case that most NMSs only released information under licences, both written and verbal, that prohibited the further transfer of the information. There was no standard form for such licences but they were all similar in that they prohibit the onward transmission of the data to third parties.

CRU’s longstanding relationship with the US Department of Energy and its transmission of station data to the US Department of Energy (which placed CRU’s station data online (e.g. here) is inconsistent with their claims that they entered into binding confidentiality agreements in the 1980s. In my 2009 appeal, I confronted East Anglia with this relationship (a matter that they discussed in email 2929).

Their refusal of my appeal was dated Nov 12, 2009 (one day before the last email in the Climategate dossier) and was sent to me on Nov 18, 2009. While the dates of these refusals have attracted some commentary, relatively little attention has been paid to the content of the refusal (reported at CA on Nov 21, 2009 here) in which East Anglia stated that the “restrictions” applying to the station data only arose after CRU provided data to the US Department of Energy (along the lines of a scenario that I contemplated in a CA post of Aug 4, 2009 entitled Dr Phil, Confidential Agent):

In regards the information provided to the US Department of Energy, my investigation has revealed that this was done in the early 1990s prior to the imposition of the restrictions now pertaining to the data pursuant to a contractual obligation at the time. Therefore, the analogy you are drawing does not apply to the data that is the subject of this request.

Despite this explicit admission, CRU’s claims that they have been subject to longstanding confidentiality agreements have been accepted by Nature and the climate science community, which has, in turn, denounced CRU critics for not believing in the existence of actionable CRU confidentiality agreements from the 1980s.

In preparing today’s post, I considered not merely the (limited but relevant) new information from Climategate 2.0, but also re-examined contemporary (1980s, 1990s) statements about provenance of station data. Prior to 2004, there is little to no evidence of the commitment to secrecy said by East Anglia to have been “invariably the case”. Quite the contrary. Prior to 2004, there is considerable evidence of a longstanding practice of making station data publicly available (both by GHCN and CRU).

World Weather Records
World Weather Records was (and is) a longstanding international effort in which station data (and metadata) have been published and, when the technology became available, digitally archived.

TD9644 , published in 1991, described its origin as follows:

In 1923, the International Meteorological Conference Committee, convened in Utrecht, Netherlands offered initial justification for the creation of WWR:

“…the conference thinks that publication of long and homogeneous series of observations in the form of monthly means of pressure, temperature and rainfall would be of the highest importance for the study of the general circulation of the atmosphere.”

In response to the Conference’s findings, the Smithsonian Institution of the United States sponsored the first series of WWR, edited by H. Helm Clayton and published in a single volume (1196 pages) in 1927. The publication included full period of record through 1920 of monthly means of temperature, pressure (station and sea-level) and precipitation for selected global stations.

According to TD9644, data was publicly available in 1991:

All available data are both published in the volume sets and digitally archived through NCDC’s tape library under DSI-9644.

Original World Weather Records report (some of which have been scanned and are available online) commendably describe the provenance of each record (something later said by CRU to be impracticable), e.g. the following:

The station names, elevations, and coordinates given in the table headings were supplied by the General Directory of Meteorology of Uruguay and were in use at the end of 1970.

World Weather Records continue to be published and are online at UCAR here.

Jones et al 1985-1991 and CRUTEM1 (1994)

Jones’ original compilation of station data, funded by the US Department of Energy, was described in three technical reports: Bradley et al 1985 (out of print and not online other than an excerpt here), TR022 (Northern Hemisphere) online here and TR027 Southern Hemisphere (online here), and two less-informative academic articles. The original version of NH data (ndp012) is no longer online, but a 1991 update is online here.

These documents indicate almost total reliance on prior World Weather Records compilations, plus archives at the UK Met Office, all of which had been made publicly available (contradicting later CRU claims of a longstanding practice of confidentiality agerements.) Jones et al 1985 (TR022) described the provenance of Northern Hemisphere station data as follows:

Most studies of global or hemispheric temperature fluctuations have relied upon the compilations of station data in World Weather Records… Bradley et al (1985) have added considerably to the WWR data using material available in published and manuscript form in meteorological archives, particularly those of the UK Meteorological Office… Full details of these improvements in station coverage are given in Bradley et al 1985. The most important improvements in coverage occur over parts of the Soviet Union and northern Europe particularly before 1881…Further improvements in coverage have also been made for the twentieth century particularly over northern Africa before 1940 and over the Peoples Republic of China…Details of each of the 2666 stations in the data bank are documented in Appendices using the formats described in Goodess et al 1985 and Bradley et al 1985.

TR027 provided similar statements about Southern Hemisphere data. It reported the compilation of 610 station records of which 293 were used in the grid. Only one meteorological service (Peru) was mentioned. In passing, two of the authors cited (Pittock 1980; Salinger 1981) turn up many years later as leaders of the campaign to censure Climate Research for the publication of Soon and Baliunas 2003.

The basic source of station air temperature data for the Southern Hemisphere land masses is the set of volumes of World Weather Records (WWR) (Smithsonian Institution, 1927, 1934, 1947, and U.S Weather Bureau, 1959–1982; available in digitized form from the National Center for Atmospheric Research (NCAR), Jenne, 1975). A considerable amount of additional temperature data for Argentina and Chile for the years 1931-60 has recently been added to this set. In WWR, these countries only have data available from 1951 (see Pittock, 1980, for further details).

Searches for data in archives as part of the present project yielded additional data for Indonesia and Australia and for some Pacific Islands, particularly Tahiti. Additional data for New Zealand was found in Salinger (1981). For Peru, the Peruvian Meteorological Service supplied information for about 10 stations covering the 1940s and 1950s. Additional data for Australia was provided by their Bureau of Meteorology. All of these sources are gratefully acknowledged.

In 1991, an update to the gridded data set (ndp020r1) re-iterated this description of provenance of station data, noting that all station data was available online.

The primary sources of these data are the World Weather Records (WWR), published by the Smithsonian Institution and the U.S. Weather Bureau, the archives of the United Kingdom Meteorological Office, and the Monthly Climatic Data for the World, published by the National Climatic Data Center (Asheville, North Carolina). Additional sources are described in Bradley et al. (1985) and in Jones et al. (1985 – TR022, 1986a – JCAM, 1986c – TR027, 1986d – JACM). The present updated version of this data set is identical to the earlier version (Jones et al. 1986b) for all records from 1851 through 1978. For the period 1979-1984, the present data set corrects erroneous data using satellite data for some sites and appends data for other sites by adding previously unavailable station data (Jones et al. 1988). The present package also adds monthly surface air temperature anomalies for the period 1985-1990, Antarctic monthly surface air temperature anomalies for the period 1957-1990, as well as the monthly mean temperature records for individual stations (Antarctic stations excluded) that were used to generate the set of gridded anomalies. Individual station data for the Antarctic (stations south of 62.5S) are not presented in this package but are given in Jones and Limbert (1989 -NDP032) and may be obtained free of charge from the Carbon Dioxide Information Analysis Center.

In Climategate 2.0 email 2929 (Sept 8, 2009), Jones said that it was a “requirement” of CRU’s contract with DOE that the data be placed online:

I don’t recall all the facts from that long ago. There is or was a version on a US Dept of Energy website from about 1990. This was a contract requirement at the time.

In 1994, Jones expanded the 1873-station data set to 2891 stations, described in Jones et al 1994 (later CRUTEM1), ascribing the provenance of new data primarily to the projects described in Karl et al 1993 (all of which had been placed online.)

In 1996, CRU placed the CRUTEM1 station data online (/projects/advance10k/cruwlda2.zip), where it remained online until CRU’s removal of the data from its website on or about July 29-30, 2009 (4270). Peterson and Vose (1997), in an article introducing the second version of GHCN, described the 2891-station CRUTEM1 dataset as “widely used”. Doug Hoyt reported that he had downloaded the file in 1999. Simister (2002) and Simister and van de Vliert (2005) report that they downloaded the data in November 2001. I downloaded the file cruwlda2 as late as July 25, 2009 (an event discussed by CRU in Climategate 2.0 4270). In 2002, Jones sent me (then unknown) the cruwlda2 dataset, mentioning that an updated dataset would be published early in 2003.

Jones et al 1999 continued use of CRUTEM1:

Here we use the land station data set developed by Jones [1994]. All 2891 station time series used have been assessed for homogeneity by subjective interstation comparisons performed on a local basis. Many stations were adjusted and some omitted because of anomalous warming trends and/or numerous nonclimatic jumps (complete details are given by Jones et al. [1985, 1986c]).

In 2009, Jones told (2929) East Anglia administrators that the cruwlda2 version wasn’t “complete” and that it was placed on an ftp site because it was “easier” to do this than to send out disks. (Jones did not explain how this related to compliance with supposed “confidentiality” agreements.)

The 1996 version (cruwlda2) wasn’t a complete version and was something we developed for a number of people in EU projects to use. We made these available to people on these projects via our ftp site, as it was easier to do this than sending disks at that time (email attachments were smaller then).

Whether the data was provided by email or ftp was obviously irrelevant to compliance with alleged confidentiality agreements. In addition, contrary to Jones’ claim to his administration, the 1996 version (CRUTEM1 or cruwlda2) was complete.

GHCN
Important light is shed on practices of the period in articles introducing the two GHCN versions.

GHCN-v1 was introduced in 1992 and was described in Vose et al 1992 NDP041. Interestingly, the 1891-station Jones network (ndp020r1) was incorporated as one of the sources for GHCN. ( The “raw” GHCN dataset lists several scribal versions. I’ve looked at a few stations in GHCN-2 and can trace one of the scribal versions to the CRU version. There’s an interesting twist here that I’ll discuss on another occasion.)

In 1997, Peterson and Vose released a second version of GHCN (the one in use until recently). Contrary to the regime of secrecy described by CRU, Peterson and Vose (1997) said that national meteorological organizations were “cooperative and enthusiastic” about contributing data to GHCN:

Because numerous institutions operate weather stations and because no single repository archives all of the data for all stations, we employed five acquisition strategies to maximize the available pool of data: 1) contacting data centers, 2) exploiting personal contacts, 3) tapping related projects, 4) conducting literature searches, and 5) distributing miscellaneous requests. In general, most parties were cooperative and enthusiastic about donating their data to the GHCN initiative, particularly since GHCN is a World Meteorological Organization (WMO) Global Baseline Data Set. As a result, GHCN version 2 contains data from 31 diverse sources (Table 1).

Whereas “personal contacts” between CRU and meteorological organizations were said to be fraught with secrecy, Peterson and Vose (1997) reported that their “personal contacts” had resulted in fresh contributions to public archives:

We also exploited personal contacts by contacting colleagues in the search for potential data sources. For example, scientists who visit or work in conjunction with the authors’ respective institutions often either have data themselves or are able to facilitate the acquisition of data from another party (e.g., by putting the authors in contact with potential sources). This was another extremely productive means of acquiring data, which yielded approximately 10 new datasets.

They reported that these efforts had yielded a much larger data set than the “widely used Jones (1994 “2961-station” dataset (a comment that further confirms the non-secrecy of the Jones 1994 station dataset.

With 7280 stations, GHCN is over twice as large as the widely used Jones (1994) 2961-station mean temperature dataset.

CRUTEM2 (Jones and Moberg 2003)

As noted above, in East Anglia’s rejection of my appeal in November 2009, they said that “restrictions” applying to the data arose after the data had been sent to the US Department of Energy. Here East Anglia was attempting to rationalize Jones’ admission (2929), in which, after admitting that he had sent station data to the US Department of Energy under a “contractual requirement”, claimed that the “restrictions” arose in the mid-to-late 1990s:

There is or was a version on a US Dept of Energy website from about 1990. This was a contract requirement at the time. Much extra data has been added since then, and this is what the restrictions refer to from the mid-to-late 1990s.

Needless to say, this position is totally inconsistent with their previous claim that they had entered into agreements in the 1980s and had lost the agreements during office moves during the 1980s.

It also raises other interesting issues that have not been examined to date: if the only relevant “restrictions” arose in the mid-to-late 1990s, this is well after the “office moves” of the 1980s. It is implausible, to say the least, that CRU should have no record of any communications with NMSs in the late 1990s relating to the confidentiality of station data. It’s hard to believe that data arising from such agreements would not have been transmitted electronically.

Jones and Moberg 2003, which introduced CRUTEM2, briefly discussed CRU’s contact with NMSs (discussed in a prescient August 4, 2009 CA post Dr Phil, Confidential Agent). Jones and Moberg 2003 gave no indication of that it had commenced a new practice of using confidential data. It complimented meteorological organizations for improvements in the quality and quantity of data exchanged and made available “for all to use”:

The majority of the world’s countries have endorsed the initiatives of the Global Climate Observing System (GCOS) to improve the quality and quantity of monthly climate data routinely exchanged. …

The second reason is that several national and other initiatives have also dramatically improved the quantity and quality of monthly mean temperature data available. Several countries have extensively homogenized their entire national holdings, releasing the results for all to use. CRU over the last eight years has also received several national and other temperature datasets.

Jones and Moberg 2003 listed these other datasets without mentioning any confidentiality issues. Whereas earlier editions of CRUTEM had not mentioned direct CRU contact with NMSs, Jones and Moberg reported that CRU had obtained station data through direct contact with some difficult or then pariah regimes: Algeria, Croatia, Iran, Israel, South Africa, Syria, and Taiwan.

CRU has collected a number of temperature records through direct contacts with the NMSs in Algeria, Croatia, Iran, Israel, South Africa, Syria, and Taiwan. Many of these records cover only the period 1961–90, but others extend over the entire twentieth century.

Jones and Moberg (2003) also reported the acquisition of data from less colorful regimes (that presumably attorned to WMO Resolution 40): NORDKLIM data from Denmark, Sweden, Finland, Norway, Iceland, Ireland, the Netherlands, Belgium and the UK itself, as well as additional data from Canada and Australia.

Accordingly, despite the elaborate charade by CRU and the UK Met Office purporting to seek consent from every NMS in the world, there were only a relatively small number of countries (Algeria, Croatia, Iran, Israel, South Africa, Syria and Taiwan) from which “restrictions” from the “mid-to-late 1990s” could have arisen.

2005
Mosher and Fuller (CRUTape Letters) concluded that Jones’ attitude towards provision of data was transformed by Mann’s unrelenting “animus”. They state:

Jones, guided by Mann’s example of animus toward McIntyre and others who question the work of climate scientists, and guided by Mann’s relentless appeals to motive is transformed from a researcher who once promised to share data even in cases where it appeared there might be legal cause to withhold it, to a researcher who cares first about motive, and second about his own reputation and who in the end will use every legal and bureaucratic means to obstruct the release of temperature data and threaten its destruction.

Climategate 1.0 (and especially) 2.0 emails contain an enormous dossier of spite following the publication of MMM2003 in October 2003, exacerbated by the publication of our MM2005a,b in late January 2005. Jones’ emails, as Mosher and Fuller observed, now become even more partisan. In an email in which Jones reported the sending of station data to Mann and Rutherford (despite supposed prohibitions on sending the data to third parties), Jones threatened to delete station data rather than provide it to us.

Just sent loads of station data to Scott [Rutherford]. Make sure he documents everything better this time ! And don’t leave stuff lying around on ftp sites – you never know who is trawling them. The two MMs [McIntyre and Mckitrick] have been after the CRU station data for years. If they ever hear there is a Freedom of Information Act now in the UK, I think I’ll delete the file rather than send to anyone.

Mann replied (apparently alluding to my finding the CENSORED directory):

Yes, we’ve learned our lesson about FTP. We’re going to be very careful in the future what gets put there. Scott really screwed up big time when he established that directory so that Tim could access the data

A few weeks later, Jones made his notorious email to Warwick Hughes (not in the Climategate 1.0 dossier, but known long before Climategate):

Hans Teunisson will reply. He’ll tell you which other people should reply. Hans is “Hans Teunissen” . I should warn you that some data we have we are not supposed top pass on to others. We can pass on the gridded data – which we do. Even if WMO agrees, I will still not pass on the data. We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider.

The email is included as a trailer to a Climategate 2.0 email from Jones to Teunissen (1299- Feb 21, 2005) in which Jones poisoned the well with Teunissen (who never replied to Hughes)

Hans,
This guy pesters me from time to time. I’ve given him your name at WMO/GCOS as someone who’ll reply. We can discuss the merits of my stance in April ! He wants to find fault with some of our station data and by default that the world isn’t warming. If you can just tell him some wmo email addresses that might respond. Why I’m helping him with emails is beyond me ! He wants to discredit what I’ve done. Why the gridded data isn’t good enough is beyond me.
Cheers
Phil

To the extent that Climategate 2.0 emails clarify the issue, in my opinion, they somewhat support Mosher and Fuller’s contention that Jones’ attitude towards data availability changed in response to Mann’s “animus” following publication of McIntyre and McKitrick (2003), though other factors are hardly precluded.

2009
Briefly re-capping 2009 events in light of the above.

In Dr Phil Confidential Agent on August 4, 2009, I had surmised that any governing “restrictions” – if they existed – arose in Jones’ dealings with Algeria, Croatia, Iran, Israel, South Africa, Syria, and Taiwan, and were very limited in application. From the Climategate emails, we know that CRU was closely monitoring Climate Audit and thus were aware of the questions raised in this post.

Nonetheless, in their “small” webpage purporting to respond to FOI requests for confidentiality agreements, East Anglia made the claim (reported above) that they were bound by confidentiality agreements from the 1980s that had been lost during office moves in the late 1980s, a claim that East Anglia resiled from in their November 2009 ruling on my appeal.

A point not discussed at the time, but a loose end: CRU’s misdirection towards the 1980s diverted attention from their unresponsiveness in respect to their communications with Algeria, Croatia, Iran, Israel, South Africa, Syria, and Taiwan in the “mid-to-late 1990s” – communications that would not have been lost in “office moves in the 1980s” and which can hardly have been entirely “verbal”. (The transmission of data, for example, must have been digital.)

On Sep 2, 2009, in my appeal of CRU’s rejection of my FOI request, as noted above, I confronted East Anglia with the inconsistency of their supplying data to the US Department of Energy with their claims that they had entered into binding confidentiality agreements in the 1980s (as well as CRU’s later online publication of CRUTEM1 “cruwlda2″). Jones’ answer in email 2929, as noted above, was both incoherent and unresponsiveness, but did introduce the new claim that the supposed restrictions arose subsequent to the supply of data to DOE i.e. in the mid-to-late 1990s. The latter position was adopted in East Anglia’s refusal of my appeal in November 2009.

Despite all of the above, East Anglia’s representations to the ICO in connection with appeals by Don Keiller and Jonathan Jones reverted to the claim that supply of station data “invariably” was accompanied by a restriction prohibiting the data being sent to a third party. The ICO decision did not address East Anglia’s inconsistent stories, deciding the issue on alternate grounds (that East Anglia had failed to demonstrate an “adverse impact” as set out in the FOI/EIR regulations.)

At this time, CRU and East Anglia have made so many different and inconsistent stories that it is impossible to draw any conclusions on the existence and language of their confidential agreements without a competent investigation to clear the air. It’s too bad that there hasn’t been one.

Finally, although East Anglia conceded to me that there were no “restrictions” arising from agreements from the 1980s and early 1990s, its webpage on data availability stated (and continues to state) the opposite. These untrue statements by the university have contributed to widespread misunderstanding within the climate science “community” of the role of the alleged (and still unseen) “confidentiality agreements” in Jones’ obstruction of both ordinary requests for data and of requests for data under FOI.

Climategate 2.0 emails contain an interesting backstory on East Anglia’s evasion of Steve Mosher’s request for something as simple as university policies that governed entry into confidentiality agreements. Palmer consulted university specialists, receiving an answer that was adverse to the line that they were taking in their CRUtem refusals. Rather than providing this information to Mosher, Palmer adopted a tactic borrowed straight from Sir Humphrey. He pretended that he didn’t understand the question and asked Mosher for clarification – undoubtedly on the off-chance that Mosher would not return the ball. Palmer’s tactic succeeded. They avoided answering the question. The Climategate 2.0 backstory, especially Jones’ candid answers, make fascinating reading, as it shows that there were indeed compulsory university policies which were related to a term of standard employment contracts – information provided directly to Palmer.
.

In response to CRU’s initial refusal (reviewed yesterday here), Mosher sent an FOI request (see July 24 – 3069) that, in addition to requesting confidentiality agreements for five countries, asked for policies and procedures governing East Anglia employees entering into confidentiality agreements:

Dear Mr. Palmer:
Pursuant to the Environmental Information Regulations, I hereby request
the following information …

In addition, I hereby request the following information:
1. A copy of policies and procedures regarding employee responsibilities regarding entering into confidentiality agreements.
2. A copy of policies and procedures regarding employee responsibilities regarding the preservation of written agreements.
3. A copy of policies and procedures regarding employees entering into verbal agreements.
4. A copy of instructions to staff regarding compliance with FOI requests.
I am requesting this information as part of my academic research.
Thank you for your attention,
Steven M. Mosher

About two weeks later (Aug 5 10:39 – 3069), Palmer drew this request to the attention of his most regular correspondents (Jones, Mcgarvie), but also Jonathan Colam-French (who was responsible for hearing FOI appeals), Annie Ogden, Head of Communications (copied on numerous emails), Andrew Mee (presently a “Learning Technologist in Information Services”) and Steve Whitehead (presently an Assistant Purchasing Officer). Palmer requested assistance:

Folks,
The next ‘other’ request relating the the CRU agreements & data.

The first part of the query will be answered in line with the answer given to other requesters for the agreements.

In regards the second part, I will need some assistance as noted below

1. A copy of policies and procedures regarding employee responsibilities regarding entering into confidentiality agreements.

Steve, do we have any contracting policy on this? Phil – anything with CRU on
responsibilities regarding entering agreements on behalf of CRU? I don’t think we wish to state that we don’t have any policies or procedures in place, but I’m not sure what to actually put here…

2. A copy of policies and procedures regarding employee responsibilities regarding the preservation of written agreements.

Ah, records management rears it’s head….We have a general statement on our website regarding our responsibilities for RM but we do lack any overarching records retention schedule or policy – Phil, does CRU have anything in-house?

3. A copy of policies and procedures regarding employees entering into verbal agreements.
See question 1; same issue here although more likely to have a `nil’ response here – consequences of that?

4. A copy of instructions to staff regarding compliance with FOI requests.

We have web guidance that can be referred to, and a brochure that I distribute that could go here…. and a statement regarding the training on offer
Cheers, Dave

Jones, with his characteristic promptness, responded two hours later (Aug 5 12:15 – 856) to all of the parties on the original email. Jones said that the “Registry” went through their agreements, but that they “sometimes ignored” Registry advice. Palmer had noted that there were responsive brochures and documents, but Jones warned him that he “wasn’t sure that [Palmer] should go down that route” i.e. provide Mosher with responsive documents, a direction that Palmer heeded.

(Jones inline answers are italicized below).

Subject: Re: FW: FOI/EIR request [FOI_09-69]
Dave,
A few responses inline
Cheers
Phil

At 11:52 05/08/2009, Palmer Dave Mr (LIB) wrote:
Folks,
The next ‘other’ request relating the the CRU agreements & data.
The first part of the query will be answered in line with the answer given to other requesters for the agreements.

In regards the second part, I will need some assistance as noted below

1. A copy of policies and procedures regarding employee responsibilities regarding entering into confidentiality agreements.
Steve, do we have any contracting policy on this? Phil – anything with CRU on responsibilities regarding entering agreements on behalf of CRU? I don’t think we wish to state that we don’t have any policies or procedures in place, but I’m not sure what to actually put here…

I don’t think there is anything – if there is I’ve never seen it. People in
CRU (not just me) enter into agreements about data and/or writing papers and getting involved in projects. UEA signs research contracts for us. UEA employees do the work, but UEA administers the grant. The various agreements that UEA signs may say things about data access, but it will vary depending on the funding body. Some are more stringent than others. The Registry goes through these. They mostly help the researchers by not letting ourselves sign away any rights and IPR. We do sometime ignore the Registry advice, preferring to fall back on the verbal agreements we have with the funders (their project officer). If we ever have a problem, we probably wouldn’t work with them again. This has happened with some scientists I have collaborated with in the past.

2. A copy of policies and procedures regarding employee responsibilities regarding the preservation of written agreements.
Ah, records management rears its head. We have a general statement on our website regarding our responsibilities for RM but we do lack any overarching records retention schedule or policy Phil, does CRU have anything in-house?

CRU has nothing in this regard.

3. A copy of policies and procedures regarding employees entering into verbal agreements.
See question 1; same issue here although more likely to have a nil response here -consequences of that?

As said in the 2 pager we’re working on, we put some of the agreements in the letters we wrote to Met Services requesting data (some of which we paid for). There has been a lot of time and effort gone into making these contacts. It seems as though this counts for nothing. Again – unlikely to be anything. People agree things with other academics at meetings. This is how science works.

4. A copy of instructions to staff regarding compliance with FOI requests.
We have web guidance that can be referred to, and a brochure that I distribute
that could go here. and a statement regarding the training on offer

I’m not sure you want to go down this route!

Palmer immediately forwarded the email thread with Jones’ troubling answer to two other UEA officials, Matthew Hume, (presently Commercialisation & Patents Administrator) and Alan Walker, Research Services Manager (see Research Enterprise and Engagement organization chart here.)

Subject: FW: FOI/EIR request [FOI_09-69]
Importance: High
Matt,
Please note Phil Jones’ response to question 1 below [about "ignoring" Registry advice] – would REE [Research Enterprise & Engagement] have anything that would be relevant to this request?

A bit of context – in response to a rejection of a request for data, we have received over 50 requests for agreements, data and a combination thereof in relation to data sets that CRU maintains/holds. This is pretty high profile and has been noted in blogs in the Guardian and Telegraph as well as in the source of all of this (see: [2]http://www.climateaudit.org). Be assure that whatever we state in response to this request is likely to be on the web, shared and very public within hours of sending….

We have a request from another individual exactly the same as below so there will be multiple recipients of the answer we give. Our deadline for a response is 21 August but as I’m on hols commencing 17 August, the ‘effective’ deadline is 14 August.
Cheers, Dave

Hume wrote back to Palmer immediately cc Walker (Aug 5 15:16 – 856) saying that they had had a “good laugh” at Jones’ answer and recommended that Palmer withhold this information from Mosher. Hume told Palmer that the University had clear policies that prohibited staff from entering into confidentiality agreements and which required all confidentiality agreements to be signed and kept on file:

Subject: RE: FOI/EIR request [FOI_09-69]
Hi Dave,
We all just had a very good laugh at Phil’s comment “We do sometime ignore the Registry advice”… If this is going to have the kind of publicity that you suggest, I would prefer if you do not quote ANY of his answer to question 1.

The UEA actually has a very strict policy on entering into confidentiality agreements, however as Phil so blithely admits, a handful of academics take it upon themselves to foul things up!

As you will note from points 1 & 2 of our policy; no UEA employee, except members of our office, has the right to sign anything on behalf of the university – the problem is that funders/other parties can be sneaky by sending the agreement in the name of the academic.

Our policy is:-

Someone from the Commercialisation & Enterprise Team should approve and sign all Confidentiality Agreements:

only our staff have the legal authority to sign agreements on behalf of the University

all agreements should be between the University of East Anglia and the party requesting the agreement (not an individual academic or school)

we will negotiate with the other party on any issues within the document that may be contentious by doing this we will ensure you the best protection of your IP rights

(In special circumstances, authorisation may be obtained from the Commercialisation & Enterprise Team allowing you to sign the agreement yourself. Such authorisation must always be obtained in advance, will only be valid for a specific instance, and the standard university agreement must be used without amendment – unless we have authorised an amendment)

In all cases, a copy of the fully signed confidentiality agreement must be retained
in our office.

FYI – we are currently finishing off the final touches to our new intranet pages – there will be a page on CDA’s with this info on it. Also, I am away on holiday next week (10th -14th), so if you do any more info on our policy regarding agreements etc, please contact Anne Donaldson, one of our Commercialisation Manages
Thanks
Matt.

Within minutes, Palmer responded (Aug 5 15:23 856) telling Hume and Walker that Hume had presented Palmer with a “conundrum”.

Subject: RE: FOI/EIR request [FOI_09-69]
Matt,
Thanks very much for this… You have given me a bit of a conundrum on how to respond but we do at least have something to work with….

What policy are you actually quoting from and is it publicly available? As the request was for the entire policy, is there any issue with making the policy publicly available? If the policy in regards confidentiality agreements is within a larger document with unrelated material, I am happy to quote but I do think we will need to provide a proper citation…
Cheers, Dave

Palmer immediately (Aug 05 15:32 856.) wrote to his original correspondents (Jones, Colam-French, Mcgarvie, Ogden) explaining the conundrum, wondering whether the policy had been in effect when the original “agreements” had been entered into:

Subject: FW: FOI/EIR request [FOI_09-69]

Folks,
In response to one of Phil’s earlier emails regarding any policies regarding entering into confidentiality agreements etc, I sent a query to REE to determine what relevant information they might have… and received the below response to which I have responded as you can see…

This does present something of a ‘issue’ in terms of drafting a response and dealing with any potential follow up request/query regarding our practices in this regard. I wonder if whether said policy was in force at the time the agreements were entered into would be a way around this… the request is for current policies clearly…. I will enquire further with Matt Hume….
Cheers, Dave

Jones then wrote to Matthew Hume (Aug 5 16:58 856). Jones told Hume that his comments had been “tongue in cheek” and that the agreements in question were “not really confidentiality agreements” – an astonishing statement given the line taken with respect to FOI requesters. Jones went on to say that “there is never any obligation on CRU or UEA” arising from these agreements which are “generally about agreeing to work together on something”:

subject: My earlier comments
Hi Matt,
My comments were tongue in cheek! The agreements that we’re talking about are not really confidentiality agreements that you’re talking about. Lots are unwritten agreements that we make scientist to scientist. Where there are written agreements they are signed between me (or previous Director’s of CRU) with other academic institutions, which were not with their central administration (but again a sub part). CRU doesn’t initiate these, but if the other side wants it and it will help us do some work then we go ahead and sign. There is never any obligation on CRU or UEA. They are generally about agreeing to work together on something.

The agreements Dave is talking about are ones that relate to us not making climate data available to third parties, which we have got from a National Met Service.

Jones went on to complain that the problem was not CRU’s disregard for university and public policy, bu that “blogsites” have alloed “climate change deniers” to find one another:

FOI is causing us a lot of problems in CRU and even more for Dave, as he has to respond to them all.

It would be good if UEA went along with any other Universities who might be lobbying to remove academic research activities from FOI. FOI is having an impact on my research productivity. I also write references for people leaving CRU, students and others. If I have to write a poor one, I make sure I get the truth to the recipient in a phone call. I’m also much less helpful responding to members of the public who email CRU regularly than I was 2-3 years ago. I’ve seen some of what I considered private and frank emails appear on websites.

Issue here is blogsites have allowed these climate change deniers to find one another around the world.

Hume responded the next morning (Aug 06 09:24 2274.), suggested that Jones would be able to deflect criticism about the confidentiality agreements by showing that university policies “removed” the decision to enter the agreements from Jones’ control. Hume noted to Jones that a term of the university’s employment contract with staff prevented staff from entering into contracts without approval, observing that he had passed this information on to Palmer (this email is not in the Climategate dossier so far):

Hi Phil,
My apologies – no offence meant – your comment amused us.

I know both you & Dave have been coming under a lot of flack from these people, and I agree the FOI rules are a pain – unfortunately they are rules we have chosen to abide by…

Myself, and the REE office, totally support you and your right to protect your research – especially when it is questioned by people who’s intentions are so antagonistic and destructive. And I surmised that these people are trying to gather information that you/the university has agreed to keep confidential. Maybe by showing these people that we remove that responsibility for this out of your control – i.e. that you were not the person who signed these agreements; but they were signed on behalf of the University; then they will have to re-think about contesting them?

I passed another paragraph onto Dave last night. This is from the employment contracts that all staff agree to when working here:

“Members of the ATR/ATS Staff shall not, in connection with any invention, patent, process or manufacture, have authority to make representations on behalf of the University, or to enter into any contract or to be concerned in any transactions on behalf of the University whatsoever without the express consent of the Council.”

It may or may not be appropriate to answer the question posed; but hopefully Dave will be able to make something with it. If I can be of any more help, please feel free to ask.
Regards
Matt.

Jones wrote back to Hume (Aug 6 10:27 2274.) warning that if Palmer goes down the route suggested by Hume (i.e. informing Mosher of the policies and procedures that prohibited Jones from entering into the alleged confidentiality agreements without university approval), this would open the university up to further criticism. Jones also worried about David Holland’s pending appeal for off-balance-sheet IPCC review comments and correspondence:

Subject: RE: My earlier comments
Hi Matt,
Thanks for the thoughts. These people don’t have logic as their strong suit. If Dave goes down the route you suggest, I think they will come back on UEA for not supervising me properly!

There is another FOI/EIR that has gone to the FOI Commissioner. This one is potentially more important as it relates to email correspondence between the authors of a chapter in the last IPCC (Intergovernmental Report on Climate Change) Report.

I think people shouldn’t be able to request my emails just because I happen to work for a government funded University. Perhaps UEA needs a ruling on requests for our emails. …
Cheers
Phil

Presumably the matter continued to be discussed among the parties, but the Climategate 2.0 curtain goes down on this topic.

Nonetheless, Palmer indeed had a conundrum: the university had clear policies prohibiting confidentiality agreements without approval and requiring retention of such agreements; staff contracts even included this requirement as a term of employment.

What to do? What to do?

Palmer’s manouevre was straight out of Sir Humphrey’s playbook and succeeded beyond any reasonable expectation. Although Palmer had all the materials on hand to respond to Mosher’s question, he pretended that he didn’t understand and sent out a “Clarification Request”.

Subject: Freedom of Information Act request (FOI_09-69) – Clarification request

Mr. Mosher,

Attached please find information on the handling of your request received 24 July 2009 for your review and action.

Cheers, Dave Palmer

FREEDOM OF INFORMATION ACT 2000 – INFORMATION REQUEST
(Our Ref: FOI_09-69)
Further to your request for information received on 24 July 2009, I have consulted relevant units within the University and, pursuant to my obligations under section 16 of the Act to provide advice and guidance, I am writing to request clarification of several aspects of your request. Apologies for the delay in responding to you on this matter, but as you may know, we have received a large number of requests for information under the Act recently and it is taking some time to deal with each request.

In your request, you have asked for a copy of policies and procedures regarding employee responsibilities regarding entering into confidentiality agreements and verbal agreements, and for a copy of policies and procedures regarding employee responsibilities regarding the preservation of written agreements.

The University does not have one, overarching policy or procedure regarding entering into confidentiality or verbal agreements. Each division within the University has policies and procedures specific to their area of work. This also applies to the preservation of written agreements. In order to answer your question for all of the University, it is highly likely that we would exceed the statutory appropriate limit of 18 person hours to locate, retrieve & review the requested information.

In order to avoid this situation, I would therefore ask you to clarify what aspect of the University’s work would be the focus of your request for such policies and procedures. For example, given the nature of the other components of your request, are you simply interested in policies in relation to research activities?

Please note that the statutory timescale of 20 working days as defined by the Freedom of Information Act 2000 will be ‘suspended’ until such time as we receive clarification of your request. Once that is received, the ‘clock’ will recommence, your request considered, and you will receive the information requested within the statutory timescale, subject to the information not being exempt or containing a reference to a third party. You will be informed of any exemptions or references to third parties.

I trust this is to your satisfaction and look forward to your reply.
Yours sincerely
David Palmer
Information Policy & Compliance Manager
University of East Anglia

Some assertions in Palmer’s letter seem untrue on their face. Palmer told Mosher that “the University does not have one, overarching policy or procedure regarding entering into confidentiality or verbal agreements. Each division within the University has policies and procedures specific to their area of work. This also applies to the preservation of written agreements. In order to answer your question for all of the University, it is highly likely that we would exceed the statutory appropriate limit of 18 person hours to locate, retrieve & review the requested information.”

However, Matthew Hume’s emails clearly state the opposite – that the University did have “overarching” policies and procedures governing confidentiality and verbal agreements. And governing the “preservation of written agreements”. However, rather than providing this information to Mosher, Palmer pretended that he required “clarification”. Palmer’s manouevre succeeded. Mosher didn’t provide the requested “clarification” and the FOI request remains pending to his day.

Maybe Mosher should send Palmer the “clarification” that he requested.

The FOIA/Mole incident of July 2009 attracted much public interest and somehow seems connected to the subsequent Climategate events, though precisely how (and even whether) remains unclear. The incident was discussed in both Mosher and Fuller’s CRUTape Letters and Fred Pearce’s Climate Files, though not in Andrew Montford’s Hockey Stick Illusion (which was mostly complete prior to Climategate.)

Climategate 1.0 emails provided relatively little context on the CRU/East Anglia side of the incident. Climategate 2.0 changes this situation dramatically, as it contains a rich (though still, at times, frustratingly incomplete) documentation of the university’s side of events. (The Climate Audit side is, of course, exhaustively documented in contemporary posts and comments, starting here and continuing in threads for the next month or so.)

The FOIA/Mole incident has been central in self-serving rationalizations of Climategate events by institutional climate science. They’ve represented the affair as an attempt to “harass” climate scientists, claiming that the events vindicated many prior years of data obstruction. However, institutional climate science and their house organs (e.g. Nature) have failed to report the origins of the incident. Its proximate cause was an institutional mendacity incident. In July 2009, CRU had refused a FOI request for station data with the flagrantly untrue claim that language in their alleged confidentiality agreements prevented them from sending station data to “non-academics” (while not preventing them from sending the same data to sympathetic academics:

Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics

This assertion was completely fabricated: there were no agreements that contained the claimed language. Climate Audit readers quickly recognized that the university’s assertion was fabricated and submitted FOI requests to the university for the supposed confidentiality agreements containing the alleged language. The university was unable to produce a single agreement with such language and quickly abandoned this line of refusal in favor of other lines of refusal – these new grounds of refusal still, however, relying on faith in agreements unseen.

The new Climategate 2.0 emails shed fresh light on the process leading to East Anglia’s fabricated claim about language that specifically prohibited the transmission of station data to “non-academics”. The emails show that the untrue claim was not made casually or by accident, but was carefully crafted through a process that included not just Phil Jones and FOI officer David Palmer, but also Michael Mcgarvie, UEA’s Director of Faculty Administration. It also seems unlikely that Mcgarvie would not have reported to his superiors in the Registrar’s office and beyond, but these reports (if they exist) are not in the Climategate dossier.

Although the university’s mendacity in this incident was one of the flashpoints that seems to have precipitated Climategate, it was not investigated by the Muir Russell “inquiry”. To do so would have required examination of Climategate 2.0 (and other) emails. Muir Russell made no effort to examine such emails until late in the review process. These late and feeble efforts were then resisted by Acton and the university administration and nothing came of them. Muir Russell nonetheless proclaimed in his press conference that, while CRU was “unhelpful” at times, the “rigour and honesty” of the scientists in question was “not in doubt”.

In preparing today’s review, I’ve made my own “edition” of the emails in which I’ve arranged threaded emails in chronological order. See here – link to follow.

I’ll start today’s chronology of events somewhat in the middle.

The Request

Contrary to a common view, prior to mid-2009, I’d never made an FOI request for the CRUtem data set. As I’ve said on many occasions, my interest in temperature data is very secondary to my interest in proxy reconstructions and I’ve been well aware that time spent on it would pre-empt topics that interested me more and which seemed more scientifically contentious.

However, I firmly supported the principle that data being relied upon by IPCC should be transparent and I had written about efforts of Warwick Hughes and Willis Eschenbach with considerable interest and had done what I could to support their efforts. In late 2006, Willis had initiated the first use of FOI legislation to overcome a non-transparency that the wider scientific community should itself have objected to. After much obfuscation by CRU, Willis managed to extract a list of stations, but not the station data as used by CRU. The progress was reported from time to time at CA.

In May 2009, I noticed that the UK Met Office held a version of the CRU station data, which was cited on one of their webpages. As someone with some familiarity with confidentiality agreements and (by then) FOI regulations, it occurred to me that FOI provisions applying to the Met Office (a government agency) might differ in some respect from the FOI provisions applying to the University of East Anglia. For example, CRU might have sent station data to the Met Office without a confidentiality agreement between CRU and the Met Office. If so, the Met Office might not be able to rely on exemptions available to CRU. In fact, this surmise proved correct though the Met Office refused anyway. Climategate 2.0 emails (not ones reviewed today) show that the Met Office understood this point and looked long, hard and unsuccessfully for such an agreement, which appears not to have existed.

In June 2009, on a blog post discussing my Met Office request, Peter Webster of Georgia Tech commented that he had readily obtained station data from CRU earlier that year (Webster’s comment was quoted by Jones in 4531):

Steve,
We have asked Phil Jones for data so that we could compare the synthesized surface temperature with actual station data. Jones has provided everything that we have asked for. This is for our study of the 1930/40 climate bump that is ongoing. Alas, these things take time. But my experience has been quite different to yours.

As you know, I have often complained that the right wing and the left wing (the absolutists of AGHW and those who do not have a bar of it) have forced us into corners in which we are not comfortable. If there is to be reasonable resolution of the climate GWH issues and the fidelity of data (both critical and reasonable questions?) I think that the questions and opinions can’t be shouted from one corner or the other…

best regards
Peter W

I immediately (June 25) sent a FOI request to East Anglia for the precisely the same data (4531).

CRU Plans the Refusal

The following day (June 26), Palmer forwarded my request (4531) to Jones, Mcgarvie and Osborn as follows:

Gents,
A request from Mr. McIntyre under EIR that arrived today. Response due by 24 July. I have acknowledged the request and confirmed that we will be handling this under EIR. Any concerns with this request? Any need for clarification?
Cheers, Dave ______________________________________________________________________________
From: Steve McIntyre
Sent: Friday, June 26, 2009 4:45 AM
To: Palmer Dave Mr (LIB)
Subject: Environmental Information Regulations
Dear Mr Palmer,
Pursuant to the Environmental Information Regulations, I hereby request a copy of any digital version of the CRUTEM station data set that has been sent from CRU to Peter Webster and/or any other person at Georgia Tech between January 1, 2007 and Jun 25, 2009. Thank you for your attention,
Stephen McIntyre

Jones immediately explained (June 26 – 4531) to Palmer (cc McGarvie, Osborn) that he sent the data to Georgia Tech in a “personal email”, asserting that “McIntyre has no right to request the data in a personal email”:

Dave,
I sent some of the station data to a Jun Jian at Georgia Tech on 15 Jan 2009. I see now that Peter Webster was a recipient on the email. I also see from looking at Climate Audit that this request results from Peter saying on CA that he’s not had any difficulty getting data from CRU (see what he said below on June 24).

I regard this as a personal email between me and this group at Georgia Tech. So, McIntyre has no right to request the data in a personal email. I only sent a small part of the dataset anyway. They asked for a specific set and said what they were going to do with the data.
Cheers
Phil

That an IPCC Coordinating Lead Author could regard distribution of CRU station data as “personal” favor rather than an obligation is an issue that the climate community has closed its eyes to.

Palmer rather gingerly explained (June 26 – 1320) to Jones (cc Mcgarvie, Osborn) that data did not become “personal” merely because Jones said so and that they needed a “valid” exemption to avoid disclosure. Palmer observed that Jones’ already sending the data to a third party presented problems; Palmer said that he was “having difficulty” making a case for not sending the data to me that had already been sent to Georgia Tech, a misgiving that he and others should have paid more attention to:

Phil [et al],
The fact that information is within an email that you consider ‘personal’ does not render the information itself personal. In order to not disclose information under EIR, we need to have a valid exception, and then also pass a public interest test that shows that the public interest is better served by non-disclosure than disclosure.

I will have a think about what exceptions are available to us, but, at this moment I am having difficulty making a case for any that would apply here. The other issue is passing the public interest test – we would, I presume be relying on some sort of public interest in preserving the confidentiality of communications between academic colleagues but there is no guarantee that the ICO would uphold this.

I will get back to you next week on this one….
Cheers, Dave

Later that day (June 26 – 2663), Mcgarvie worried that providing data would set a bad precedent for other cases, particularly David Holland’s pending appeal of their refusal of IPCC correspondence not in the IPCC archive. Mcgarvie recommended that they invoke one of the exemptions that they had discussed – but only after stretching the time period as long as possible:

Dave et al,
As we are testing EIR with the other climate audit org request relating to communications with other academic colleagues, I think that we would weaken that case if we supplied the information in this case. So I would suggest that we decline this one (at the very end of the time period), with one of the valid reasons that you, Jonathan [Colam-French] and I disucssed, and let him go through appeal.

Happy to discuss further (but not for a couple of weeks since my diary is pretty full next week and the week after).
Regards
Michael McGarvie
Director of Faculty Administration

Jonathan Colam-French, who was responsible for hearing appeals, was involved from time to time in the initial strategy on how to avoid compliance.

A few days later (June 30 – 2663), Palmer reminded Mcgarvie and Jones that refusing station data presented other difficulties relative to refusing emails. Palmer noted that there was no general academic principle opposing the exchange of data – indeed, Palmer observed that archiving of data was even regarded as meritorious as some parts of the academic community – a further point to overcome in constructing a refusal:

Phil/Michael,
I can understand your reluctance to deal with Mr. McIntyre’s request but we do need to have justifiable grounds for claiming an exception under the EIR in order to do so…

To address your point Michael, I think that there might be a difference seen between personal correspondence between academics and actual data which has a life/role outside that correspondence. In regards the public interest test that we have to address, once again, I would think that whilst there is a good argument for protecting the ability of academics to communicate freely and openly, the underlying data that may comprise part of that communication might well fall into another category. One only has to look at the JISC funded projects on national scientific data repositories and exchange to see that there appears to be a perception in the academic community that the exchange & re-use of data is a good thing.

We also have to remember that, much like FOIA, the exception regarding ‘confidentiality’ is in relation to a person providing the information to the organisation – it does not touch upon correspondence from the organisation. That is covered either by ‘internal communications’ exception, or as in the other case with the IPCC, an ‘adverse effect’ on international relations (which I believe to be entirely justifiable)

As you are both quite busy over the next couple of weeks, I would be happy to discuss this further w/c 13 July with you, Michael and verify our approach the following week prior to the deadline of 24 July.
Cheers,. Dave

Jones then searched EIR legislation for exemption ideas. Later that day (June 30-1473), he suggested three ways of refusing the data, one of which was exemption 12(5)(f) – confidentiality. Jones told Palmer and Mcgarvie that “some of the data was supplied to CRU on the grounds that we didn’t pass it on”. (Here Jones was conspicuously silent on how CRU circumvented these supposed conditions when it recently passed data on to Georgia Tech and, for that matter, when it passed data on to the US Department of Energy which then created a public archive of station data as it stood in the early 1990s.)

Dave,
I’ve done something I thought I would never do – I’ve printed off the EIR for 2004!

Here’s a few thoughts.

1. I don’t have the exact data that I sent in January 2009. I’d have to recreate it. The data are part of a larger database. What I’d recreate would be different from what I sent in Jan 2009 (12.4a).
2. The requester has no idea what I sent on January 2009 (12.4c).
3. If I do have to recreate it, then it will contain data where 12.5fi-iii apply. Some of the data was supplied to CRU on the grounds that we didn’t pass it on. These conditions were put on it by some of the National Met Services around the World (including the UK).

On a related matter and back to Michael’s point. The next IPCC process will start in 2010. It is possible that UEA people will be involved in the author writing teams. The members of these teams will be available through IPCC. What is to stop people asking for emails I might write to some or all of these authors. This, in effect, is the purpose of the appeal in the other issue with Keith and Tim and IPCC correspondence.
Cheers
Phil

Palmer wrote back to Jones and Mcgarvie (June 30 – 1473), rejecting Jones’ first two excuses but encouraging the third, with Palmer welcoming Jones to the “‘dark side’ of FOI”. However, Palmer pointed out that confidentiality agreements in themselves would not suffice – they would still have to “overcome the obvious fact that some data was passed to a fellow academic so therefore would need to draw a distinction between that type of disclosure and that requested by the applicant”. This, of course, was the nub of the problem.

Phil,
Ah, now we are getting somewhere…. (and I will turn you to the ‘dark side’ of FOIA/EIR yet! lol)

I don’t’ think Reg. 12(4)(c) will hold water as I think we know exactly what he is asking for but it’s our ability to provide it that is at issue. However, even if we think it is too general, Regulation 9 mandates us to provide advice and assistance to the requester and indeed, subsection (2) specifically notes that if we do feel that 12(4)(a) applies, we must ask the applicant to provide more particulars & to assist the applicant in providing those particulars. I would think it likely that the applicant in this case would simply ask for the entire base file….?

As to point 1 below, if we don’t have the original email nor any record of what was sent, then there may be a case for the application of 12(4)(a). However, being contrary (and that’s part of the job description), Regulation 9 would also raise it’s hoary head here and we would need to tell the applicant out problem in ‘reassembling’ the data. I suspect the outcome would be exactly the same as above; namely a request for the entire base data.

However, point 3 has definite promise. We would have to demonstrate an adverse effect on the interests of the party providing the information/data, and then pass the public interest test, overcoming the presumption of public interest in disclosure. Clearly, if the data was given to us on terms that forbade its further disclosure to persons/instructions that would exclude the applicant (and we have evidence of that), then we can also assume some presumption of adverse effect (although once again, thinking ahead, evidence of this would be useful).

We would have to overcome the obvious fact that some data was passed to a fellow academic so therefore would need to draw a distinction between that type of disclosure and that requested by the applicant.

I do not disagree with your final point which is why I was drawing the potential distinction between data and private correspondence. Our case before the ICO is all about the confidentiality of information coming to us and the adverse effect its disclosure would have on the persons providing it, and the international relations we have with bodies such as the IPCC. (and I think we might have a case under EIR ‘manifestly unreasonable’ grounds as that definition is wider than that for ‘vexatious’ requests under FOIA).
Cheers, Dave

Climategate 2.0 emails are quiet on developments for the next two weeks. (Presumably further information is in the locked tranche.)

The curtain rises again on Thursday, July 16 (and this passed unnoticed at CA at the time) when Andrew Montford sent an FOI request (2786) presciently requesting copies of the confidentiality agreements governing the station data. Montford’s request did not refer to my pending FOI request, though that was obviously in the air, but referred back to claims that Jones had been in correspondence with Keenan in 2007. Palmer notified (2786) Mcgarvie and Jones of this new development and proposed adding this to the agenda of a meeting on FOI that they had scheduled for Monday July 20.

Miraculous Language
At the July 20 meeting, Jones, Mcgarvie and Palmer appear to have agreed on the refusal strategy employed in the July 24 refusal letter a few days later: to reject the FOI request on the (untrue) basis that the supposed confidentiality agreements contained language that prohibited the sending of data to “non-academics”. On June 30, Palmer had been concerned about how to justify not sending data to me when the same data had been sent to third parties at Georgia Tech. It was shall-we-say little short of miraculous that Jones (presumably) reported to Palmer and Mcgarvie that his confidentiality agreements entered into so many years ago contained language dealing with the situation, language that prohibited CRU from sending data to “non-academics”.

Palmer appears not to have challenged Jones’ miraculous claim or carried out the mundane due diligence of examining the supposed agreements (or else events would have unfolded differently). On July 21 (3334), Palmer distributed a draft refusal to Jones and Mcgarvie as they had discussed at the meeting (presumably along the lines of the July 24 refusal). In the covering email (3334), Palmer made a last plea to Jones and Mcgarvie to consider the (logical) alternative of simply providing me the data under conditions that prevented my re-distribution (rather than refusing):

A draft response along the lines discussed yesterday. I would expect an almost immediate appeal of this decision by Mr. McIntyre.

Phil, as your concern is the publication of the requested information, I wonder if a possible alternative is to release it but place conditions on it’s use. This will ONLY work if UEA has some rights in the data itself or in the database. ‘Copyright’ in the contents of a database would require some personal creative input by ourselves to the data or database that would render it different from preceding external versions and ‘original’.

However, even if the contents aren’t ‘original’, there is a ‘database right’ where the contents of the database are assembled as the result of substantial investment in obtaining, verifying, or presenting it’s contents. It is the framework, not the contents, that attracts the rights. These rights exist for 15 years from the completion of the database BUT any substantial change to contents will ‘renew’ the database rights for another 15 years. The owner of database rights has the right to prevent the extraction or reuse of all or a substantial portion of the database.

There is ‘fair dealing’ in database rights to the extent that anyone has a right to extract & reuse an insubstantial portion of the database (not really defined in law but it’s very small) for any purpose, or where the portion is substantial, extract and use data for non-commercial research or private study. What can’t be done is re-issuing this information to the public under a different guise.

The upshot of all of this is that, if we have a ‘database right’ in this information, then we can release it BUT insist on our exclusive right to re-use the information – BUT the issue is actually ‘enforcing’ those rights…… more difficult in practice than in law or theory….

Just thought I would proffer this as an option in place of the refusal and the inevitable appeal.
Cheers, Dave

Palmer’s sensible suggestion was rejected by Jones (780, 3712). On July 24, Palmer sent out the refusal (untruthfully) claiming that language in their confidentiality agreements prevented the sending of data to “non-academics”:

Regulation 12(5)(f) applies because the information requested was received by the University on terms that prevent further transmission to non-academics

As noted above, CRU’s claim about contract language pertaining to “non-academics” was fabricated. As matters would show, there were no agreements. Presumably their strategy was simply to force me into a lengthy appeal process which I might not pursue (a tactic that had been successful with Eschenbach a couple of years earlier.)

However, it’s one thing to be “unhelpful” and quite a different thing to make fabricated claims about their so-called confidentiality agreements. As someone with actual experience with real confidentiality agreements (in the mineral exploration business), I didn’t believe East Anglia’s excuse for a minute. It was inconceivable to me that CRU’s “confidentiality agreements”, even if they existed, contained the claimed specific language about “non-academics”. I immediately (July 24) responded at CA here as follows about this surprising development:

This is the first time that we’ve heard that their supposed confidentiality agreements merely restrict “further transmission to non-academics”. A couple of observations on this. I’m sure that CRU will soon receive a similar request from someone to whom this excuse does not apply.

However, aside from that, there are other troubling aspects to this refusal. If there actually are confidentiality agreements, I would expect the relevant language to be framed in terms of “academic use” as opposed to guild membership i.e. I’d be surprised if the language were framed in terms of institutional affiliation as opposed to use. I’ve published relevant articles in peer reviewed literature, acted as an IPCC reviewer, been cited in IPCC AR4, been invited to present to a NAS panel – my use of data is “academic” by any legal standard.

Secondly, over at the Met Office, they say “it cannot be determined which countries or stations data were given in confidence as records were not kept.” But over at CRU, they purport to “know” nuanced details of the contractual language of the confidentiality agreements – clauses that have the effect of justifying the refusal of the data.

I and other Climate Audit readers were offended not simply by CRU’s refusal to provide data, but by the arrogance of an institution that had no compunction about fabricating excuses for non-compliance. Requesting the alleged confidentiality agreements containing language about “non-academics” was an obvious next step. Recognizing that a request by me as an individual was likely to be marginalized, I invited Climate Audit readers to show the university that others were concerned about CRU obstruction by submitting their own FOI requests for confidentiality agreements. Quickly Climate Audit readers submitted nearly 60 FOI requests for the so-called confidentiality agreements, each reader specifying five different countries. In addition, a number of what East Anglia called “legit academics, rather than the usual suspects” submitted FOI requests for the The data set that had been refused to me.

In effect, Climate Audit readers had staged a protest against institutional mendacity.

Considerable interest was added to the protest by the “Mole Incident” (see here) and CRU’s subsequent deletion of numerous files from their FTP site, including two different versions of the supposedly top secret station data that had been online since 1996 and 2003 respectively. (Yet a third version was online at the US Department of Energy since 1991, CRU having sent the data to the US Department of Energy who had funded them.)

However, those are stories for another day. Today, I”m going to follow the gradual realization by East Anglia administrators that there were no confidentiality agreements containing language specifically restricting the transmission of data to non-academics and their seamless and unapologetic change of argument to one that was less transparently untrue.

On July 28 (490), Palmer outlined a strategy for the various FOI requests. Palmer recommended that they place “any/all [confidentiality] agreements” on the CRU website, while continuing to oppose requests for data. At this time, Palmer was presumably unaware that Jones’ “confidentiality agreements” would not be producible or that they would contain no language restricting access to “non-academics”

Folks,
Just to summarise our approach to the various requests we have received to date that we have agreed:
A. ‘Country’ requests
1. Respond to Montford request as normally – cite s.21, information available (see point 2)
2. Place any/all agreements (or links thereto – Met Office) on the CRU website
3. Acknowledge & respond to all 44? Country requests by citing s.21 and pointing them to the CRU website
B. Data requests
1. Acknowledge requests
2. Deal with as per normal, cite Reg. 12(5)(f) re agreements and Reg. 12(4)(b) ‘manifestly unreasonable’ on the grounds that the data is already available publicly via the ClimateAudit.org website, and note that a format of the data (gridded) already is publicly available
3. Note that raw data is available from the Met Office and other national weather services (also goes to ‘manifestly unreasonable’)
.C. McIntyre appeal
1. Maintain position regarding Reg 12(5)(f) re confidentiality agreements and point him to published versions on website
2. Add ‘manifestly unreasonable’ on basis that he already has the requested information in his possession & is also available elsewhere
3. Handle as per published protocols with initial ‘informal’ approach, followed by review by JCF

D. ‘Other’ requests
1. Acknowledge requests
2. Deal with as usual, citing whatever section is appropriate above to the requested information

E. General points
1. Interaction with any media to be handled by Press Office
2. Approval of transfer to Georgia Tech would be good to find
3. We are NOT citing s.14 for the ‘country’ requests
4. Estimated time to locate ALL agreements regarding data transfer is within the 18 hour appropriate limit
5. Any correspondence to go out will be circulated prior to transmission
I hope I have captured what was agreed – please comment if your understanding is different than mine
Cheers, Dave

The next day (July 29 09:49), Jones agreed (1131) to Palmer’s proposal that the “agreements” be placed online:

Here’s what I propose to do over the next week or two. I will get the agreements scanned and write some text about them and the others that we have had.

A scan of four agreements (none of which pertained to station data) was completed by 11:47 on July 29 (see the document properties of the document placed online on August 11). Later in the day (July 29 17:19 1248902393.txt), in a Climategate 1.0 email, Jones told Peterson of NCDC in the U.S. said that he planned to deal with the FOI requests with a “small [webpage] document”:

Anyway enough of my problems – I have a question for you. I’m going to write a small document for our web site to satisfy (probably the wrong word) the 50 or so FOI/EIR requests we’ve had over the weekend. I will put up the various agreements we have with Met Services.

Jones also consulted (July 30- 1916) with Peter Thorne of the Met Office, who strongly discouraged any attempts by Jones and/or East Anglia to be “opaque” in their responses, a recommendation that East Anglia unwisely disregarded:

Generally the more opaque this is the more work eventually will accrue for you (and us, but mainly you). If we’re forced to wash our laundry we may as well use Daz brilliant white … so be inanely pedantic in the details so that you minimise the angles for attack. It will take longer right now but I suspect in the long run be less work which means you can do science if you can still remember what that is (I vaguely do myself …).

Thorne also recommended that Jones try to locate past evidence of his “championing open access” – not an easy task given Jones’ actual views and prior conduct:

Is there any documentation in GCOS reports we can point to, especially AOPC where we can show that you are championing open access? That would shut them up slightly so is VERY DEFINITELY worth looking for and linking to as it shows you in the correct light. I think some text showing taht CRU is fighting for openness in relevant forums would be a gold dust addition.

By the first week of August, the Climate Audit protest, in combination with the Mole Incident and CRU’s deletion of website documents, had attracted wider attention. Between August 7 and 10, Olive Heffernan, a reporter with a journal (Nature), that sympathized with the trade interests of the climate community, carried out an email interview with Phil Jones, which I’ll discuss on another occasion.

The webpage contemplated in late July was published on August 11 here, prompting an immediate response at Climate Audit here. CRU’s webpage did not include ANY confidentiality agreements concerning station data. Instead, they provided two request letters from 1993-94 for 1961-90 climate normals for 9 variables (to the UK Met Office and Spain) and two response letters regarding 1961-90 climate normals (Bahrain, Norway) – only Bahrain within the tropical zone of the Georgia Tech data. CRU stated that they were unable to locate any actual agreements because they “moved offices”:

Below we list the agreements that we still hold. We know that there were others, but cannot locate them, possibly as we’ve moved offices several times during the 1980s. Some date back at least 20 years. Additional agreements are unwritten and relate to partnerships we’ve made with scientists around the world and visitors to the CRU over this period.

CRU claimed that the language in these letters was representative of the supposed “agreements” that they were unable to find:

Since the early 1980s, some NMSs, other organizations and individual scientists have given or sold us (see Hulme, 1994, for a summary of European data collection efforts) additional data for inclusion in the gridded datasets, often on the understanding that the data are only used for academic purposes with the full permission of the NMSs, organizations and scientists and the original station data are not passed onto third parties. …In some of the examples given, it can be clearly seen that our requests for data from NMSs have always stated that we would not make the data available to third parties. We included such statements as standard from the 1980s, as that is what many NMSs requested.

Even if the documents were to be construed as “actionable” confidentiality agreements (which they weren’t), they contained no language about “non-academics”. To the extent that there were any restrictions, they applied not just to “non academics”, but to all third parties, including not just Georgia Tech, but the US Department of Energy, Mann and Rutherford and perhaps even the Met Office.

CRU also volunteered the embarrassing information that they no longer held the original “raw” data, only “value added data”, (laughably) attributed to limited storage capacity in the 1980s. (This admission provoked much satire and independent criticism):

Since the 1980s, we have merged the data we have received into existing series or begun new ones, so it is impossible to say if all stations within a particular country or if all of an individual record should be freely available. Data storage availability in the 1980s meant that we were not able to keep the multiple sources for some sites, only the station series after adjustment for homogeneity issues. We, therefore, do not hold the original raw data but only the value-added (i.e. quality controlled and homogenized) data.

CRU’s embarrassing document was immediately criticized at CA here. Willis Eschenbach, whose 2007 FOI request for data had been stonewalled, wrote in angrily (Aug 11 – 752) asking that the 2007 refusal be reconsidered, pointing to CRU’s failure to produce a single confidentiality agreement restricting the station data. Jones defended the webpage to Palmer (752), claiming that there were many other faxes that contained restrictive language that he hadn’t included on the webpage because they said the same thing:

He has missed the point. I could have put in loads of the faxes similar to the British Territories one, as all the requests that Mike Hulme and I sent in the mid-1990s included the statement :

The data will not be used unauthorised for any other project and will not be passed onto any third party.

I didn’t include all of these as they just say the same thing. I only included those that reiterated this point when they sent us the data.

This, of course, was inconsistent with statements in the webpage that they had just published. It stated unambiguously that they had listed “the agreements that we still hold.” If Jones had other agreements saying “the same thing”, then he should have said so in such a sensitive document.

Jones added:

UEA is not promoting this dataset as a suitable basis for making billion-dollar decisions on what we should do on regarding the ‘global warming’ supposedly shown by your dataset. This is simply NOT TRUE.

I presume that Palmer must have felt sandbagged when he saw that none of the extant “agreements” contained the language to which Palmer had affixed his name in the refusal letter. Most “non-academics” would have sent Jones a strongly worded WTF letter asking for an explanation. No such letter from Palmer is in the Climategate dossier thus far. Perhaps there’s one in the unopened dossier, perhaps Palmer never sent one.

In any event, East Anglia never again mentioned their fabricated claim about restrictions against “non-academics”.

Instead, East Anglia adopted a new line of refusal. They gave up trying to walk the tightrope between refusing data to potential critics, while giving data to fellow travellers. They now argued that Jones’ still unseen “confidentiality agreements” prevented access not simply to non-academics, but to third parties in general. They now said that the data had been given “in error” to Georgia Tech and that the “same error” should not be repeated. This line was used to refuse requests for data from Ross McKitrick, Jonathan Jones and Don Keiller (and my appeal), using language along the lines of the following:

We do concede that information was provided to Georgia Tech without securing consent of the institutions that provided it, and, upon reflection, this is an action we would not choose to take again. However, having made one error does not, in our eyes, justify making the same error again.

“One error”.

In fact, if there ever were such confidentiality agreements, then CRU had breached them right from the start – by sending the 1991 version of the data to the US Department of Energy which published the station data online; by placing the 1996 version online at CRU as part of the ADVANCE/10K program; and by sending station data out on request (not just to Georgia Tech, but to others, including Mann and Rutherford in 2005 and even to me in 2002 before I was identified as a potential critic).

The inconsistencies in CRU’s position were clearly set out as part of my appeal (Sept 2 – 2929):

I am in possession of three earlier versions of the CRU station data. The 1990 version has been posted at a US Department of Energy website for many years. In September 2002, I requested a copy of this data from Dr Jones. He sent me a 1996 version (cruwlda2) – a version that was also posted at the CRU website until recently – and indicated that the then revised version would posted up when Jones and Moberg (2003) was published, which, according to the date-stamps at your FTP site was done in Feb 2003, as Dr Jones had undertaken to do (the data set newcrustnsall recently removed from your public directory).

Notwithstanding this, when Warwick Hughes and Willis Eschenbach requested station data, for some reason, CRU failed to provide then with this information. As you noted in your letter, following your recent refusal to provide station data to me, I examined the CRU FTP site and determined that the newcrustnsall was the version of the station data for Jones and Moberg 2003 that Dr Jones had previously undertaken to post up on the Internet.

While these data sets are of interest, my request was for the current version of the data set and I do NOT wish to withdraw my appeal of your ruling. Could you please advise me of Mr Colam-French’s email so that I may submit further particulars of the basis of my appeal. In the mean-time, I would appreciate it if you reflected further about the apparent contradictions in your present refusal, given Dr Jones’ previous provision of an earlier version of the dataset to me, the posting of two versions of the data set on your website, one from 1996 to the end of July 2009 and the other from 2003 to the end of July 2009 and the provision of a version to the US Department of Energy, where it has been posted on the internet since 1990.

Jones’ reply (2929) to Palmer was incoherent. Jones now said that he didn’t “recall all the facts from that long ago”. Jones admitted that there was a version at the DOE website and even that this was a “contractual requirement at the time” – not explaining how a contractual requirement to make the data available was consistent with the confidential agreements that he had supposedly entered into with the NMSs:

I don’t recall all the facts from that long ago. There is or was a version on a US Dept of Energy website from about 1990. This was a contract requirement at the time. Much extra data has been added since then, and this is what the restrictions refer to from the mid-to-late 1990s.

Jones’ “explanation” of cruwlda2 being on the CRU website since 1996 was no more coherent. Jones said that it was on their website because this was “easier” than sending disks to the parties – again inconsistent with supposed confidentiality. Jones (falsely) stated that the dataset wasn’t complete(it was):

The 1996 version (cruwlda2) wasn’t a complete version and was something we developed for a number of people in EU projects to use. We made these available to people on these projects via our ftp site, as it was easier to do this than sending disks at that time (email attachments were smaller then).

Jones (falsely) added that he didn’t “have a copy of that file” – this is only a few weeks after he’d ordered the removal of the file from the FTP site during the early August carnage.

Jones’ attitude to the supposed confidentiality agreements is further evidenced in an email to Palmer and Mcgarvie (Sep 23 – 2840) about Jonathan Jones’ FOI request in which he said that they were “quite adept” in the past at getting around any conditions set by NMSs in the alleged confidentiality agreements:

The other issue is that Met Services putting conditions for the use of the data was common in the mid-1980s and 1990s. We were just quite adept at getting around the conditions.

None of Jones’ admissions caused the university to reconsider their refusals. It fought appeals by Jonathan Jones and Don Keiller at every step. Last summer, nearly two years after the original request, the ICO rejected the original UEA refusal. And, despite CRU’s serial violation of the supposed confidentiality agreements now said to be so important, the university closed their eyes to these prior violations rather than reprimand CRU.

Update Dec 31, 2011: In the university’s rejection of my appeal, they conceded that the reference to “non-academics” was an “error”:

In response to your first point in your email of 24 July regarding the non transmission of data to non-academics, I have concluded that the reference to non-academics was in error and that the status of yourself, or any other requester, is irrelevant to the factors to consider regarding disclosure of the requested information.

In response to comments about previous availability of the data set, they stated:

Turning to the points you raised in your email of 2 September, you note that other earlier versions of this data are available on the US Department of Energy website and that Dr. Jones had sent an earlier version of the data to you and had mounted it on FTP server.

In regards the information provided to the US Department of Energy, my investigation has revealed that this was done in the early 1990s prior to the imposition of the restrictions now pertaining to the data pursuant to a contractual obligation at the time. Therefore, the analogy you are drawing does not apply to the data that is the subject of this request.

Richard Muller sent me the BEST list of “very rural” sites – see www.climateaudit.info/data/station/berkeley.

I took a quick look at the “very rural” stations in two tropical countries – Peru and Thailand – in order to groundtruth their classification methodology. These two examples were chosen because several years ago, I looked at Hansen’s “rural” Peru stations, most of which were actually urban; and because one of my sons is in Thailand.

Berkeley classified only two stations in Peru as “very rural” – Huanuco and Quince Mil. However, the population of Huanuco, according to GeoNames, is 147959; it was classified by Hansen as urban. Why the Berkeley algorithm classed it as “very rural” is unclear. Quince Mil is a small town recently featured in Time magazine here. The Berkeley record only has a few sporadic values after the mid-1980s. The BEST urban-rural comparison in Peru looks to me like it has no meaning.

The Berkeley classification in Thailand also looks very suspect. As in Peru, many of the “very rural” sites are small cities. Some locations look suspicious. “Bangkok Pilot” is shown as “very rural”, but when googled is associated with Suvarnabhumi, the name of the very urban Bangkok airport (data is sparse, so it may be something else, but anything associated in any way with Bangkok can hardly be “very rural”)

There are five “very rural” Thailand sites with a minimum of 120 values: PHETCHABURI (population 46,501), KO LANTA (population 20,000), TAKUA PA (population 35,337), KO SAMUI (population 50,000) and MAE HONG SON, a relatively long record in the northwest hill country near the Myanmar border. (We visited Pai which is near there a couple of years ago.) A picture Mae Hong Son airport is shown below:

The Bangkok metropolis data set is the longest series in the area. For reasons that remain obscure, the Berkeley version for Bangkok runs hotter than the CRU version:

The CRU stations in the CRU gridcell are all highly urban: BANGKOK METROPOLIS (!), ARANYAPRATHET (population 73813), CHANTHABURI (population 488397), SIEMREAP (population 85,000), KAMPOT (poluation 39186), PHNOM-PENH (!). Although we keep hearing of the unimportance of UHI, Bangkok has increased 0.24 deg C/decade relative to Mae Hong Son.

In March 2010, Eugene Wahl admitted to the NOAA Inspector-General that he had destroyed his email correspondence with Keith Briffa about changes to IPCC and falsely stated that “all” the emails were in the public domain. This was untrue. The attachments were not in the public domain. Not only were the attachments not in the public domain, but Wahl had actively opposed disclosure of these emails. Following Wahl’s disinformation about the topic, I submitted an FOI request in April 2010 for the attachments to Wahl’s emails. I provided a review of this and subsequent events earlier this year here – worth re-reading if you wish further context on today’s post.

Briefly, East Anglia refused my request for the attachment to the Wahl-Briffa emails on the grounds: “Information not held”.

Later in the year, after Muir Russell admitted to the Parliamentary Committee that he had not investigated Jones’ email deletion enterprise, Acton told the committee that he had “investigated” the incident – the first public mention of the “Acton investigation” – and, in answer to a direct question from Graham Stringer whether all the emails were available and could be read, told the Committee “Yes”.

Acton’s answer was inconsistent with the excuse used by East Anglia to refuse my request for the attachments to the Wahl-Briffa emails. I accordingly appealed the decision last year. I followed up on several occasions. In July 2011, I was informed that the decision was in the process of being drafted. In early November 2011, I was told that the decision had been drafted and was awaiting sign-off. But still nothing.

Climategate 2.0 brings interesting new information on the attachments – information which may also shed some light on the provenance of the Climategate dossier itself.

On October 12, 2009, about a month before the release of the Climategate dossier, Tom Melvin emailed Mike [Salmon] describing a procedure in which Melvin had copied Briffa’s complete Eudora file together with 3.5 GB of attachments (apparently going back a number of years) to his portable. These arrangements were presumably made to facilitate Briffa’s access to his email history while Briffa was recovering from a serious illness.

3939. 2009-10-12 12:07:03
______________________________________________________
cc: Keith
date: Mon Oct 12 12:07:03 2009
from: Tom Melvin
subject: Keith Email
to: Mike
Mike,
For Keith’s Email :
1. Copied the full C:\Eudora directory to my portable.
2. Deleted the 12000 temporay .gif files from C:\Eudora\Embedded.
3. Copied 3.5 gig of attachments (1 year or older) from C:\Eudora\Attach to C:\OldAttach – this will need to be copied back to his PC
4. He is left with a 1.5 gig C:\Eudora directory on my portable which can be copied back to his PC and readily be moved from PC to portable etc.
5. When using my portable (via yellow cable (in office) or various WiFi networks) Keith logs in to VPN.
Tom
PS. I need to take my portable to a conference w/c 26th Oct.

Bishop Hill, in an article today, refers to another CG2 email that sheds new light on CRU’s handling of emails. In email 0021, Jones informed Manola Brunet:

Hola Manola,
I’ve saved emails at CRU and then deleted them from the server. Now I’m at home I just have some hard copies.

Email 21 was on Sep 12, 2009, only a month prior to Melvin’s email. CA readers will recall that, in August 2009, as a result of the “mole” incident, Jones had ordered the removal of many documents from CRU’s FTP server (many of which were presumably placed on another server not intended for public access.)

Maxim Lott of Fox News has an interesting article drawing attention to the co-opting of US Department of Energy funders by CRU and their associates – co-opting in the sense that the US Department of Energy totally failed to ensure that their grant procedures complied with US federal policy for requiring grantees to archive data.

In the article, Lott quoted extensively from Jones’ emails between 2007 and 2009 in which he assured correspondents that the US Department of Energy had assured him that he didn’t have to archive data:

Making that case in 2009, the then-head of the Research Unit, Dr. Phil Jones, told colleagues repeatedly that the U.S. Department of Energy was funding his data collection — and that officials there agreed that he should not have to release the data.

“Work on the land station data has been funded by the U.S. Dept of Energy, and I have their agreement that the data needn’t be passed on. I got this [agreement] in 2007,” Jones wrote in a May 13, 2009, email to British officials, before listing reasons he did not want them to release data.

Two months later, Jones reiterated that sentiment to colleagues, saying that the data “has to be well hidden. I’ve discussed this with the main funder (U.S. Dept of Energy) in the past and they are happy about not releasing the original station data.”

A third email from Jones written in 2007 echoes the idea: “They are happy with me not passing on the station data,” he wrote.

I was contacted by Lott and drew his attention to correspondence in 2005 between Warwick Hughes and the US Department of Energy, which was reported at CA in October 2005 here. Earlier in February 2005, Jones had famously refused Warwick Hughes as follows:

Even if WMO agrees, I will still not pass on the data. We have 25 or so years invested in the work. Why should I make the data available to you, when your aim is to try and find something wrong with it. There is IPR to consider

In my October 2005 post, I reported Hughes’ email to Tom Boden of the US Department seeking Jones’ data, together with Boden’s response advising Hughes that the terms of Jones’ grant did not require him to provide the funding agency with the data. Curiously, in the early 1990s, under earlier Department of Energy contracts, Jones had provided the US Department of Energy with a complete station data base which was published online by the DOE. (The University of East Anglia has never provided any theory of how alleged confidentiality agreements from the 1980s remained “actionable” given that Jones had either not been prevented by these agreements from providing station data to the US Department of Energy for their online publication or had done so without complaint by the NMSs at the time.)

The October 2005 post is well worth re-reading as it reviews US federal policy which would appear to have required the DOE to ensure that grantee CRU archived data. It also quotes policy from UNFCCC which also clearly required NMSs to have ensured that a system for archiving station data was established. The position of Nature and other defenders of CRU as justified due to elusive confidentiality agreements entails that IPCC participants who rely on these supposed agreements have failed to live up to these UNFCCC obligations. In my opinion, Jones, in his capacity as IPCC AR4 Coordinating Lead Author, ought to have reported such NMS consent refusals to IPCC participants as breaches of UNFCCC commitments. Jones’ failure to do so is hard to separate from his expressed desire as CRUtem proprietor to prevent critics (for reasons that remain unclear) from examining this data.

In my October 2005 post, I wondered at the tactical wisdom (from the Team’s standpoint) at obstructing access to this (and other) data, noting then (as on numerous subsequent occasions) that I did not necessarily anticipate large changes in the temperature history, but equally, given the wide public interest, why the record should not be made available for scrutiny.

David Holland, in a guest post at Bishop Hill, shows that Climategate 2.0 has provided more context on Phil Jones’ efforts to organize the deletion of documents.

Jones’ May 29, 2008 request to Michael Mann that he ask Eugene Wahl to delete his IPCC emails (and that Mann delete his own emails) was one of the more notorious emails. It read as follows (as readers know):

subject: Re: IPCC & FOI

Mike,
Can you delete any emails you may have had with Keith re AR4? Keith will do likewise. He’s not in at the moment – minor family crisis.

Can you also email Gene [Wahl] and get him to do the same? I don’t have his new email address.

We will be getting Caspar [Ammann] to do likewise.
Cheers
Phil

As is also well known, Mann immediately confirmed that he would contact Wahl about this as soon as possible. As is also well known, Wahl (earlier this year) confirmed to a NOAA Inspector General that he deleted the emails promptly after being contacted by Mann:

subject: Re: IPCC & FOI
Hi Phil,
…I’ll contact Gene about this ASAP. His new email is: generwahl@yahoo.com. talk to you later,
mike

Although the UK Parliamentary Committee directly asked Muir Russell to examine this incident, Muir Russell refused to do so, and, worse, made a false finding that Jones had not sought to delete emails subject to a prior FOI request. Muir Russell did not satisfactorily explain his false finding when recalled by the UK Parliamentary finding, but admitted that he had not asked Jones about the incident because that would have been asking Jones if he had committed a crime. In the fall hearing, Acton claimed that Acton himself had done the investigation that Muir Russell had failed to carry out and told the Parliamentary Committee that Jones had been able to provide all the emails – thus nothing had been deleted. (See Holland’s article for references.)

Acton’s testimony was inconsistent with assertions made by the UEA in refusing my request for the attachments to Wahl’s surreptitious correspondence with Briffa. The university refused my FOI request on the basis that they did not have the attachments i.e. the version of the email complete with attachments had been deleted/destroyed. Following Acton’s testimony, I appealed this refusal to the ICO (pointing out the inconsistency between Acton’s evidence to the Parliamentary Committee and the grounds of the UEA refusal) and am awaiting a decision. The ICO official in charge says that the decision was drafteda while ago and is awaiting final approval.

Back to the new evidence.

A day prior to the notorious deletion email, in a Climategate 1.0 email that received too little attention, Jones told Palmer and Mcgarvie (BTW Mcgarvie is considerably more prominent as a recipient of Climategate 2.0 context emails than in the original tranche – a point that I shall return to) that Briffa should give an (untruthful) answer about his surreptitious correspondence with Wahl:

Keith should say that he didn’t get anything extra that wasn’t in the IPCC comments.

As David Holland observed, Climategate 2.0 email chain 2526 from June 4 to June 6 sheds further light on the enterprise. Another email (4885), not noted by David, shows Jones gossiping with Wigley and Santer about the events.

On June 3, Holland was notified (2526) of the refusal of his FOI 08-23 (a request curiously omitted from the Muir Russell list of FOI requests.) See contemporary CA post here. The primary excuse was that releasing the information would supposedly be an “actionable breach of confidence” (though such reasoning did not prevent East Anglia employees from distributing Holland’s emails to numerous people unaffiliated with the University of East Anglia without asking Holland’s consent):

s.41, Information provided in confidence: The release of this information would constitute an actionable breach of confidence…

we hold that the s.41 exemption applies to all requested correspondence received by the University. We have consistently treated this information as confidential and have been assured by the persons and organisations giving this information to us that they believe it to be confidential and would expect to be treated as such.

Holland immediately appealed (June 4) – also see contemporary CA post here. CG2 email 2526 includes an acknowledgement of Holland’s appeal and forwarding by Palmer to Jones, Osborn, Briffa, Mcgarvie and Colam-French (the latter would later consider the appeal):

Subject: FW: Freedom of Information Act request (FOI_08-23) – Appeal
Gents,
The expected response from Mr. Holland. We now need to invoke our complaints/appeal procedure contained within the UEA Code of Practice for Responding to Requests https://www1.uea.ac.uk/polopoly_fs/1.2750!uea_manual_draft_04b.pdf). We have a multi-stage process; first, I try to resolve it informally, then it goes to the Director of ISD, and finally, it can go to an internal ‘Adjudication Board’ – we have never reached this latter stage on any appeal to date – we usually resolve the matter or it ends up with the ICO prior to this stage. Please note, that the ICO will not hear an appeal unless all avenues of internal complaint process have been explored/undertaken.

It should also be noted that neither the Act, nor any Code of Practice mandates what sort of complaints process we should have; merely that one should be in existence, be fair, be known to the requester, and used in situations such as this. Please also note that we have to provide a time limit for response – in our Code, we give ourselves 4 weeks. (28 calendar days) I will acknowledge the complaint/appeal and will assess whether there is any room for an informal resolution of this request. I doubt that there is much room here but I’m bound to look for it…

Should it go to the Director of ISD, I will prepare a briefing paper setting out the issues for him from a FOIA perspective; and then he will ‘rule’. I have not had the opportunity to review Mr. Holland’s response closely as yet but will do so and get back to you… there are some aspects that you may be in a better position to comment on than I (e.g.. issues of confidentiality of correspondence; amount/location of correspondence) and I would appreciate your input. Michael, once we have had an opportunity to review, worth another meeting?
Cheers, Dave

The policy manual referred to in the email is still online here. A Bishop Hill reader observes that it links in turn to a document entitled “Management of Records under the Freedom of Information Act 2000″ that is password protected.

Later on June 4, Jones wrote the following to Jean Palutikoff, formerly of CRU, who was then at the UK Met Office as part of the TSU for IPCC WG2. In this email, as David Holland observed, Jones told Palutikoff of IPCC that Briffa and Osborn had moved all their correspondence off their PCs onto memory sticks. (Jones also referred to an earlier incident where John Mitchell of the UK Met Office “conveniently lost” his emails pertaining to his conduct as Review Editor of the IPCC paleo chapter in which Briffa had been Lead Author:

subject: FW: Freedom of Information Act request (FOI_08-23) – Appeal
to: “Palutikof, Jean”
Jean
Sorry to bring you back down to earth! Can you read the attached – a glass of wine would help, perhaps several.

Keith got an initial request which developed in CRU01.pdf. We then got the second one CRU02 which was replied to with the refusal letter (the incorrect date was changed before this went). We have just got CRU03. What Keith and Tim did was to email all the CLAs and LAs on Ch 6, to ask if they would be happy for Keith/Tim to send emails relating to Ch 6 discussions. They all refused, hence the refusal letter.

What we’re now considering doing is to refer him to WG1 and/or IPCC in Geneva. know that there is likely only Melinda Tignor left in Boulder. Susan changed her email address a few months ago. John Mitchell did respond to a request from Holland. John had conveniently lost many emails, but he did reply with a few. Keith and Tim have moved all their emails from all the named people off their PCs and they are all on a memory stick.

So any thoughts on how to respond? TSU for WG2 was mentioned on the first request! As you and Tom know Keith and I are nowhere near the world’s best for structured archiving – working as we do on sedimentary sequencing!
Cheers
Phil

The next day (June 5 – 4885), Jones forwarded Holland’s email (not extending Holland the same confidentiality as they claimed for their pals) to Wigley and Santer – neither of whom had any duties in respect to the FOI request:

Tom, Ben,
An annoying email from yesterday is attached!

We will likely be replying in a similar vein to our earlier, saying emails between CLAs and LAs for Ch 6 were in confidence. We have emails from all in Ch 6 to say the group doesn’t want emails made available. We will refer Holland to WG1 in Boulder – knowing that there is likely only one person there keeping things ticking over till the TSU closes – which it may have.

IPCC will have to alter those work guidelines to stop this sort of thing next time. I’ll be raising it with whoever is the next Susan. Decision in early Sept – news is it will be one of Tom Karl, Ram, Brian Hoskins or Thomas Stocker.

What will amuse is the paragraph about structured archiving. As you both know Keith and me work on the sedimentary sequence approach to filing!
Cheers
Phil

Here is a copy of the search warrant. The name of the detective who swore the information giving rise to the warrant has been whited out at Tall Bloke’s request. (Tall Bloke said that he had “promised anonymity” to the detective. I don’t know why he would do this, but he was the one surrounded by six detectives.) The information (affidavit) supporting the warrant doesn’t seem to have been provided to Tall Bloke so far.

The warrant entitles the police to enter and search the premises for “evidence of an indictable offence” referring to section 15 of the Police and Criminal Evidence Act. In its Climategate series, even the Guardian was unable to conclude that there had been a crime. So I wonder how the Detective Inspector came to the conclusion that the computers at Tall Bloke’s residence would provide “material that is likely to be relevant evidence and be of substantial value to the investigation of the offence”.

The warrant does not include “special purpose material”, which,under section 14, is said to include “(b)journalistic material, other than excluded material.” “Journalistic material” is defined in section 13 as follows:

13 Meaning of “journalistic material”.
(1) Subject to subsection (2) below, in this Act “journalistic material” means material acquired or created for the purposes of journalism.
(2)Material is only journalistic material for the purposes of this Act if it is in the possession of a person who acquired or created it for the purposes of journalism.
(3)A person who receives material from someone who intends that the recipient shall use it for the purposes of journalism is to be taken to have acquired it for those purposes.

In California, bloggers are legally included as journalists. I wonder what the UK situation is.

Tall Bloke reports that he was raided by UK police and his computers seized. Read his blog post for further details, including actions in the US involving WordPress. Also see Jeff Id here. It is not a coincidence that this story is reported first on these three blogs.

In an unrelated story, according to Richard Tol, IPCC has told its WG2 scientists that they are above the law on freedom of information.

Another episode in the continuing series – Tiljander and Upside Down Mann.

As CA readers are aware, the “big news” of Mann et al 2008 was its claim to have got a Hockey Stick without Graybill’s bristlecone chronologies (camouflaged as a “no-dendro” reconstruction). CA readers are aware that this claim depended on their use of contaminated modern portion of the Tiljander sediments and that the original claims for a “validated” no-dendro reconstruction prior to 1500 fell apart, even though no retraction or corrigendum to the original Mann et al (PNAS 2008) has been issued.

As we learned (from an inline comment by Gavin Schmidt in July 2010), Mann et al have conceded that these claims fell apart, but did so using a “trick” (TM- climate science.) Instead of acknowledging the false assertions at the journal in which the assertions were made (PNAS), they acknowledged the failure of the no-Tiljander no-bristlecone reconstructions deep in the Supplementary Information of a different paper (Mann et al, Science 2009) – a trick for which the term “Mike’s PNAS trick” is surely appropriate (though the term “Mike’s Science trick” also merits consideration.)

The failure to retract Mann et al 2008 from PNAS has, in turn, either tricked IPCC Chapter 5 authors (or been ignored by them.) CRU’s Tim Osborn, one of the most visible Climategate correspondents, is one of the Lead Authors of Chapter 5. (Had the climate “community” given a damn about public opinion, they would have insisted that IPCC locate a Lead Author for this section who was not quite so intimately involved in Climategate. By failing to do so, they have more or less given the finger to the rest of society.)

Here’s how CRU’s Tim Osborn and his Chapter 5 coauthors assess upside-down Mann:

Mann et al. (2008) used an expanded database of proxy records and two statistical methods and found enhanced amplitude of millennial NH temperature variations, a warmer MCA than in some previous work though perhaps still cooler than the modern warm period, and that similar findings were obtained without using tree-ring data.

Thousands of blog readers are aware that the “similar findings…without tree-ring data” were obtained only by including upside-down contaminated data. It’s disquieting that IPCC coauthors are unaware of this. The failure of Mann and his coauthors to retract or correct the PNAS 2008 article lingers on.

As CA readers are aware, key findings of Santer et al 2008 do not hold using updated data. Ross and I submitted a comment to IJC showing this. The comment was rejected twice, with one of the reviewers (as in the case of the comment on Steig et al) being a Santer coauthor (who was not identified to us as such). Ross eventually managed to get similar results published in another journal.

Jean S points out in a comment on the Steig thread that our findings were completely misrepresented by IPCC chapter 10 (also the source of disinformation about Steig).

Our article stated that there was a statistically significant difference between models and observations in the tropical troposphere. Instead of citing our articles as rebutting Santer’s assertions, IPCC cites us as endorsing Santer’s false assertions:

The claim by Douglass et al. (2008) that modeled and observed trends in the tropical troposphere for the period from 1979 to 1999 are significantly different is contradicted by Santer et al. (2008) and McKitrick et al. (2010). The findings of Santer et al. (2008) are based on analyzing updated radiosonde and satellite datasets, considering observed and simulated trend uncertainties due to natural variability. Santer et al. (2008) also provide evidence based on synthetic data that the consistency test applied by Douglass et al. (2008) leads to incorrect conclusions.

They go on to make the following absurd statement:

Taking these studies together, we conclude, that apparent differences between tropical free tropospheric temperature trends in models and observations and differential warming in the tropics over the period 1979–1999 are unlikely to be statistically significant after fully accounting for observational uncertainties.

Watch the pea. The issue with Santer was that key results fell apart over the longer period of 1979-2009 (or 2010 or 2011) as opposed to the 1979-1999 period. As noted above, realclimate spoke out strongly against Courtillot’s analysis which didn’t use up-to-date data. Pierrehumbert alleged that such analysis was dishonest. Why should different standards apply when employed by IPCC chapter 10?

Jeff Id has an excellent post on IPCC AR5 use of the highly flawed Steig et al 2009. Despite Steig’s efforts to block the publication of O’Donnell et al 2010, O2010 shows clearly that whatever is new in Steig et al 2009 is not only incorrect, but an artifact of flawed math and whatever is valid was already known.

As Jeff observes, Steig coauthor Josefino Comiso (who was very uncooperative, to say the least, in providing data underpinning Steig et al 2009) is Coordinating Lead Author of AR5 Chapter 4 on the cryosphere, where Steig et al 2009 is cited, but not O’Donnell et al.

AR5 chapter 10 cites Steig et al 2009 four times, without any citation of O’Donnell et al 2010. Jeff quotes the following from AR5:

It was concluded that the pattern of mean surface temperature trends in both West and East Antarctica are positive for 1957–2006, and this warming trend is difficult to explain without the radiative forcing associated with increasing greenhouse-gas concentrations (Steig et al., 2009).

As readers of CA and tAV are aware, Steig’s methodology smeared warming from the Antarctic Peninsula into other parts of Antarctica. Jeff observes of the IPCC draft (with considerable frustration):

That last sentence is too much for me. The positive trends are very easy to explain – bad math!!

In Chapter 5 (Paleoclimate), Steig et al cited as authority for polar amplification. In this case, they have a placeholder noting O’Donnell et al 2010 as follows:

[Reference needed: O'Donnell et al., ?]).

O’Donnell et al 2010 was the only reference among the 1000 or so citations for this chapter where the authors had not bothered locating volume citation information. It was the only reference returning the term “null” as follows:

O’Donnell, R., N. Lewis, S. McIntyre, and J. Condon: Improved methods for PCA-based reconstructions: case study using the Steig et al. 2009 Antarctic temperature reconstruction. Journal of Climate, 0, null.

One doesn’t get the impression that the IPCC authors had read, let alone assimilated, the findings.

Check out Jeff’s post.