We are told that the science has been settled on the basis of “an immense edifice of painstaking studies published in the world’s leading peer-reviewed journals…vetted and documented in excruciating detail…” And a climate modeler claims that the input to the reports “has been scrutinized to the highest level possible” (p 33). [Tell that to Steve McIntyre, in an episode to be described in another post]. And “A core principle of the IPCC is that only peer-reviewed literature is cited” (p 41).
The Chairman himself made that claim in a speech to the legislators of North Carolina.
Leaving aside what we have learned about the corruption of the peer-review process, claims about the exclusive use of peer-reviewed literature are bogus. In a chapter on “the peer review fairy tale” Laframboise described a collaborative project involving a worldwide team of helpers who checked all the cited references in the 44 chapters of the 2007 report, counting how many were peer-reviewed and how many came from the “gray” literature.
Her suspicions were aroused by reports from IPCC expert reviewers (not insiders to the writing) that some items were being submitted which did not have scientific status. These even included some press releases, however their concerns were dismissed and the reports were listed as input to the final report (p46).
The final score for 18,531 references in the 2007 report was 5,587 (one third) not peer reviewed. In 21 of the 44 chapters the score for peer reviewed references did not reach 60%. This would not be so bad if it was admitted up front and in public, also if there were clearly defined and properly policed rules for vetting the grey matter (not peer-reviewed) for use by the inner circle of authors.
Among the sources used to support IPCC recommendations were newspapers and magazine articles, unpublished theses, Greenpeace and World Wildlife Fund documents, and yes, press releases.
While Chairman Pachauri had declared an Indian government discussion paper fit only for the rubbish bin, we found that the Climate Bible cites dozens of discussion papers. In one case the decument relied on by the IPCC was clearly labelled as ‘version one’ of a draft. (p 48)
After the release the Chairman wrote a piece referring to “approximately 18,000 peer-reviewed publications”, with the concession that “a limited amount of gray literature” was used “in cases were peer-reviewed literature was unavailable”. One third is a limited amount? And what happened to planning and consultation with the profession? How did all the thousands of climate scientists in the world leave the IPCC short of peer-reviewed material for vital parts of the report?
This discovery suggests that the great mass of journalists and environmental commentators have been irresponsible and unprofessional in failing to report on this situation so the myth of universal peer-review remains alive.
The rules for using “gray” literature
Apart from the “rule” that limited quantities of non peer-reviewed material are used in special circumstances, there was a rule that such items have to be clearly identified in the list of references. One of the helpers found that a total of 6 (six) of almost 6,000 gray items were flagged.
Some people close to the action suggested that more efforts should be made, perhaps using a different colour to flag gray material. After some internal discussion there was a policy statement that peer-reviewed material should have priority and all mention of flagging gray matter has vanished (p. 53).
Relevant stuff on Quadrant on Line.