You may have missed the excellent article by Frank Furedi published in Saturday’s Australian (see below) about the corruption of the peer review system.
For too long it has been asserted that the gold standard in scientific research is peer review.
That’s false. Peer review is second best to the true gold standard: where scientists undertake repeated research in controlled circumstances and present their analysis and full data for scrutiny. That is, that they seek to have their findings falsified. And the more times in which the analysis is subjected to rigorous testing and remains extant, the more confident we can be in the analysis. But it survives only until it is disproved.
Karl Popper is famous among other things for his falsifiability / refutability view. That is, something is not a theory if it cannot be refuted. My theory: all swans are white can be disproved by showing me a black swan. Obviously anthropogenic global warming is therefore not a theory – there is no evidence that could be presented that would be classed as refuting the AGW “theory”.
But it wasn’t Popper who first thought of this. It seems it was Galileo (although one cannot exclude the possibility it was some Ancient). In his Sidereus Nuncius, Galileo writes of his observations of the moon. He showed that it wasn’t perfect – it contained mountains, valleys and craters. Indeed, Galileo was about – using geometry – to calculate with remarkable precision the height of mountains by their distance from the terminator. He was surprised to find that the highest observable lunar mountains were around six kilometres high.
Of course this all offended the authorities, since the Heavens were clearly perfect. Therefore the moon had to be a perfect sphere.
Johann Georg Brengger of Bavaria, an astronomer who was contemporary to Galileo, said the moon appeared to have a rough surface, but it didn’t. In fact it was covered with a transparent crystal substance that filled every valley and crater. So the surface of the moon was perfectly smooth.
Galileo wrote:
the hypothesis is pretty; its only fault is that it is neither demonstrated nor demonstrable. Who does not see that this is a purely arbitrary fiction that puts nothingness as existing and proposes nothing more than simple non contradiction?”
So Galileo sets out falsifiability.
And when quantum physics was postulated by Neils Bohr and Werner Heisenberg among others, it was opposed by Einstein who famously quipped that ‘God does not play dice’. Einstein then posed question after question probing the quantum physicists in an attempt to falsify their theory. He failed.
Today’s climate scientists have much to learn from Galileo, Einstein and Popper.
==========================
FRANK FUREDI
20 February 2010
There’s a noble tradition of scientists scrutinising research to establish integrity, but the process has become corrupted
SUDDENLY the esoteric peer-review system has hit the headlines. The Lancet, a leading British medical journal, has acknowledged that it made a serious error in publishing a study that sought to link the MMR (measles, mumps and rubella) vaccine to autism and bowel disease.
Earlier this month a group of leading stem-cell researchers wrote an open letter pointing out the systematic abuse of peer review by a small cabal of scientists, whom they accused of using their position to slow down the publication of the findings of their competitors.
And the scandal surrounding the leaked emails of the Climatic Research Unit at East Anglia University and the dubious data published by the UN Intergovernmental Panel on Climate Change exposes a worrying trend towards the corruption of the system of peer review.
Peer review is a system that subjects scientific and scholarly work to the scrutiny of other experts in the field. Ideally, it ensures that research is approved or published only when it meets the standards of scientific rigour and its findings are sound.
At its best, peer review guarantees it is disinterested science that informs public discussion and debate. Through peer review, the authority of science helps to clarify disputes and injects into public discussion the latest findings of research.
Peer reviewing depends on a community of experts who are competent and committed to the pursuit of an impartial review. It depends of the commitment and collaboration of scientists and scholars in a given field.
However, the individuals who constitute a community of experts are also scientists and scholars who are preoccupied with their own position and status. Often the colleagues they are refereeing are their competitors and sometimes even bitter rivals. The contradiction between working as a member of an expert community and personal interests can’t always be satisfactorily resolved.
Unfortunately, even with the best will in the world, peer reviewing is rarely an entirely disinterested process and all too often it is not free from vested interests.
As many of my colleagues in academe know, peer reviewing is frequently conducted through a kind of mates’ club, between friends and acquaintances, and all too often the matter of who gets published and who gets rejected is determined by who you know and where you stand in a particular academic debate.
Nevertheless, peer reviewing is probably the most effective way of exercising quality control over the proposals and output of the scholarly and scientific communities.
However, peer reviewing cannot be immune to the preoccupations, agenda and interests of the individuals who carry out the refereeing. Even with the best will in the world academics and scientists can overlook errors and be blind to the importance of a new but maverick contribution. They are ordinary mortals who possess their fair share of prejudices and are often no less petty or self-centred than their fellow citizens.
The experience of the past month indicates there are at least three ways peer review can be undermined.
First, there is the genuine mistake. An example of this was the failure of The Lancet’s refereeing process to spot the flaws associated with the study associating the MMR vaccine with autism and bowel disease. Now that the journal has retracted this flawed study, questions need to be asked as to whether the imperative of gaining publicity for The Lancet was a factor in influencing its decision to rush into print.
Second, there is the damaging influence of nepotism and professional jealousy. Academics and researchers are all too conscious of the prestige and career opportunities that can be enhanced through getting their work published in an important academic journal. Sometimes reviewers regard the research they are refereeing as the work of their competitor and adopt the tactic of delaying or preventing its publication.
This is the accusation made by 14 stem-cell researchers in a letter to several journals in their field. The researchers claimed that the peer-review process was corrupted by reviewers who deliberately stalled and even stopped the publication of new results so they or their associates could publish the breakthrough first. They also blamed the journals for not doing enough to prevent this behaviour from happening.
The third, and in recent years the most disturbing, threat to the integrity of the peer-review system has been the growing influence of advocacy science. In numerous areas and most notably in climate science, research has become a cause that is politicised and moralised.
Consequently, in climate research, peer review is sometimes perceived as a moral project where decisions are influenced not simply by science but by a higher cause. The scandal surrounding Climategate is as much about the abuse of the peer-review system as it is about the rights and wrongs of the various claims made by advocacy researchers in and around the IPCC.
The usual problems associated with peer review have been exacerbated through the transformation of peer review into a public ritual of authorisation. Increasingly, peer review is used as a form of unquestioned and unquestionable authority for settling what are in fact political disputes.
Consequently, the findings of peer review are not simply represented as a statement about the quality of research or the status of a scientific finding but as the foundation for far-reaching policies that affect everything from the global economy to the lifestyle of the individual.
Increasingly peer review has been turned into a quasi-holy institution that signifies a claim is legitimate or sacred. From this perspective, voices that lack the authority of peer review are by definition illegitimate. Peer review provides a warrant to be heard. Those who speak without it deserve only our scorn.
You can almost visualise peer-review dogmatists waving their warrant and demanding that their opponent be silenced. George Monbiot, the British climate alarmist journalist, represents peer review as the equivalent of a holy scripture.
Boasting of his encounter with an opponent who challenged him to a debate on speed cameras, he wrote that “I accepted and floored him with a simple question”. And predictably the question was: “Has he published his analysis in a peer-reviewed journal?” In a world where opponents can be floored because they lack the authority provided by the ritual of peer review there is considerable incentive to manipulate the system.
Andrew Dessler, a climate-change researcher, also sought to floor his opponent, who apparently wrote a “denier op-ed” in The Wall Street Journal, by dismissing its value on the grounds that this newspaper is not peer reviewed.
Since “the only place” where this “denier” can write his views is in “non-peer-reviewed venues like conferences and press releases”, he is worthy only of censorious contempt.
Climate alarmists do not simply boast of their monopoly over peer-reviewed outlets, they also do their best to call into question peer-reviewed outlets that dare publish research that challenges their crusade.
When Cambridge University Press dared publish Bjorn Lomborg’s The Sceptical Environmentalist, it faced bitter criticism from campaigners who hinted that something had gone wrong with this publisher’s system of review.
Stephen Schneider, a professor in environmental studies, asked why “a publisher with so excellent a reputation in natural sciences (it even published the IPCC reports) publish a polemic under its imprimatur” and demanded to know if the press had “the book completely reviewed”? It appears that as far as Schneider is concerned it is simply unthinkable that a publication that questions the prevailing consensus could have been properly reviewed.
The zealous policing of peer-review by campaigners is directly encouraged by the modus operandi of the IPCC itself. As Reiner Grundman argued in the peer-reviewed journal Environmental Politics, the IPCC “characterises outside critics as unscientific as they do not publish in peer-reviewed literature”.
With so much moral resources invested in the authority of peer review it is not surprising that some supporters of the IPCC consensus adopt an almost casual attitude towards the violation of academic protocols.
The leaked email shows how one UEA scientist, Keith Briffa, wrote to a colleague to ask for help for keeping a paper that he did not like out of an academic journal that he edits. US climate scientist Michael Mann proposes that a journal should be ostracised for daring to publish a paper criticising his work.
“I think we have to stop considering Climate Research as a legitimate peer-reviewed journal,” he writes.
Phil Jones, the central figure in Climategate, promises to keep out two research papers from the IPCC report. “I will keep them out somehow, even if we have to redefine what the peer-review literature is,” he notes.
Sadly, there are far too many researchers for whom science has become an instrument for the realisation of a higher cause. As a result they are scientists in name but moralisers in practice.
The manipulative exploitation of peer review is underwritten by a culture where campaigners are permitted to have a cavalier attitude towards facts.
While the IPCC insists that its critics should be judged by the most rigorous standards of peer review, it has a more relaxed attitude towards its own publication. In recent weeks there has been a series of damaging revelations about how conclusions drawn by the IPCC’s 2007 report were based on speculation and anecdotes.
So claims made about disappearing mountain ice were cobbled together from information drawn from a student’s dissertation and an article published in a mountaineering magazine. Other claims were based on information based newsletters, press releases and reports produced by environmentalist advocacy groups.
It is not surprising that those involved in the corruption of peer review are also happy to use anecdotes and speculation as the moral equivalent of hard scientific data.
However, it is important to understand that these people fervently believe in their cause and are convinced that far from deceiving the public they uphold a higher truth.
As with the authors of the British government’s dodgy dossier on Iraq, they are convinced they are absolutely right. It is this sense of righteousness that allows them not to let the absence of a few facts stand in the way of promoting their arguments as hard intelligence or peer-reviewed science.
It was the moral conviction of former US defence secretary Donald Rumsfeld that allowed him to respond to a question about the existence of weapons of mass destruction in Iraq by stating that “the absence of evidence is not evidence of absence”. In a similar manner the absence of evidence does not deter climate alarmists from practising their art.
The philosophy of the Noble Lie permits its practitioners to stretch the truth in good conscience. Tragically, they can count on the authority enjoyed by climate science to avoid having to engage with the criticism of the public.
That is why even when the emperor is caught out without his peer review, the IPCC can carry on by blaming the young boy for his scepticism.
Frank Furedi is professor of sociology at the University of Kent. His latest book is Wasted: Why Education Isn’t Educating (Continuum Press).
The logic of testing was probably realised in antiquity, the value of Galileo, Einstein and Popper was to apply the principle to important issues at the time. [on a point of detail, the more important innovations by Popper were the "conjectural turn" and also the "social turn"].
The practical value of the falsification or testing approach is that it enables you to find out early in a discussion whether the other party is prepared to take evidence seriously, and even better, to find out what kind of evidence would make a difference to their position. Many people don’t like that question because they are too committed to their position.
For researchers the “testing” approach is the cost-effective way to go, saving time, money and effort in chasing data. It imposes the valuable discipline of geting clear about your theory in relation to rival theories so the experimental effort (or whatever kind of data collection is required) has a chance of coming up with a result.
Rafe
22 Feb 10 at 8:23 pm
Furedi and the Spiked crowd are always worth reading
http://www.spiked-online.com/
Except for Rundle.
ken n
22 Feb 10 at 8:48 pm
So, it’s okay to reproduce entire feature articles from the Australian?
daddy dave
22 Feb 10 at 9:26 pm
Probably not.
Sinclair Davidson
22 Feb 10 at 9:42 pm
“Frank Furedi is professor of sociology”
I thought they were looked down upon round here?
Jarrah
22 Feb 10 at 9:50 pm
There are exceptions Jarrah.
Even among sociologists.
ken n
22 Feb 10 at 9:52 pm
not any more!
daddy dave
22 Feb 10 at 9:53 pm
and a former Marxist too, I understand.
Sinclair Davidson
22 Feb 10 at 9:56 pm
Yes they are an interesting bunch.
I asked one whether she still considered herself to be on the left and she said “Of course. I believe society can be improved – it’s just that the techniques needed are different”.
Tho Living Marxism was not really a Marxist magazine. Even then heading towards something resembling libertarianism,
ken n
22 Feb 10 at 10:00 pm
Sadly they lost a bundle of money in a court case and had to close the magazine. They are barely surviving at present.
They tried to kick off a sydney chapter a few years ago but there was not enough interest.
The laws on libel or slander are scandalous in Britain, a whole book had to be pulped after threat of legal action by an Oxbridge academic who was described as “incompetent” for grossly misrepresenting the ideas of a group of scholars. He tried to do the same with the US edition of the book but the publisher was not intimidated.
Rafe
22 Feb 10 at 11:01 pm
It was good for Furedi to recall the Lomborg case. Is this fully written up somewhere? He faced a wall of opposition from environmental activists and doomsayers (many of them properly certified scientists) who did not want to hear any good news about the environment. He produced a heap of evidence to show that post-1970s concerns about the environment and consequent actions had resulted in significant areas of improvement (take the Parramatta River and the Sydney Harbour as an example). He was subjected to dreadful abuse and one supposedly reputable magazine (Scientific American?) published five critical reviews of the book and did not allow him the right of reply.
Rafe
22 Feb 10 at 11:10 pm
If we keep on pouring CO2 into the atmosphere and if it doesn’t get hotter then the AGW theory is falsified. So the theory can be falsified. So what’s the complaint?
TerjeP (say tay-a)
23 Feb 10 at 2:03 am
@terje: ‘If we keep on pouring CO2 into the atmosphere and if it doesn’t get hotter _over the next 30 years (to make it statistically significant)_ then the AGW theory is falsified.’
Fixed it for you. ‘Course that is a little akin to testing whether a gun is loaded by pressing the barrel against your temple and pulling the trigger. Dilemma, right?
JasonW
23 Feb 10 at 5:45 am
Indeed. The problem is to get a fix on the temperature. If there is no trend then we have a very good falsification. An even better falsification would be a trend down. But even if there is a statistically significant trend upward(say at the 5% level)there are still questions of the size (oomph) and the reason, given the complex of factors involved.
Rafe
23 Feb 10 at 7:14 am
And of course, questions of risk management. So, you could look at what’s been (geological record of sea levels over the past few hundred thousand years, ice cors’s and what have you), and try to find out what conditions were prevalent then. You extrapolate to today. You measure and weigh, you look around at nature and how it’s behaving. You come to the conclusion, along with a couple of thousand of your peers, that the risk of something adverse to our current equilibrium is more likely than not. A lot more likely. You also know that human society has reached a point at which is not as flexible anymore as it used to be. What do you do?
And then there’s the reason. Let’s bring in the gun again: Say you’re a copper at a murder scene (sorry for the morbid example). There’s a body lying on the floor, it’s not quite clear how it’s been killed. You look at an iron poker but that looks pristine. Kitchen knives are in their drawers, the medicine cabinet with the sleeping pills looks untouched. And then there’s a gun that was found not far from here, near the murder scene, with blood on it. In the forensic lab they find out that this blood is from the victim and fingerprints found on the gun are from the victim’s neighbour.
Now, wouldn’t that make it the probable murder weapon and the neighbour highly suspect? This would warrant building a case against the neighbour around that. Contrary evidence would have to be pretty strong to clear the bugger.
The analogy isn’t perfect but stay with me on this. The body is GW. The burglar with the iron poker, the jealous wife and kitchen knives or the victim himself with the sleeping pills are the sun, cosmic rays, Milankovich cycles etc. The gun is CO2. The neighbour is fossil fuels. And the fingerprints on the gun are the radioactive carbon isotope (or rather the lack of it) that links the excess CO2 in the atmosphere to burnt fossil fuels.
This is the case for AGW, except there are some considerable more lines of evidence. A defence lawyer for the neighbour would have to do better than to hold up the untouched kitchen knives and say “Actually, it was this, your honour – and the wife did it.”
JasonW
23 Feb 10 at 8:19 am
Raf, two interesting spin-offs from Spiked are
http://www.instituteofideas.com/index.html (Claire Fox)
http://www.manifestoclub.com/ (Josie Appleton)
A few of us have tried to start an Australian version of Manifesto Club: History is still young.
ken n
23 Feb 10 at 8:47 am
Rafe: The Lomborg case and other examples of environmental exaggeration are written up in the excellent book Science and Public Policy: The Virtuous Corruption of Virtual Environmental Science by Aynsley Kellow of UTas.
http://www.amazon.com/Science-Public-Policy-Corruption-Environmental/dp/1847204708
It has the great example of the Kting Voar creature that may well be mythical that is listed as endangered. http://en.wikipedia.org/wiki/Kting_Voar
Karl Kessel
23 Feb 10 at 9:30 am
JasonW, your story does not convince. The warmenist case is being increasingly exposed as riddled with frauds.
We may well be in a plateau on the cycle or on a very slightly upward path.
There is nothing like a smoking gun in sight,unless you are talking about leaks from the fortress of warmenists.
Nothing that we do in Austrlia will make a scrap of difference and yet you would have us unleash a feeding frenzy of rent seekers and green bureaucrats. You have got to be joking!
Rafe
23 Feb 10 at 10:37 am
The warmenist case is being increasingly exposed as riddled with frauds.
.
I’m not sure that’s true Rafe. Apart from the glaciers gaff what else is there but hot air and conspiracy theories? I read The Australian this morning. No anti-AGW sctick just debate on policy. There was a column that, ha ha ha, advocated Europe’s approach.
Adrien
23 Feb 10 at 10:48 am
If we keep on pouring CO2 into the atmosphere and if it doesn’t get hotter then the AGW theory is falsified. So the theory can be falsified. So what’s the complaint?
.
It’s already been falsified.
There were reasonably specific predictions about what would happen between 2000 and 2010 and those predictions were wrong. End. Of. Story.
Look, in 2020 it will be even more falsified, but people will still be running around going, “what’s the evidence against AGW?”
I guess we are in a post-falsification era of science.
daddy dave
23 Feb 10 at 11:01 am
The glacier thing was not fraudulent merely sloppy research not an unknown thing to happen at Catallaxy nor to Rafe.
We do know glaciers are melting faster then anyone thought.
Butterfield, Bloomfield & Bishop
23 Feb 10 at 11:08 am
DD – The failure of predictions based on computer models, and there are a wide range of predictions, does not falsify a theory. And despite the notion that the AGW wagon has collapsed so prevalent here, I’m afraid in the wider world it’s taken for granted.
Adrien
23 Feb 10 at 11:19 am
Homer – We do know glaciers are melting faster then anyone thought.
.
Well except for that lot at the IPCC.
Adrien
23 Feb 10 at 11:19 am
Frank Furedi’s inclusion of the Wakefield paper from the Lancet is somewhat odd since the paper was retracted after the General Medical Council found that Wakefield had acted unethically and possibly fraudulantly in the research conducted for that paper. Peer review can review the described methodology, processes and data described in the paper but is unable to assess whether or not a researcher is lying about the results – in fact one of the primary assumptions of peer review is that the author has been honest. In the Lancet case I think it is erroneous to suggest that this was a failure of peer review as on the basis of the study as described the results and conclusions were plausable. The fact that the hypothesis was not honestly derived or investigated is outside the peer review process.
This does not negate criticisms of the peer review process, which like any process that involves humans is going to have flaws – the assuption of honesty for one. This is why scientific reputation is so critical.
Grendel
23 Feb 10 at 11:29 am
The failure of predictions based on computer models, and there are a wide range of predictions, does not falsify a theory.
.
In that case I’d like to know what would falsify the theory.
.
I’m afraid in the wider world it’s taken for granted.
.
So are star signs.
daddy dave
23 Feb 10 at 11:38 am
In that case I’d like to know what would falsify the theory.
.
Evidence that says that the theory is wrong. The computer models use a bunch of variables emplaced to fill in data gaps. There are gaps. The theory is not a fact.
.
So are star signs.
.
You’re obviously a Scorpio.
Adrien
23 Feb 10 at 11:45 am
As Adrien notes there are a vast many theories. One incorrect prediction means merely that a part of the body of theories that you are relying on is wrong.
For example when we found that predicted orbits of Uranus were wrong we didn’t falsify Newtonian gravity but rather the theory that there was only seven planets.
Showing that increased Co2 levels do not trap additional heat would be the cleanest test to falisify the theory.
Other than that what you are likely to be falisfying is various theories about the strength of the positive and or negative feedbacks in the earth’s climate.
Steve Edney
23 Feb 10 at 12:21 pm
As Adrien notes there are a vast many theories. One incorrect prediction means merely that a part of the body of theories that you are relying on is wrong.
.
Thank Steve, nice illustration. I’m conversant with how falsification works and its vagaries. However, you have to be careful with endless ad hoc modifications to a theory thats simply not working. In the case of AGW, temperature rises are a fundamental feature of the theory. Newton had a ton of other observations to support the theory (although Mercury was always a strange one); all we’ve got in this case is historical data, plus emerging temperature trends.
.
Showing that increased Co2 levels do not trap additional heat would be the cleanest test to falisify the theory.
.
Lack of temperature increases despite C02 increases will also falsify the theory. The question is, how long do we wait before declaring the theory wrong. The answer is – I think – somewhere between 10 and 100 years.
daddy dave
23 Feb 10 at 12:34 pm
While it will be the most nauseating 2 minutes of your life, you will understand that the true proof of the futility of ‘peer review,’ is a sheltered workshop whose leaders claim arts are far more than just another industry
Peter Patton
23 Feb 10 at 1:16 pm
Peter:
Kate is allowed to say whatever the fuck she pleases even if it doesn’t make any sense at all and is unadulterated crap.
You’re in fact only allowed to agree with her , as are the rest of us.
Any woman that good looking is simply always right. Always.
JC
23 Feb 10 at 1:25 pm
jc
Indeed she is. And it is that context that we must all understand what is meant to ‘peer-review’ in 2010.
Peter Patton
23 Feb 10 at 1:30 pm
jc
Oh, hold on. I thought the article title was:
Climate Scientists are far more than just another Scientist
Peter Patton
23 Feb 10 at 1:33 pm
PP if you are a pretty looking gal with tatts JC will always give you a free pass
tal
23 Feb 10 at 1:34 pm
Oh, sorry. I didn’t realize JC posted from Long Bay!
Peter Patton
23 Feb 10 at 1:36 pm
Long Bay?
Yes Tal’s right to say that about me. Every pretty gal gets a free pass to say whatever stupid thing comes into her head. It’s the right thing to do.
JC
23 Feb 10 at 1:41 pm
JC
I am sorry if you are actually a Pentridge Boy, and I have insulted you!
Peter Patton
23 Feb 10 at 1:42 pm
Peter , Frankly I don’t get it. What’s the association with prisons?
JC
23 Feb 10 at 1:46 pm
tal says you like ‘girls’ with tatts!
Peter Patton
23 Feb 10 at 1:51 pm
Peter JC likes real girls with tatts not jail bitches. He is very shallow
tal
23 Feb 10 at 2:12 pm
tal
Oooooohhhhh! Now I geddit!
Peter Patton
23 Feb 10 at 2:19 pm
Lack of temperature increases despite C02 increases will also falsify the theory. The question is, how long do we wait before declaring the theory wrong. The answer is – I think – somewhere between 10 and 100 years
.
True and the rub as too.
Adrien
23 Feb 10 at 2:40 pm
Peter:
Gals with tats is pure heaven, i reckon. I’m not talking about big crap all over their bodies type thing.
But a tiger on the small of the back, or a single one on an arm etc. is very sexy.
The best one would be a tatt on the lower part of the tummy.
My better 1/2 thinks I’m a loon for even suggesting such a thing.
JC
23 Feb 10 at 2:47 pm
I read that as “beer review”,
.
Rationalist
23 Feb 10 at 7:10 pm
So in summary we’re expected to accept the views of cranks because the poor little wilting flowers are “outside the club” and can’t get their views published?
Really?
JM
23 Feb 10 at 9:56 pm
JM
How would you know if they’re cranks if they haven’t published. Furthermore your stand sounds more than a little intolerant of people holding views contrary to your own. Why are you like that?
A paper could reach the wrong conclusion and still get through peer review. Many theoretical papers that later turn out to be wrong in fact go this way.
Show a little more open mindnedness and a little less narrow vision, JM. It could help through life.
JC
23 Feb 10 at 10:02 pm
… and not just tatts.
Sinclair Davidson
23 Feb 10 at 10:05 pm
So in summary we’re expected to accept the views of cranks because the poor little wilting flowers are “outside the club” and can’t get their views published?
Really?
.
No, we’re not expected to “accept the views of cranks.” Now, read it all again and try to understand what’s going on a second time. To make it easy for you, I’ll give you a hint. This passage is very important:
daddy dave
23 Feb 10 at 10:19 pm
That’s still very long, so I’ll chop it down for you to the most salient bit, JM.
In numerous areas and most notably in climate science, research has become a cause that is politicised and moralised.
There are logically only three positions.
1. you agree and you are alarmed.
2. you agree but you are not alarmed.
3. you don’t agree.
daddy dave
23 Feb 10 at 10:22 pm
JC: A paper could reach the wrong conclusion and still get through peer review.
Of course. And I think your observation points out the central problem with the poster’s premise.
Peer review is not the gold standard; it is the minimum standard. If you can’t get your ideas through the somewhat shaky peer review process, why should I pay them any attention at all?
(And believe me, crank science is like self-published fiction. You have no idea how bad it can be unless you’ve seen some of it.)
Yeah, you can wave Popper at me, but that’s a very different and far more esoteric philosophical argument. But the fact remains.
Peer review is a minimum. It’s not a guarantee of correctness, it’s just a condition that the views expressed meet minimum professional standards.
JM
23 Feb 10 at 11:50 pm
JM
Doesn’t this comment kid of contradict your last one?
JC
23 Feb 10 at 11:54 pm
Terje: If we keep on pouring CO2 into the atmosphere and if it doesn’t get hotter then the AGW theory is falsified.
Good point. We’ve been doing it for about 150 years in serious amounts and have a result already.
What’s say we keep going for another 150 years and see if we can repeat those results? Falsification you know, very important.
What’s that? You say that’s not a good idea because we’ll have wrecked the climate, our economy and our civilization by the time the confirmation is in?
Well I say “Karl Popper” to you.
JM
24 Feb 10 at 12:03 am
JC, ahh no. I don’t think I’m contradicting myself. Could you explain? Maybe I am.
JM
24 Feb 10 at 12:05 am
Dave I’m not entirely sure what your point is. It seems to boil down to 2 propositions:
a.) do I “agree”. I presume you mean do I think AGW is real
b.) am I “alarmed”. I presume you mean do I think our response should be precautionary and proactive (aka “mitigation”) or relaxed and reactive (aka “adaptive”).
If so my answers and simple, and I would have thought obvious by now.
I think AGW is real and the proof is in. I think mitigation is the only responsible course.
In my view, anything else is stupid beyond imagining.
Feel free to disagree.
JM
24 Feb 10 at 12:19 am
You said:
So in summary we’re expected to accept the views of cranks because the poor little wilting flowers are “outside the club” and can’t get their views published?
Really?
I read this to suggest you didn’t want to allow sceptics entry through the peer review door as you don’t think they are worthy.
The second last comment suggests you’re not that concerned with peer review because of its weaknesses.
How do you know after the climategate emails and DOc Pach’s IPCC fiascos what is good science and bad science in terms of being able to predict.
JC
24 Feb 10 at 12:23 am
I read this to suggest you didn’t want to allow sceptics entry through the peer review door as you don’t think they are worthy.
Then you read it wrong. Anyone is able to say anything they like, but I’m not going to bother paying attention to it unless they can get at least the minimum respect from their peers that enables them to publish. If they can’t meet minimum professional standards I don’t think their views are worth any more than the dentistry skills of my young son armed with a Dremel.
The second last comment suggests you’re not that concerned with peer review because of its weaknesses
Peer review has flaws you could drive a truck through. But it is a minimum standard.
The poster here is arguing for tearing even that minimum down and letting the cranks have the floor.
Sorry. Can’t agree with that.
JM
24 Feb 10 at 1:19 am
JM is correct. Given the ubiquity of scholarly/academic ‘peer-reviewed’ journals in every discipline imaginable, a peer-reviewed publication should be the necessary, even if not sufficient prerequisite.
Peter Patton
24 Feb 10 at 8:18 am
“Peer review has flaws you could drive a truck through. But it is a minimum standard.”
I agree with that but one of the flaws is that a field can be captured by one school of thought so that no-one with a dissenting view can get published.
Leaving aside AGW, how could someone who believed that pomo and its variants are nonsense be published in a sociology journal?
They could do a Sokal type hoax I suppose.
ken n
24 Feb 10 at 8:24 am
ken n
That is not true. It is not all that difficult to start new academic journals. Run a JSTOR or Web of Science test and see countless journals in every academic discipline going.
Peter Patton
24 Feb 10 at 8:30 am
Yep, done that.
Can you suggest a climate journal that would accept a paper casting doubt on the consensus of a sociology or cultural studies journal casting doubt on pomo?
The Jones emails showed how hard some scientists work to keep stuff they don’t like out of journals.
Do you doubt that some fields and their journals are captured by a prevailing view? And that that is a problem for scholarly enquiry?
ken n
24 Feb 10 at 8:41 am
ken
I would say most journals in most fields are captured to some degree or other. But this is not so surprising, Kuhn and all that. That is why new journals start up every week.
Establishment journals, by definition, tend to become mouthpieces of “normal science,” by focusing on within-paradigm “puzzle-solving” incremental research.
OTOH, newer journals focus on the “anomalies” that ‘fringe’ scientists see in the “normal science.” As these published anomalies increase, more within-paradigm scientists become converts to the “revolutionary science” being published in the newer journals.
Given how relatively new climate science is, it is not surprising that there are not a lot of journals in the ‘anomalies’ space as yet, but clearly an increasing number of scientists are taking notice of anomalies. And the fact they are receiving communication of these anomalies in the first place suggests that the peer-review process is operating as expected.
As for critiques of pomo as used in cultural studies, sociology, history, literature, anthropology, psychology, biology, or wherever, come on, there are literally thousands of journals publishing critical research every day.
Peter Patton
24 Feb 10 at 8:58 am
JM…
Dave I’m not entirely sure what your point is. It seems to boil down to 2 propositions:
a.) do I “agree”. I presume you mean do I think AGW is real
b.) am I “alarmed”. I presume you mean do I think our response should be precautionary and proactive (aka “mitigation”) or relaxed and reactive (aka “adaptive”).
.
no. Let me spell it out to you again. Do you agree with the following statement from the article?
.
In numerous areas and most notably in climate science, research has become a cause that is politicised and moralised.
.
If you agree, do you think it matters?
daddy dave
24 Feb 10 at 9:19 am
just in case you missed it JM, I’m wondering if you agree with this claim:
daddy dave
24 Feb 10 at 9:20 am
Good point. We’ve been doing it for about 150 years in serious amounts and have a result already.
.
Wrong. That’s inventing the hypothesis after the fact.
daddy dave
24 Feb 10 at 9:22 am
“If they can’t meet minimum professional standards I don’t think their views are worth any more than the dentistry skills of my young son armed with a Dremel.”
So I assume you deride the academic and dentistry skills of the characters in the CRU scandal?
Semi Regular Libertarian
24 Feb 10 at 9:43 am
[...] made this comment in the Peer Review thread. The laws on libel or slander are scandalous in Britain, a whole book had [...]
The British threat to free speech at Catallaxy Files
24 Feb 10 at 2:47 pm
Dave I didn’t miss it, you don’t need to repeat yourself.
In numerous areas and most notably in climate science, research has become a cause that is politicised and moralised.
I disagree with the use of the word research, otherwise the statement is obvious and trivial.
Substitute the phrase “some interpretation of research” and I’ll agree with it.
I don’t think the research is politically or morally driven:-
a.) much of it is data collection and is incapable of being influenced by political or moral considerations – numbers are just numbers.
b.) heaps of the theory comes from non-climate related areas
c.) most of the research and theory predates any moral and physical considerations. (I bang on a lot about radiative physics, that comes from the early to mid 20thC, and the other side of the AGW model – thermodynamics – comes from the 19thC for example.)
The results of the research have been manifest for a long time (in the form of implemented technology in many cases) long before we noticed that we have a problem with CO2 emissions.
The moral and political interpretation of the results is relatively recent.
But yes, I’ll agree that AGW has moral and political implications.
JM
24 Feb 10 at 8:42 pm
Sorry Dave I forgot your question:
If you agree, do you think it matters?
Yes I think it matters. But I don’t think you should object to the cause if you don’t like the effect. Don’t argue that “I don’t like the political implications therefore the science isn’t real”
As some character once said – “If you don’t like my answers, you should cease asking scary questions”
JM
24 Feb 10 at 8:46 pm
c.) most of the research and theory predates any moral and physical(sic) considerations.
I agree.
More seriously, what you say avoids DD’s point; to say that they have been moralised/ politicised is to say that in the past research was conducted in a manner free from moral and political considerations. The fact that “the theory” predates moral or political considerations doesn’t diminish the moral and political considerations that have since exerted themselves upon the research over the last twenty-odd years.
dover_beach
24 Feb 10 at 8:53 pm
JM:
I disagree with the use of the word research, otherwise the statement is obvious and trivial.
.
well, I stand by the word research. It’s getting to the point where entire topics are just not trustworthy (not just climate).
.
a.) much of it is data collection and is incapable of being influenced by political or moral considerations – numbers are just numbers.
.
data collection is highly corruptable. It’s a likely source of biases.
daddy dave
24 Feb 10 at 10:07 pm
Dave and Dover
Let me address both of your points – theory and data – from the perspective of timescale (with a little bit of peer review chucked in)
Data collection is not highly corruptible. Not when it is simple like thermometers and has been collected from a wide variety of sources over about 400 years. You can’t say that measurements of temperature taken continuously in central England over a period of centuries were corrupted by a moral and political argument we’re having now.
The theory is similarly not corruptible. The basis for the energy balance argument that I usually mount hinges on two things:-
a.) thermodynamics from the 19thC, particularly as formalized by Lord Kelvin
b.) radiative physics where the way forward was first proposed by Max Plank in 1900 with black body radiation and Einstein in 1905 with the photoelectric effect. The combination of those two led to the atomic bomb, lasers, the semi-conductors in modern computers, the whole box and dice.
Now. Were Kelvin, Planck and Einstein corrupted by our modern problem with CO2? You’ve gotta be kidding me.
But perhaps you notice that Einstein’s 1905 photoelectric paper (for which he got the Nobel) was peer reviewed by only a couple of people – one of whom was Max Planck (who got the Nobel for his 1900 paper that proposes the quantum as the basis for the black body effect, Einstein identified the mechanism that makes quantum theory plausible)
If you were a denialist operating today, and these papers were published today; you would now be screaming “conspiracy”. Planck was only promoting Einstein because Einstein supported Planck’s theory – looks like mutual backscratching doesn’t it?
But you would be wrong. In 1905 I would have taken the opinion of Planck over the top of any number of people who thought Einstein was wrong (and there were a lot).
And today I’ll do the same with today’s senior scientists who are reviewed by their senior colleagues.
People like retired TV weatherman Anthony Watts I’ll just ignore (at least until the point where I see intelligent people being taking in by them).
JM
24 Feb 10 at 11:04 pm
“Data collection is not highly corruptible.”
Okay, so why did they cherrypick 1/36 trees in Yamal?
Semi Regular Libertarian
24 Feb 10 at 11:16 pm
JM,
The bottom line is that I am far more cynical than you, and far more pessimistic about the ability of the bulk of scientists working today to produce clean, unbiased, easily replicable findings.
.
I can indulge naivete, but you’re in error in a couple of ways. First,
Data collection is not highly corruptible. Not when it is simple like thermometers and has been collected from a wide variety of sources over about 400 years. You can’t say that measurements of temperature taken continuously in central England over a period of centuries were corrupted by a moral and political argument we’re having now.
.
I don’t say that. However, even seemingly simple data collection operations can have hidden complexities, especially when they are on a large scale. Measurements fail, or get collected at the wrong time; errors are made in transcription; adjustments are made using secondary data sources that are less reliable… and so on.
.
In short: data is hugely corruptable. I’m not going to do any further selling of this fact. We’ll simply have to agree to disagree, because if you don’t understand that, then you’re not even at first base in understanding how real science works.
.
The theory is similarly not corruptible. The basis for the energy balance argument that I usually mount hinges on two things:-
.,
However, none of those men predicted global warming. This tells us that there are additional steps between their solid theoretical work and the AGW hypothesis.
.
And today I’ll do the same with today’s senior scientists who are reviewed by their senior colleagues.
.
I’m confident your view on this will change over the coming decade.
daddy dave
24 Feb 10 at 11:19 pm
As an aside, JM, I’m actually a bit envious of your boundless faith and optimism in science and scientists.
daddy dave
24 Feb 10 at 11:22 pm
errors are made
Yes Dave, they are, and they have many sources. We know they’re there, even if we can’t identify them all.
That’s why we have statistical methods to analyse their magnitude and give confidence levels on our conclusions. Regardless of the underlying errors and their source.
none of those men predicted global warming
True enough. But global warming was predicted empirically before them, their work explained the mechanism (aka theory) even if they didn’t realize it.
JM
24 Feb 10 at 11:25 pm
Dave I’m actually a bit envious of your boundless faith and optimism in science and scientists.
Here’s why
That’s the microwave radiation from the Big Bang. The solid line is theory. All the tiny little dots on the line that you can barely see because they fit so well, are the empirical data.
Science works.
JM
24 Feb 10 at 11:28 pm
And what happens when your “peers” are Media Studies and ‘Education’ types? Well, you get people arguing that Galileo was in error, not a “dissenter,” the sun really does circle the earth.
Enjoy.
“Dissent” is possible only in matters of opinion, politics, ethics, philosophy, etc. It is impossible in principle to “dissent” from fact. If I wish to assert that the sun did not rise this morning, that is error, not “dissent”.
Peter Patton
25 Feb 10 at 12:06 pm
Oh, I forgot the little prat’s conclusion to his Own Goal 101 lecture.
Until then, they’re just crank denialists who no more deserve to apply to themselves the noble labels of skeptics (Gallileo, Copernicus, Columbus) or dissenters (Mandela, Gandhi, Walesa) than do the dust bunnies under my bed.
Peter Patton
25 Feb 10 at 12:09 pm
Data collection is not highly corruptible. Not when it is simple like thermometers and has been collected from a wide variety of sources over about 400 years. You can’t say that measurements of temperature taken continuously in central England over a period of centuries were corrupted by a moral and political argument we’re having now.
No one is saying that those who collected data three centuries earlier were corrupted by contemporary moral or political considerations, JM. But speaking of central England temperatures:
http://motls.blogspot.com/2010/01/warming-trends-in-england-from-1659.html
Now. Were Kelvin, Planck and Einstein corrupted by our modern problem with CO2? You’ve gotta be kidding me.
What utter nonsense. Radiative physics and thermodynamics do not exhaust climate science, JM. The models are likely to be wrong even if many of the physical relationships, etc. they piggy-back on are right because they are likely to be wrong in respect of those other relationships, etc. of which Kelvin, Planck, and Einstein had nothing to say.
And today I’ll do the same with today’s senior scientists who are reviewed by their senior colleagues.
This is disingenuous tosh. What you need to do, JM, is to compare the work of people like Koutsoyannis, Pielke, etc. who are themselves senior scientists in their field with the work of their alarmist senior colleagues rather than mouthing on about Watts. You really must lack confidence in your chosen “senior scientists” if you only ever compare them with Bolt or Watts.
dover_beach
25 Feb 10 at 1:08 pm
[...] The repeated use of the argument from authority by environmentalists 23 02 2010 H/T Catallaxy blog. See http://catallaxyfiles.com/2010/02/22/peer-review/ [...]
The repeated use of the argument from authority by environmentalists « Tim R
26 Feb 10 at 11:50 am
[...] a proper scientist is always sceptical and an interesting article on your beloved peer review Peer review at Catallaxy Files "That government is best which governs least." "This is a sharp Medicine, but [...]
Hansen again
4 Jun 10 at 8:04 pm