If you can’t settle the science, settle the policy

So we hear a lot about the need for ‘evidence-based policy’. But there is a huge elephant in the room:

The replication crisis (or replicability crisis or reproducibility crisis) is, as of 2019, an ongoing methodological crisis in which it has been found that many scientific studies are difficult or impossible to replicate or reproduce. The replication crisis affects the social and life sciences most severely.

That can’t be good.  Mind you, I can’t complain, I’ve done well out of replicating empirical policy relevant analysis and found them wanting.

The (one-sided) Conversation has a piece justifying why evidence- based policy is good even in the absence of the very evidence the policy is being based upon.

The National Party, Queensland farming lobby group AgForce, and MP Bob Katter have banded together to propose an “independent science quality assurance agency”.

To justify their position, Liberal-National MP George Christensen and AgForce’s Michael Guerin specifically invoked the “replication crisis” in science, in which researchers in various fields have found it difficult or impossible to reproduce and validate original research findings. Their proposal, however, is not a good solution to the problem.

Now another review board may or may not be a good idea. But have a look at the arguments:

In various scientific areas, including psychology and preclinical medicine, large-scale replication projects have failed to reproduce the findings of many original studies. The rates of success differ between fields, but on average only half or fewer of published studies were successfully replicated. Clearly there is a problem.

So far, so good. But:

It is tempting to respond to politically motivated attacks on science by simply pointing to the excellent track record of scientific knowledge, or the good intentions of the vast majority of scientists.

How do you reconcile “excellent track record”, with “on average only half or fewer of published studies were successfully replicated”?

But moving along? What does The Conversation recommend?

Second, establish a national independent office of research integrity to allow errors in the scientific literature, whether deliberate or accidental, to be corrected in a fair, efficient, and systematic way. 

Hmmmm. This differs from the independent science quality assurance agency” in what way?

This entry was posted in Education, Hypocrisy of progressives. Bookmark the permalink.

43 Responses to If you can’t settle the science, settle the policy

  1. Angus Black

    It differs in that it will cost more, achieve … well I was going to say less, but I’ll settle for nothing.

  2. Tim Neilson

    Hmmmm. This differs from the “independent science quality assurance agency” in what way?

    Very good question.

    Maybe Katter and the Nats mean “independent” in the English language sense, and the Monologue means “independent” in Middle-Class Pinko-Speak, i.e.
    “Independent” – utterly reliant on taxpayer funding and therefore deeply psychologically committed to a high tax and spend big government statist worldview (see e.g. “the ABC is an independent news source”).

  3. cuckoo

    “Second, establish a national independent office of research integrity”… to shut down academics who question the climate change agenda.

  4. Behind Enemy Lines

    I’ve been involved in this ‘evidence-based policy’ first hand, ever since it became a buzzword within the Commonwealth. Here’s a brief outline of how it works.

    Step 1: senior people arrive at policy decision without doing any research

    Step 2: junior people assemble evidence that supports that decision, and ignore evidence against that decision

    Step 3: presto – evidence based policy!

  5. Ivan Denisovich

    How do you reconcile “excellent track record”, with “on average only half or fewer of published studies were successfully replicated”?

    https://www.thecatholicthing.org/2018/12/21/science-in-the-unmaking/

  6. Behind Enemy Lines

    cuckoo
    #3167196, posted on September 25, 2019 at 2:51 pm
    “Second, establish a national independent office of research integrity conformity”… to shut down academics who question the climate change agenda.

    There you go.

  7. RobK

    Step 3: presto – evidence based policy!

    The department for the environment works this way with commissioned reports. Unreviewed.
    The subject of this post is important and unfortunately somewhat intractable as politics and money complicate things.

  8. jupes

    In various scientific areas, including psychology and preclinical medicine, large-scale replication projects have failed to reproduce the findings of many original studies.

    Richard Feynman:

    If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.

    We live in the age of ignorance.

  9. jupes

    The National Party, Queensland farming lobby group AgForce, and MP Bob Katter have banded together to propose an “independent science quality assurance agency”.

    I have zero confidence in anything The Mad Katter is involved with.

  10. Frank Walker from National Tiles

    If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.

    Hello dark matter…

    If commenters can understand some linear regression based statistics, this paper has basically destroyed AGW as a concern.

    https://www.earth-syst-dynam.net/3/173/2012/

    Polynomial cointegration tests of anthropogenic impact on global warming
    M. Beenstock1, Y. Reingewertz2, and N. Paldor3

    1Department of Economics, the Hii brooo University of Jerusalem, Mount Scopus Campus, Jerusalem, Israel
    2Department of Economics, the George Washington University, 2115 G St, Washington DC, USA
    3Fredy and Nadine Herrmann Institute of Earth Sciences, the Hiii Broo University of Jerusalem, Edmond J. Safra Campus, Givat Ram, Jerusalem, Israel

    Received: 06 May 2012 – Discussion started: 16 Jul 2012 – Revised: 24 Oct 2012 – Accepted: 30 Oct 2012 – Published: 21 Nov 2012

    Abstract. We use statistical methods for nonstationary time series to test the anthropogenic interpretation of global warming (AGW), according to which an increase in atmospheric greenhouse gas concentrations raised global temperature in the 20th century. Specifically, the methodology of polynomial cointegration is used to test AGW since during the observation period (1880–2007) global temperature and solar irradiance are stationary in 1st differences, whereas greenhouse gas and aerosol forcings are stationary in 2nd differences. We show that although these anthropogenic forcings share a common stochastic trend, this trend is empirically independent of the stochastic trend in temperature and solar irradiance. Therefore, greenhouse gas forcing, aerosols, solar irradiance and global temperature are not polynomially cointegrated, and the perceived relationship between these variables is a spurious regression phenomenon. On the other hand, we find that greenhouse gas forcings might have had a temporary effect on global temperature.

    How to cite: Beenstock, M., Reingewertz, Y., and Paldor, N.: Polynomial cointegration tests of anthropogenic impact on global warming, Earth Syst. Dynam., 3, 173–188, https://doi.org/10.5194/esd-3-173-2012, 2012.

  11. I_am_not_a_robot

    In climatology and I guess other specialties experience shows that a field can be captured by a prevailing view and so-called peer-review becomes pal-review, it gets worse with any government-sponsored supervising agency.
    The problem in government-funded climatology and related fields is uniformity of view has been enforced (see climategate emails) causing a Lysenko-style lack of diversity of opinion, competition or rivalry.

  12. Pyrmonter

    More a Rafe point, but … isn’t the idea of an ‘independent … agency’ fundamentally anti-scientific? What would Karl Popper say?

  13. Bruce of Newcastle

    establish a national independent office of research integrity

    Translation:

    research with integrity = left wing fads
    research without integrity = results unpopular with the Left

    If you don’t believe me, ask yourself who will be the star chamber appointees staffing this government funded critter? One guess.

  14. I_am_not_a_robot

    Government-funded meteorologists and climatologists ought to stick to their knitting viz. meticulously collecting and organising unadjusted raw data not theorising, in climate that can be left to future generations, maybe after a couple of centuries.

  15. Mother Lode

    Independent?

    Like the ABC?

    Demands the right to embark on political adventures as if beholden to no one but their own political biases?

    What could possibly go wrong.

    The ABC pretends there exists on their independence – which is to say what everyone else owes them*. The actual obligation is impartiality – what they owe us.

    Yet no politician will insist on this distinction.

    Once a new ‘science body’ is established we can expect much the same. Can we at least call it ‘The Peoples Lysenkoist Science Directorate’? At least we can get a little of our money’s worth in the form of an honest, non-1984 name.

    *(Why would we have an entity established which serves no purpose except itself?)

  16. Mother Lode

    The ABC pretends there exists on an obligation regarding their independence

  17. As a synthetic chemist (yes, really! I was synthesized) prior to publication in various journals, rule 1: repeat the synthesis (start to finish), until reproducible.

    Inconceivable that people would ignore rule 1.

  18. Mak Siccar

    I sent the following article to my former boss and much admired colleague, a now-retired Professor of Structural Engineering whose many national and international peer-reviewed publications centred around very complex modelling of both simple and complicated engineering structures.

    https://quadrant.org.au/opinion/doomed-planet/2019/09/a-climate-modeller-spills-the-beans/

    This was his response:

    Mak Siccar
    A great article on how models are abused and distorted to fit a political agenda.

    Re GIGO: I like this quote from the book “Buckling of bars, plates and shells by R.M. Jones:

    “My friend Jack Martinelli said the common acronym GIGO does not mean just ‘Garbage In, Garbage Out’, but really means ‘Garbage In, Gospel Out’ because many young engineers (as well as some who are not necessarily supposed to know better) tend to bless any results that come from a computer even if the input and the understanding are garbage.”

    Re Nonlinear effects: I said this in a previous email to John (another colleague) and an unnamed believer in the scam:

    “If you can get such different results for such a simple structure as a two-bar truss, imagine what happens when you try to model the earth’s climate, which we have known since the 1960s to be chaotic in the mathematical sense, when you have poorly-understood models, missing significant factors, crude to non-existing data with large error bars, non-uniqueness of solutions due to inherent nonlinearities in the system, poor numerical resolution, etc, etc. The non-uniqueness aspect is particularly important. If you could somehow wave a magic wand and change the CO2 level back to the level it was in your favourite year, there is essentially zero chance that the climate will revert to what it was in that year. It also beggars belief that such a complex system as the earth’s climate is essentially controlled by one factory only, namely the C02 level.”

    I’m hopeful that with more scientists calling out this scam, sanity will finally prevail.

    Needless to say, I share his views but at the moment, not his optimism that sanity will prevail in the near future.

  19. RobK

    Frank Wf T,
    John Reid has similar conclusions from his analysis and went on to write a book explaining where he thinks it went wrong.

    THE FLUID CATASTROPHE
    http://fluidcatastrophe.net
    AVAILABILITY, CITATION AND CONTACT
    The title, The Fluid Catastrophe, refers to the failure of the Navier-Stokes equations of Fluid Dynamics to properly account for the entropy associated with turbulence. These equations describe a mythical fluid “continuum” which is everywhere smooth and continuous. In this book a granular description of the natural world is proposed which is more realistic, more closely related to observation and consistent with the Second Law of Thermodynamics. This shift in emphasis has repercussions for a number of disciplines as summarized in these pages viz:

  20. 2dogs

    The solution here requires a change in the way science is funded.

    Research funding should be paid for producing datasets, not papers.

    A proposed dataset should be put out to tender for various research bodies to collate it. Research bodies that have a history of poor quality datasets will, of course, find themselves at a disdvantage, or even excluded, from tendering.

    Writing useful papers get you a say in what datasets get proposed, but doesn’t give you funding.

    In any case, a segregation of duties should exist between dataset proposers, and those evaluating research tenders.

    Of course, you will be paid the same for the dataset regardless of what it ends up saying. The incentive to fiddle the figures to get a particular result must be removed.

  21. Frank Walker from National Tiles

    Very interesting Rob.

  22. Tim Neilson

    Of course, you will be paid the same for the dataset regardless of what it ends up saying. The incentive to fiddle the figures to get a particular result must be removed.

    Regrettably it’s part of human nature that the government will give hints as to the desired outcomes, and will steer funding to those who develop a track record of producing the “right” data.

    Just like how, in the unlamented days of RGR, members of the AAT who found in favour of lots of “refugees” somehow seemed to have longer tenure than ones who didn’t.

  23. egg_

    If it disagrees with experiment, it’s wrong. In that simple statement is the key to science.

    Hello dark matter…

    Isn’t there dark heat hiding deep in the Ocean?
    /ManBearPig

  24. egg_

    As a synthetic chemist (yes, really! I was synthesized) prior to publication in various journals, rule 1: repeat the synthesis (start to finish), until reproducible.

    Marxists + ABC Charter = Dogma

  25. Nob

    This differs from the “independent science quality assurance agency” in what way?

    It will be an institution more amenable to leftist capture. Which would presumably be its purpose.

    It’s weird – every leftist I know who is still a leftwinger, started out with contempt for scientists engineers etc, except maybe in the soft sciences like biology and zoology, aka “Nature Studies”.

    Then along came Global Warming and suddenly they’re all about The Science.
    Helpfully avoiding any specifics about fields or specialisations so that anyone from amateur paleontologists to psychologists can assume the sacred mantle of Science (and be counted among the mysterious 97%).

    The exception is of course, “Climate Scientist” a field that never existed before the panics and only exists to service these panics.

    Hands up if you went to a Uni with a department of Climate Science before 1995..

  26. Behind Enemy Lines

    RobK
    #3167227, posted on September 25, 2019 at 3:31 pm
    Step 3: presto – evidence based policy!

    The department for the environment works this way with commissioned reports. Unreviewed.
    The subject of this post is important and unfortunately somewhat intractable as politics and money complicate things.

    This is true, Rob, always has been. Money and politics won’t be denied. But at least in the old days one could distance himself professionally from imbecile policies that were imposed despite his advice. These days the juniors are expected to justify the higher-ups’ decision with “evidence” and then own the result. It’s a bit different from Sinc’s point about reproducibility, but the cause is the same: the higher-ups want to keep making imbecile policies, but escape responsibility for them.

    The public service has been turned into a machine for creating lies, and evidence-based policy is a meta-lie.

  27. Dr Fred Lenin

    Dont the government write the results of “indepenent inquiries “ months before the inquiry ?
    “We will appoint you to chair the inquiry ,we have already chosen the other members , here is your copy of the results we want ,ypou will be well rewarded for your time and be on the list for future inquiry chairperson jobs “. I thought that was standard procedure in politics ,always get the result you want ,good politics , its the Westmonster way .

  28. C.L.

    there is a huge elephant in the room

  29. RobK

    there is a huge elephant in the room
    Sez elephant gravitar.😀

  30. C.L.

    Ahahahahahaha.
    So then: leftism has polluted science and the scientific method, rendering most ‘scientific’ studies false and worthless. Enter the NATIONAL PARTY (er, bastion of disciplined thinking) – which actually proposes a solid ameliorative response to the crisis. Which leftists resent and want to block lest 1) the guilty ‘scientists’ are too unkindly repudiated; and 2) their ideology (masquerading as science) is publicly undermined (to the detriment of leftism).

  31. Mak Siccar

    C.L. Snigger snigger.

  32. Rex Mango

    Would be great to see an evidenced based study about conscripts who were sent to Vietnam against their will.

  33. Rex Mango

    Perhaps if created a position of say Chief Scientist, they would bring evidence based thinking to the table, which is free of politics.

  34. mem

    Mak Siccar
    #3167307, posted on September 25, 2019 at 5:01 pm
    “The non-uniqueness aspect is particularly important. If you could somehow wave a magic wand and change the CO2 level back to the level it was in your favourite year, there is essentially zero chance that the climate will revert to what it was in that year.”

    What a brilliant insight. Thank you for passing that on to us.

  35. W Hogg

    2 major differences:

    1) it’s not the solution that Katter666 proposed
    2) it doesn’t have scare quotes around it

  36. Entropy

    Hmmmm. This differs from the “independent science quality assurance agency” in what way?

    Hmm yes. They hadn’t thought their strategy all the way through: it would have been cleverer to appear to agree completely with Bob no detail under the big white hatKatter, then structure and staff the star chamber however they want.

  37. Tel

    You know how countries that call themselves “The Democratic …” typically are not very democratic?

    Yeah, well when a branch of government is called “The Independent …” you can be confident it’s intended to be ironic.

  38. Not bad from a dismal scientist. In decreasing order of laws of various disciplines to theories it goes something like this: mathematics – physics – chemistry – engineering – biology – geology/geophysics – (large gap) then climatology/law/geography bringing up the rear.
    Having used the rotary lie detector on many occasions I have had the occasional success interspersed with glorious failures. Severe counselling has been required on more than one occasion

  39. Nob

    You’re too harsh on yourself IRFM. I’d put geology/geophysics above biology in that hierarchy.

  40. Howard Hill

    But aren’t so many people guilty of this? Behind Enemy Lines spells it out as clear as a sunny day. Then you have people like Frank Walker from National Tiles, sprouting thorium reactors in nearly every post, which we all know is bullshit. Not to denigrate him in any way I actually agree with almost everything he posts and respect his opinions. BUT! even the most sensible of people can be corrupted by ideology. How do you prevent this human flaw from influencing the lives of countless people? Sometimes these people have everyone’s best intentions on mind.

  41. Nob

    You’re talking about O’Sullivan’s First Law.

  42. CameronH

    “This differs from the “independent science quality assurance agency” in what way”? You may well ask?

    When a national bureaucracy is set up the left can take it over and corrupt it for their own purposes in the same way that the CSIRO, The BOM, most of our public service, their ABC, the Universities have been taken over and corrupted.

    The best way to do this is to hire private contractors under a specific well defined specification. This should not be a shotgun approach but for each specific item. For example: To review the collection and analysis of the temperature records by the BOM. This should include everything from the standard of the raw sampling process and standards all the way through to the statistical analysis performed to do such things as “homogenisation”.

    This should be done by appropriately qualified people for each step. So called “climate Scientists” should not be involved. The review of the statistical analysis methodology, for example, should be done by a statistician who has no previous involvement with the climate science industrial complex.

  43. John A

    CameronH #3167793, posted on September 26, 2019 at 9:31 am

    The best way to do this is to hire private contractors under a specific well defined specification.

    No – the best way is to NOT create another body of administration.

    The best way is to make sure JCU loses AGAIN in their stupid fight against Peter Ridd.

    And to make sure that Izzy Folau wins his fight with the stupid RA Board who are bullying the International Rugby Federation to stop him playing elsewhere.

    And to ensure that George Pell’s appeal to the High Court is allowed and is won so that we get some sanity back into the legal system, where once again Truth might be allowed as a realistic defence.

    And follow up similarly with sold determination in every similar case that follows.

    This will signal the people can actually tell the truth without fearing for their livelihoods.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.