Damn statisticians (and accountants)

I suppose an inquiry into the Bureau of Meteorology is not a bad thing, but I’m wondering what George Christensen hopes to find?

He seems to be suggesting that any process involving statistics (or accounting) is hopelessly compromised. Now that is simply not true. He also seems to think that an inquiry will show the Bureau is up to no good.

What such an inquiry will find is that index construction is a complex task that involves many more or less arbitrary value judgements. That if you make a whole bunch of different value judgements that you might get a different outcome, or might not. I very seriously doubt that any inquiry into the temperature record methodology is going to find a deliberate effort to show rising temperatures in the present relative to the past. That may well be the outcome of the actual methodology employed to create the time series. More likely we would find that the temperature time series isn’t particularly useful for policy purposes given all adjustments that need to be made to generate “consistent” data.

So the community would learn a lot of the limitations of aggregate data and the uses of those data, but not much about the intentions of the Bureau of Meteorology.

The overall lesson would be that we don’t know as much as we think we do – certainly not enough to be spending billions of dollars on direct action, or taxing carbon emissions and that the Bureau should stick to forecasting the weather.

This entry was posted in Global warming and climate change policy. Bookmark the permalink.

58 Responses to Damn statisticians (and accountants)

  1. MemoryVault

    any inquiry Inquiries by private researchers into the temperature record methodology is going to find have found a deliberate effort to show rising temperatures in the present relative to the past.

    FIFY Doomlord.

  2. Token

    The overall lesson would be that we don’t know as much as we think we do – certainly not enough to be spending billions of dollars on direct action, or taxing carbon emissions and that the Bureau should stick to forecasting the weather.

    Isn’t that a good outcome?

  3. closeapproximation

    Great. Just what we need, more polarised dumbass philistine witchhunting. Of course, BOM, CSIRO and every other scientific body on the planet have more or less brought this upon themselves.

    Further transparency/open data & process is what’s needed; anything less will be a waste of everyone’s time.

  4. blogstrop

    It’s worth doing to put BOM, CSIRO, and any other group of crypto-warmies on notice.

  5. stackja

    Wikipedia:If you lie down with dogs, you get up with fleas
    From Wikipedia
    If you lie down with dogs, you get up with fleas, or in Latin, qui cum canibus concumbunt cum pulicibus surgent. “He that lieth down with dogs shall rise up with fleas” has been attributed to Benjamin Franklin’s Poor Richard’s Almanack. The Latin has been unreliably attributed to Seneca, but not linked to any specific work.
    The quote has a large almost universally agreed meaning of “You should be cautious of the company you keep. Associating with those of low reputation may not only lower your own but also lead you astray by the faulty assumptions, premises and data of the unscrupulous.”

    Both BOM, CSIRO need cleansing.

  6. closeapproximation

    Both BOM, CSIRO need cleansing.

    So you want everyone to dig their heels in even more, or would you rather know the truth ?

  7. outsider

    Appoint Don Aitkin and Jo Nova to oversee the inquiry.

    I recall quite a few years back a helpful employee of the BOM wrote a simple program to report on the degree of errors of forecasting made by his employer, thinking it would assist in improving their performance. The bumbling incompetents sacked him. Nothing is beyond these zealots, I believe, they are the CFMEU of the ‘scientific’ community.

  8. peter cavanagh

    There is no doubt the start of the whole exercise deliberately excluded the federation drought and the so called homogenisation of data to lower previous figures is tantamount to fraud and heads must roll

  9. incoherent rambler

    Professor, when is it acceptable to adjust the raw data in your data sets and refuse to publish the original data?
    It may be ok for economics, but not for science.

  10. Sinclair Davidson

    … refuse to publish the original data?

    That is a problem (even for economics 🙂 ) – but has that actually happened? Yes, I know that Phil Jones etc. got up to this – but George Christensen is quoting original raw data.

  11. Infidel Tiger

    Bloody hell. What size suit is he wearing?

  12. Jack Lacton

    I very seriously doubt that any inquiry into the temperature record methodology is going to find a deliberate effort to show rising temperatures in the present relative to the past.

    Then you will be wrong. CSIRO/BoM use the same methodology that is used by all climate centres including NOAA, Uni East Anglia CRU etc. The effect of the methodology is to lower early 20th Century temperatures. A couple of years ago in NZ they sacked their chief climatologist (who was ex-CRU) due to the corruption of their temperature data that took an originally flat 20th C record and turned it into a +0.3C result. The original temperature data was lost in the process. NZ government allocated $10M and 3 years to recreate the original data.

  13. incoherent rambler

    but has that actually happened?

    Yep. The early temperature records. I would need to search for links at JoNova and Jennifer Marohasy.
    Related to this is the blustering answer from the BoM about publishing their algorithms for the production ‘high quality data sets’.

    Dishonest science is more than a spat between academics. It is a matter of critical national importance.

  14. DaveA

    I’m not so sure. Australia is Large, and much of it suffers from low historical temperature station density. The national mean is area weighted, as it should be, with many ‘blocks’ a product of high extrapolation from distant stations. So you end up with a few stations having a large impact on the national mean. Fiddling a station here and there can have high impact on the national mean time series.

  15. .

    incoherent rambler
    #1497757, posted on October 30, 2014 at 1:26 pm

    Professor, when is it acceptable to adjust the raw data in your data sets and refuse to publish the original data?
    It may be ok for economics, but not for science.

    It’s not. Most journals these days get you to submit data, source code or nominate the programme you use, along with the results and write up.

    Using anything but raw data is also frowned upon.

    The Darwin, NT adjustments of temp data are a bloody joke, along with the adjustments of other remote stations…using places hundreds of kms away as a proxy, the only pattern emergent is that the highest possible temp was chosen and this turned a shallow downward trend at Darwin for example, into a steep upward trend.

  16. Up The Workers!

    B.O.M. – Bureau of Mendacity.

    C.S.I.R.O. – Crap Science Inspired by Religious Oratory.

    I wonder what could possibly have aligned the “Scientologists” in both these bodies, with the A.L.P. and Brown Movement?

  17. Rob MW

    Sinc – it’s more a story of how BoM and other Climate Scientists nutters abuse the Bayesian methodology with their own made-up flim-flam (as yet unheard of) methodologies.

    Your statement: – “I very seriously doubt that any inquiry into the temperature record methodology is going to find a deliberate effort to show rising temperatures in the present relative to the past.” – is void given that Steve McIntyre, Nic Lewis and others have already conclusively proved beyond doubt that that is exactly what they have been doing. Take the Mann hockey-stick; the moron ‘spliced’ “present relative” temperatures onto the front end of “past” tree-ring proxies and called it science based statistical analysis of what appears to be a fraudulently plotted computer model.

  18. Tropical

    Actually Joanne Nova has an excellent article at her website.
    http://joannenova.com.au/2014/08/the-heat-is-on-bureau-of-meteorology-altering-climate-figures-the-australian/
    Not sure the link thingy will work – apologies if it does not work.

  19. Biota

    The problem is that there are a number of weather stations around Australia that have long-term records but these stations are not evenly geographically distributed; there are large areas in between with no data. There are methods that can be used to guess at what the data in between is such as various forms of Kriging, Spline Curvature etc. Some methods don’t honour the known data points and some do. Whatever method BOM uses does not honour the known data and this has resulted in a revision of what actually occurred at the known locations. The outcome almost always seems to be that early temperatures are lower and more recent temperatures are higher than the actual record. Rutherglen is a classic example.
    http://joannenova.com.au/2014/09/explain-this-rutherglen-homogenized-with-17-stations-including-hillston/

  20. Dr Faustus

    As DaveA and Biota point out, Australia has a sparse and incomplete meteorological recording system.

    Given Australia’s huge landmass, to obtain information for providing local weather forecasts and approximate historical climatic records at a usefully tight scale, data needs to be extrapolated between wide spaced recording stations. Similarly, with changes in stations and recording methodologies across time, raw data needs to be adjusted to create a normalised and consistent working data set.

    There is nothing conceptually wrong with this, the same approach is taken in many other geospatial data sets. But problems arise when the working data set is used out of context.

    If you are using the data to grid temperature/pressure gradients for weather forcasting purposes, normalised data estimated between stations is fine. What you doing is mapping estimated temperature changes and differences over a short time frame (days or weeks) and modelling how that should affect the weather. In this case, the relative consistency of the contemporary station measurements is at a premium and abolute station temperature values across longer time frames is unimportant to the task.

    This causes problems when you use the same normalised data set to measuring changes across longer periods of time, not days, but decades, to estimate changes in climate. Suddenly, the adjustment and extrapolation techniques, that don’t matter much on a daily time frame, become signals and artefacts in the working data set when viewed over 100 years.

    I suspect that has happened here and, although it is convenient from a warmist perspective, I doubt it is an organised conspiracy. The better approach for the BOM would be to be clear about its data adjustment processes and present the error bars on the working data set – rather than implying a precision that is not there.

  21. Elizabeth (Lizzie) B.

    Showing that data ‘created’ by such august bodies (in the public mind at least) at best produce indications, with uncertainty, not absolute factual numeric truths, wouldn’t be such a bad thing.

    If that was the message that clearly came out of it. Which it probably wouldn’t be.

  22. Streetcred

    Before you write on this topic again, Sinclair, I would recommend a conversation with the relevant researchers who have exposed the mendacity of the BoM … other commenters here have already mentioned them. There is no excuse to be ignorant of the what has been perpetrated by these charlatans around the world.

  23. MemoryVault

    Streetcred
    #1497913, posted on October 30, 2014 at 4:21 pm

    Wot ‘e sed, Mr Doomlord Sir.
    But with the utmost deference and servility.

  24. Brian

    Biota, I know two Queensland stations, both of them early immigration ports, that had temperature records going back to the late 19th century. The earliest records on the BOM site for both of these stations now start in the mid to late 1960’s. Where are the earlier reports?

  25. Ian G

    Check out this ‘Australian mean temperature anomaly’ graph to show the story behind many people’s concern with the BoM homogenised data set, ACORN.
    http://www.bom.gov.au/climate/change/acorn-sat/#tabs=Adjustments
    Prior to 1965 AWAP temps have been reduced.
    Between 1965 and the early ’90s, temp adjustment fluctuate.
    Since 1994, ACORN temps have been adjusted upwards.
    Ken Stewart has done a lot of research on this. His work is at:
    http://kenskingdom.wordpress.com/2014/10/04/adjustments-grossly-exaggerate-monthly-and-seasonal-warming/

  26. jupes

    I very seriously doubt that any inquiry into the temperature record methodology is going to find a deliberate effort to show rising temperatures in the present relative to the past.

    Well I would be gob-smacked if it doesn’t.

    Jennifer Marohasy has shown this is exactly what they have done with the ‘homogenising’ of the temperature record.

  27. buckshot

    This temperature record forms part of the Global Temperature Anomaly (GTA) on which the Anthropogenic Global Warming (AGW) theory is based. So even if the independent investigation does nothing other than increase the doubt associated with the readings, then it still may be sufficient to reject the theory.

    If it is reported that the GTA has increased by 0.8C over the past 160 years, but that the margin of error of the measurements and adjustments is now +/- 1.0C, then no conclusion can be drawn about whether the GTA has increased over this time => AGW theory not supported.

    There is also the possibility that movement in GTA from year to year is actually a random variable and the plots shown of GTA are just random walks. Is 0.8C rise over 160 years any different to me going to a casino and placing 160 bets and ending up $80 ahead? Other weather/climate phenomena exhibit randomness, like where lightning will strike next, or how much rain will Melbourne get next year.

    I can test whether my success at the casino is random, or causal, by recording the variance from the mean of each wager. But the inherent variance in the global average temperature is averaged out in deriving the GTA each year, and now this homogenisation process my be doing its part to try and make the temperature record look as monotonically increasing as possible.

  28. Sinclair Davidson

    Gentlepeople – what many of you are pointing to is that the adjustments have resulted in older temperatures being revised downwards. I’m happy to accept that this may have happened. What remains to be shown, however, is that is was a deliberate attempt to falsify the historical record or if it is a consequence of the methodology employed. There may well be good reason why that has occurred.

  29. James in Melbourne

    or if it is a consequence of the methodology employed. There may well be good reason why that has occurred.

    Oh, My God. They’ve got to the Doomlord.

  30. chrisl

    Truncated data. Homogenization.Does this not ring any alarm bells?

  31. .

    Sinclair Davidson
    #1497999, posted on October 30, 2014 at 5:39 pm

    Gentlepeople – what many of you are pointing to is that the adjustments have resulted in older temperatures being revised downwards. I’m happy to accept that this may have happened. What remains to be shown, however, is that is was a deliberate attempt to falsify the historical record or if it is a consequence of the methodology employed. There may well be good reason why that has occurred.

    What do the leaked emails say about this? They would be instructive.

  32. Biota

    Sinc, so you (yeah, I know, not really you) have actual records i.e. real raw data and stick that in the statistical blender coming up with a theoretical figure that you say is what the raw data really meant to say. That these records generally result in lower early temps and higher recent temps might not be deliberate, it could well be an artefact of the method. But it is a tad convenient for the AGW crowd. Maybe subconscious confirmation bias?

  33. jupes

    What remains to be shown, however, is that is was a deliberate attempt to falsify the historical record or if it is a consequence of the methodology employed.

    Ask yourself this: is there any chance whatsoever that they would have revised the older temperatures up? Any at all?

    There may well be good reason why that has occurred.

    Indeed.

  34. jupes

    More on my point above. Every time warmists have changed a record anywhere in the world, the result is exactly the same as their predictions. Always erring to support their (debunked) theory. Always.

    How could you possibly think that the BOM be any different?

  35. Ian G

    A new peer reviewed paper has recently been accepted re a review of New Zealand temps. It concludes that temps there have not risen by the amount the NZ bureau has stated. Maybe the same result may happen if there was a review of the BoM’s data.
    http://www.warwickhughes.com/agri14/deFreitasetalNZtrends.pdf

  36. Eyrie

    So DeFfreitas et al got +0.28 deg C per century PLUS OR MINUS 0.29 deg C. Could have cooled.
    Different from the +0.91 +/- 0.3 eh?

  37. Alfonso

    The Alfred P. Sloan Professor of Meteorology at MIT since 1983, Richard Lindzen, could NEVER be named to head any half-arsed Australian CSIRO or Met Dept relevant to CAGW.

    The only reason can be: he doesn’t reflect the proscribed / required CSIRO institutional agitprop.

    Or maybe he isn’t qualified for a low level Australian “scientific ‘ collective?

    “From 1972 to 1982 Lindzen was a professor of dynamic meteorology at Harvard University. From February to June 1975 he was a visiting professor of dynamic meteorology at MIT, and during part of 1979 Lindzen was a visiting professor at the Hebrew University of Jerusalem, before switching affiliations to MIT as the Alfred P. Sloan Professor of Meteorology in 1983.

    During this time, Lindzen published research on gravity waves,[42] as well as Hadley circulations.[43] Lindzen is named as one of 16 Scientific Members of the team authoring National Academy of Sciences’ 1975 publication, Understanding Climate Change: A Program for Action, full text and pdf scan available at the Internet Archive.[44]

    Lindzen is a recipient of the American Meteorological Society’s Meisinger and Charney Awards, American Geophysical Union’s Macelwane Medal, and the Leo Prize from the Wallin Foundation in Goteborg, Sweden. He is a member of the National Academy of Sciences (NAS), and the Norwegian Academy of Science and Letters, and was named Fellow of the American Academy of Arts and Sciences, the American Association for the Advancement of Sciences, the American Geophysical Union, and the American Meteorological Society. He is a corresponding member of the NAS Committee on Human Rights, and a member of the United States National Research Council Board on Atmospheric Sciences and Climate. He was a consultant to the Global Modeling and Simulation Group at NASA’s Goddard Space Flight Center, and a Distinguished Visiting Scientist at California Institute of Technology’s Jet Propulsion Laboratory. Lindzen is an ISI highly cited researcher,[73] and his biography has been included in American Men and Women of Science.[74]”

  38. I am been running quality control tests on the ACORN-SAT station data and the first results illustrate a number of the points raised here. The stats test itself may be of interest are its a panel test out of econometrics.

    A scorecard of sites shows Rutherglen, Deniliquin, and Kerang have trends where adjustments have made them inconsistent with the neighbouring 20 or so stations (i.e. >0.1C/decade warmer 95%CL) . (http://landshape.org/enm/scorecard-for-acorn-sat-quality-assurance-the-score-so-far/)

    Some stations probably obviously need adjusting like Williamtown RAAF (http://landshape.org/enm/williamtown-raaf-an-interesting-case/)

    Sinc:

    … refuse to publish the original data?
    That is a problem (even for economics 🙂 ) – but has that actually happened?

    The table of adjustments that Christensen was waving during his speach was available 2 years ago (I know because they sent it to me) but it was not published on their website as required by the peer-review of ACORN-SAT until just recently. Another example of hidden data is Kerang, where even though daily data is available to 1910 in ACORN-SAT it is truncated to 1965 in the raw database called Climate Data Online. The Bureau are now uploading hand-waving summaries attempting to justify the adjustments with cherry-picked comparitors (report at http://landshape.org/enm/bom-publishing-station-summaries-justifying-adjustments/) rather than the full analysis. There are a lot of stations opened prior to 1910 and the data is either not available from CDO or its still in the form of scanned sheets, yet ACORN-SAT only goes back to 1910. Makes claims of the ‘hottest on record’ tenuous at best.

  39. Rob MW

    “I’m happy to accept that this may have happened. What remains to be shown, however, is that is was a deliberate attempt to falsify the historical record or if it is a consequence of the methodology employed.”

    Sinc – put that the other way around. They have used their own made-up, u-beaut methodology in a “deliberate attempt to falsify the historical record”. To imply that they have not done this deliberately makes a mockery of their PHD’s. The rest of their premeditations is just lies and bullshit 🙂

  40. cohenite

    Thanks David; I was just going to link to your site. David’s methodology is straightforward and transparent; a quality sadly lacking from the BOM site.

    Also of note is this paper from New Zealand which substantiates claims made against the New Zealand equivalent of BOM, NIWA. Previously NIWA had successfully defended litigation brought against it for doctoring temperature data to increase the trend/warming. In that litigation the guts of the decision against the suitors against NIWA was their expertise was not equal to the scientific expertise of NIWA. The new paper by de Freitas et al now supplies the scientific gravitas missing from the prior litigation.

    David’s study along with Jennifer Marohasy, Jo Nova and members of a study group, supplies sufficient gravitas in this country to justify a close examination of the BOM.

  41. Andrew

    Christiensen, although a National, is better than at least half of the Liberal MP’s in the Parliament. He actually fights against the Nanny State.

  42. .

    He’s pretty good save for his idea that welfare ought to be drug tested.

    Drugs are cheap, food, housing and fuel are not.

  43. Thanks Cohenite. Its good to see that paper but I tried to find it in Google Scholar to see if it had been cited and its not even listed! As you know I have a couple of audits of the BoM published in E&E and even they are in Scholar (though they have never been cited or acknowledged by BoM). But you write them a letter and they say they will only engage in debate in the peer revewed literature.

  44. jumpnmcar

    He actually fights against the Nanny State.

    Yep.
    My tick went next to his name on the ballot paper in Dawson.
    He’s got a good man in his office too, Denis O’Riely, prompt replies to my questions.
    I deliberately worded my emails to hide any ” leaning ” yet was treated great ( even got passed up the chain resulting in a Minister reply [McFrarlane] )
    He fries lefties on twitter at times also.

  45. cohenite

    Yes David, BOM, like NIWA and every major, official temperature network, simply does not concede any doubt about their results and methodology. The fact that BOM are not transparent or even acknowledge manifest problems which people like you raise casts doubt on their temperature network.

    And don’t forget BOM is the lynchpin of government policy which costs $billions. If the Abbott government was genuine about balancing its budget it would question BOM and the policies which are based on it.

  46. DaveA

    I can’t see why the whole process couldn’t/shouldn’t be completely mathematically objective and script driven (‘turnkey’ as McIntyre’s calls his). No one should be eyeing charts to make decisions. Obviously a few parameters involved, and can allow different approaches for individual steps (e.g. neighbor inclusion). Known station changes can be defined and feed in.

    If they didn’t gate the data behind a web interface I suspect there would be a lot more public work done.

  47. Tel

    And don’t forget BOM is the lynchpin of government policy which costs $billions. If the Abbott government was genuine about balancing its budget it would question BOM and the policies which are based on it.

    At very least, Abbott could simply publish ALL of the BOM internal data, at zero cost to the budget and then stand back and let the chips fall where they may. If the Greens bitch he just says, “transparency”… job’s done.

  48. Tel

    So you want everyone to dig their heels in even more, or would you rather know the truth ?

    What’s that supposed to mean? BOM data is owned by government, you wouldn’t be suggesting that BOM employees (i.e. servants of the public) are likely to destroy property that does not belong to them. Perish the thought, that would be a crime.

  49. Blogstrop

    The same people who might “hide the decline” would want to omit inconveniently high 1930s temps, or other similar periods way back.

  50. Ian G

    No, Tel.
    The BoM might just ‘archive’ the data – just as GISS NASA has done with its v1 and v2 raw data set of individual w/stations.

  51. handjive

    “The IPCC (and CSIRO) relied heavily on the Mann paper in coming to their global warming conclusions.
    The paper’s climate curve was nicknamed the “hockey stick”: relatively flat from 1550 to 1900, with a sharp rise as greenhouse warming lifted global average temperatures.”

    http://www.theage.com.au/articles/2004/01/16/1073878029212.html

    Would the BoM like to dis-associate themselves from that statement?

  52. handjive

    October 21, 2014
    “Bureau of Meteorology officials, meanwhile, told Senate estimates on Monday that Australia was on a clear warming path, with temperatures rising between 0.71 and 0.76 degrees since 1960, depending on the methods used.

    Pre-Post homogenisation?

    http://www.smh.com.au/environment/climate-change/threat-of-air-pollution-to-worsen-along-with-global-warming-warns-climate-council-20141020-118u3k.html

    Meanwhile …

    Climate change a CSIRO priority as new chief looks to secure funding
    Larry Marshall says global warming is high on the public’s agenda because ‘the science is so compelling’

    http://www.theguardian.com/australia-news/2014/oct/15/climate-change-a-csiro-priority-as-new-chief-looks-to-secure-funding?CMP=twt_gu
    . . .
    What a bunch of 97% certified climate change quacks.

  53. Tel

    No, Tel. The BoM might just ‘archive’ the data …

    I reckon the best type of archive would be some sort of web based archive. Something along the lines of public data, for public use, paid for out of the public pocket. The sort of project a public spirited Prime Minister would want to be involved with.

  54. Just consider this fact re the BoM documentation for their ACORN SAT dataset –
    Canberra is classified as ‘Non-urban’.
    Just ponder the large arc of high rise offices, shopping precincts and terminals constructed post 2000. Apparently the BoM thinks that air artificially warmed by that large volume of concrete, increased albedo and associated energy output could never migrate a few hundred metres to warm the weather instruments just across the runways to the east.
    Just one example of a BoM blindspot.

  55. Rob MW

    Sinc – Bishop Hill is reporting that the New Zealand’s official temperature record has been refuted by a peer reviewed paper and accepted by Environmental Modeling & Assessment.

    From Bishop Hill

    New Zealand’s temperature record

    A Reanalysis of Long-Term Surface Air Temperature Trends in New Zealand was produced by principal author C.R. de Freitas with M.O. Dedekind and B.E. Brill.
    Abstract

    “Detecting trends in climate is important in assessments of global change based on regional long-term data. Equally important is the reliability of the results that are widely used as a major input for a large number of societal design and planning purposes. New Zealand provides a rare long temperature time series in the Southern Hemisphere, and it is one of the longest continuous climate series available in the Southern Hemisphere Pacific. It is therefore important that this temperature dataset meets the highest quality control standards. New Zealand’s national record for the period 1909 to 2009 is analysed and the data homogenized. Current New Zealand century-long climatology based on 1981 methods produces a trend of 0.91 °C per century. Our analysis, which uses updated measurement techniques and corrects for shelter-contaminated data, produces a trend of 0.28 °C per century.”

Comments are closed.