ABC Fact Check: Error-ridden and misleading I

Cats will recall that Chris Berg and I recently calculated that ABC journos are nearly 5 times (precisely: 4.74 times) more likely to vote Greens than the general Australian population. I wrote about it recently here at the Cat.

To remind ourselves this is what Chris and I did:

What Chris and I did was match that peer reviewed research with data from the Australian Electoral Commission results for the 2013 election and then report that ABC journalists were almost 5 times more likely to vote Greens than the general population.

This has caused a huge kerfuffle and now is the subject of an RMIT-ABC Fact Check analysis.

Their conclusion: Flimsy.

Now it isn’t quite clear what “flimsy” means – I’m sure they would have loved to have concluded “False”, “Wrong”, “Misleading” etc. Anyway let’s have a look at what the ABC Fact Check unit did.

It is important to have a clear understanding of what Chris and I did: we compared results from a peer-reviewed survey of journalists (conducted by Folker Hanusch) to the actual election results. When the ABC Fact Check unit asked me what we had done I explained that to them, sending them a link to the Cat post where I described it, and also a cut and paste from our book where we discuss it.

Okay – so what did they do?

Step 1: Ask psephologist Kevin Bonham is this is a valid comparison. He suggests that rather than compare voting intentions to an actual election that we should have compared the voting intentions to Newspoll.

Dr Bonham suggested using an average of the Greens primary vote taken from the Newspolls conducted in the period in which Professor Hanusch’s survey took place.

Step 2: Ask Associate Professor Jake Olivier about that comparison. What comparison you ask? The comparison that Chris and I did, or the comparison suggested by Kevin Bonham? Well look at the answer Jake Oliver gives (emphasis added):

I think the quality of the data makes it difficult to draw strong conclusions here. This is compounded now by comparing results from very different surveys (Hanusch’s versus Newspoll). 

Just to remind everyone – we didn’t compare Folker Hanusch’s sample data to Newspoll.

Step 3: Ask Folker Hanusch what he thinks (emphasis added):

In particular, I would caution against comparisons to the results of this study with much larger and far more robust studies of the voting intentions of the general Australian public, in the way that appears to have been done here.

I really want to belabour the point; we did not compare Folker Hanusch’s survey results to Newspoll.

So all the talk so far about Newspoll etc. so far is all a distraction. I don’t know if the ABC Fact Check people are hoping that nobody would have read that far to realise its bunkum, or if they managed to get themselves confused. But let’s soldier on.

Step 4: Estimate using the Folker Hanusch data and Newspoll how many ABC journalists vote Greens. So it looks like the ABC Fact Check people commissioned Jake Olivier to do some modelling for them

Step 4a: Simply add back the non-responses in the original data to the analysis assuming they are non-Greens voters and calculate some confidence intervals and conclude:

“If I compare the ABC and Newspoll results for Greens preference in a simple analysis, ABC journalists are 2.4 times as likely to prefer Greens than the general public,” Associate Professor Olivier said.

Associate Professor Olivier gave his calculation a 95 per cent confidence interval of between 1.5 and 3.8.

This means that it can be said with 95 per cent confidence that the real figure falls in between these two values.

“You’ll notice that [the number five] is not in the confidence interval,” Associate Professor Olivier said.

At this point smart people would have quit while they’re ahead.  This would have left Chris and me having to argue that ABC employees being 2.5 times more likely than the general population voting Greens is still a big number and we disagree with his adding back non-responses and blah, blah, blah. People would lose interest, stats is hard, and everyone would move on.

But no.

Step 4b: Do a more sophisticated analysis.

Associate Professor Olivier also calculated an odds ratio, using a logistic regression model, which includes all outlets (ABC, Fairfax and News Corp) as well as Newspoll.

Using this model, he calculated that ABC journalists have a 2.9-fold increase in the odds of preferring the Greens over Newspoll respondents, with a 95 per cent confidence interval between 1.6 and 5.2.

To paraphrase:

This means that it can be said with 95 per cent confidence that the real figure falls in between these two values.

You’ll notice that [the number 4.74] is within the confidence interval. 

So the ABC Fact Check commissioned research confirms that our guesstimate of how many times ABC employees are more likely to vote Greens than the general population falls within the 95% confidence level of their best guess.

Maybe “flimsy” means “not robust” and I’m happy to admit that Chris and I did a quick analysis (we actually focussed more on voting intentions across the three media groups ABC, Fairfax and News than against the general population in our book).  But I’m happy to accept that our figure 4.74 (or nearly five as we described it)  has been subjected to a more sophisticated analysis and found to be robust. We’ll be citing that robustness check in our second edition.

So while we said that ABC journalists are nearly five times more likely to vote Greens than the general population, the ABC Fact Check shows that ABC journalists could be more than five times more likely to vote Greens than the general population.

There is heaps more to say about this “Fact Check” but this is a long post already and so it’ll keep for the next few days.

This entry was posted in Books and writing, Media, Shut it down. Fire them all., Taking out the trash. Bookmark the permalink.

43 Responses to ABC Fact Check: Error-ridden and misleading I

  1. Roberto

    The ABC is hoist with its own petard…

  2. Tim Neilson

    Simply add back the non-responses in the original data to the analysis assuming they are non-Greens voters

    That alone is enough to disqualify them from being trusted.

  3. MJ

    Often “back of envelope calculations” give obvious answers. Similarly just listening to most of them (with a few exceptions ) tells you they are likely to vote green.

  4. Farmers Market Stall: There are six apples in this basket.

    Their ABC Fact Check: It appears that the basket had contained oranges previously and may do so in future, so to assert that the basket contains six apples is somewhat be misleading (Their ABC Fact Check also couldn’t ascertain whether the apples were organically grown).

  5. Up The Workers!

    These are people whose political Party is too dumb to spell its own name correctly.

    Labor(sic) – It has no place for “U”!

  6. H B Bear

    Actual idiots (aka j’ismists) consult IYIs.

  7. Rococo Liberal

    How many taxpayer dollars were wasted on this farrago?

  8. John Constantine

    Flimsy is the feeling word that reassures the gang that all those that oppose them are unsound.

    This is the easiest storytelling.

    Comrades.

  9. Roger

    Labor(sic)

    King O’Malley’s legacy – an American interloper.

  10. Squirrel

    “So it looks like the ABC Fact Check people commissioned Jake Olivier to do some modelling for them”

    If so, this would be particularly interesting in light of the ABC getting stuck into the ATO today for spending a million dollars on client satisfaction surveys of people who have had a formal dispute with the ATO –

    http://www.abc.net.au/news/2018-07-17/million-dollar-ato-survey-finds-we-think-tax-office-is-unfair/9999744

  11. Rebel with cause

    It’s their ABC.

  12. Herodotus

    I don’t know if the ABC Fact Check people are hoping that nobody would have read that far to realise its bunkum,

    This would be the SOB.

  13. duncanm

    I don’t know if the ABC Fact Check people are hoping that nobody would have read that far to realise its bunkum,

    every .. single .. fact check which doesn’t match the ABC agenda is approached this way.

    Pages and pages of waffle, to justify a ‘flimsy’ or ‘not really’ verdict.

    .. and you notice they completely avoid the point. It is not that 4.7 is the correct answer, it is (and they confirm) that ABC journalists are much more likely to vote green.

  14. Sydney Boy

    ABC journos are 100% more likely to vote for the Greens than the average person. Anyone who says differently is lying.

    Perhaps the people who authored this Fact Check are the same who declare the average Q&A audience is composed of 40% conservative voters?

  15. Oh come on

    It’s pretty astonishing that the ABC’s fact check unit would investigate a claim against the ABC. Any conclusions it draws are going to be highly questionable.

  16. nemkat

    Their ABC Fact Check: It appears that the basket had contained oranges previously and may do so in future, so to assert that the basket contains six apples is somewhat be misleading (Their ABC Fact Check also couldn’t ascertain whether the apples were organically grown).

    While pretending to have a go at the ABC, you’ve had a free whack at Organic Farming.
    The facts are that the ABC has always been hostile to Organic Farming, and The Greens official policy is that it is elitist, and shouldn’t be supported.

  17. .

    The facts are that the ABC has always been hostile to Organic Farming

    Oh dear god. The stupid is weapons grade.

  18. While pretending to have a go at the ABC

    Idiocy beyond the pale.

    I wasn’t pretending to have a go at Their ABC, I was having a go at Their ABC.

    And on organic farming, Most of it is just a scam.

  19. Tel

    .. and you notice they completely avoid the point. It is not that 4.7 is the correct answer, it is (and they confirm) that ABC journalists are much more likely to vote green.

    It’s a classic Trump-trolling technique. First make a claim of something true but put some simple spelling mistake in it or small exaggeration. Then the bleating herds are struck with the compulsion to make a big deal over that little mistake so they can crow about how smart they are, and thus distribute your message to all the normal people who get the gist of it and don’t give a crap about the difference between 2.5 and 4.7 or whatever.

  20. nemkat

    And on organic farming, Most of it is just a scam.

    Maybe.
    That’s nothing to do with the topic, which is the astonishing propensity of ABC staff to vote Greens, coupled with their extreme sensitivity over being called out on it.

  21. nemkat

    Assuming that ABC staff are quite well informed, and on a pretty good wicket salary wise, i’d wonder what they find attractive about The Greens.

  22. Siltstone

    The “ABC Equity and Diversity Plan 2016-2018” has “Focus Areas”, one of which is to “Maintain or improve the representation of diversity groups by Division and review the areas of under-representation for targets.”
    The non-Green sector is clearly under-represented and, if one were to delve more closely, the non-left side of “diversity groups” is, in effect, not represented at all by the ABC. Hypocrisy and the ABC are not strangers.

  23. Nerblnob

    Nobody except the complainant reads an ABC “Fact Check” with any aim other than confirming their prejudices were right.

    They will not read any further than “flimsy” if they were brave enough to even get past (trigger warning!!!) IPA.

    Even if they did, they would be faced with (trigger warning!!!) numbers and statistics which is like Martian to them

  24. nemkat

    Hard to know what to make of the heavy ABC/The Greens connection
    The Greens are an Authoritarian Party, their Party Conferences are private, no part of the Media appear concerned.
    Their Policies are pitched to uninformed younger people, and are no less incoherent than Hanson’s, so what’s the attraction for presumably well informed ABC people?

  25. Here’s a fact for you.

    The left and their acolytes will spend as much time, effort and taxpayer funds to try and discredit their bias and lies.

  26. stackja

    ABC doesn’t fact check. I am shocked! I assume ABC supporter disagrees. Scrolled by.

  27. eb

    My God the troll is so stupid.
    We gotta get a better class of troll I tells ya!

  28. Oh come on

    thus distribute your message to all the normal people who get the gist of it and don’t give a crap about the difference between 2.5 and 4.7 or whatever.

    The trouble with this is that the vast majority of readers don’t read beyond the second or third line of articles. With these fact checker-type outfits, they read the fact being checked and the verdict and move on. Very few people bother drilling down.

  29. Slayer of Memes

    I wonder if Matt Martino (who wrote the Fact-Check article) was one of those who answered Prof Hanusch’s original survey and, if so, what his answer was?

    And, if he indicated he voted ‘Greens’, would that in any way influence any article he was to then write (and publish) debunking any claims that the ABC leans ‘Green-Left’….

    Curiosity and all…

  30. duncanm

    Assuming that ABC staff are quite well informed, and on a pretty good wicket salary wise, i’d wonder what they find attractive about The Greens.

    They’re not well informed. They live and breath (in) an elitist inner-city bubble circle-jerk.

  31. Bruce in WA

    This would have left Chris and I having to argue

    … Chris and me …

    [Yes. Thank you. Sinc]

  32. J.H.

    Well, considering that the ABC and our ecofascist Greens, define all Facts that make them feel uncomfortable or upset as “Hate Facts”. It’s amazing you haven’t been investigated by the HRC….. But then again, you probably won’t know for another year.

  33. Within a few days that RMIT-ABC Faux Check will probably appear as an article on The Conversation.

  34. MichelLasouris

    Well Done ! I DO love it when these clever dicks are shown up to be the manipulative frauds they are….

  35. None

    This is gold. Even if the ABC were twice as likely to vote Green it is BIG NEWS and shows that its workforce lacks diversity.

  36. Malcolm Thomas

    Nice take down Sinc. I smelled a rat at the outset, but couldn’t be bothered to read the Faux Check and your original closely enough to yell exactly how. Perhaps that’s what Their ABC is counting on.
    Now that you’ve done the hard work, how about sending a complaint, and seeing if and how they respond.

  37. Baldrick

    It used to be YourABC, then it became OurABC and it’s now firmly TheirABC.

  38. NuThink

    Assuming that ABC staff are quite well informed, and on a pretty good wicket salary wise, i’d wonder what they find attractive about The Greens.

    What they are really voting for and find attractive are not the Greens but the Greenbacks, which are in abundance and over represented in the salary packages at their ABC.

  39. Entrop

    Yes. It is a bigger government they are really voting for including a bigger ABC and the bigger salaries that go with it. They are actually voting for watermelons.

  40. Tom

    So while we said that ABC journalists are nearly five times more likely to vote Greens than the general population, the ABC Fact Check shows that ABC journalists could be more than five times more likely to vote Greens than the general population.

    Excellent work, Professor Davidson.

    You have proven beyond reasonable doubt that most journalists are leftists who, without strict adherence to a professional code of ethics, can never be trusted to report news dispassionately in the public interest.

    In Australia, journalism’s code of ethics has been abandoned by all but the News Corp formula tabloids, Seven West Media’s The West Australian, News Corp’s broadsheet The Australian and Fairfax Media’s Macquarie radio news network (which nevertheless regularly uses strong opinionation in its “news” content).

    The ABC regularly flouts the code of ethics in its news reporting, has had a number of instances in recent years of false reporting/fake news driven by an ideological desire to believe rather than the facts and its current affairs content is off the charts as an offender.

    So of course the ABC would want to go the extra mile in using sophistry to obfuscate and attempt to discredit your findings.

    This is crucially important work for those of us studying the factors behind the news media’s loss of credibility and public trust.

  41. Des Deskperson

    The correct tactical response by the ABC would have been simply to point out:

    that it has a code of professionals standards that include a statutory duty to ensure that the gathering and presentation of news and information is impartial according to the recognised standards of objective journalism.

    That ABC journalists are obliged to comply with this standard

    that the private voting intentions of these journalist are therefore irrelevant.

    The fact that the ABC has instead spent a good deal of taxpayers’ time and money attempting to discredit Davidson and Berg is actually a de facto admission that it has a problem.

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.