After the WTO plain packaging result was leaked last year – Australia won the case – it has taken a long time to actually release the reasoning. That happened yesterday our time. The judgement is over 800 pages long, but I have taken some interest in examining what the WTO said about the efficacy of the policy. There is a huge difference between a policy being legal and a policy actually achieving its stated policy goals.
My RMIT colleague Ash de Silva and I have looked very closely at the evidence surrounding plain packaging and have concluded that the policy did not achieve its stated policy goals. We have peer reviewed papers here and here and working papers at SSRN here and here. None of that work is cited in the WTO report. Rather it relies on peer reviewed work that supports the efficacy of the policy and then submissions as to the validity of various statistical approaches and techniques.
What was particularly interesting is how the WTO interpreted the so-called Wakefield studies that Ash de Silva and I had critiqued. We covered these studies at the Cat and particularly drew attention to the $3 million price tag associated with the studies. This is what Ash and I conclude:
Our results suggest that the independent, and combined, influences of plain packaging and health warnings had no statistically significant impact on smokers’ quit attempts as a result of the introduction of the plain packaging policy.
What did the WTO conclude (emphasis added)?
120. With this in mind, and based on the studies and expert reports before us and discussed above, the empirical evidence available to us regarding quitting-related outcomes and other distal outcomes, which is sometimes scarce, suggests that:
a. The impact of the TPP measures and enlarged GHWs on adult cigarette smokers’ quitting intention and quitting-related cognition reactions is limited and mixed.
b. The TPP measures and enlarged GHWs have had a statistically significant positive impact on avoidant behaviours, such as pack concealment, among adult cigarette smokers, while their impact on stubbing out and stopping smoking is much more limited and mixed.
c. Although the TPP measures and enlarged GHWs have statistically significantly increased calls to the Quitline, the observed impact of the TPP measures and enlarged GHWs on quit attempts is very limited and mixed.
d. The empirical evidence of the impact of the TPP measures and enlarged GHWs on adolescents’ quitting-related outcomes is limited. This evidence suggests that the impact of the TPP measures and enlarged GHWs on adolescents’ refraining from smoking cigarettes and thoughts about quitting is statistically not significant. No empirical evidence has been submitted to us on pack concealment among adolescent smokers.
Well – that is $3 million down the drain. Compare what the WTO said about the Wakefield studies to what the Australian population was told at the time.
The first comprehensive evaluation of Australia’s ground-breaking plain packaging tobacco laws shows they are working, the Victorian Cancer Council says.
Fourteen separate studies on the impact of plain packaging in its first year were published today in a special supplement to the British Medical Journal.
The research found after the laws were implemented, there was a “statistically significant increase” in the number of people thinking about and making attempts to quit smoking.
Cancer Council Victoria researcher, Professor Melanie Wakefield, said before plain packaging about 20 per cent of people made attempts to quit over the course of a month.
I’m also a bit concerned that some perjury may have occurred. The WTO claims:
113. We note that Brennan et al. 2015 is the only peer-reviewed study analysing the association between proximal outcomes and distal outcomes based on the NTPPTS data.
We have covered this point before. The Victorian Cancer Council was required to investigate immediate and distal outcomes of the policy and when Ash and I had suggested that they had not done so, the Victorian Cancer Council responded with a long press release that included this explanation (emphasis original):
The NTPPS was quite explicitly not designed to assess quitting success or change in smoking prevalence but rather focussed on the immediate impact of the legislation on perceptions of the pack, effects of health warnings and understanding of product harmfulness.
So why the WTO thinks that the NTPPS data does what the Victorian Cancer Council explicitly says it does not do is mystery.
Anyway – long story short (well not too short) the Wakefield survey and the papers that flow from that survey were not convincing to the WTO.
To remind ourselves: Chipty manufactured a change in trend lines in this graph by simply choosing the start and end dates to coincide with the introduction of plain packaging.
She also had an econometric analysis where she finds over a 34 month period after the introduction of plain packaging (and increased graphic health warnings) that the policy had contributed to a 0.55% decline in the prevalence of tobacco consumption. Sounds impressive – and the WTO are impressed – yet she never reported that the sample error in her data was 0.6%. That is bigger than the effect that she reports to have found. The WTO does not report that sample error either. Furthermore it makes no mention of the pseudo-R^2 of her model being 0.091. Nor the unusual base case being an unmarried, male, Australian born, 14 – 17 year old, with a tertiary qualification, employed full time, but with an income less than $6000, and living in Victoria.
So the WTO looked past the peer-reviewed research after, quite rightly, finding it to be unconvincing but accepted the non-peer reviewed Chipty analysis that to my mind is even more unconvincing.
So while the WTO has ruled that the policy is legal, it still seems to me that the efficacy of the policy is in doubt. The WTO only looks at evidence up to 2015 (pretty slack given that more evidence of failure has appeared in Australia since then and also in France and the UK).