Last year Sarah Durkin, Emily Brennan, Kerri Coomber, Meghan Zacher, Michelle Scollo and
Melanie Wakefield published a paper in Tobacco Control that concluded:
These findings provide some of the strongest evidence to date that implementation of [Plain Packaging] with larger [Graphic Health Warnings] was associated with increased rates of quitting cognitions, microindicators of concern and quit attempts among adult cigarette smokers.
Sounds awesome, sounds definitive. Maybe they weren’t expecting anyone to actually read the paper – especially tables 1 and 2. Not that there is a table 3, but you get the idea.
In April 2012, the Department contracted the Centre for Behavioural Research in Cancer, Cancer Council Victoria, to conduct a national cross-sectional, monthly tracking survey of smokers and recent quitters for the purpose of assessing the short to mid-term effects of tobacco plain packaging.
Word is that this multi-million dollar contract was not put out to tender. In addition, despite the Department of Health stating:
To request access to data collected in the National Tobacco Plain Packaging Monthly Tracking Survey, please email the Tobacco Control inbox.
I have been emailing since the beginning of January and still haven’t had even an acknowledgement of my emails let alone a decision on my request, or even the data. Perhaps I shouldn’t be surprised – if the published results are as weak as I’m about to demonstrate, I suspect the unpublished (and until now secret) data may reveal greater weaknesses and problems.
Turning our attention to Durkin et al.
This is how they describe their sample:
Respondents were aged 18–69 years and were either smokers (n(unweighted)=8755) or recent quitters (quit in the past 12 months, n(unweighted)=1553). Those who smoked daily or weekly were classified as smokers, while those who smoked monthly or less-than-monthly were allowed to self identify as a current or ex-smoker.
For the current study, we restricted the sample to baseline current smokers of factory-made or roll-your-own cigarettes …
We further restricted the sample to those who completed their follow-up survey before the implementation of the 12.5% tax increase for tobacco products that occurred in Australia on 1 December 2013 … [this] allowed us to better isolate the effects of the packaging policy from those of the tax increase.
They also required respondents in the sample to have complete data responses – i.e. no missing data. So far, so good.
What they are looking for in the data is attempts at quitting – not actual quits. That may seem like something of an oversight, but as will become clear they had good reason to be modest in their research question.
Those who were smoking at baseline were asked at follow-up if they were still smoking and if so, whether they had made any attempts to quit smoking over the past month. We created a binary variable … this variable allowed us to predict the proportion of smokers making quit attempts, irrespective of whether these attempts were successful.
Emphasis added. Get that? They are not interested in success, only attempts at quitting.
So what they do is offer a series of before and after comparisons. They have four periods of time:
The pre-PP phase included those who completed both baseline (10 April–1 September 2012) and follow-up surveys (7 May 2012–30 September 2012) prior to implementation of the packaging changes …
The early transition phase included those surveyed at baseline in the pre-packaging changes period (20 August–28 September 2012) and followed-up during the transition to the new packaging (20 August–28 September 2012) and followed-up during the transition to the new packaging (1 October–11 November 2012 …
The late transition phase included those first surveyed during the transition to the new packaging (1 October–30 November 2012) and followed-up either during the transition or soon after the full implementation of the new packaging (29 October 2012–20 January 2013 …
The PP year 1 phase included those who completed both surveys in the first year of full implementation of the new packaging (baseline surveys: 1 December 2012 to 4 November 2013; follow-up surveys: 2 January 2013–30 November 2013 …
So they slice and dice the data and produce Table 1 showing summary statistics for each time period under consideration.
We are interested in the row “Weighted, n”. Then read across the columns. “BS” stands for Baseline Smoker – someone who was smoking at the time of the initial call. “CS” stands for Continuing Smoker – someone who was still smoking at the time of the follow up call about a month later. Now while Durkin et al don’t actually say so, the percentage change between the BS figure and the CS figure is the short-term quit rate. That percentage difference captures the people who had stopped smoking sometime over the month between the two calls. Of course, the call itself may have induced smokers to quit – but let’s abstract from that and just look at the numbers. What I have done below is break out the numbers from their table 1 and then add in an additional row where I calculate the percentage change and then I also add in some columns where I add up the Early transition, Late transition, and PP year 1 data to create a new time period that I label Post-PP.
Before the introduction of plain packaging the quit rate was 5.9%. The Early transition quit rate was 7.97% (looking good there), but then the rate collapses to 3.57% in the Late transition period and then is 5.66% in the first year of plain packing implementation. Now because they haven’t reported this information they haven’t tested for statistical differences – I think I know why. The thing is 5.66% is lower than 5.9%. Now there may be no statistical significant difference between those two numbers but it does make a paper somewhat redundant if you want to argue that people thought about quitting but actually didn’t. Or worse, that quit rates actually fell.
I am surprised that the referees didn’t pick up that quit rates could be calculated from the data in table 1. Perhaps they didn’t want to know.
Now look at the columns I have labelled Post-PP. Here I have summed the data over the whole period where smokers could have come into contact with a packet of plain packaged tobacco. Durkin et al have sliced the data finely, I want to have a look at a coarser cut of the data. The quit rate in the Post-PP period is 5.5% down from 5.9% in the Pre-PP period. Again I don’t know what the statistical significance of that difference might be, but it is inconsistent with the hypothesis that quit rates would have increased as a result of the introduction of plain packaging.
It gets worse. Let’s look at their actual results reported in their table 2.
So what they have done is created seven indicators of the intention to quit smoking and then sliced and diced the data by time period to tease out any implications between these variables and smoking behaviour. “OR” stands for odds ratio – it is a coefficient from a logistic regression. Unfortunately they don’t report their actual regressions or the regression diagnostics so it is difficult to understand or interpret quite what they have done. in any event very few of the ORs are statistically significantly different from the Pre-PP ORs. What we are doing here is looking for asterisks; * means the number is statistically different from the Pre-PP value at the 5% level of significance and ** means the number is statistically different from the Pre-PP value at the 1% level of significance.
The first thing to notice is that there is only one instance where a result is statistically significant at the 1% level. All other significant results are only significant at the 5% level. The second thing to notice is that the results are not consistent. Some indicators, “Daily thoughts about quitting in the past week” and “Firm date to quit in next month”, have no statistically significant results at all. Other indicators have one or two but distributed across different time periods.
What is interesting though is that “Intend to quit in next month” is strongly statistically significant in the Late transition period. Yet we know from table 1 (reading between the lines) that this period exhibited the lowest actual quitting behaviour. So smokers told researchers that they intended to quit next month but then actually didn’t. Another referee oversight.
Then we get to the best and strongest and most consistent and coherent results in the paper. Smokers during the Late transition and PP year 1 periods “Concealed or covered pack several or many times in past month”. Well yes. Of course they did. The pack are deliberately designed to be ugly and unattractive. They are covered in medical porn. Small wonder that smokers concealed the packets.
The other thing to notice is that the results are highly statistically significant for “Stopped from smoking several or many times in past month” in the Early Transition period. Again, I’n not surprised. That period includes 20 August through to 30 November 2012. Cigarette prices are automatically indexed in March and August every year (then to CPI, now to AWOTE). So in that period tobacco prices had just increased and some smokers tried to quit. That’s called “responding to price signals” and it explains the 7.97% quit rate in that period. So despite Durkin et al wanting to keep their study uncontaminated from tax increases they were unable to entirely do so.
So long story short: The evidence for plain packaging having an effect on smoking quit rates is poor. The evidence for plain packaging having an effect on the intention to quit rate is poor. What evidence there is suggests that excise increases effect quit rates.
As the authors concede:
Results indicated that short-term changes in quit intentions were no different after the full implementation of the new packaging than in the pre-PP phase. The absence of these effects on next month quit intentions and having a firm date to quit, in the presence of effects on microindicators of concern and quit attempts, may indicate that plain packs with larger GHWs continue to trigger increased concern in the smoking moment—reflected in increased stubbing out before finishing a cigarette and pack concealment—but do not continue to increase deliberate plans to quit well after the shock of the transition phase.
That is a very positive spin on a very poor result.