My last post challenged a New York Times op-ed’s view that the media industries should give up fighting piracy. Implicit in the Times editorial’s argument is the idea that piracy has little or no impact on media sales — a view shared by many in the tech community and seeming by some journalists as well.
My colleague, Rahul Telang, and I recently finished a paper reviewing the academic research on the impact of piracy on sales. Our review finds that, when viewed as a whole, the academic literature strongly suggests that piracy harms media sales: the vast majority of academic papers — particularly those published in peer-reviewed academic journals — find evidence of harm from piracy. This conclusion is consistent with reviews of the academic literature by Stan Liebowitz in 2006 and by Felix Oberholzer-Gee and Koleman Strumpf in 2009, but includes more recent studies — and we believe these recent papers make the case of harm from piracy even stronger than what the literature suggested just a few years ago.
One recent paper in particular caught our eye. As we were finalizing our review, George Barker and Tim Maloney of the Australian National University posted a paper to SSRN contradicting the results of a prior paper by Birgitte Andersen and Marion Frenz. Andersen and Frenz used data from a 2005 Decima Research survey on piracy and purchasing habits of more than 2,000 Canadian citizens. Their original report to Industry Canada used this data to conclude that increased piracy actually helps music sales: “for every 12 downloaded songs, music purchases increase by 0.44 CDs.” While this finding was softened to a finding of “no association between” piracy and CD sales by the time the paper was published in the Journal of Evolutionary Economics, Andersen and Frenz’s work was widely touted in some circles as conclusive evidence that piracy doesn’t harm, and can in fact can help, media sales.
The Barker and Maloney paper, however, used the same data as Andersen and Frenz did, but came to the opposite conclusion: “a 10% increase in P2P downloads reduces CD demand by around 0.4%” (emphasis mine). The change in results was driven by several changes in Barker and Maloney’s empirical analysis, the most interesting of which was their decision to include a large number of respondents excluded in Andersen and Frenz’s study. Specifically, the Andersen and Frenz paper excluded all respondents who did not purchase CDs in 2005 (about 20% of the total sample) under the rationale that “these consumers may never have been active in CD purchasing” and therefore were irrelevant to an analysis of the impact of piracy on CD sales.
Barker and Maloney noted that the original survey data included questions on the number of CD sales in both 2005 and 2004, allowing the authors to partially test whether consumers who had no CD purchases in 2005 were “never active in CD purchasing.” Interestingly this test showed that about one-third of consumers who did not purchase CDs in 2005 had purchased CDs in 2004. This one small fact radically changes the likely importance of these consumers to the analysis. While consumers with no interest in CD purchases might be irrelevant to the analysis, consumers who stopped purchasing CDs between 2004 and 2005 might be exactly the sort of consumers whose purchases were most influenced by piracy.
I believe there are two important lessons here. First, that small changes in the understanding of the data can make a large change in a paper’s conclusions, a lesson that Joshua blogged about previously. Second, that in the context of social science research we should be careful not to take any one paper as conclusive evidence of an effect. Rather, because of the limitations inherent in any social science research, when trying to understand what the academic literature “says” about a topic, we should look at the literature as a whole. Applying this standard to the academic literature on piracy one finds paper after paper examining different datasets from different industries in different timeframes all reaching the same conclusion: piracy harms sales.