Surprise: Problems with Widely Touted Anti-Gun Research

“I am not interested in giving any serious thought to John Lott or his claims.”

Those are the words of University of Alabama associate professor Adam Lankford in response to Fox News after economist John Lott called Lankford’s highly publicized study into question.

Lankford’s study was published in 2016 but was touted by President Obama and a fawning media anxious for any “evidence” that gun ownership is somehow evil even before it was officially published. Lankford’s anti-gun perspective is evident early on in his paper. His eighth paragraph starts, “Less positive may be the fact that, according to a comparative study of 178 countries, the United States ranks first in gun ownership…”

Despite all of the attention it received, Lankford’s study is troublesome. He claims to have found that 31% of global mass shooters attacked in the United States between 1966 and 2012. He states that the U.S. suffered 90 offenders during this period while only four other countries had more than nine offenders. Obama took this alleged finding and ran with it, claiming “The one thing we do know is that we have a pattern now of mass shooting in this country that has no parallel anywhere else in the world.”

The problem is that Lankford’s study is lazy and sloppy, if not deliberately limited. He used the New York Police Department’s 2012 Active Shooter report supplemented with the FBI’s 2014 Active Shooter Report and “data gathered on incidents from other countries.” Lankford used the same methodology as the NYPD to gather additional information.

The NYPD only used open-source material – i.e., Google. They didn’t use subscription-based research services like LexisNexis, government databases, or any of the resources available to professors at well-funded world class universities. The “NYPD limited its internet searches to English-language sites, creating a strong sampling bias against international incidents.” That methodology works for the NYPD’s purpose of developing recommendations for risk mitigation but it doesn’t work for a cross-national study because most of the world speaks a language other than English. The FBI 2014 Active Shooter report was limited to incidents in the United States.

President Obama and others eager for anything that casts gun ownership as fundamentally dangerous took Lankford’s 31% claim at face value. Other researchers, academics, and journalists who questioned Lankford’s work – or even asked to see his data – were rebuffed by the UA professor.

John Lott thought that 31% seemed high, so he asked if Lankford would share his dataset – a common courtesy among academics and researchers. Lankford refused. Repeatedly. Lankford also refused to explain how he measured (or counted) mass shootings. Lankford refused to tell journalists how he collected his data, despite his claim that he found complete data for 171 countries – somehow without using foreign language sources. Journalists at Real Clear Politics asked Lankford questions about his methodology and for access to his raw data; he refused. Lankford’s paper does not include a list of the number of shooters in each country, only providing the totals for five countries including the United States.

So Lott built his own dataset using the University of Maryland Global Terrorism Database, Nexis, and web searches for mass shootings. Lott hired people who spoke foreign languages to help with this effort. Unlike Lankford, Lott provides the search terms he used as well as a list of the cases in his dataset. Lott has been as transparent as possible with his study and even acknowledges that his monumental effort likely undercounts shootings in foreign countries due to current and historical news coverage of such events in the developing world. Lott looked at the years 1998-2012, likely to ensure the availability of better data.

Lott found at least fifteen times more mass public shooters than Lankford in less than a third the number of years (1998-2012). Lankford claimed to find 292 mass public shooters over 47 years while Lott found more than 10,000 such shooters around the world in the last 15 years. Professor Carl Moody at the College of William & Mary confirmed Lott’s counts for The Washington Times and added, “By the way, anybody can do this. The GTD database is free and available to all.”

Lott’s most important finding is that 1.43% of mass public shooters attacked in the United States. That is starkly different than Lankford’s 31% claim. Lott extended his work to the number of attacks and found that 2.88% of mass public shootings between 1998 and 2012 were in the United States.

Since Lott’s paper was published, his findings have been covered in The Washington Times and in a comprehensive article on RealClearPolitics.com that touches on the importance of allowing other researchers or journalists to verify one’s data (including infamous cases within the context of firearms-related research).

Lankford’s conclusion – that reducing gun ownership would reduce mass shootings – falls apart when one considers the undercounted foreign cases. One would hope – even expect – that a professor would give serious thought to valid questions about his work and to more comprehensive studies that refute his findings. Petty, dismissive, and evasive answers do not reflect poorly on the genuine questions raised or those asking them.

© 2018 National Rifle Association of America, Institute for Legislative Action. This may be reproduced. This may not be reproduced for commercial purposes.