Terrorism studies and the struggle for primary data

This massive spike in attention for the subject, as well as the new sources of funding that came with it, brought many new scholars to the field and ensured a tremendous increase in output. Unfortunately, as authors like Andrew Silke and Avishag Gordon have shown, newcomers’ interest in terrorism was often short-lived; up to 83 percent of articles published in the 2000s were by one-time contributors. While some of these were of high quality, many lacked the in-depth knowledge of the field to truly help move it forward, especially where it came to issues like resolving the long-acknowledged scarcity of primary data.

Unfortunately, a relatively small group of dedicated scholars was not the only obstacle to a timely resolution of the primary-data problem. The long and short of it is that terrorism is in many instances simply a difficult subject to gather primary data on. One issue here is that many Western states have been fortunate enough not to suffer from a large number of terrorist attacks. This also means that terrorism is often a relatively ‘small-N’ problem, which increases data collection challenges. Then there are practicalities like a) finding (former) terrorists who could, for instance, be interviewed, b) convincing them to cooperate and c) making the time necessary to take these steps within research projects that are often conservatively budgeted in time and resources. Moreover, while it’s been argued that the potential dangers of engaging directly with (former) terrorists can often be managed, personal safety is still an issue that must be considered, as are the ethical guidelines for working with what are often vulnerable individuals and groups.

Concerns over the field’s apparent inability to gather more primary data, despite the problem having been known for decades, arguably peaked in 2014 when Marc Sageman’s provocatively argued that research on terrorism had “stagnated.” At first glance, such a damning statement from one of the most influential terrorism researchers in the post-9/11 period supports continued pessimism about the field. Yet, there was also considerable push-back; other leading scholars were quick to qualify Sageman’s piece as overly critical, and pointed to areas of real progress in the study of terrorism. Since then, both established and up-and-coming researchers have begun proposing that, actually, the primary-data issue may no longer be quite so problematic as it was. The question, however, is what the data says on the issue. Is ongoing pessimism warranted or has there been a change for the better? With the last in-depth assessment of the field dating back to Silke’s groundbreaking work in the early to mid 2000s, a fresh look was needed.

Over a two-year period, and with help from several research assistants and interns, I collected data on the circa 3.500 articles that had been published between 2007 and 2016 in the field’s nine leading journals. Like the work by Silke that had inspired this effort, I was principally interested in whether these articles used any kind of primary data, whether they utilized statistical analyses, and what their method of data collection was. Unfortunately, the number of articles with a specific ‘data’ (or even a separate ‘methodology’!) section was relatively small, meaning that in most cases my team and I had to trawl through the notes to ascertain what type of information had been utilized. After two years of this kind of masochism, we were happy to draw the analysis to a close with a decade of research scrutinized. The results have since been published in Terrorism and Political Violence and are available for early view.

The results strongly suggest that the study of terrorism has made significant strides towards finally addressing the shortage of primary-sources based research that has plagued it for so long. Not only did the study indicate a slow but steady upward trend in the percentage of articles using such data, but by 2016 a slim majority of all pieces published used some kind of primary data. Of course, there are qualifications; the inclusion threshold for counting something as primary data was purposefully set rather low. As a result, both years of field work and a single five-minute telephone interview suffice to tick the primary-data box. Still, I’d argue that the overall picture warrants considerable optimism.

Unfortunately, the field has not made similar progress in other areas. Qualitative research designs still reign supreme, with little growth in terms of the percentage of authors using statistical analyses. Moreover, some forms of data collection are still virtually nonexistent, such as experimental ones or clinical assessments of terrorists’ mental health. Most problematic, however, appears to be the enduring problem of one-time contributors and a lack of collaboration. Some 75% of all articles published between 2007 and 2016 are the work of single authors. Such limited collaboration cannot but constrain the scope of the research questions being tackled, as larger teams will generally have more resources in terms of time, finances and intellectual capital. Similarly, some 75% of all publications were the work of one-time contributors. While lower than the 83% noted by Silke in the 2000s, the relatively small group of dedicated researchers arguably continues to limit the speed and extent to which the field will actually be able to move forward on key issues.

The significant steps taken towards addressing the scarcity of primary data signal the maturation of terrorism studies as a field of research. Still, there is a lot of work to be done. Not only must we ensure this positive trend is perpetuated, but the quality and quantity of primary data being used still frequently leaves considerable room for improvement. No less important is that we continue to build a larger core of dedicated terrorism researchers, and that collaborative efforts become more frequent. There are good reasons to be positive about the prospects of terrorism studies as a field of academic inquiry. But there’s no room for complacency.

— Read more in Bart Schuurman, “Research on Terrorism, 2007–2016: A Review of Data, Methods, and Authorship,” Terrorism and Political Violence (1 March 2018) (DOI: 10.1080/09546553.2018.1439023); see also Bart Schuurman, Becoming a European Homegrown Jihadist: A Multilevel Analysis of Involvement in the Dutch Hofstadgroup, 2002-2005 (Amsterdam University Press, March 2018).

Bart Schuurman is an Assistant Professor at Leiden University’s Institute of Security and Global Affairs (ISGA), based in The Hague. The article is published courtesy of the University of Oslo.