People protest against the recent cases of mob lynching of Muslims who were accused of possessing beef, in New Delhi, India, June 28, 2017.

People protest against the recent cases of mob lynching of Muslims who were accused of possessing beef, in New Delhi, India, June 28, 2017.

When official stats aren’t good enough





When BJP President Amit Shah said that there had been more lynchings during the UPA government than during the Modi government‘s rule, he was echoing a debate that has raged on Indian social media over the last week–do we have data to show that lynchings are on the rise, or is this a liberal media conspiracy?

First, here’s what the official data says. India does not record “lynchings” or “mob violence” separately, but does record incidents of communal riots, promoting enmity between people on the basis of religion, and communal incidents.

The first two come from the National Crime Records Bureau, a collection of FIRs from across the country. In addition to being subject to data recording issues–for example, the NCRB operates on the ‘principal offence’ principle, meaning that only the “most heinous” of all offences relating to one incident is recorded. The NCRB also runs with a time lag; the latest data is for 2015.

The NCRB only began to record “communal” riots separately from 2014 onwards; the data shows that that number of communal riots declined from 1,227 incidents in 2014 to 789 such incidents in 2015, along with an overall decline in the total number of riots over the few years before that.

HINDUSTAN TIMES VIA GETTY IMAGESCitizens hit the streets in support of the campaign ‘Not In My Name’ against lynching of a Muslim teenager Junaid, in Mumbai’s Carter Road.

Similarly, the NCRB only began to record “promoting enmity between different groups” from 2014; those cases rose from 336 to 424 between 2014 and 2015, but we know little about the period before that.

Finally, the Ministry of Home Affairs records “communal incidents”, which it publishes annually. (This number, by the way, differs substantially from the number of communal riots recorded by the NCRB, and the MHA does not have an answer for why this is so.) These numbers do not show any substantial increase post-2014 either.

For lynchings or mob violence, specifically, then, media organisations and interested individuals have turned to news sources; the data journalism website IndiaSpend did an analysis based on a Google search of English news websites, economics writer Rupa Subramanya‘s analysis created a dataset from news reports, and on Twitter, scientist and writer Anand Ranganathan sought to point out that as many news reports on lynchings in 2013 existed as those on lynchings in 2016.

Are analyses based on news reports worthwhile? To be sure, there is scholarly pedigree to the enterprise. Political scientist Ashutosh Varshney, Sol Goldman Professor of Political Science and International Studies and Social Sciences and the Director of the Brown-India Initiative at Brown University, with Steven Wilkinson, assembled a dataset on Hindu-Muslim violence between 1950 and 1995 based on news reports in the Mumbai edition of The Times of India. That dataset, publicly available, is widely regarded as a standard in the field and has been replicated many times over. For the current question, Varshney says this:

“The first question we’d need to ask is–are there better alternatives? The answer to that is no, for government statistics are wholly unreliable both because the process itself does not generate faith in the accuracy, and the category construction is bizarre,” says Varshney. What we are then left with is newspapers, he says, and while realising that newspapers too will not be accurate, in a competitive atmosphere, newspapers are unlikely to be as unreliable as government statistics.

“Forget issues of bias and political control, sometimes an important story may be dropped for as random a reason as ‘the paper looks too depressing today'”

What’s key, however, is that such an analysis cannot merely involve a literal search of a selected newspaper or set of papers. In Indonesia, for example, where Varshney and his colleagues have just concluded an even more ambitious dataset of violence, the researchers used national, regional and local newspapers. For these datasets, an important step was triangulation–randomly selecting some large riots and cross-checking with local knowledge centres: NGOs, knowledgeable activists, in-depth local reports. In a riot dataset, large incidents drive the overall results, says Varshney–large riots need special investigation. “At the end of it, what you would realistically hope to achieve is statistical reasonableness rather than statistical precision,” Varshney says.

  1. Ram, former Editor-in-Chief of The Hindu, agrees that in his experience, “one can safely assume that every serious lynching would have been covered by the news media, some more reliable than others”. Such an analysis, however, would need to involve a wide enough search and be done systematically and scientifically, he added.

HINDUSTAN TIMES VIA GETTY IMAGESCitizens protest in New Delhi’s Jantar Mantar.

However, some within the media sounded a note of caution. Two senior Delhi-based journalists, who asked not to be named, said that institutional issues in newspaper journalism can make them unreliable on occasion. “Forget issues of bias and political control, sometimes an important story may be dropped for as random a reason as ‘the paper looks too depressing today’ or ‘we need an important figure in the government to come give to our event,” one of them said. “The paper is shutting editions in non-metros and giving very little focus to reporters who are not in Bombay or Delhi,” a former senior editor with another newspaper said. “Having worked inside the paper, I would not trust it to be counted on as a newspaper of record,” he added.

Defects in news reporting is something Ranganathan frequently points out himself on Twitter. Then why use news reports now? “It is not an ideal method, certainly. But it is the only option, given the non-availability of such data from NCRB,” he says. “Then again, it is worth pointing out the obvious–that not only there would be bias in the data collected thus, projecting such data as an answer other than to prove hypocrisy or to allay a selective argument would be foolhardy,” he says.

And then there is the question of whether we need data to answer the question at all, or whether with anti-Muslim sentiment being fanned by members of the ruling party, a sense of impunity about mob violence against Muslims–and the resulting fear and insecurity among Muslims and those who worry about them–is to be expected. Yes, says Ram — even if it is early days, it would be good to have data, if obtained properly, to pin the phenomenon down. “There is no doubt that many members of the ruling party have fanned anti-Muslim sentiment, and it is to be condemned unequivocally,” says Ranganathan. However, given the defects that he sees in the analysis of the most recent incidents–the data is not normalised and is a relatively small number–Ranganathan believes that “it would be unwise to link scaremongering/hate speeches/anti-community sentiment, with the actual occurrence of the crime itself in this case.”

There is an ethical answer and a social science answer,” Varshney says. “The ethical answer is that even one life lost this way is too big a loss and there is no particular reason to establish trends if, according to our observational judgement, the chilling, gruesome and ominous quality of the violence must be highlighted. Precise stats, however important in a social science framework, are really quite besides the point in such reasoning. But the social science approach would look at broad trends and broad departures from it and this might require more data, and might give different insights. Both modes of inquiry are justified and necessary,” he says.