The idea for this piece came from a May 13 tweet by G Raghuram. In this tweet Raghuram talked about the Goodhart’s law in the context of the way Covid numbers are being reported.
In a 1975 article, the British economist Charles Goodhart had stated: “Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.” This came to be known as the Goodhart’s law. Of course, like many other laws in economics, the Goodhart’s law has also not been stated in simple English.
As Carl T Bergstrom and Jevin D West write in Calling Bullshit—The Art of Scepticism in a Data-Driven World: “While Goodhart’s original formulation is a bit opaque, anthropologist Marilyn Strathern rephrased it clearly and concisely: When a measure becomes a target, it ceases to be a good measure.”
As Bergstrom and West further explain: “If sufficient rewards are attached to some measure, people will find ways to increase their scores one way or another, and in doing so will undercut the value of the measure for assessing what it was originally designed to assess.”
Examples of this phenomenon can be seen across different facets of life. A business school I used to work for had started dozens of journals and magazines, without much quality control, to drive up its rankings and it briefly did succeed. This was because business school rankings gave some weightage to research carried out by the faculty of a business school and by having its own magazines and journals, it was easier to publish. This helped in driving up the ranking.
Now what does the Goodhart’s law have to do with the covid pandemic? As the covid pandemic struck and spread, different measures have been used to get an idea of its strength (for the lack of a better term). These include daily increase in covid cases, the total number of tests carried out in a district and a state, the total number of covid deaths, etc.
As per Goodhart’s law, these different measures have become targets. And that has led to different state governments trying to game these measures, in order to make themselves look good and tell the world at large that they have the covid pandemic under control.
Before I get into data and news reports, let me explain this through a very simple example. For a while, the daily increase in the number of covid cases in Nagpur in Maharashtra was much more than the increase in the entire state of Madhya Pradesh.
Anyone who knows Indian geography would know that Nagpur is right on the border that Maharashtra shares with Madhya Pradesh. It is not an island. People can move between the states. This anomaly wasn’t really explainable unless one looked at the Madhya Pradesh numbers from the lens of the Goodhart’s law.
One parameter that has been managed (or should I say fudged) by different states is the number of people dying of covid. The idea as I explained earlier is to tell the world at large that they have the situation under control. The trouble is that the governments may be able to manage the data, but they can’t always hide the dead bodies.
Crematoriums across the country have been working overtime. Public health expert Ashish Jha, offered a straightforward argument in a Twitter thread on May 9. As he wrote: “During [the] non-pandemic year 2019, about 27,000 Indians died on [a] typical day. Crematoriums handle that level of deaths every day. Additional 4,000 deaths won’t knock them off their feet. Crematoriums across the country [are] reporting 2-4X normal business.”
He further writes: “So best estimate [of] 55,000 to 80,000 people dying daily in India, If you assume baseline deaths of 25,000-30,000, Covid [is] likely causing additional 25,000 to 50,000 deaths daily, not 4,000.” As Anirban Mahapatra writes in Covid-19 – Separating Fact from Fiction: “During the pandemic many of these excess deaths are due to COVID-19.”
Many journalists and newspapers have found ways of going beyond the official numbers. Let’s take the case of Gujarat. The Divya Bhaskar newspaper has reported that the state has issued 1.23 lakh death certificates between March 1 and May 10 this year. It had issued around 58,000 death certificates during the same period last year. So, the number of deaths has more than doubled this year. As per Gujarat government’s data only 4,218 deaths happened due to covid during the period. This suggests massive underreporting. The Gujarat government has called this report inaccurate.
It would be unfair to suggest that this trend of underreporting covid deaths is prevalent only in Gujarat. An April 15 report on NDTV, during the early days of the second wave, said that for Lucknow, the “cumulative official covid death count released by the government in the last seven days is 124.” Nevertheless, as “per the records maintained by the city’s crematoriums, over 400 people who died because of the virus had been cremated,” during the period. The government explained away this difference by saying that those dying in neighbouring districts and states were also being cremated in the city.
A similar thing happened in Bhopal as well. Over a period of 13 days in April, the official covid death count stood at 41. Nevertheless, a survey carried out by The New York Times of the main covid-19 cremation and burial grounds in the city, revealed that more than 1,000 deaths had been handled under strict protocols. There was a similar newsreport on Kanpur as well.
In fact, the Financial Times, collected news reports across seven districts and found that the number of covid victims who had been cremated are ten times larger than the official covid numbers in the same districts. (Click on the above link to look at the graph).
Of course, other than such news stories, there have been a spate of photographs and videos lately, showing bodies washing up and then later buried on the shores of the Ganga river, flowing through Uttar Pradesh and Bihar. A Dainik Bhaskar news report puts the number at more than 2,000 bodies, with Kanpur, Unnao, Ghazipur and Ballia being worst hit. (Those who can read Hindi, I suggest please read this report).
Journalists have also been counting paid obituaries being published in newspapers, again suggesting a huge difference between the reported numbers and the actual state of things.
As Bhramar Mukherjee, an epidemiologist at the University of Michigan told the New York Times: “It’s a complete massacre of data… From all the modeling we’ve done, we believe the true number of deaths is two to five times what is being reported.”
As per the Institute for Health Metrics and Evaluation, which is based in Seattle, United States, the total covid deaths in India as of May 6, stood at 6.54 lakh, around three times the official figure.
There are several ways in which the undercounting happens. In Uttar Pradesh, in order to get admitted into a hospital, the patient required a reference letter from the Chief Medical Officer “who heads the Integrated Command and Control Centres set up by the government in all districts”. Due to this rule, patients were turned away from hospitals. And if such a patient died he or she wouldn’t be counted in the covid deaths.
A medical officer in Krishnagiri in Tamil Nadu told The Hindu: “We have been told orally in the meeting that only deaths within 10 days of admissions will be taken as covid-19 deaths.” MK Stalin, the new Tamil Nadu chief minister, has asked the state government officers not to fudge data.
The number of deaths also depends on how the counting is carried out. Take the case of West Bengal, where in May 2020, the “official’ coronavirus death toll… doubled in the five days since the state virtually shelved its Covid-19 death audit committee.”
Then there are cases where an individual dying of covid had not tested positive (hence, it was a case of a false negative). There are examples of such cases not being counted as well.
There are also cases of covid deaths being attributed to other health complications that individuals had when they got infected by the virus. These include diabetes, hypertension, cancer etc., which increase the risk of severe covid.
A news report on BMJ.com published in July 2020, pointed out that in Vadodra “death audit committees attributed nearly 75% of deaths in covid-19 positive cases to other causes such as complications from diabetes or following organ transplants.” All this is happening against the prevailing guidelines of the Indian Council of Medical Research.
People who die outside hospitals or on their way to one, aren’t counted in the covid deaths. Two thirds of registered deaths in India happen at home. In all around 86% of deaths in India are registered.
Even here there is a great deal of variation across states. In Bihar and Uttar Pradesh, only 34.6% and 60.8% of the deaths, respectively, are registered. As the disease spreads across rural Bihar and rural Uttar Pradesh, massive undercounting of both active covid cases and deaths, is happening.
The reluctance of politicians notwithstanding, the system itself is not geared up to count the dead, from covid or otherwise, in these states.
The biggest evidence of undercounting comes from the fact that the Prime Minister Narendra Modi recently said that the “states should be encouraged to report their numbers transparently without any pressure of high numbers showing adversely on their efforts”.
There are several reasons why the governments need to count the number of people dying because of covid, correctly.
First and foremost, people have a right to know what is happening in the country. It tells us clearly how the disease is progressing and helps us prepare accordingly, mentally, physically and financially.
Second, as I have often said in the past, if we don’t recognise a problem how do we work towards solving and/or containing it. With regard to this, Bhupinder Singh Hooda, a former chief minister of Haryana, made an important point in a recent column in The Indian Express, where he said:
“The Union government is allocating oxygen on the basis of the severity of the second wave in the state. If the state government underreports the numbers or fudges the data, it will harm, rather than help, the state as it will get a lower allocation of oxygen and more deaths will follow.”
Third, counting covid death numbers as accurately as possible is important for the overall health security of the world. No herd immunity can be achieved if the disease keeps spreading across India.
Fourth, the correct data helps epidemiologists run their models properly and then make projections that should help policy.
It also needs to be said here that historically during a pandemic, data is not always accurately collected. As Chinmay Tumbe writes in Age Of Pandemics (1817-1920):
“Death figures are collected on the basis of ‘registration’, which is a process that usually breaks down in a period of crisis, as observed by the health officials of those times. It leads to serious underestimation of the number of deaths, especially in poorer countries with weak data collection systems. In India, the Census of 1921 noted that due to ‘the complete breakdown of the reporting staff, the registration of vital statistics was in many cases suspended during the progress of the epidemic in 1918’.”
The mortality statistics of those who died in the pandemic that happened between 1918 and 1921, have been updated through various studies over the years.
Having said that, when it comes to data and data collection, things have improved by leaps and bounds over the last 100 years. Hence, even with the pandemic being on, data collection and management, needs to be carried out in a much better way.
Of course, all this is lost on a central government, which is primarily interested in narrative management. It is currently busy spreading the narrative that it had warned the states of a second wave.
But then it did nothing about it… Didn’t order enough vaccines… Didn’t make sure that there was enough stock of oxygen… Exported the vaccines being produced… Continued with the kumbh mela and the elections, both big super spreader events… And also told the world that India had managed to defeat covid.
In between all this we were also asked to bang utensils and eat dark chocolate.
