An opinion on polls

With Assembly elections round the corner in Bihar, very soon we will be flooded with opinion polls and exit polls, with each one making estimates about which party or alliance will win how many seats. It will be business as usual — political parties shown as winning the elections or being ahead will applaud the polls while parties shown as losing will question the poll results. Poll-bashing continues even after results are declared and even when the actual results may be close to what opinion poll or exit poll predicted. Poll-bashing is more a matter of habit or on the basis of the perception that the polls must be wrong rather than on the basis of a simple reality check of comparing the poll prediction with the actual results.

Ask anyone how accurate the exit polls are and there will be an automatic reply that they are horribly wrong. Old memories die hard, and perhaps that is primarily what plagues(bother,परेशान) exit polls today. Even though the 2004 Lok Sabha elections are about a decade old now, people still remember that the exit polls got it horribly wrong then, when they predicted victory for the BJP-led National Democratic Alliance but the actual winner turned out to be the Congress-led United Progressive Alliance.

More often right

The myth that exit poll forecasts are always wrong needs to be broken, as election forecasting in Indian elections has certainly come a long way since 2004. In the last few years, most surveys have consistently managed to make fairly accurate seat projections. Let us look at projections for state assembly elections in the last few years to understand this. How accurate have surveys been in the past few years? Is there a pattern in the error made by pollsters? Can this be connected to any parallel trends in Indian politics?

We analyse here seat forecasts for elections held in 19 States in the last five years by a common set of agencies to comment upon the accuracy of seat forecasting. A ‘poll of polls’ — basically an average of multiple projections by various agencies — has been compared with the actual results to estimate the accuracy of the polls. The pollsters failed to correctly identify the trend in only two elections — Assam in 2011 and Punjab in 2012 — when most agencies predicted the winner wrong. Even in these elections, there were some agencies that went against the common trend to identify the correct winner. In terms of accuracy, we find that the accuracy rate was greater than 90 per cent in eight elections. These were essentially the smaller States with clear bipolar contests. In most elections, the forecasts were fairly correct and the accuracy rate fell between 70 per cent and 90 per cent. In terms of seats, pollsters were off the mark on only three occasions — Assam (2011), Manipur (2012) and Delhi (2015).

Getting the margins right

What stands out is that the accuracy of pollsters has been relatively lower in elections that saw decisive(crucial,निर्णायक) verdicts. Analysts seem to be clearly underestimating the seat share of the winning party. In 8 out of the 11 elections in which the accuracy rate was lower than 90 per cent, pollsters had underestimated the seats of the winning party. In the last few years, there has been a rise in the seat share of the winning parties in assembly elections, and parties have started getting decisive victories. As elections become more one-sided, it becomes easier to project the winner but the challenge of getting the scale of victory correct remains difficult.

There could be two possible explanations why pollsters have not been able to perfectly project the magnitude of victory in case of decisive verdicts. The first is that pollsters often tend to be conservative in their estimates and consciously refrain from making ‘bold’ projections. Even with a correct vote share estimate, most pollsters remain wary of predicting that a party is likely to get more than 90 per cent of seats in a keenly contested three-way contest. The second explanation mainly revolves around an unconscious omission by pollsters. In 7 out of the 8 elections in which pollsters underestimated the winner, there was a rise in the seat-to-vote ratio. What happened probably was that many pollsters got the seats wrong despite correct vote shares because they had been unable to anticipate a change in the seat-vote ratio.

Election analysis and estimating vote shares and seats are not akin(similar,समान) to gazing(stare,घूरना) into a crystal ball. Conducting a survey for recording people’s vote choice is just one stage of the larger process of making projections. Estimating seats is a completely different stage and requires distinct statistical skills. The process is dynamic, as political formations and alliances keep changing.

The upcoming Bihar election, for instance, is going to be very difficult for pollsters to predict. The alliances of parties have changed dramatically between 2010 and now. The two main regional parties Janata Dal (United) and Rashtriya Janata Dal, which have contested several elections against each other over the last few decades, have formed an alliance this time. The BJP, which had been a junior partner till the last assembly election, is now leading an alliance of its own, and will be contesting the highest number of seats among all major parties. Ram Vilas Paswan has also changed sides between the two assembly elections, all of which pose challenges for poll predictions.