Do you trust the opinion polls?

Yesterday there was something of a minor earthquake in political circles when the figures were released for the latest opinion poll by Lord Ashcroft – which showed a quite incredible six point lead for the Conservatives over Labour. To date most polls had showed either a narrow Labour lead, or the two parties running neck-and-neck, so the figures were a quite literal bolt from the blue. But as Mike Smithson on Political Betting pointed out, the figures were also highly confusing, as an opinion poll by Populus showed a five point lead for Labour. They were conducted at the same time – so surely they can’t both be right, can they?

Opinion polls have taken a bit of stick over the years – not least since they predicted a Labour win in 1992 and completely missed the evidence of a fourth Conservative victory. But as Lord Ashcroft himself points out – polls are snapshots, not predictions, and they are only intended to give an indication of the general view of the public. So for those readers who don’t follow the polls, I wanted to share a few remarks to help you untangle this often heavily misused political device.

First of all, let’s clear up the Margin of Error. Politicians often bandy this about to dismiss polls that they disagree with, but it misses the point. The margin of error stresses accuracy – you only need to sample 1,000 respondents (provided the sample is balanced) to get a result that is accurate to within 3% either way – which isn’t bad at all (for those who wonder, you need a sample size more than double to reduce the accuracy to within 2%, and therefore pollsters decide it isn’t worth the expense). Taking the Ashcroft poll for example – the figures show the Tories on 34% and Labour on 28% – but it may well be both parties on 31%, or indeed the Tories on 37% and Labour on 25%. The margin of error simply means that it is not a perfectly accurate measure – but you can’t simply dismiss the figures out of hand.

Secondly, polls are snapshots of opinions. The question respondents are usually ask is “How would you vote if there were a General Election tomorrow?” – but there isn’t a general election tomorrow! If you look at historic polling (I recommend the UK Polling Report website for this – it is very user friendly!) you will observe that incumbent governments do very badly while in government but then recover as the election draws nearer. That is because voters don’t often like what governments do, but when forced to choose at the ballot box, prefer the government to the opposition!

All of these mean that the most important thing is not a specific poll but the overall trend. Politicians are very prone to take an isolated poll and shout about it from the rooftops when it suits them, and then turn around and dismiss another poll when the figures do not suit them. The truth is that polls can be useful for indicating whether a party’s message is sticking or having any impact – if their numbers are generally going up for example, even if the poll is not accurately guessing what percentage they will get in the end, the poll is demonstrating the party is becoming more popular. Based on that, the party that should have the most to worry about is Labour. As I anticipated in an earlier post, there is a swing heading back towards the government, and the Labour lead in the polls has been falling month on month. Phrased another way – their lead is nowhere near big enough for a party looking to win an election in less than five months.

A final word of warning is this – polls predict a national share of the vote, but elections are decided locally at each constituency, and do not account for variations in vote change – while the Lib Dem vote is likely to collapse from 23% to 10% (or worse) their vote isn’t necessarily going to collapse in every constituency – especially where they have sitting MPs. You should not be surprised if there are huge swings to Labour in some of their heartlands for example, but much smaller swings in the marginal seats. So we have to treat opinion polls with care, yes – but they can also be a useful barometer into just how well a party is resonating with the electorate. And at the moment, the barometer is not showing the kind of support that makes me think we will wake up to a Labour majority in May.

The Christian heritage of voting

2015 has been marked red in my diary for the last four years with one clear message: ELECTION YEAR.

With the election only 124 days away, I am opening this crucial year with a feature on one of the very first articles I read for my course on Voting Theory, which acknowledged a rather surprising contributor to development of electoral democracy – the adoption by the medieval Church of the conclave to elect a new Pope. Now, with my reformed Protestant theology I was not expecting that I would be very warm towards the article, but the explanation of how the Conclave came about was a fascinating insight into how the historic church used a form of democracy to achieve unity – while of course acknowledging that when the church was to divide in the reformation, one of the issues at stake was Papal Primacy! If one sets aside the religious aspect however, the problem faced by the medieval church was rather similar to what all modern democracies face – making a choice.

If you will forgive a brief history lesson – the Conclave was introduced in the first instance because the Church had to reconcile the fact (notionally at least) that God had appointed one person to the role of Pope, but that the Cardinals held sincerely different beliefs whom that man should be. A democratic vote was judged the means of acknowledging there was a division, and achieving a united result. In actual fact, the subsequent development of the distinct features of the Conclave vote, namely the necessity of a two-thirds majority, a secret ballot, and the Cardinals being locked away until a decision is reached, were all intended to achieve a final result that would be clear, achieved quickly, and would not be challenged afterwards.

The church may have been authoritarian in nature, and the era described many centuries previous to us, but there are some good lessons to learn from the choice by the medieval church to adopt a voting mechanism to choose their leader:

1. Division is to be expected
One of my favourite quotes is attributed to former Conservative Prime Minister Arthur Balfour: “Democracy presumes a people sufficiently united as to bicker.” There is a worrying trend in modern politics to criticise opponents for daring to have a contrarian view – which in my view is both an attack of freedom of expression, and also what Sir Humphrey would refer to in Yes Prime Minister as ‘playing the man, not the ball.’ Let’s not be disheartened that division occurs, but rather be glad that we can enjoy earnest debate, reassured that every government will be challenged and held accountable for the decisions it has taken, and embrace the opportunity each of us has to assess the personal performance of our local MPs.

2. Elections are meant to ultimately unite
The intention of the Papal Conclave was (and still is) to unite the church in support of the man elected as the Pope, regardless of whom the man might be. The General Election of May 2015 will be exactly the same – it will tell us whom the country has chosen to entrust to the task of governance for the next five years, whether they should govern alone or in some form of coalition; and potentially even whom the principal opposition should be, or with whom the largest party should form a coalition. At the moment we can only guess, but by the morning of Friday 8th May 2015 we will know.

3. The choice is ours
With that in mind, we should remember what a great privilege it is that we have the opportunity to choose who represents us in Parliament, and therefore to determine indirectly whom will form the next government. The obvious flaw of conclave was the limitation of the franchise to certain Cardinals – whereas the vast majority of us enjoy the right to vote for our MP. We’re going to be divided in our sincerely held beliefs, but we’re also going to achieve a clear outcome by the end.

I would not be so arrogant as to attribute sole success of democracy to this part of the Church’s history – but when we ask what Christians have done for us, we can certainly learn from this aspect of their history, and take to heart the lessons for today.

The Labour lead is soft

Around this time last year there was plenty for Conservatives to be gloomy about, with the opinion polls showing Labour hovering persistently in the high 30s and the Conservatives struggling to break out of the low 30s. The latest opinion poll by Lord Ashcroft shows the two parties more or less neck-and-neck on 30% and 31% – and most other polls show a similarly small Labour lead, but basically too-close-to-call. It reflects an assertion that I made last winter which is worth repeating now: The Labour lead is ‘soft.’

For the benefit of the vast majority who (very sensibly) don’t usually bother with opinion polls, a ‘soft’ number is an unreliable number that is highly likely to change. The classic example of this is in fact the polling numbers for the main opposition party during the middle period between elections – lots of voters indicate that they want to vote for the opposition because the main thing governments usually do is annoy the electorate – an astonishing example is from September 2008 when an Ipsos-Mori poll had the Conservative opposition on an amazing 52%! When you consider that the Tories ended up with 37% of the vote in May 2010, you can see that the number was not a reliable estimate but rather a reflection of voter dissatisfaction with the Labour government of that time (and, given that most other polls weren’t that high, also likely to be an outlier!)

So one year on the polls have narrowed, and the main news I am taking is this – the Labour lead was soft, and probably still is soft. Lots of voters were angry and disappointed, and probably still are – but they aren’t sure they are ready to trust Ed Miliband and Labour. The impact of this is potentially huge – using electoral calculus I did a base prediction of seats assuming that the Conservatives had 30% of the vote, Labour 31%, the Lib Dems 8% and UKIP 17%. In this model, the Conservatives would have 273 seats and Labour 324 – to all intents and purposes, a Labour government. I personally think that the polls will shift between 2 to 3 points simply because of the soft Labour lead – adjusting for a 2.5% swing from Labour to Conservative (ie. Conservatives on 32.5% and Labour on 28.5%) puts the Tories on 308 and Labour on 292. I took the model one stage further and supposed that a reasonable percentage of the UKIP vote swings back to the Tories – not impossible given that Lord Ashcroft’s findings showed that over half the UKIP vote placed the Tories second. Reducing UKIP to 10% and giving an extra 7% to the Tories put the Conservatives on 352 and Labour on 254 – a clear majority for the Conservatives.

Of course all of this is supposition – in all of this the Lib Dem vote remained stubbornly at 8% – but it may recover. Labour may in fact recover voters from the Green party – and this model did not account for the rise of the SNP in Scotland. But it does show that a soft poll lead can massively hide what is actually happening. It is also my theory that if the Tories begin picking up reluctant UKIP voters, and/or Labour begin picking up reluctant Green voters, that will accelerate a poll trend away from the smaller parties and towards the two largest parties. At the moment the polls are telling voters it is too close to call and no party is likely to win a majority – that incentivises voting for a smaller party to hold the balance of power. If either Labour or the Conservatives begin pulling away into a clear lead and there is the possibility they can form a majority government, then the smaller parties mean begin to lose support in the polls as the voters focus instead on whether they want David Cameron or Ed Miliband in Downing Street – and the lower those numbers go, the greater the likelihood more wavering voters will leave the smaller parties and vote for the big two.

It could go either way and it is still too close and too uncertain to call – but I am absolutely certain that the Labour lead remains soft, and the Conservative vote is certain to increase between now and the election.

The Constituency Effect

In a number of posts so far I have referenced tactical voting and the distorting impact of the First-Past-The-Post system we use for Westminster elections. I did not want to assume that everyone is automatically familiar with why this is the case, so I am very briefly going to touch on how votes turn into seats. If I were to compare my previous post on the likely composition of the House of Commons with the actual projected vote shares of the parties, there would be an obvious disparity – that is to say, the Greens and UKIP would only have a collective 4 seats with around 21% of the vote, while the Lib Dems with a projected 8% might end up with 32 seats! This of course is deeply ironic given the Lib Dems past history – in 1983 (as the Liberal/Social Democrat ‘Alliance’) they finished 2% behind Labour on 24% of the vote, but won a mere 21 seats to Labour’s 220! The reason is simple – they finished a strong second in lots of seats, but you don’t get seats for finishing second!

On Political Betting there is an excellent article explaining how the Lib Dem vote could collapse, but they could still remain major players post 2015. While their vote is likely to collapse everywhere, and especially where a sitting MP does not run again, they only need to poll about 35 to 40% of the vote to stand a chance of holding on, especially if UKIP or the Greens take votes off Labour and the Conservatives – they only need to finish with more votes than any other party in these core seats. It is a bizarre truth that the Lib Dems might crash to 8% of the vote overall, and in many seats finish with less than 5% of the vote – but in a crucial 30 odd seats, sneak a win with 40% of the vote. That’s the constituency effect – as ABBA would have put it: “The Winner Takes It All.”

Now it is not the purpose of this particular post to visit the long-running debate on whether our present electoral system is unfair and should be replaced by a proportional system, as is used for devolved elections in Scotland, Wales and Northern Ireland. It is simply to highlight that if you do vote for a particular party, it only impacts your given constituency – and even if a party is bombing nationally they still might pull out a strong local showing. I suspect this will contribute to a lot of disappointed voters based on current trends – imagine for example if UKIP and the Greens outpoll the Lib Dems, but Nick Clegg ends up choosing the next coalition partner because the Lib Dems have enough MPs to do so?

This is why my advice on making your own vote count is very simple – judge the incumbent MP. If they have done a good job, and you are happy with the party leader they would endorse as Prime Minister, vote to re-elect them. If not, then vote for the candidate who is best placed to defeat them. As I mentioned in my last post, you do not gain bonus prizes for voting for a different opposition party member. In much the same way, it doesn’t actually matter (directly – but I will revisit this!) if you decide to vote for the candidate who will finish 3rd (or 4th, or 24th) in the election. Your vote still won’t impact the final outcome, except to prevent the candidate in second from beating the winning candidate. Although you can vote expressively for a particular party I would encourage you to make your vote count – endorse your MP, or vote for the candidate most likely to hand them their P45!

The dangers of the split electorate

Recent opinion polling has shown just how much the British electorate has fragmented – and perhaps the biggest shock of all is the extent to which the SNP is potentially poised to breakthrough in Scotland, and deprive Labour of several seats it had presumed to be safe. Based on latest polling averages which I entered into Electoral Calculus, I got the following projection of what the composition of the House of Commons might look like:

Hung Parliament

Projected seats based on opinion poll averages.

Having observed in my earlier post that a government needs 326 seats to win a majority you will notice the evident problem – not only do Labour and the Conservatives fall far, far short of a majority, it is mathematically impossible for two parties (apart from an unlikely grand coalition of Labour and Conservatives!) to collectively form a voting bloc of 326 seats. This is genuinely new territory in British politics.

It isn’t my intention to use this post to comment on the polls or on how votes convert to seats – those are two different stories! Instead I would make the following observations:

1: It will be almost impossible to form a stable coalition. The government would be formed by whichever one of the two largest parties could pull together a deal with enough MPs to support the government. The most simple would be with the SNP and Lib Dems – but that presumes that both parties are willing, not only to join a coalition, but moreover to sign to a coalition agreement with concessions made to every party, and the policy platform largely dictated by the largest party. That is highly unlikely given their diverse platforms.
2: A confidence and supply agreement is likely. This simply means that a party agrees to support (or in the very least, not vote against) major government items of business, such as the Budget or votes of no-confidence. The Lib Dems have been stung by a coalition government, and are unlikely to want to be a human shield again for either Labour or the Tories. The SNP meanwhile are entirely motivated by achieving maximum benefit for Scotland, and they know full well they are most likely to get that by winning concessions case-by-case.
3: Such a Parliament would be highly unstable. Our best comparison is the 1974-79 Parliament, where Labour had a minority government. When they lost the support of the Liberal Party, they had to broker deals with the smaller parties (Ulster Unionists, Scottish/Welsh Nationalists) to get their business through – and it was exhausting and unstable. It would be next to impossible to plan a coherent programme of government without having the security of a cohesive majority
4: There would be a huge risk of side-deals. To scrape together the necessary 326 for a given vote, the government may well just have to buy off as many MPs as required. With UKIP, the Green Party, and the various Northern Irish parties involved, that could quickly become complicated and or expensive – again, you would not have a coherent, cohesive or wise programme for goverment.

In view of this, I predict that if the result was as outlined above, the government would be forced to hold a fresh election very soon after the first election, to ensure some form of stable government. I’m still inclined to think however that the dangers set out above will concentrate the minds of the electorate – and however angry voters may be, they do not want to wake up with either Ed Milband or David Cameron being held hostage to fortune by the smaller parties.

Your MP has less choice than you think

I commented earlier in this blog about the increased fragmentation of the British electorate. This is not a new development, but it is a striking one – in 1951 the two largest parties won 97% of the vote between them; by contrast in 2010 they won 65%! Rather than comment on how this has come about, I’d like to argue that voting for a party other than the big two is less of an alternative choice than you might think.

One of my lesser celebrated modules during my master’s course was called “The Theory of Voting” – but being a political anorak I thoroughly enjoyed it! The core of the course is how you use voting mechanisms to reach decisions – recognising that voting is the fairest way to resolve differences. And one thing that we have to bear in mind is this – when decisions are reached, it ultimately comes down to a binary decision of yes or no – either you support the measure, or you don’t. Let us take the Budget – in practice you cannot accept 60% of the Budget bill. You can vote yes or no on amendments, but when the whole Budget is submitted, it is either accepted or rejected.

Let me briefly illustrate why this is important. There are (usually) 650 MPs sitting in the House of Commons – so you need 326 to have a majority. Advocates for third parties like to argue that they can argue for a moderate position compared to the bigger parties – to use a fictitious example: Party A argues for a 20% base tax rate, and Party B argues for a 40% tax rate. Party C, which is smaller, argues for a 30% tax rate. However, let’s consider what that would look like based on a House of Commons loosely based on the 2005 election results:

Party A has 202 MPs
Party B has 355 MPs
Party C has 62 MPs
And there are 31 ‘Others’ belong to other parties

Even if you put Party A, Party C and all the ‘Others’ together, you can’t come to 326 votes. In short – Party B will always get their way. Now – I recognise the capacity of sections of Party B to rebel against the government and help the opposition to defeat a government motion – but by and large they won’t do this – they represent the party they do because by and large they agree with their party’s platform! It doesn’t matter that there are (at least) three different parties in opposition – their collective opposition cannot defeat the government and doesn’t impact the outcome.

Of course, we currently have a so-called ‘hung Parliament’ in which no party has a majority. I still maintain that despite this, variance in opposition has no real impact. In the first instance – you still have the ‘majority’ problem. The coalition parties (at least on government business) vote as one bloc. That bloc has more than 326 MPs, and so even if you put all the opposition MPs together, they won’t win. Secondly however, if the smaller coalition partner is willing to rebel against the larger party, as the Lib Dems did when they voted with opposition MPs to amend changes to the Spare-Room-Subsidy reform, this makes for extremely unstable government. No government can plan fiscally responsible government if they consistently have to accommodate demands from Parliament. The upshot of this and final complication, is that a coalition government therefore will seek tight and binding agreements when they form a coalition to effectively prevent such rebellions – again bringing back the majority vote problem.

Your MP really has only one choice – to support the government or to oppose it. That is why I have argued, and will go on arguing, that you should choose your MP based on whether they will endorse the government or not. Having a different flavour of opposition MP won’t actually make any difference to the outcome in Parliament.