Social Europe

politics, economy and employment & labour

  • Themes
    • Strategic autonomy
    • War in Ukraine
    • European digital sphere
    • Recovery and resilience
  • Publications
    • Books
    • Dossiers
    • Occasional Papers
    • Research Essays
    • Brexit Paper Series
  • Podcast
  • Videos
  • Newsletter

5 Things Everyone Should Know About Opinion Polls

Peter Kellner 21st August 2015

Peter Kellner

Peter Kellner

1) Poll Results Are Snapshots, Not Predictions

Public opinion is seldom fixed. Views change. Good polls tell us what people think at the time they were interviewed. Someone reviewing polling results might make a prediction based on the data, but that is a personal judgement, not a poll finding. Suppose you know that a runner is leading by ten metres in a 1500m race, with one lap to go. You might predict that the runner is likely to win – and you would make your prediction with more confidence knowing the state of the race at that point. But you cannot be certain of the outcome. If you want to know the future, don’t commission a poll. Buy a crystal ball.

2) Good Polls Are Seldom Exactly Right, But Seldom Badly Wrong

Polls obtain responses from a small fraction of the population – typically 1,000-2,000 out of a population of millions. Good polls seek to match their samples to the characteristics of the population as a whole – by age, gender, region, social class etc. But statistical theory warns us that even the best survey is subject to a margin of error.

Suppose a coin is tossed 1,000 times. We would expect it to land heads roughly 500 times, and tails roughly 500 times – but it would be a fluke if it landed EXACTLY 500 times each. Likewise with polls. If 50% of the whole population hold a certain view, a well-conducted poll should produce a finding between 47% and 53%  – but the laws of probability tell us but one time in 20, even a “good” poll will produce a result outside that range. But it’s vanishingly unlikely to be, say, ten points adrift of the truth.

3) A Small, Representative Sample Is Always Better Than A Big, Unrepresentative Sample

Newspapers and television programmes sometimes invite their readers/viewers to text, phone or email their views. They sometimes then say something like: “We now have the verdict of more than 100,000 people  far more than any opinion poll”, and claim that the size of the exercise makes it better than a poll of just 1,000 people.

Nonsense. Here’s a cautionary tale. In the 1936 US elections, a magazine, Literary Digest, elicited the voting intentions of more than two million Americans and said that President Roosevelt would be buried after just one term as President, and that his rival, Alf Landon, would win by a landslide. Never heard of Mr Landon? That’s because he lost badly. Gallup Polls conducted a far smaller, but properly representative, survey – and rightly showed Roosevelt well ahead.


Our job is keeping you informed!


Subscribe to our free newsletter and stay up to date with the latest Social Europe content. We will never send you spam and you can unsubscribe anytime.

Sign up here

An unrepresentative sample is an unreliable sample – simply making it bigger makes no difference.

4) The Details Matter – Dates, Question-wording, Client

So, you see some polling information. You note that it comes from a reputable company and therefore likely to be properly conducted. But does it mean what it seems to mean? Here are some tests to apply.

When was it conducted? Polls often look at current controversies, at a time when public opinion might be volatile. If the fieldwork for the poll was, say, two weeks ago, it might be a less accurate guide to current public opinion than one conducted two days ago.

Where does the report of the poll findings appear? If it’s a media report, it might tell only part of the story. If it’s put out by a campaigning organisation, it might select only those findings that suit its case. And/or they might – wittingly or unwittingly – abbreviate the questions and/or results in a way that ends up being misleading. To be certain about what questions were asked and what results obtained, the best advice is to go to the polling company’s own website.

Who commissioned the survey? Polling clients often have their own agenda – to promote a cause, a party, a candidate, a product or a point of view. This fact does not necessarily invalidate research they commission. Reputable polling companies make sure that the questions they ask are fair and balanced. But when clients have an agenda, it’s especially important to look under the bonnet and check exactly what questions were asked and what the full results show.

5) Apples And Pears Must Be Compared With Care

Here is a fictional example of real problem. A poll asks people whether they would prefer to spend a sunny summer day in a city or at the seaside. By 60-40%, people say they prefer the seaside. A year later another polling company asks people where would they prefer to spend a sunny summer day: in a city, at the seaside or in the countryside. 45% say the seaside, 30% say the countryside, 25%  say a city. The next day, a new report says that the seaside has slumped in popularity, with the number saying they would like to spend a sunny summer day there, down from 60% to 45%.

It’s obviously nonsense, as the first poll offered only two options while the second offered three. It’s just one example of why it is unwise to compare the findings from different polling companies asking different questions. That example is particularly egregious; sometimes the differences are more subtle – for example, telephone surveys often find different numbers of “don’t knows” than online surveys; as a result the numbers for each of the answer options are liable to be different.

The only safe way to be sure of movements in public opinion is to compare surveys through time by the same polling company, using the same interviewing method (online, phone or face-to-face) and asking the same questions each time. And even then, small differences (say by two or three percentage points) may reflect sampling fluctuations rather than real change.


We need your support


Social Europe is an independent publisher and we believe in freely available content. For this model to be sustainable, however, we depend on the solidarity of our readers. Become a Social Europe member for less than 5 Euro per month and help us produce more articles, podcasts and videos. Thank you very much for your support!

Become a Social Europe Member

Peter Kellner

Peter Kellner is a journalist, political commentator and President of the YouGov opinion polling organisation in the United Kingdom.

You are here: Home / Politics / 5 Things Everyone Should Know About Opinion Polls

Most Popular Posts

Russian soldiers' mothers,war,Ukraine The Ukraine war and Russian soldiers’ mothersJennifer Mathers and Natasha Danilova
IGU,documents,International Gas Union,lobby,lobbying,sustainable finance taxonomy,green gas,EU,COP ‘Gaslighting’ Europe on fossil fuelsFaye Holder
Schengen,Fortress Europe,Romania,Bulgaria Romania and Bulgaria stuck in EU’s second tierMagdalena Ulceluse
income inequality,inequality,Gini,1 per cent,elephant chart,elephant Global income inequality: time to revise the elephantBranko Milanovic
Orbán,Hungary,Russia,Putin,sanctions,European Union,EU,European Parliament,commission,funds,funding Time to confront Europe’s rogue state—HungaryStephen Pogány

Most Recent Posts

reality check,EU foreign policy,Russia Russia’s invasion of Ukraine—a reality check for the EUHeidi Mauer, Richard Whitman and Nicholas Wright
permanent EU investment fund,Recovery and Resilience Facility,public investment,RRF Towards a permanent EU investment fundPhilipp Heimberger and Andreas Lichtenberger
sustainability,SDGs,Finland Embedding sustainability in a government programmeJohanna Juselius
social dialogue,social partners Social dialogue must be at the heart of Europe’s futureClaes-Mikael Ståhl
Jacinda Ardern,women,leadership,New Zealand What it means when Jacinda Ardern calls timePeter Davis

Other Social Europe Publications

front cover scaled Towards a social-democratic century?
Cover e1655225066994 National recovery and resilience plans
Untitled design The transatlantic relationship
Women Corona e1631700896969 500 Women and the coronavirus crisis
sere12 1 RE No. 12: Why No Economic Democracy in Sweden?

Eurofound advertisement

Eurofound webinar: Making telework work for everyone

Since 2020 more European workers and managers have enjoyed greater flexibility and autonomy in work and are reporting their preference for hybrid working. Also driven by technological developments and structural changes in employment, organisations are now integrating telework more permanently into their workplace.

To reflect on these shifts, on 6 December Eurofound researchers Oscar Vargas and John Hurley explored the challenges and opportunities of the surge in telework, as well as the overall growth of telework and teleworkable jobs in the EU and what this means for workers, managers, companies and policymakers.


WATCH THE WEBINAR HERE

Foundation for European Progressive Studies Advertisement

The winter issue of the Progressive Post magazine from FEPS is out!

The sequence of recent catastrophes has thrust new words into our vocabulary—'polycrisis', for example, even 'permacrisis'. These challenges have multiple origins, reinforce each other and cannot be tackled individually. But could they also be opportunities for the EU?

This issue offers compelling analyses on the European health union, multilateralism and international co-operation, the state of the union, political alternatives to the narrative imposed by the right and much more!


DOWNLOAD HERE

Hans Böckler Stiftung Advertisement

The macroeconomic effects of re-applying the EU fiscal rules

Against the background of the European Commission's reform plans for the Stability and Growth Pact (SGP), this policy brief uses the macroeconometric multi-country model NiGEM to simulate the macroeconomic implications of the most relevant reform options from 2024 onwards. Next to a return to the existing and unreformed rules, the most prominent options include an expenditure rule linked to a debt anchor.

Our results for the euro area and its four biggest economies—France, Italy, Germany and Spain—indicate that returning to the rules of the SGP would lead to severe cuts in public spending, particularly if the SGP rules were interpreted as in the past. A more flexible interpretation would only somewhat ease the fiscal-adjustment burden. An expenditure rule along the lines of the European Fiscal Board would, however, not necessarily alleviate that burden in and of itself.

Our simulations show great care must be taken to specify the expenditure rule, such that fiscal consolidation is achieved in a growth-friendly way. Raising the debt ceiling to 90 per cent of gross domestic product and applying less demanding fiscal adjustments, as proposed by the IMK, would go a long way.


DOWNLOAD HERE

ILO advertisement

Global Wage Report 2022-23: The impact of inflation and COVID-19 on wages and purchasing power

The International Labour Organization's Global Wage Report is a key reference on wages and wage inequality for the academic community and policy-makers around the world.

This eighth edition of the report, The Impact of inflation and COVID-19 on wages and purchasing power, examines the evolution of real wages, giving a unique picture of wage trends globally and by region. The report includes evidence on how wages have evolved through the COVID-19 crisis as well as how the current inflationary context is biting into real wage growth in most regions of the world. The report shows that for the first time in the 21st century real wage growth has fallen to negative values while, at the same time, the gap between real productivity growth and real wage growth continues to widen.

The report analysis the evolution of the real total wage bill from 2019 to 2022 to show how its different components—employment, nominal wages and inflation—have changed during the COVID-19 crisis and, more recently, during the cost-of-living crisis. The decomposition of the total wage bill, and its evolution, is shown for all wage employees and distinguishes between women and men. The report also looks at changes in wage inequality and the gender pay gap to reveal how COVID-19 may have contributed to increasing income inequality in different regions of the world. Together, the empirical evidence in the report becomes the backbone of a policy discussion that could play a key role in a human-centred recovery from the different ongoing crises.


DOWNLOAD HERE

ETUI advertisement

The EU recovery strategy: a blueprint for a more Social Europe or a house of cards?

This new ETUI paper explores the European Union recovery strategy, with a focus on its potentially transformative aspects vis-à-vis European integration and its implications for the social dimension of the EU’s socio-economic governance. In particular, it reflects on whether the agreed measures provide sufficient safeguards against the spectre of austerity and whether these constitute steps away from treating social and labour policies as mere ‘variables’ of economic growth.


DOWNLOAD HERE

About Social Europe

Our Mission

Article Submission

Membership

Advertisements

Legal Disclosure

Privacy Policy

Copyright

Social Europe ISSN 2628-7641

Social Europe Archives

Search Social Europe

Themes Archive

Politics Archive

Economy Archive

Society Archive

Ecology Archive

Follow us

RSS Feed

Follow us on Facebook

Follow us on Twitter

Follow us on LinkedIn

Follow us on YouTube