Political Polls Have Consequences
Ever since the Literary Digest poll of 1936 infamously predicted that Alf Landon would smother Franklin D. Roosevelt in that year’s presidential election, pollsters have been working to refine their art.
This column looks at why polls have become more difficult to do well and of the consequences the polls can have for election outcomes.
When I commissioned polls in the 1970s and ’80s for U.S. Senate and other candidates I was working for, we had the gold standard of random digit dialing of the universe of nearly 100 percent of households that had telephones. The pollster was confident he could achieve a sample of voters that was representative of the whole population.
Today that is much harder to come by. First, there is the problem of the non-response bias. With caller ID, many people refuse to answer calls from unknown numbers.
And nearly 40 percent of households no longer even have a land line, so good pollsters buy lists of cell phone numbers and call them. But many cell phone users refuse to take time for pollsters because they are often paying for the calls on their dime.
The people who do respond to pollsters entreaties on the first call are different from those who never respond. They tend to be people with little to do. These respondents are often passionate party faithful. They are different overall from busy people who won’t respond.
So pollsters call non-respondents four or more times in efforts to get them to respond, which drives up the cost of polling.
After many tries, pollsters take the responses they have garnered and weight them. An older white woman may be given a weight at 0.7 of a response because the pollster has more of them than is representative of the whole population. On the other hand, a young black male may be given a weight of 1.7 because there are too few like him in the response pool.
This can distort the sample’s representativeness because the pollster has to assume that the young black man, from a very small sample, is representative of the whole of his group.
For non-presidential election years such as this one, there is also the “likely voter problem.”
For example, the Paul Simon Public Policy Institute at Southern Illinois University recently conducted a state-of-the-art, 1,000-respondent sample survey of the Illinois electorate.
Seventy percent of the final sample swore up and down to the pollster that they were going to vote, yet we know from experience that only about half of them actually will vote in the November election. Which half? Pollsters again have to make assumptions.
“People lie,” observes Charles Leonard, who supervises polling for the Paul Simon group. Some people will say what they think the interviewer wants to hear.
A really reputable poll is expensive. It may take 10,000-20,000 calls to get 1,000 usable responses, at $30 per usable land line response and $45 per cell phone. The latter is more expensive because pollsters have to buy lists of cell phone numbers (and you thought nobody knew your number) and dial the numbers manually.
That means $40,000 or so per survey.
As a result, many who want poll data are turning to automated polls, with robot voices conducting the polls. Federal law prohibits automated dialing of cell phones, so such pollsters either ignore cell phones or use panels of cell phone users and/or internet respondents they have recruited. Then they do plenty of weighting.
I checked with one automated polling firm, and the website said a 500-response survey, based on a 5 percent response rate at 10 cents per call made would cost only about $1,000. So you can see the appeal. But maybe you get what you pay for.
Polls have consequences. U.S. House Republican leader Eric Cantor would have behaved differently this past spring if not lulled into complacency by his own poll which showed him with a 34 point lead over his opponent; he lost.
And supporters and fundraisers for GOP Illinois gubernatorial aspirant Kirk Dillard may have been discouraged from further activity by an automated poll that found him trailing Bruce Rauner by 17 points this past April; he lost by 3 percentage points.
Polls are at best approximations of reality. In reading a poll, look for the sample size; be wary of any sample less than 500. Never rely on one poll alone. And the more a pollster tells you about his methodology, the better.
But now we can take our own poll, by going into the voting booth November 4.