26, 2008 — Comedian
Robert Orben once asked, "Do you ever get the feeling that the only
reason we have elections is to find out if the polls were right?"
Orben was joking, but for polling firm Ekos Research Associates, the
upcoming federal election is a chance to prove that their controversial
new polling method can stand up to scrutiny.
|Intead of employing hundreds of call centre workers,
Ekos Research Associates can now conduct their opinion polls electronically
through an automated phone system.
Ekos is the first Canadian polling company to use Interactive Voice
Response (IVR) or automated polling to track how Canadians intend to
vote on Oct. 14.
IVR is a new version of phone polling where, instead of speaking to
an interviewer, respondents hear an automated voice asking them to participate
in the survey. They respond to the short poll – only five to six
questions are asked – using the keypad to indicate their preferences.
For example, when asked which party they intend to vote for, respondents
might press one for the Conservatives, two for the Liberals, and so
Paul Adams, executive director of Ekos, says IVR has allowed the company
to reach more people every night than through traditional
For its daily election tracking poll, Ekos surveys about 1,000 Canadians
every evening, whereas phone polling reaches only about 300 or 400 per
Adams says he knows some pollsters are skeptical about the new method,
but believes that IVR is as reliable as traditional phone
polls, if not more so.
"You can think of all sorts of theoretical objections, that a
person won't respond to automated voice," he says. "But the
fact is that it's had a very successful run in the United States."
IVR south of the border
Ekos decided to make the switch after seeing how successful the technology
was in predicting the outcomes of the U.S. primaries.
SurveyUSA and Rasmussen, the two companies using automated
polling in the United States, have a good track record for accuracy
and were ranked second and third in their polling during the primaries
by Nate Silver of the popular polling website fivethirtyeight.com.
Charles Franklin, the co-founder of pollster.com and a professor of
political science at the University of Wisconsin, says there are some
drawbacks to IVR. You can't tell who's answering the phone –
a child or an adult – and the surveys must be kept short and simple
so that people can respond through the keypad.
But Franklin says that IVR is cheaper than traditional telephone surveys
and allows pollsters to respond to breaking news quickly.
"Theoretically, voters might even feel more comfortable reporting
a vote to a machine than to a possibly judgmental human," he says
in an email. "That might matter more if the campaign involves
potentially sensitive topics, as in the case of race in the current
This is a recording
But George Bishop, a professor of political science at the University
of Cincinnati and author of the polling textbook The Illusion of Public
Opinion, is more critical.
|'Theoretically, voters might even
feel more comfortable reporting a vote to a machine than to a possibly
Unlike most phone polls where the interviewer can randomize
who they speak to, by asking for the person in the household
with the latest birthday for example, automated polls are stuck with
whoever picks up the phone.
"What if you don't understand the question? There's no opportunity
for clarification," he says. "There's also no one there to
probe you to answer the question if the person isn't sure."
Other Canadian pollsters aren't ready to jump on the automated bandwagon.
Timothy Woolstencroft, managing partner of The Strategic Counsel,
the pollster for the Globe and Mail, says the debate over IVR is part
of a broader controversy within the polling industry about the reliability
of new technologies, such as online polling.
| Critics of IVR say the automated system has no
control over who responds to its polls.
Woolstencroft says his firm, which uses traditional telephone
polling for its election surveys, takes its cues from pollsters
for other major media organizations, such as NBC, ABC and the New York
Times – and it's sticking with what it knows.
"It's interesting and novel, but it's not yet looked at by your
peer group as an accepted methodology," he says.
Tim Olafson of Angus-Reid says his company hasn't considered making
Angus Reid uses online polling for its election coverage and
Olafson says he's happy with the results.
Olafson says that from his limited experience with IVR he's found that
it can work well, but isn't appropriate for all types of polling.
"I've seen IVR be successful in election campaigns and I've seen
it fail. I've seen telephone be successful and fail and I've seen online
be successful and fail," he says. "Would we use it in our
political polling? No, we wouldn't."
But Adams says he's confident in Ekos' election data so far.
Ekos' numbers have been within the range set by traditional phone polls.
Adams says that as the election nears and as people decide whom they're
going to vote for, the polling data from each company tend to tell the
"Let's say we don't converge [with other pollsters], which I would
be surprised by but we've never done an election with IVR so I can't
be certain," he says. "I guess the proof of who has the better
methodology will be who's closer to the actual results."