Two polls on Quebec voting intentions were revealed on March 21 by political columnist Jonathan Trudeau during the morning show hosted by Paul Arcand of Montreal’s Cogeco station.
They have surprised many observers, both by their similarities and their differences, but above all by what they suggest.
First, the elephant in the room, the Mainstreet Research poll that gave Éric Duhaime’s Conservative Party of Quebec (PCQ) second place! This study was commissioned by an oil and gas company seeking to exploit the shale gas reserves in the St. Lawrence Valley, Utica Resources. The participants were asked about energy issues with questions such as: “Quebec imports all the oil and gas consumed here. Does this concern you? and “Do you think it’s a good idea for Quebec to produce natural gas for export to Europe to replace Russian gas?” The study was conducted while the National Assembly was considering Bill 21, which would ban all hydrocarbon exploration and exploitation in Quebec, a piece of legislation that the study’s sponsor is strenuously opposed to.
So far no problem.
However, the sponsor took the opportunity to also measure respondents’ voting intentions. And the results are surprising to say the least: 36% for the Coalition Avenir Québec (CAQ) against 24% for the PCQ. The other opposition parties are lagging far behind.
However, the recent average of support for the PCQ – favorable to the exploitation of shale gas in Quebec – hovers around 14%. How do you explain what appears to be a true statistical anomaly? Mainstreet Research surveys are conducted by automated phone calls (known in the industry as IVR – interactive voice response) and are therefore truly “probabilistic”, ie the sample of respondents is random (in this particular case with a margin of error of ±3%, 19 times out of 20). The probabilistic nature of these kinds of polls makes them susceptible to statistical fluctuations, but not of this order.
The president of Mainstreet Research, Quito Maggi, also expressed some doubt about the results of his research. “We have been measuring growing support for the PCQ for over a year, mainly in the Capitale-Nationale region. However, this data seems well outside current norms compared to what we’ve measured in recent months, showing us some volatility, especially among young voters. As with the PPC at the federal level in 2021, support for the PCQ has likely inflated due to pandemic fatigue. †
We are dealing with a classic example of selection bias, as the poll would never have made it to the media if it had not been favorable for the PCQ. Imagine an actor – interested and partisan – doing polls, while the latter is on the move, with everything happening in the news. A first poll gives 14%, another 18% and a third 24%. Because the only data offered to the public is the highest, we are therefore faced with a biased perception of reality.
To illustrate this point, here is a small 500 point graph representing random data around a value of 18 (with a standard deviation of 3). The vast majority of points are just below or slightly above 18, and only a few are further from the mean (arrows point to values of 12 and 24, six points from the mean).
Now look at the exact same distribution, but with the value points less than 20 subtracted from the graph.
The perception is different, isn’t it?
And it is for this reason that the poll in question in the list of Qc125 polls was considered biased (see full list here). It’s not that it was compromised or manipulated, but it was only made public because it was appropriate for the parties involved.
The second study comes from Synopsis Research Marketing. National figures for the Liberal Party, Parti Québécois and Québec Solidaire are about the same as Mainstreet’s, but for the CAQ and PCQ the data shows real differences.
According to Synopsis, the CAQ is alone at the top at 44%, the highest share of support for François Legault’s formation since last fall. As for the PCQ, it is at 16%, which is statistically equal to the PLQ and QS (but still above the current average).
However, this research was commissioned and provided by… the CAQ. In addition, only partial information was disclosed to us, namely national data and the breakdown by mother tongue and age groups.
Of course, the CAQ wouldn’t have given a columnist a poll that didn’t favor him. This again creates a selection bias. This is why he too was judged on a partisan basis. It has nothing to do with the poll’s performance or reliability, but with the source that leaked this poll.
It was the same with the Army Survey performed this winter in driving Marie-Victorin, which resulted in a draw between Shirley Dorismond of the CAQ and Pierre Nantel of the PQ ahead of the April 11 by-election. It was commissioned and then broadcast by the Parti Québécois. Would it have been made public if he measured a 10-point delay for Pierre Nantel? Of course not. The same goes for the CAQ and its home survey Synopsis.
I often say that we should be careful when interpreting political opinion polls, especially those filtered for partisan purposes, published not for the purpose of informing, but of persuasion. That’s why I believe it’s crucial to re-emphasize how important it is to keep the focus on the whole landscape of public opinion, not just a tree in the woods.
Visit the Qc125 webpage here for all Quebec voting intent polls.