News, views and commentary on Long Island, state and national politics
BloggersDan Janison Rick Brand Tom Brune Robert Brodsky Paul LaRocco Yancey Roy Celeste Hadrick
Nassau: Suozzi consultant, Siena pollster tangle over numbers
Bruce Gyory, an upstate-based consultant to Democratic campaigns, calls it “most odd” that the latest Newsday/Siena poll on the Nassau executive race shows Ed Mangano up to 52 percent against challenger Tom Suozzi. He says so in a memo, shown below, to Suozzi campaign manager Danny Kazin. While calling Siena polls honest, nonpartisan and transparent, the memo questions several approaches in the survey.
But Donald Levy, the poll’s director, defended the methodology, including an aspect Gyory targets in his memo — how Siena uses “likely” versus “registered” voters as Election Day draws closer. “The truth of the matter,” Levy says, “is that whether Democrat or Republican, the candidate trailing in virtually any public poll finds a way to blame the pollster.”
Below is Gyory’s memo, followed by Levy’s response, in full.
“You asked me to analyze the latest Siena poll on the Nassau County Executive’s race. After seeing all the polling data throughout the Summer and early Fall showing a tight race between Mangano and Suozzi, with Mangano in the low 40’s against Suozzi (Siena’s August 30th poll) and other data showing Mangano capped at about 45% in terms of generic ballot strength, I find it most odd that Mangano would surge to 52%.
Especially when that surge came from a poll taken in a week (poll taken October 6-9th), when Mangano had his roughest week of press coverage. The school boards, the PTA’s and a Newsday editorial calling Mangano out over the fact that his assessment policies directly led to sticker shock on school tax bills not to mention that the GOP’s ploy to put and finance Hardwick on the ballot to split the Black vote was at once exposed and Hardwick was knocked off the ballot.
Here is what I suggest. Let’s hold our assessment until our pollster Global Strategy Group (GSG) next goes into the field with a poll to track for a change in the trendline, knowing that GSG takes great care to properly sample crucial subsets (e.g., the minority vote and female voters), to make sure the cross tabs results are not scientifically suspect (e.g., a margin of error in the 15-20% range for the cross tabs underneath a poll with a macro 4% margin of error can be common in these public polls).
I do not presume Siena’s work is shoddy, because they are serious pollsters. But I will tell you, this poll doesn’t feel right. For example, in their August 29th poll of registered voters, Siena’s data shows Suozzi sweeping the minority vote with a landslide (up by 41% amongst Blacks and by 20% amongst Hispanics), whereas the October 13th likely voter poll shows a dead heat amongst non-white voters (44% for Suozzi and 43% for Mangano) when the GOP brand has never been weaker amongst minority voters.
That is really odd. Moreover, if Mangano’s internal polling data showed him surging with minority voters why would they have embarrassed themselves on Hardwick? That finding just doesn’t add up. An error on minority voters may foreshadow weaknesses in other places, when Siena’s October 13th poll shows striking Mangano gains (e.g., amongst women, Independents and Jewish voters) from Siena’s August 30th poll.
For example, this latest poll shows women at only a 52% share of likely voters, when historically women are at least 53-54% of Nassau’s November vote and it places Jewish voters at a 17% share when they usually are in the low 20% range. It also shows minority voters at 26% share of registered voters but only 20% of likely voters. Siena could be correct, but they could also be very wrong.
Let’s take a step back. Siena has a very good track record in their polls of registered voters (e.g., their early October poll in the 2010 NYS gubernatorial race projecting a Cuomo landslide proved far more accurate than Quinnipiac’s likely voter poll of October 6th poll showing Cuomo with only a 6% lead). But Siena has a far less stellar record in the accuracy of their likely voter sampling. In Siena’s last 2010 poll here in NYS, a likely voter survey, here in NYS they showed Cuomo heading toward 55% against Paladino and they also showed Schneiderman and DiNapoli in dead heats. Their likely voter sampling had the state’s aggregate minority vote at a 15% share of the electorate rather than the 29% found in exit polling data. That error, underweighting the minority vote by almost 100%, plus an underestimation of gender gap, is why their likely voter poll missed Cuomo crossing 61%, Schneiderman winning by 12% and DiNapoli by 4%, when the votes were actually counted.
I find it worth noting that Siena’s August 30th poll of Nassau was of registered voters (showing a dead heat), whereas their October 13th poll was of likely voters (a large Mangano lead). Simply put, empirically folks should have more confidence in their August 30th than their October 13th data.
Then last month, Siena projected in a likely voter survey, Mayor Richards of Rochester with a lead of 36% the Sunday before his primary against Lovely Warren (63-27%), only problem was that Warren beat Richards by 16%. So with a margin for error of 4.4% in the Rochester race, Siena’s projection was off by 52%!
Moreover, Siena has had chronic problems polling here in Nassau County. In 2008, Siena projected in a likely voter sample, State Senator Kemp Hannon leading Kristen McElroy by 26%, on November 2nd. But two days later, Hannon won by only 52-48% (that poll had a margin of error of 4-8% and it was off by 22% - 5 times the margin of error).
Then in 2009, less than a month before the 2009 Suozzi/Mangano dead heat, Siena projected Suozzi ahead 53-31% (a 22% lead) in a likely voter sample. On election day both had 48% and in a recount Mangano won by 386 votes.
Yet in other races Siena has been spot on (e.g., Foley over Trunzo in 2008 Senate, catching the late break and this year’s mayoral races in Albany, Syracuse and Buffalo). All were likely voter samples. So Siena is not always wrong in their likely voter samples.
Here is what I think. Siena is very proficient in their polls of registered voters, but something can go very wrong in their likely voter methodology. Not always, but too often for comfort. I do not know why. I suspect they use a multi-question screen for registered voters to create their likely voter sample. I further suspect that Siena changes their assumptions on how to measure the level of a respondent’s score for placing them in the likely voter pool. Those assumptions could create flawed samples. But I do not know their underlying methodology.
Bottom line. The Siena polls are honest, they are strictly non-partisan and they have always been at the forefront of transparency – all to their great credit. But their likely voter sampling can be very off. In fact, Siena’s likely voter polling reminds me of the old nursery rhyme about the little girl’s behavior, “when they are good, they are very good, but when they are bad, they are horrid.”
While I would not castigate Don Levy, their polling director, for he is both learned and a good guy, I would also not give him the free pass he implicitly asked for after last month’s Rochester mayoral debacle. Levy said of that Rochester poll, that Siena simply “pulled the wrong marbles out of the poll.” Their blunder rate, particularly here in Nassau County, means there is something systemic that pops up. In fact, when they are wrong Siena tends to inflate the percentage of white males and discount both the size and break of gender gap and minority voting.
So let’s not internalize this Siena poll showing Mangano ahead by 17%, as accurate unless and until we see that explosion coming out of nowhere for Mangano, tracked by our own polling. Our polling by Global Strategy Group takes great care to accurately measure the electorate by gender, race, ethnicity, partisanship and ideology.
Quite frankly, I was surprised to see Siena pop out a poll so different here in Nassau from August 30th to October 13th, with so little change in events on the ground. Should Siena blow this Nassau CE’s race after their Rochester debacle, they could have a lot of explaining to do.
Perhaps the polling gods from Mount Olympus are smiling on us. I recall back in October of 2009, we relaxed after Siena’s poll showed Tom up by 22% over Mangano, while Mangano’s campaign became energized by the challenge of proving Siena wrong. I suggest we do the same in 2013 as the Mangano folks did in 2009.
The best case scenario for us will be if our next internal poll comes back with a much different picture, while team Mangano relaxes. In 2009, we saw an early poll has all the staying power with actual voters, of a walnut in the batter of eternity.
“I can’t wait to analyze our internal data, to see if this Siena poll is a trailblazer or an outlier. The real downside of this poll could be perception, but only if we let it repress our campaign’s energy level. Let’s wait to see if there is a Mangano surge or just another case of what Siena calls picking the wrong marbles.”
"While I will not challenge Mr. Gyory's points one by one, it is fair to say that he is right on some points and very wrong on others (for example, he cites a poll during the 2009 Nassau County Executive race as being of likely voters less than a month from Election Day, when in fact it was a poll of registered voters.
“The bottom line is this: we work our collective butts off to get it as right as possible and we are happy to match Siena's record against any other pollster when it comes to number and scope of state and local races in New York State we have polled. Are we wrong sometimes? Absolutely. About five percent of the time, right around the statistical probability.
“But the truth of the matter is that whether Democrat or Republican, the candidate trailing in virtually any public poll finds a way to blame the pollster. We look forward to our final Newsday/News 12/Siena College poll in this race. Stay tuned.”