(Update Feb. 27, 2015: Since some time before Dec. 12, 2014, I have been blocked by Darren Rovell. It might be because I questioned his Johnny Manziel autograph story (admittedly minor questions answered by others).  It might have started with this blog, from May of 2013.)

I recently had a dust-up with ESPN’s Darren Rovell about the reliability of Poptip poll results.

Poptip is a program that allows users to conduct quick surveys on Twitter, and then compiles the results.

No harm, nothing foul.  But apparently some folks got into it with Rovell about his references to a “margin of error” in his Poptip polls.

So being a professor and a math nerd, I thought I would convene a quick session on surveys, if for no other reason, to explain why some people (like me) get so agitated when Poptip polls claim a margin of error.

A margin of error is one measurement of how closely a sample’s opinions reflect the population as a whole.  You might also see it called a “plus/minus.”  It relates to the survey’s reliability in predicting the results of an election, for example.  It does seem to give a poll a certain gravity and authority, but it’s not always allowable.

The issue here involves not the size of the sample or the quality of the survey, but how the sample was drawn.  As anyone who has studied statistics will tell you, the margin of error is provided only when the sample is drawn from the population by some mathematical formula — random or systematic.

Poptip polls — like the quick polls on the front page of ESPN — create what are known as volunteer or convenience samples.  These are not mathematically drawn samples, and they cannot reflect the population as a whole, because they are drawn from people who happen to be on the ESPN page, or see a Darren Rovell tweet, and vote.

They are fun, and they give often thousands of people a chance to voice their opinions.  But that is as far as it goes.  The results reflect only the people who voted, not sports fans or voters as a whole.  Even responses in the tens of thousands (“mass,” as Rovell described it to me) have no statistical meaning beyond those who vote.

Again, that’s OK.  No problem.  Just don’t present quote a “margin of error” in a survey to make it into something authoritative when it is not. That’s my issue.

When a pollster uses a random sampling method, he/she has mathematical tools available to predict how close the results are to the population as a whole.  Not perfect.  Not always right.  But as Nate Silver demonstrated in the 2012 election, carefully drawn data in skillful hands can yield rich information.

I’m not here to spoil anyone’s fun. If you want to do a PopTip poll, have at. It looks like fun.  But don’t talk margin of error. That’s just putting lipstick on a statistical pig.