The voting on the first round of 10Questions closed last night at midnight Eastern time, and what a run it's been! We topped out at 121,614 votes, of which approximately 23,000 came in during last 10 hours of the day. We also ended up with more than 31,000 individual participants, way exceeding our expectations.
A bunch of new video questions were submitted at the last minute, and one of them, on the size of government, was powered into the top ten by the concerted efforts of its maker, Patrick Ruffini, and his friends among conservatives online. And that development is worth discussing in some depth, as it has raised the hackles of some users, judging from some of the emails we've gotten today.
Not only did Ruffini (a former director of e-campaigns for the Republican National Committee and a prominent contributor to our techPresident group blog as well as his own site) put out a call on his blog, he also posted a call on talk radio host Hugh Hewitt's blog, where he has guest-blogging privileges, and on techRepublican. As he explains in a post he cheekily calls "Hacking 10Questions," he also emailed his contact list, alerted some Facebook contacts, and did some Facebook status and Twitter updates to spread the word. He reports that RedState, Instapundit and Race42008 all posted about his video in the last two hours. (And other sites definitely picked up his call, like Flopping Aces.)
Ruffini describes how he undertook to push his video in under the noses of what he perceives to be a predominantly liberal user base at 10Questions, by waiting until the last minute and then swooping in with a crowd of positive votes before anyone could notice and counter-mobilize against his question. I'm not going to focus on his political assumptions (suffice it to say that I think he's wrong, and the mix of leading questions on the site as well as the major feeders of traffic show a wide range of views, and certainly can't be pigeonholed as narrowly liberal or left wing; also, his question was hardly that ideological), and instead I want to get at whether the process around his question rising so quickly was fair or not.
A look at our site traffic from yesterday shows 13,367 unique visits from more than 700 sources. The top referrers, other than direct visitors (who numbered nearly 5,500) were Hugh Hewitt, Patrick Ruffini, Crooks and Liars, RonPaulForums, Townhall, organic Google search, the Marijuana Policy Project, Digg, the Politico, ConservativeGrapevine, the New York Times, Why Tuesday, and Fair Elections Now. The sites Ruffini mobilized generated about 2600 unique visits out of that total. Throw in another 1000 visitors from random sources like email or Facebook that he presumably generated. That's not even a third of all the visitors we had during the day.
In the end, Ruffini's question garnered 1789 net votes (1931 for and 142 against), and assuming we don't find irregularities in the voting, it made it into the top ten despite being on the site for less than half a day. That has got some people asking, was this fair? For example, a 10Questions user named Adrian Haynes wrote us this morning to complain. He's given me permission to quote his email in full:
Patrick Ruffini used unfair tactics to get a question that would be voted down if it was subjected to a substantial voting period.
His blog documents his intent to do so [....]
"About two weeks out, I spotted an opening. The thought of an eBay auction came to mind. If you're bidding in an auction that lasts a week, it makes no sense to play early in the process. Your early bid will just be fodder for someone to one-up you, and could trigger a bidding war that increases the eventual going price. The idea is to show as little interest as possible — until the final seconds of the auction when you swoop in with (hopefully) the final bid."
Please subject the question to a fair voting period, or remove it from the running.
Adrian Haynes
I wrote back on behalf of the 10Questions team:
We respectfully disagree with your reading of what took place here. Patrick Ruffini didn't do anything outside the rules of the 10Questions process (assuming our audit of the votes on his video doesn't uncover signs of ballot-stuffing). He didn't do anything anyone else couldn't have done. In that respect, what he did was not unfair, just smart.
Now it may be true that the video would have been voted down with more time, but that becomes a slippery slope. Unless we get everyone to vote on every video, the same argument you are making against his question could be made against others, i.e. that not enough people had a chance to look at it to ensure that its existence in the top ten reflected a wide community consensus. Developing that kind of intense community involvement doesn't happen quickly, though we do think it IS developing around 10Questions and it will be precisely these kinds of episodes that help everyone evolve an understanding of how to make the site work best for all of us.
One thing that this episode does suggest to us is that perhaps we should close the voting on Round Two at an earlier time of day, when more people are in a position to be watching the site for last minute developments (if that is actually something they care about).
Again, please recognize that this is an ongoing, organic process of development and that we will continue to listen to feedback and make changes accordingly.
I don't think that reply fully satisfied Haynes, and he may want to add more in the comments below. But what do you think? Do we need to change the 10Questions process in some way to deal with last-minute swings in voting? Or should we accept that this is a natural part of any voting system and that as long as everyone is above-board in how they earn votes, the process is basically fair?
print
email
delicious
digg
technorati
Thanks for this post
In a continued spirit of transparency, let me suggest one method you could have used to "defeat" my methodology: close the question submission period before the end of the voting period. This probably would have given the votes on my question ample time to "correct" themselves.
As you say, though, this leads to a slippery slope whose effect over the long run is to chill participation. You need ever more complex methods to defeat ever more sophisticated attempts to vote up questions. And there is probably no way you could defeat a MoveOn or an American Family Association, with 3M+ email lists, if they decided to ask their members to vote up a question in unison -- no matter when they did it.
I will say I find the reaction to this pretty interesting. When MoveOn sent an email to a segment of its list promoting the net neutrality question (which remains the #1 question), there was hardly a peep of complaint from the user base and the tone of the explanation did nothing to raise questions about the fairness of the process:
It's also interesting that questions fueled primarily by advocacy campaigns that were once #1, 2, and 3 ended up as #1, 4, and 5. That I can find, no one found these concerted campaigns troubling -- and the longer voting period (the solution your correspondent suggests) did nothing to dislodge these questions from the top 10.
It's certainly possible my question could have survived a longer period on the top 10. It had a bit of a libertarian streak that could have earned support from Ron Paul supporters. The very early voting for it (before I did anything to push it) was 2-to-1 in favor.
But for the purposes of this experiment -- undertaken to see how a minority group could effectively mobilize when outnumbered -- I wasn't going to leave anything to chance.