A Question – and also Fimacademics, or what you've been missing out on if you don't follow Bad Horse and PoweredByTea · 3:05am Mar 25th, 2013
One of the things I love about Fimfiction is that there's so much more here than just fanfic. When I first registered with the site, I looked at the blog feature somewhat askance, but at this point about half of the people I follow are on my watchlist because I like their blogs or comments and want to keep track of them and provide encouragement. That's not to say they aren't good writers, too – just that their stories aren't what got me following them initially.
A few days ago, Bad Horse published a blog post looking at information theory and writing, or to put it another way, how word novelty impacts our impressions of story quality. It's a really interesting idea, if dense – and, to be complete, was inspired by the musings of PrettyPartyPony on her own blog. Today, PoweredByTea busted out with some full-on research into this matter, and if you don't think it's indescribably
Or just go watch this.
awesome that we have people actually researching these things right here in our midst, I suggest you go watch some Neil Degrasse Tyson videos until you've been suitably indoctrinated into the awesomeness of science.
Anyway, as a statistician, this gets me all sorts of excited and I couldn't help thinking about other ways to look at this problem, as well as a few other fun pony story data issues[1]. But it raised a question to me, one whose answer I've been taking for granted without really considering. What do people see as a better barometer of story quality: total likes or like-to-dislike ratio?
The "Top Stories" list for Fimfiction uses a type of adjusted like-to-dislike ratio, which lets stories that have lots of good feedback but not that many views compete with stories that have thousands of likes, but a number of dislikes as well[2]. I've always considered this a more reasonable way to gauge story quality, but I hadn't really sat down and thought about it before. So I'm curious, my wonderfully clever readers, what you think about this. Are you more likely to read a story if it has a high total like count, or do you care more that it's like-vs-dislike meter is almost entirely green[3]? Or do you really not care, and judge what you want to read solely based on title and description, or on what shows up on EqD and/or in the feature box?
Inquiring minds want to know![4]
[1] Bad Horse has done some of this, too. He has secret files about how this whole thing works, kept in a safe in his concrete bunker, I'm sure, where he and his minions meet to plot the destruction of all that is good.
[2] It also uses something called Wilson Scores for figuring out this ranking, which boggles my mind, because it's so much more computationally complex than evaluating this as a Bayesian with a non-informative prior. In short, the like-to-dislike ratio you should care about is probably (Likes + 0.5)/(Likes + Dislikes + 1). Anything more complicated to calculate than that is kind of a waste of time, since it's going to give you approximately the same thing and require lots more thought.
[3] Apologies to any red-green colorblind folk. Blame Knighty, not me.
[4] And on a somewhat unrelated note, inquiring minds might also want to read this, which I was tipped off to by Jesin on Powered's blog post. It's not incredibly novel, but it's something we don't often think about and definitely fascinating.[5]
[5] And yes. My firmware has become corrupted by the virus that is GhostOfHeraclitus.
I like to look at the views per comment ratio, if the positive feedback is at least 90%. I'd rather read a less-than-stellar story that gives room to a lot of talk than a very well received, but completely straight forward one.
Of course, no metric is perfect, but this has served me well so far.
I judge based on descriptions almost exclusively. Recommendations weigh into that, but I've seen far too much awful writing amass hordes of upvotes and comments for those to sway me much, if at all.
I generally go to the top rated list or feature box to gather more for the read later list or occasionally browse the new stories (I love finding under-appreciated gems in there) but the like/dislike doesn't matter to me that much- if my interest is piqued I'll read anything
I actually do use the likes/dislikes ratio straightforwardly, with a caveat. The higher the likes total gets, the more relatively sensitive I get to dislikes. A story with a thousand upvotes and a hundred downvotes looks sketchier to me than a story with ten upvotes and three downvotes. Ten upvotes and three downvotes is fairly normal for a new story or an inexperienced author, whereas a thousand upvotes and a hundred downvotes indicates a story that had to rub a lot of people the wrong way.
There are a few massively up-voted stories on this site that really aren't good.