I've been dealing with life for the last couple days and had to step away from the write-off for a bit, but I see there's been a pretty major explosion of voting discussion in the time I was out. As well as some calls to get it off the already-prohibitively-large "All In" thread. That seems like a good idea.

I'm making a new thread for people to discuss this subject, in part because I've got a fair amount I'd like to say about it. I'll be back for that later today. At the moment, though, I'm just hoping to get people moved over here for discussing this.

For me, I really really would love to see a consistent 'Voting should be done in this style' metric. For my first writeoff I voted based on how I felt the stories rated relative to fiction as a whole, meaning my average was in the 6s somwhere, because getting a 0/1/2 would be really really hard since those would be the shit that just barely passes moderation on Fimfic.

Since then, I've tried to normalize things so each # from 1-10 has an equal number of fics in it. (For that matter, I don't know why we have 0s on the scale; we may as well make it go to Eleven at that point, because 12 is a nicer number than 11). So, stories are voted relative to one another; if every entry were the most amazing thing possible I would consider upending that but generally it works out okay, though it does mean that stories that are 4s and 5s in the general populace are getting 1s and 2s here. But that's fine, that's just reflecting strength of competition.

The issue I see is that we're not consistent in this application. Some of us are grading where each point gets an equal number of entries; some grade on a bell curve; TD grades on curve where 0 is the baseline and the most commonly used score. And yea, I find this frustrating if the voting system is supposed to have some kind of meaning because it does invite gaming it in various ways, and a 10 from him remains a much more potent 10 because most other people are getting like 3 or below.

So, what I would like to see is 'How to vote' codified into the ruleset. I do feel grading a 'Equal # of each, round as needed' or Bell Curve-like voting is the best way to go, but as long as it's codified in some way I will be much happier than 'Do this however you like'.

Finally! I've be avoiding responding in the other thread because I knew saying anything would just keep it going. But I have a couple of very important things to say:

First, I'm NOT a judge of whether your story is good or bad. For God's sake, don't trust any word or number I give for that.

Second, upon thinking about it I can give a scale that will always apply to all of my votes, yet says exactly the same thing I was trying to say on the "All In" thread:

10
9 - I loved it.
8
7 - I liked it.
6
5 - I felt indifferent.
4
3- I disliked it.
2
1 - I hated it
0 - It was unreadable/it was genius but boring.

(With stories gaining or losing a point on the 1-10 scale based on strong or weak points.)

Genius but boring is my high crime of literature. It's the reason my friends online here divide themselves between "Writers" and writers, because the writers I know equate literary writing with genius but boring fics and stories and avoid the Writers like the plague. That's not true "good writing," and in fact it's worse than bad writing, because a) the writer obviously had the potential to write something interesting and instead decided to try to show off by writing something deep and boring, and b) other people are encouraging the poor reputation of a craft I love.

But notice my first point: I can't tell if a story is good or bad. Enter reviewers. There are a few specific ones whom I trust to tell me if something is well written. If they say that a story is genius, but I found it boring, the fic is genius but boring. If they don't think it's very good, it's probably just boring, meaning it falls under "I didn't like it." Fics that are boring don't insult the field of literature, anyone can write a boring story. Fics that are genius but boring deserve the lowest score I can give.

Is this subjective? Of course! I don't know how to give points that aren't subjective without reading the reviews of those people I trust. But it's not about winning. If it was about winning me, or any specific other fic winning, I wouldn't be top loading my scoring with almost every story falling five or above. It's about what I value stories for, and my firm belief that there's something worse than a story that's poorly written.

3956382
For the reason stated above, I don't think there should be, nor can there really be a codified "how you should vote" metric. Everyone will think some flaws are worse than others, and I doubt you can force people to agree on those.

From what can see, this is a competition about objective opinions on fics (too many for the average person to read) that needs a consistant across the board method of voting so that everyone's votes will count practically the same way and not cause any 'curving' of the system due to value interpretation.

So what needs to be done?
1 - The value of a vote needs to be established and made clear (Either spoken, or forced by the voting system)
2 - The value of the input votes needs to be standardized (either all top votes are equal and bottom votes are equal, or the average vote value per fic is equal from one user to the next)
3 - Accommodations need to be made for voters not having read all the stories (This has already been done in one form, simply exclude them from the voting, but that can be unfair (title or list position bias when fics presented) if they don't receive any points while the read fics do.)

I have a few interesting thoughts of possible systems that accomplish these things, but I have to go do something for a bit. I will finish this comment later today.

3956405

Genius but boring is my high crime of literature. It's the reason my friends online here divide themselves between "Writers" and writers, because the writers I know equate literary writing with genius but boring fics and stories and avoid the Writers like the plague. That's not true "good writing," and in fact it's worse than bad writing, because a) the writer obviously had the potential to write something interesting and instead decided to try to show off by writing something deep and boring, and b) other people are encouraging the poor reputation of a craft I love.

What I think you're trying to say is something I generally agree with. However, what you actually said appears to be quite the opposite, especially given the next paragraph. Would you mind clarifying that paragraph?

Okay, so, as most people know, I'm a statistician. I'm also pathologically interested in things like ethics (it's not just for gaming journalism!). So I'm going to talk about voting from two different perspectives, and see where this goes. I'll do them as separate posts. This one will be...

1. The Statistical Side

I've seen people, like 3956382 just now and Trick Question in the "All In" thread, discussing distributions of scores, and whether it makes more sense for individual users to do something like a bell curve with their votes, or a roughly flat distribution where each score shows up an equal(ish) number of times. People have also discussed how not using the full 0-10 span decreases the variability in your votes and thus the impact of your votes.

Now, for anyone who's ever taken a statistics class, hopefully you've heard of a result called the Central Limit Theorem (CLT). In terms of the scores stories get at the end of the competition, this is the big driver of what happens. Briefly, the CLT says that however our individual votes are distributed, as we get more and more of them in the system, the average of those votes gets closer and closer to a nice bell-shaped curve (Normal or Gaussian distribution), and the spread of that curve (standard deviation) goes down in proportion to the square root of the number of voters. At the end, all the votes are getting shrunk toward something like this.

If we want to get a little technical, each story is like a random variable, and votes on that story are observations on that variable. We've got 93 random variables in the "All In" minific competition. Each of these stories will have its own mean and distribution of scores, so the CLT idea breaks down slightly since the stories aren't all identical, but the idea still bears out. When we look at the competition as a whole, and the story rankings, what we're looking at is a distribution of the average votes for each story. If the stories were identical and voting were random, this would be where the CLT really kicks in, and we'd have something called a sampling distribution for the mean vote on stories. None of this matters a whole lot, but for people who like thinking about statistics with some lingo, this is basically the problem we're talking about.

Okay, so what are the key points here?

(1) No matter how individual readers vote, the overall scores are going to wind up approximately normally distributed—as long as there are a sufficient number of voters, and voters aren't forming cabals to game the system. The first isn't really under our control. The second isn't something anyone has discussed, and I feel comfortable saying that, knowing a lot of the people involved in these discussions so far, I don't think it's something anyone here would do. That's not to say anything about individual voters "gaming the system", and I'll have more to say on that later. But the real problem would be if we got voting blocs doing it, because that effectively reduces the size of our voter pool and screws with the CLT. If we had 100 voters, almost none of this would matter, but with more like 20-30 voters, a bloc of 3-5 voters intentionally doing the same thing could be a problem.

(2) The more voters we have, the narrower the distribution of final votes is going to get—assuming that most stories are roughly interchangeable. Again, this bit is based on an assumption that all the stories are functionally equivalent, which doesn't pan out, so we're really expecting a high-kurtosis distribution: something very peaked in the middle, with fatter tails running off toward 0 and 10.

Does this all say anything about how people should vote? I think yes, sort of. And I think, of all people, Titanium Dragon has this closest to optimal.

So let's begin with the following assumption: we all want to pick the best story in the write-off as the winner. We can think of user votes as having two parts. First, there's shared information being used by all the voters—grammar, spelling, some baseline level of quality stories may or may not have, etc. Basically, the shared information reflects what everyone is picking up on when they read the story. Second, there's information unique to the individual voter—Bad Horse's perception of story depth, or bookplayer's analysis of characterization, or PresentPerfect's assessment of overall story quality, or anyone's particular liking/disliking of any given story. We can think of user-specific information as being very similar to shared information but often more refined. Readers will probably have a shared general sense of whether or not a story "worked", but for each individual user, whether or not they liked the story is a deeper level of information on the same general subject. Writers who are really good with particular tools and styles will probably be better able to analyze how well a story uses those tools.

If we're interested in picking the best stories, then we should recognize that every voter will be using this shared information in building their votes, and so the shared information is going to tend to count for a lot in the overall results. Each individual voter contributes her or his own unique information through her or his own votes, but a lot of what's going into the results is repeated application of this shared information. Now, to some extent that's probably reasonable. Stories that have obviously perceptible flaws everyone is seeing should probably get rated lower; stories that do a really good job and impress most readers should get rated higher. But when we combine up the information contained in votes, if we assume half of a vote is shared information and half is voter-unique information, we're basically drowning out voter-unique information with the shared information.

Finding a balance here isn't super easy, but I would strongly suggest that voters try to focus on their unique information when deciding on their votes. If you feel like you've got a better handle on characterization than other writing skills, it makes sense to heavily weight your votes based on how you feel about the characterization in each story. You're never going to escape the shared information, but if you focus on your own unique information, your votes provide more meaning to the overall process. And, I think, this helps us pick better stories overall (assuming that voters' unique information is in fact useful, but I'm pretty content to buy into the assumption that most people here are voting with the intention of trying to pick the best stories).

So when Titanium Dragon gives a lot of 0's and 10's, or when bookplayer says she may look at whether she thinks a story is being received better than she feels it deserves, I see these both as applications of voter-unique information. The consensus is powerfully driving the end results, probably more than any of us would really like it to be, because focusing on shared information is going to wash out important unique information, which is an issue Pascoite obliquely touched on in the last write-off. I'm not saying anything about how ethical it is to vote in these ways. I'll do that in a bit. But in terms of maximizing our ability to detect the best stories in the competition, I actually think what TD and bookplayer have discussed doing (as long as it's not mean-spirited or designed to cheat, which I don't believe of either of them) is really one of the better approaches that can be taken overall, in a statistical sense.

If you want to know how nerdy I am, the foregoing discussion owes a lot to a conversation my dad and I had while watching Fritz Lang's "The Ministry of Fear". It's an excellent movie that you should watch, and it involves a cake-weight-guessing game at a British country fair, which is a bit more natural fit for these ideas, but I think they mostly extend here as well.

3956405
Huh. I philosophically deeply dislike your 'genius but boring', even if I understand the point and practically speaking do not see it a big deal that you choose to vote that way here.

Basically, I find it problematic because much of the media that I have found most worthwhile and non-boring (expanding this outside of just fanfiction) has been things that I know for a fact other people at times criticize as pretentious or boring or trying-too-hard. And that's usually because we expected different things from a given work, and it's fine that they didn't enjoy it – but it's really unsatisfying for them to make sweeping statements about a thing's worth (or worse, its 'authenticity', which you verge on doing here with 'writers' v. 'Writers').

Often, the kinds of work that fall into this category are indeed flawed, but ambitious in a non-obvious way, whether thematically or structurally or technically. And while if you didn't enjoy something, you didn't enjoy something, and that's a completely legitimate thing to express, I find it baffling that you grade something that other people find a lot of worth in much harsher that something that's just all-around mediocre. To extend this in two ways: First, I think if everyone approached criticism this way, it'd be very punishing to any works that aim at being deep, because there will always be someone on a level to think it's 'genius-but-boring' (see: Bad Horse's levels of deep illustration, which I like). And second, from my own experience, there have been works that at one point in my life I viewed basically in that way, that upon reconsideration later on, I found I really really enjoyed.

Ultimately, this is more arguing general perspectives than anything else though. I don't actually think you voting in this way is particularly problematic in the context of the writeoffs. (In part because I know from your blogs & stories that you in particular generally consider things on a nicely deep and nuanced basis) But I don't like the concept, not one bit.

I really don't have a problem with the many different scoring methods. As long as a voter applies the same system to all stories, then there is no distortion. Someone who trends high or low is still going to apply that trend to all stories. The changing tide shifts all boats up or down. Problems crop up when someone is internally inconsistent.

Besides, establishing quotas or other hard-and-fast requirements on how to vote would kill a good chunk of the fun. I'd rather read and evaluate each story in turn than worry about the macro-scale distribution of my ratings.

3956408
You may want to rephrase "objective opinions." An opinion is, by definition, subjective.

I've said my "rant" in the main thread already, but here's my voting process, for those curious:

The story I liked the most gets a 10. The story I liked almost as much as the first one gets a 9. Other stories I liked start with a base score of 6, and if something makes a story stand out, it can become a 7 or an 8. Stories I don't like start with a base score of 4 and have gone as high as 6 and as low as 2.

Personally, I don't like the whole voting system at all. Ideally, we'd do away with it entirely, but that's not likely to happen. Do I think it's wrong to skew scores to prevent/ensure victory for a story? Yes, but that can't be regulated well, so I accept that it exists.

In place of ranking, awards like "best use of prompt," "best characterization," or "best story idea" could be implemented. We'd still have to vote on those, of course, but it does give better feedback than an arbitrary numbers scale and shows authors how they stack up in that sense. This helps them understand what they need to work on in their story/stories and is more appropriate to a workshop like this.

3956440
Not at all.

I have two circles of friends. One I refer to as writers, because they love to write. The other group I refer to as Writers because they also love to write, but take it far more seriously than the first group. I don't mind this usually, I also take writing seriously which is why I hang out with them. I like to analyze what works and what doesn't in writing as much as the next girl. But I still feel that the primary goal of almost all fiction is to entertain[1], somehow.

Now, occasionally a story will come along that the Writers really love, because it has tons of stuff to analyze. And that's cool, because that's what entertains them! But the problem is that often, when they find a story like that, they'll make a value judgement about that story being somehow intrinsically better than other stories, regardless of whether the story actually entertains anyone else.[2] Often stories like this turn Show, Don't Tell into a treasure hunt for the plot or theme, forget that well developed characters in difficult moral conflicts can still find joy in other parts of their lives, and generally take the view that if you can tell exactly what a character is feeling without reading a passage twice they're being too obvious. Once again, this is fun for some people! But the fact that it's fun for people who take writing seriously does not mean that it's a better story than one that's fun for people who don't take writing seriously.

My personal problem is that making those value judgements give the analytical side of writing a bad reputation among lower-case-writers who actually do enjoy meatier stories. They feel like the whole world of trying to write deeper stories is closed off to them because they also (or primarily) care about writing something that's just fun to read. I don't think that's what the upper-case-Writers intend, but I see it all the time. And I can't blame them, because they're being shown "yes, an entertaining narrative is nice, but it's not nearly as important as this stuff."

Now, personally, I sometimes can't tell when a story is just boring, or when a story is boring because the author was trying to hide everything interesting for Writers to unpack. That's what I mean by not being able to tell if a story is "good" or not. So my base assumption is that it's just boring. But if I see a bunch of Writers unpacking all of this stuff, I can see that it's boring, well, almost on purpose, and that it's one of those value judgement stories where it's better than a story that's "just" an entertaining narritive, because it's entertaining to Writers. This doesn't happen a lot, but when it does I feel the need to strongly lodge my disapproval of this way of writing stories.

Obvious disclaimer: Of course this doesn't apply to stories that are entertaining narratives and have a lot to analyze. That is a better story, because it's fun for a wide variety of people. That's what we should be aiming for.

[1] Please note that when I talk about stories being "entertaining" I don't mean happy, sweet, funny, or nice. Some people are entertained by horror, tragedy, watching two intelligent characters have a conversation, watching characters fight, analyzing pieces of the story, or a million other things. However when I talk about a narrative being entertaining, I generally mean "stuff actually happens."

[2] But don't the lower-case-writers also make value judgements like that by finding the stories that Writers like boring? Sometimes, and I shout them down just as loud. But mostly, no, they don't. Most lower-case writers I know will happily recognize that a story with stuff to analyze is better than a piece of fluff, they just still expect it to be entertaining as a narrative. The less serious writers I know are actually better at giving weight to both sides of a story when judging its worth.

3956453

And that's usually because we expected different things from a given work, and it's fine that they didn't enjoy it – but it's really unsatisfying for them to make sweeping statements about a thing's worth (or worse, its 'authenticity', which you verge on doing here with 'writers' v. 'Writers').

I hope the above provided some clarification, and perhaps some insight into why people perceive things that way.

Okay, on to the next part.

2. The Ethical Side

If we want to talk about what is and isn't proper behavior, I think we need to think about what we want from the write-off.

For me—and I think for horizon, and for a lot of other writers—the feedback cycle is really the most important thing for us. Speaking for myself, I'm interested in the craft of writing, I want to get better, and I'm pretty sure that having my name attached to a story is going to skew people's reactions to that story, because at this point a lot of readers just assume I'm a "good writer" and that my stories will necessarily be "good stories". When most of your readers give you the benefit of the doubt, you can miss out on a lot of insightful responses, either through commenter self-censoring, or through readers thinking as they read, "Oh, I'm sure this is what he meant, because he's too good a writer to have done something as stupid as I was just thinking he did." Reviews of anonymous stories are the big reason I joined the write-off, and voting matters a lot less to me.

Also, I am aware (or deluded enough to believe) that I really am a pretty good writer, and that if I keep participating in these things, it's probably just a matter of time until I medal. I bring this up because I know this is skewing my own perspective on the write-off. I don't care as much about voting because I feel like I don't need to care much about voting. Getting a high-rated story is nice, but it doesn't mean as much to me as it does to other people, because I've already gotten a lot of fandom approval for my writing. I'm not trying to toot my own horn here, I'm just trying to establish that I'm probably going to fail to understand some people's perspective on these issues. I'm aware of this, and I'd ask that y'all help me understand. I'm also pretty confident, given some of the people participating, that I'm not going to be alone in this boat.

Okay, so, for me voting doesn't matter as much. It matters a lot to a lot of people. And in my mind, voting should have two purposes here: to provide writers with another layer of feedback about their stories, and to choose the stories that we collectively think are the best in the write-off. That's my natural stance coming into this, and it's going to shape my opinions about behavior. Now I'd like to talk about some of the behaviors that have been discussed.

Using lots of 0's and 10's in voting:
3956408 mentions standardizing the values of votes, because someone voting like Titanium Dragon with a lot of extreme votes and fewer in the middle is having a bigger impact on the results than a user who doesn't vote in this way. Personally, I don't like the idea of standardizing the values of votes, though my reason for it may come off as kind of elitist. Please hear me out; hopefully the elitism side will get mitigated toward the end by the practical side. So, elitism—frankly, I think that some users' votes are worth more than others. I trust horizon's and Bad Horse's opinions about stories more than I trust the opinions of a first-time participant I've never met before. That's not egalitarian, no, but I do think it's pretty fair. horizon and Bad Horse have a long track record of saying very insightful things and thinking hard about story quality. (PresentPerfect does too, but he hates all my stories, so I'm going to ignore the fact that he exists here. Hi, PP!) So standardization doesn't seem intuitive to me because I genuinely feel like one person, one vote isn't actually the fairest or best way of judging stories.

Do I want to impose a scheme where votes from horizon and Bad Horse actually count more than other people's votes? God no. That'd be a nightmare—just as standardization itself would be a nightmare. We'd have to do something like restricting how many of each vote a user had available, or telling everyone they had to vote using a particular distribution, or do some sort of score-recalibration which would get extremely dodgy. No, I personally think the most reasonable solution is just for individual voters to be aware of the issue here—that more extreme votes have a marginally higher weight overall—and to ask that all voters vote based on how strongly they feel about stories. If someone feels like a story might be below-par but they don't feel like they really know storycraft well enough to pick apart the problems, it makes sense to give a somewhat-low score instead of an extremely low score. On the other hand, if someone who knows storycraft well thinks a story is just doing a terrible job, a 0 makes a lot of sense. And even for someone who doesn't feel up to serious story analysis, if a story just falls dead for them, it makes sense to vote with the 0. I think voters should be aware that more extreme votes can matter more, and should vote based on their own opinions about stories. You're always going to have people like Titanium Dragon who are just very strident about what they think works or doesn't work, but I don't think there's a good way to avoid that, and I don't think anyone else has the right to begrudge his perspective that he's really making an important contribution by 0-voting stories he think deserve it.

So my attitude is, people should try to be informed about how voting plays into the results, but we should let everyone vote as they like, as long as they're not genuinely trying to screw things up.

Also, back to my point about trusting some people's views more than others, this is already kind of built into the system in the story reviews a lot of writers provide (and which I could totally be doing right now, instead of discussing this, but I've got screwed up priorities). Reviewers can color other readers' opinions of stories, and the power of a reviewer to do so is going to be directly proportional to how much other readers value that reviewer's viewpoint. But this brings us directly into the next point I wanted to discuss.

Changing one's votes based on what other readers have said:
Honestly, I'd do this in a heartbeat. I'd say, "I do this all the time," but this is only my second write-off and I don't remember having encountered this situation yet. But if I read a reviewer discussing a story and they've noticed things about it that totally went over my head, and if I think those things are important and I was dumb for missing them, then of course I'm going to let that impact my vote. I want to identify the best stories, and I'm not always going to be the best judge. This goes back to my stats discussion a bit, too. Reviews are one form of some voters' unique information. I think unique information is likely to be structurally downweighted too much. I'm perfectly content to use other voters' unique information, if I think it looks like good information that bears on how I'd want to evaluate stories myself.

But I'm arguing easy mode here. Some people have suggested using other people's reviews in deciding your own votes is blanket-bad, and I feel like that's clearly not the case. But a lot of people seem troubled by bookplayer's statements that she'd down-voting a story she saw getting a very positive reception, if she thought that story was being over-valued. So that deserves some directed discussion.

On the one hand, I think bookplayer's attitude fits pretty well with what I said in my stats post about maximizing unique information. If a lot of public commentary on a story is good, then that commentary is probably slipping into the space of shared information, and unique information against that story (including something as simple as a voter's perception that a story is just pretentious and not very good) is going to get increasingly drowned out by the consensus positive. Down-voting is a natural balance to that.

I think it still feels wrong to a lot of people, and one of the reasons (which I think bookplayer has tried to clear up, to middling success) is that she wasn't initially as clear as she could have been that she was talking about voting more negatively on stories she already actively disliked, as opposed to trying to game the system so some good stories couldn't win. Even if a story is good in many readers' minds, it may not be good in everyone's mind, and it makes sense to give low votes to stories you don't think are good.

Tangential Rant
Personally, I'm willing to go farther than that, because I tend to agree with 3956405 when she talks about genius but boring stories. I think there are people who learn literary writing skills as a way to escape criticism. I'm not making that judgment here, but I definitely feel that way about the literary fiction community as a whole. There's some really wonderful stuff out there, I've grown to realize. But there's also a lot of ugly, repulsive nonsense that I'd rather we all forget ever happened, including a few major literary figures of the 20th century. It's a problem Bad Horse has blogged about, too—though in slightly different terms. And it's a view I've taken a bit of oblique flack for, from time to time, I think. But my attitude is, if you can't make your writing interesting, you're not doing a good job as a writer. Unless, like, your objective is to bore your reader—but except in some very rare circumstances, that doesn't seem like an objective I can stand behind.

This is why people like my old creative writing teacher bug the hell out of me, with his insistence that no good story can involve conflicts beyond what the reader could expect to experience in their own life; or the way some authors view plot as a distraction that real writing should avoid so it can better focus on characterization. To me, people who hold attitudes like this are a huge problem for writing, moreso when they get into positions of some critical authority, as seems to have happened in the world of American literary fiction. I firmly believe that if you want to write a character-centric story, and you can't figure out how to structure a narrative to highlight the themes you want to bring out with your character work, you're just not a very good writer. I'm not saying I'd have an exceptionally easy time doing that myself, mind—but I'd try. And that's what bothers me about some of these people—the casual dismissal of plot because some authors are unwilling to actually think and work to tell better stories. Did Victor Hugo strip the plot out of Les Miserables so he could dig into the character of Jean Valjean better? No, I think we can all agree that's not what happened.

So when I see a story that's trying for deep ideas and themes but doesn't bother with being a story, or making itself interesting, I react about like bookplayer. That's a type of writing I just don't support, however well executed it may be. I don't think anybody in a fanfiction community is likely to be prone to too much of this, but I do think that there can be a certain degree of imitation of "high art" style writing that some writers occasionally shoot for. And I have no problem with that, as long as it's grounded in a story that's in some sense enjoyable to read. But trying for deep themes and meanings without building a good story is like trying to learn cubism without knowing how to draw an accurate scene. If you don't understand how to apply the basics, or if you're not careful about using them, it's going to undermine any higher-level work you attempt.

To sum up, as long as the voter is interested in trying to find the best story and not in gaming the system, I'm totally cool with the idea of responding to other voters' impressions. I think it might even be advisable, on the back of the unique information argument. As with all things, though, I think it needs to be guided by a strong sense of what it is you're trying to do. If you're trying to pick the best story, great. If you're just trying to undermine a story because you don't want to see it do well, that's a very different situation. I think we're all on board with the latter being unethical. The former, though, I think is fine. Yes, I'm big on the intentions behind actions. I'm Confucian at heart—I think the best way to encourage good behavior is by modeling it and discussing it.

Voting for or against a particular type of story:
I've already kicked out a mountain of text here, so I'll keep this short. Bad Horse has said he does this with comedies. He's also been very explicit about the fact that he's essentially doing it as a shared-information reaction, the same as bookplayer's comments about down-voting "boring genius" stories that draw a lot of praise. My feeling tends to be the same. As long as voters are acting in an internally consistent method that highlights the unique information they bring to the voting process, and as long as they act from a sincere motivation to help find the best stories in the write-off, I think the actions discussed are entirely reasonable, and I'd be fine with anyone engaging in those actions.

Haven't been following the All In thread, but I've got enough info to catch up:

93 stories

*pulls up his pants to hide plumber crack, spits tobacco* Well see that there's where yer problem is, y'got all these here fics clogging yer pipes and that's like 2 or 3 more fics than there should be in there. I can sell ya a new filter on the spot, only $500, real good bargain. *cough* No but really, my butt clenched when I heard there were so many fics in this round, even if the wordcount wasn't super super high. Already I saw people in the threads going into triage mode: I can't review all these fics in full, so here's a succinct review. That's a crying shame in my mind, since getting all of that peer contact is such a valuable part of these contests. I sure as hell steered clear of it.

Roger's website already supports the ability to do a preliminary voting round. That is, everyone gets a few fics to review and score, and then the highest rated fics from that round move on to the final round. This means that we'll quickly cut down on some of the lower-rated feghoots, and everyone will only have to look at the higher rated stuff, yet even the lower rated fics will get reader feedback because people will leave feedback in the prelim round. (IIRC feedback in the prelim round was mandatory for this reason, but feedback in the final round was optional as usual.)

------

On another point... like, we're discussing rigid voting policies and statistical analyses and all sorts of stuff like that. Mmmaybe we're getting a bit too serious about internet points? I get it, I get it, the scoreboard exists for a reason and people want the voting to mean something. Plus, I'm talking here with a few gold medals on my shelf, so I definitely know how rewarding that group validation can feel. All the same, I'd be worried about introducing too much structure and rules, and choking the fun out of the group. I've definitely run into that case before, where you take a lax group and add more rules, more rules, more rules on top of it, until everyone's being strangled by the red tape, and hating the group they used to enjoy.

Specific to the point of rules enforced on voting... The thing is, everyone has their own opinions on a fic. Certain things matter more or less to an individual. Someone's going to be very nitpicky on grammar since it shatters their immersion. Someone will care less about execution and more about the heart of the matter. Someone will feel that execution is everything. And the thing is, this is accurate. This is how readers work. Your readers here on Fimfic aren't a uniform mass, and I feel like there's value in having the voting (to some extent) and the comments (to a large extent) reflect this reality. Thus, I get iffy about telling people to force their thinking and feeling into some sort of rigid rubric.

3956441

[Voters forming cabals to game the system] isn't something anyone has discussed, and I feel comfortable saying that, knowing a lot of the people involved in these discussions so far, I don't think it's something anyone here would do.

On top of this not really being a large concern given the participants in this group, I furthermore don't see how a grading rubric would prevent this, anyway. Furthermore, this is a large part of why the submissions are anonymous... unless the accusation is simply that people would cabal against the strong contenders, rather than fics written by Author X. Even so... voters gonna vote. As I said before, every reader is looking for something different in your story, so it would be very difficult to prevent—or even detect—foul play. Though that said, Roger is always able to look at the voting records and could theoretically look for foul play, if that was a huge concern.

3956558
Heh. The funny thing is that practically speaking, I think I totally agree with you. And the kind of story that frustrates you is exactly one that would frustrate me, and which I would give a fairly low score to, and not feel bad about. (My biggest complaint with so many stories was 'what's the point?')

There's just something about framing it so clearly in the terms you provide that really unsettles me. Thinking about it, I guess it's because I could see that attitude being distorted into relying solely on your first impressions and writing off fics that might -actually- be genius if you gave them another try and dug into them slightly more. Particularly if the other person who's expressing why it's genius has a point that you hadn't considered and allows you to recontextualize everything. (And yes, theoretically you should be able to get this stuff on the surface, not dig for it, but that discounts the fact that sometimes you just flat out miss something that is there for whatever reason. I know I do.)

I guess it doesn't bug me within this specific context because I trust you overall, and having a bookplayer acting in this manner is fine, as sort of a check against form overreaching narrative function. But having everyone take it upon themselves to do the same? It's a fairly scary thing to have everyone trying to deliberately police the 'artsy' stories and being deliberately harsh if they don't fit their personal standards of worth. Usually you're already taking enough of a risk with writing that sort of thing - I don't know that it needs additional discouragement.

Another thing to consider is whether these are treated as a writing contest or a story contest, at least as far as the voting is concerned. I do the former, but I suspect most voters do the latter. If I didn't like a story, but I can tell that my reaction to it is personal taste more than anything else, and the quality is otherwise good, I'll give it a high score. Many voters, if they don't like a story, leave it at that and give it a low score. Others at least give some credit for the writing quality and vote it middling at best. I mean, there are basically comments being made that someone is giving a low score because they wanted a happy ending or because it's a shipping pair they don't like. In a story contest, I can't exactly fault them for doing so. In a writing contest, I can.

3956679
One thing I think you have to remember is the group we're working with here. I think the attitudes you're talking about are more prevalent among readers, absolutely. But I also think that most, if not all, of the voters in this contest are writers (or Writers) or people with a deeper interest in the craft of writing. So it doesn't concern me as much if other people vote like me, because I think most of us here are self-selected to be looking for a certain amount of art, or trying to learn to look for that art, and are happy to find it. We're in more danger of falling into the form over function trap, from a lot of what I've witnessed over the past few months.

On the other hand, sometimes people surprise you with multiple stories about poop, so who knows.

3956689
This is an interesting distinction, though I'm not really sure I know where I stand on it. Probably somewhere in the middle? Particularly, I'm thinking about A Nugget of Wisdom here, which I found surprisingly well written, despite the fact that I had pretty much zero interest in it. I don't think I could ever bring myself to 10 a story like that, no matter how good the writing, even if it explored deep human themes of loss and social insecurity. But I still gave it a fairly good score on the back of some nice metaphor work, IIRC, because I could appreciate that absent context.

On the other hand, I definitely dislike the idea of people voting stories down because they have a shipping pair the voter may not like, regardless of how it's executed in text, or because they didn't like the ending. Then again, it's easy for those to be writing critiques too—if a writer ignored obvious bits of characterization that'd play into the shipping pair, or if he/she laid the groundwork for one ending without noticing it and just whiffed instead of actively trying to subvert, for example.

So not only am I not entirely sure which I do, I'm not entirely sure what I think people should be doing either.

3956408
So to address what I have said earlier, here are a few examples of systems that could fit the requirements of an evenly distributed and easily maintained voting system.

PLEASE AT LEAST READ MY FINAL NOTE AT THE END!

The gallery is randomly organized. (Solves list order bias. Should be combined with the voting methods.)
This means that each user that goes to read is presented with a different order of the fics in the gallery than another user (and perhaps each visit to the page is randomized). This will hopefully allow each fic it's moment in the spotlight as the top of the list, or the bottom, or the middle, depending on how the voters end up reading. This also helps to evenly distribute the votes across all entries as I am pretty sure that very few people read all of the entries before voting on them. If there were any uneven distribution of votes before due to this issue, then this would fix that. CONS:A little programming required... not much else.

Votes are specifying the order of most liked to least liked.[impractical] (Forces the same standard of vote interpretation)
Users could be forced to order the stories they read from 1 (the best) to the number of fics they read (the worst), and all unread fics marked as N/A. The fics that they voted on will then be given a score from 0-10 relative to their votes, for example the #1 fic would get a 10, and the last numbered fic gets a 0. CONS:In order for this to work with decent distribution and vote weight, there would need to be a minimum number of fics read and voted on, something that cannot be expected of the voters. It could also cause voters that say loved two fics, have to say that one is lesser than the other, the degree of this difference being determine by the number of fics they voted on, hence incorrect vote weights.

The 0-10 numbered votes are given specific interpretations.[Unevenly weighted] (Encourages similar interpretation of votes)
Like what 3956405 said (without the genius but boring specification. Users should get to determine what that is worth.). Simply specify using phrases like "I Loved it", "I Hated it", "I'm indifferent about it". CONS:Allowing this kind of voting (0-10 voting) allows for biases in vote weight. An example that 3956604 gave (that he believes is good for reasons I only partially disagree with, something I'll address later in this post), is that a voter who wants his/her votes to count more could simply vote using only 10s and 0s, which would severely influence the weight of his/her votes, whereas the voter who evenly distributes his votes honestly would have less impact on raising or lowering a score.

Categorical voting. [Extremely accurate voting] (forced voting standard)
Voting by categories such as "Original", "Grabs My Interest/Gets Me Invested In It", "Grammar/Spelling/Etc.", and others (excluding "Overall Rating"). This would allow for super accurate interpretations of the value of the story. CONS:It is very difficult to use as a decider as to which fic is the winner because each of those categories has different weights to different readers. For example, in my opinion, the "Gets me Invested" would weight the most while "Grammer/Spelling/Etc." weighs the least. Someone else might feel differently, and therefore the voting standardization still lacks complete standardization. This system also allows for voter abuse just as the 0-10 voting systems do.

We use a voting system of thumbs up, thumbs down, indifferent/undecided, or N/A. (Forces similar vote interpretation and heavily moderates the vote weight problem)
This is simple. You can either say "I like it", or "I don't like it", similar to what we already have here on Fimfiction. The final score would be calculated as (upvotes)/(upvotes+downvotes), N/A votes being excluded from the formula. And we could specify to the voter beforehand that N/A votes are also used as "I don't feel strongly either way". This essentially gives us a percentage of the upvotes out of the total "valuable" votes. Cons(but not really): It only solves 99.9% of the vote weight issue because of the allowance of an N/A-Indifferent-Undecided option, but the pros of the N/A-Indifferent-Undecided option is that it allows for more accurate feelings to be expressed by voters, and allows a sort of "cancel vote" option if they vote on a story by accident or later decide they don't want to affect the score of the vote.

These are just a few ideas on the different ways that the voting system could be set up. My personal preference is the upvote/downvote/(N/A-indifferent-Undecided) system and the use of a randomly ordered reading gallery.

3956604
Now to address your views on the vote weight. While yes, in reality, some people's votes might be considered by some to be worth more than others, and the scale allows voters to truly express the degree of their like or dislike (or what have you) of a story, sadly, it allows for serious voter abuse, especially by those who think that their vote is worth more than others, whether or not it actually is. While I love Titanium Dragon and his reviews, I do not believe that his votes should be weighed any more than mine simply because he feels they should, or because he has much stronger opinions about things than I do (I don't believe he votes that way, this is just an example).

Everyone's votes should count the same because we can't allow the voters to be the judge of how much their vote is worth, because that clearly allows for bias. And yes, I understand that opinion based voting is already bias in and of itself, but the bias I'm speaking of is the bias of vote weight, something that should be the same for everyone for the reason that I stated above: We don't know what our votes are really worth no matter how strongly we feel about them, therefore to avoid allowing bad judgement, everyone gets the same vote worth.

Now while I would love for the 0-10 voting system to be used so that I could accurately express my feelings, it has the ability to be used fairly and unfairly. And so to prevent the possibility of unfairness, more robust systems (like upvote/downvote/(N/A)) would be required, which still allows voters to somewhat accurately express their feelings on certain types of stories, like "genius but boring" stories.

And now a...
FINAL NOTE:
I don't think any change is required. I think the voting system is fine the way it is, and I think that most if not all voters use it properly, but if you want to be very exact about making sure it's fair and balanced no matter what, then I have written the above for that reason.

You can't enforce any sort of "perfect" voting system. You really can't. People will vote the way they want to vote. Weighting things this way or that just creates a new system that people will vote the way they want to vote within.

I appreciate that a lot of this discussion is just kind of for the sake of thinking about the subject out loud, but it's very much a moot point. There is no way to "fix" things so that one particular person's way of voting becomes the way that everyone votes.

People will vote the way they want. People will write the way they want. People will participate the way they want. (Or in my case, not at all the way they want, just the way they have time for. I'm glad I slightly randomized my reviewing, because I don't think I"ll be able to add any more reviews to the pile before this is over.)

Anyhow, have fun talking about it. But please, don't get too invested in making everything work perfectly. We live on Earth, not Equestria.

3956807

Re: Most Liked to Least Liked.
If you use the full scale of the vote (from 0 to 10), then you're already in effect doing this. Like I've said before, the writeoff vote score is only a relative score in relation to the other stories. It's not feasible to compare scores between writeoffs or assess scores on their own. The numbers are already fundamentally linked to the story's general standing in relationship to the other stories in each event.

Re: Specifically scaled votes
This also makes it even easier for someone to game the system by throwing in unwarranted 0's. In effect, it enables exactly the kind of behavior we're trying to prevent.

Re: Categoric votes
No. It should be the voter's own discretion how much each category comes into play, or if any unlisted categories significantly impact the story as a whole. This could even be different when the same reviewer considers different stories. If this system is enacted, the easiest way to win is to write a story that intentionally avoids objectionable fault in each listed category. This stifles creativity.

Re: The upvote/downvote idea.
This looks great on paper, but it's unfeasible for a very important reason. Writeoffs tend to get only 10-25 votes per competition, which means that even if every voter never lists a single "N/A", the stories can only be divided in increments of 4-10% (based on the number of votes above). This is simply not differentiating enough. When you consider a writeoff with the number of stories that this month's brought in, that means that unavoidably, each story on average will share an identical standing with 3-10 other stories. This does not accomplish the job of differentiating which stories were better received.

3956807
I feel like the Up/Down/Neither system only takes away the granularity of the current system. In a way it exacerbates problems: If I have a fic that's ~7 scored, is that thumbs up or neither? ...What about considering that it's very clear that 'neither' indirectly helps my own fic in comparison? I don't think people are that tempted to bump things because it helps their own fic, but when the edge cases suddenly matter a lot more, it's only encouraging that. And your handling of N/A there is bad. If one fic has a super catchy title or comes up in the reviews a lot and thus is read by 100 people, half of whom upvote, is that truly better than a fic that is only read by 70 people, but 50 of whom upvote?

I would also suspect that it means inoffensive/non-divisive/comedy fics would do drastically better, because they'd be more likely to earn 'weak' up-thumbs and no down-thumbs where a really great work might get 'strong' up-thumbs and a few down-thumbs. (And/or encourage 'overrated' downthumbs in response to that phenomena). These are problems that exist in the current system, but I think you're making them worse, not better.

And I really feel bad for the guy who gets nothing but downthumbs for his flawed first attempt at the contest.

3956807
I'm not going to critique the specific ideas, since you said you're perfectly content with the way things are. I will say, I think your cons are ignoring some important issues.

Generally, I most agree with you when you say:

I understand that opinion based voting is already bias in and of itself

because to me, that's the beginning and the end of it. Constructing a system that can't be abused is prohibitively difficult, and the option you seemed to think most above-board is also the option that results in the least interesting feedback, i.e. the Fimfiction system. Personally, I think it's a lot more reasonable—especially in a group like this—to choose a good system that allows people to express their views with a degree of flexibility, to monitor that system to make sure it's not being abused, and to discuss that system periodically (as we're doing) to consider whether there are problem points or ways to refine it that we would collectively like to see.

For my money (which is $0, because thank you Roger), I'm very content with how the voting works right now. I suppose we could all try to talk Titanium Dragon into being less of an asshole using the middle scores more, but that's about it. And I'm personally perfectly comfortable allowing some people to vote in a way that causes their votes to have a bigger impact. I don't do it—I shoot for something resembling a uniform distribution or a very slight bell curve—but I don't personally feel like it's anyone's place to tell other people how to vote, or to try to control those votes. Not in this sort of community (the write-off), where nuanced feedback is really at a premium. A more restricted system makes sense for Fimfiction because monitoring for abuse becomes more difficult the more complex your vote system, and "I liked it" and "I didn't like it" are the only feedback many users are prepared to provide.

I made a comment earlier about how I think it's important that everyone understands how the voting system comes together, in statistical terms, and the effect of heavy 0/10 voting. Mostly, I think that's important so users can calibrate their own votes, because while Titanium Dragon seems to be a bit of an outlier in terms of how he votes, I actually think a better response would be for more people to shift their votes toward looking like his, than to worry about him biasing the results himself. The CLT is going to pull everything together in the end, anyway, and having a bigger spread in scores allows for a finer ability to detect differences between stories. Voters who are using something like a bell curve in their own votes are basically voting N/A to almost every story. If that's what they want to be doing, that's fine. But if they're wanting to actually register votes, then they should probably be pushing more toward the 0's and the 10's.

I don't think I'd encourage anyone to go past what TD is doing, and I absolutely don't encourage pure 0/10 voting, because I think there is a lot of nuance to story quality that people experience, and that we should try to capture in our votes. But, at least for myself, I'm very comfortable with people having their own unique voting standards, and with people pushing away from middle values to try to improve the rankings' ability to discriminate higher quality stories from lower quality ones. I personally think TD's vote distributions look pretty sound.

Though, of course, this is all just my opinion.

There's this funny little parable which I think is relevant to the discussion here, and it goes like this:

“Holmes and Watson are on a camping trip. In the middle of the night Holmes wakes up and gives Dr. Watson a nudge. "Watson" he says, "look up in the sky and tell me what you see."

"I see millions of stars, Holmes," says Watson.

"And what do you conclude from that, Watson?"

Watson thinks for a moment. "Well," he says, "astronomically, it tells me that there are millions of galaxies and potentially billions of planets. Astrologically, I observe that Saturn is in Leo. Horologically, I deduce that the time is approximately a quarter past three. Meterologically, I suspect that we will have a beautiful day tomorrow. Theologically, I see that God is all-powerful, and we are small and insignficant. Uh, what does it tell you, Holmes?"

"Watson, you idiot! Someone has stolen our tent!”

Guys, I love all of you, seriously. New and old. Been here two years. I've said that a lot recently, but it's to hopefully get you to realize I've stuck around for a reason.

But I feel I ought to say: be careful here. Be careful when you bring mathematics and science into the realm of art. Like, seriously. I say this as a physicist. Are we in a laboratory here? I don't think so. But if you personally do, then please, at least follow proper laboratory etiquette and recognize what assumptions many of you are making (assumptions like: all stories ought to have differing final scores). Science and its methods for discovering the truth go wrong when you fail to do this.

I'm not saying this discussion can't be fulfilling, fruitful, or that people ought not to care about it. But I see what I consider to be an awful lot of "missing the point" in all of this, and have for a long, long while. I know everyone is going to ignore me on this, and as I said in the story thread, this discussion is never going to go away. And that's okay.

Just, don't get too hung up on it, okay? This is from one brother to all his older and younger siblings out here.

Well, off to review more stories!

3956604

But when we combine up the information contained in votes, if we assume half of a vote is shared information and half is voter-unique information, we're basically drowning out voter-unique information with the shared information.

Changing one's votes based on what other readers have said

Sorry, forgot to comment on this part but it's important too. There's been plenty of contests on here where there's been reader disagreement and discussion on a fic. And that's good! And healthy, and super valuable to the author. It could potentially sway the vote, yes, but people are expected to at least primarily follow their own heart. I mean, if I didn't read any of the fics, only read Present Perfect's reviews, and voted exactly like his reviews told me to... what even? What's the point of that? I wouldn't cast my own vote blindly on someone else's opinion instead of my own. No, for the most part, reading peer reviews is likely to only sway you plus-or-minus one point.

Unless, of course, you missed an important element in the story. Some people assume Celestia was being cold and heartless in this one scene, but other readers remember that there's a callback reference to an earlier scene, so the correct interpretation is actually blah. And the first groups reads the reviews of the second group, slaps their collective foreheads, understands the fic, and increases the score by several points. I've definitely done this on several occasions, but I feel like I should not be. I missed a fact. Missing this element severely hindered my enjoyment of the fic, and my score should reflect the actual text. If someone punches me in the gut, but tells me that it was supposed to be a gentle massage, I don't tell them that the massage felt wonderful. It's essentially an execution failure. Even if I rationalize "yes but I'm stupid", it's still the author's attention to smack me upside the head and make me think about the correct solution. I don't necessarily mean "spoonfeed me", but I at least mean "there should be enough foreshadowing to clue me in that 'something is here, stop and think.'"

Back in the ponychan days, where you can post as Anonymous or as Author Of Bitter Batter Bitter, we used to get in trouble with this. I was rather guilty of this on one occasion. As people didn't quite understand the full meaning of A,B,C in my fic, I posted on ponychan under alias Author of X (so no one knew I was Pav yet, but I was still clearly identified as the fic's author), and explained my authorial intent before the voting closed. Back in the ponychan days, this was a very common practice. And so a lot of people had an "aha!" moment, and voted my fic higher. And I felt highly guilty about it, since they were voting not on my actual fic, but on my explanation of the fic. I wasn't intentionally trying to game the system or anything; I was just green and eagerly excited to discuss my fic idea. But I definitely felt scuzzy about it in hindsight, and was glad when we moved to Fimfic forums and the temptation of anon posting was removed.

So yeah, I'm sort of saying that people should not take other reviews into account when doing their own scoring. But people definitely will. So am I going so far as to suggest that all reviews should be withheld until scoring is over, for a week of silence? Ugh, no. The authors need their feedback, and readers enjoy discussion. Let's not futz with that. I guess I'll lamely say that this falls back into the "everyone has their own personal criteria" sentiment, but nevertheless I strongly feel that people ought to vote according to personal feelings, and not be swayed by external factors.

3956689
3956741
3956405
Show of hands here, who has sent a fic to an editor and gotten feedback like the following: "Well, this was well written, no mechanical issues, I can tell you put a lot of work and heart into this, but... God this was so boring. I'm sorry but it's just vanilla paste. You've done way better in the past." It's one of the most scathing yet fair critiques you can get, and in a group like this, absolutely it needs to be conveyed to the author, so that they can improve. Score should reflect this, yes. How much the score should reflect this is up for debate... I would take issue with someone who said "funniest fic gets a 10, regardless of other mechanics" or "my OTP automatically gets a 10".

I would also like to think that we're mature enough here to not do spite-downvotes related to shipping pairings. This is not an issue for me, as I literally ship everything. I can think of exactly three pairings that I do not ship. Then again, religion has popped up in our contests now and again. It's not necessarily a kiss of death (Appletheosis was more or less "Satan meets the Mane Six", and it was a gold medalist here plus an upcoming RCL featurist) but religion can absolutely swing votes negatively if handled in certain ways. But again, I feel like that's indicative of readers as a whole, so it's meaningful feedback as long as it's honest.

3957060 Boring's one thing. Just not liking it is another. If I don't like it, but I can point out concrete reasons as to why, which are linked to things the author actually did wrong, that's justified. But going on a gut reaction, which is pretty much how all up/downvote systems work for the majority of participants, doesn't put aside personal taste. It's up to individual voters as to whether they can or should do so.

I don't disagree with many particular points in this thread, and I've come in a bit late so replying to specific posts is going to prove a but cumbersome for me. In that case, I'm just going to speak my mind to how I view the whole issue in general.

First, I apologize if I made it clear on the other thread that I am looking for strictly objective votes and reviews. I know that such a thing would be impossible even in a perfect world, because even among well versed reviewers there are differing opinions about what makes a story "objectively" good. All that I was asking for was a good spread of reviews and an accurate scoring via the votes. But while I am plenty satisfied with the former, this discussion about how some people are tweaking their scores in ways that has nothing to do with the merits of the stories themselves makes me wonder if I am truly getting the latter.

I will say first off that I understand why certain people are doing what they are doing. Just by their nature, certain genres like comedies might get an influx of bloated high votes because they are inherently more likable. I think that it's not by any means a condemnable desire to "balance" the scales out by giving those genres a slightly reduced score to account for those higher votes. Same goes with the stories that are well received, but a person feels like might be overrated for whatever reason. That being said, these kinds of concerns should be kept to the reviewing process, not the voting process.

Reviews are the place where people should be expressing their concerns like this, not in the voting process. With a review, a person can explain their decision as to why they did or didn't like the story as well as give insight to why they gave it a score that they did. A vote is just an anonymous number among a spread of another numbers without any explanation or context. Even if the vote is to try and counterbalance a story's over-popularity, that does nothing to explain why you think that story is over-popular in the first place. All the author sees is some random low votes thrown in with the rest, and if it were me I would be thinking hard about what those low votes could possibly mean. And if it would've happened before I learned about this practice of altering the votes, I would be thinking about why the voter thought my story was worth a 4, not that they actually think my story is worth a 7 but because it's a comedy or something they feel that they need to counteract all the 9's and 10's they assume I'm going to be getting. While I feel like the motives behind these practices are noble enough, I just don't feel like they are having the effect you guys think they are having.

Then there's the practice of making sure the votes are weighted as correctly as possible by making sure you give at least one 0 and one 10. This strikes me as an incredibly poisonous way of approaching the system, because it guarantees that there will be at least one author who thinks his story is terrible in every conceivable way and one who thinks his story is a literary equivalent of the coming of Christ. But that might not necessarily be the case at all. A vote of 0 or 10 should always be reserved for those very special edge cases of exceptionally good or irredeemably bad, but going in with not only the expectation but the actual desire to have at least one 0 and one 10 is going to affect how a person votes. If nothing else, it's going to make someone go through their votes, then at the end of it all make sure they have that 0 and 10, and if they don't, they will be compelled to arbitrarily change some of their votes to fit this artificial voting strategy. The only "fair" way the voting system would work among everyone who isn't an experienced author or critic is to say "on a scale of 'I could barely make it past the first few sentences' to 'this story has changed my entire outlook on life', how would you rate this story?" And I realize that this is going to be subject to a lot of opinionated factors, but even so, these kinds of votes are a lot more meaningful to me than votes that are based on an arbitrary system that commands votes be spread a particular way to make each vote "weigh" more.

I realize that there are different ways than this that you guys are basing your votes on, but I'm saying this not as someone who is casting these votes, but as someone who is going to be receiving them, and how I am going to interpret what I see. A 0 from one person will mean something different than a 0 from someone else, but to me all I would see is a 0, and like I said before, these numbers come without explanation or context. For example, I respect (even if I don't agree with) the practice of giving a 0 to a story that is "genius but boring", but only she knows that's what that particular 0 means. All the author is going to see is a 0, which is going to be an incredibly disheartening thing to see.

I just feel like it's a bit hypocritical for people to be talking about the weight of votes and how they need to account for certain stories being inflated unjustly while at the same time saying that the votes don't really matter and the focus should be on the reviews anyway. At the moment, it feels like the votes are being treated in some bizarre mixture of utmost importance and not important at all, and it's my opinion that such a state is unsustainable and is doomed to collapse at some point. Either the voting system should be somewhat standardized (even informally) or done away with completely.

3956807

I most like the idea of having each person receive the list of stories in a randomized order, but I think the only issue that would solve is the problem of the stories at the top of the list getting the vast majority of votes/reviews. As far as the scoring process itself, I think that a combination of the "Like/Didn't Like/Indifferent" and the "Categorical Scoring" methods might work. I mean, the system keeps the 0-10 scale for overall score, but also had a list of optional categorical "Like/Didn't Like" for things the reader thought the author did particularly well or poorly, such as Narrative, Characterization, Flow, Technical Skill, Concept, and so on. (I'm not a veteran author, so others could probably come up with a more concise set of categories than I could.)

3957106
I'm fairly new to the write-off myself, but I don't believe we actually get to see the votes on our stories. We get to see the average over all the votes cast, and occasionally the standard deviation if our stories are controversial enough, but I think that's it.

Could someone correct me please, if I'm wrong?

3956992

I forgot that I wanted to address my thoughts on letting other people's reviews affect my voting, but you pretty much said what I wanted to say about it. If as a reader I cast a low vote on a story, then from reading later reviews realized that I based my vote on a crucial plot detail that I forgot about or didn't pick up on, I don't view that as a failure on my part as much as a failure on the author's part to make that detail more prevalent in the importance of the overall story. A score should be made as fairly as possible, yes, but first impressions are just as important as the deep analysis.

3957133

If that's the case, I feel like on a certain level that's even worse. Rather than having a big grouping of context-less numbers, you only get one context-less number. To me, it just feels like, for all the people in this event are emphasizing the reviews and peer evaluation and learning what aspects of my story-writing need the most work in an anonymous and therefore as unbiased as possible setting, having a story end up with just a number without much explanation to it is a bit of a slap in the face.

3957133
Correct.

3957159
Fair point, this has come up in the past too. "I didn't really get any bad feedback, or perhaps even glowing feedback! But then the contest ended, and I got a 4.2/10... wat?"

Sometimes (based on what my own score for them was), it's just a matter that "hey, this was alright, but many of us are voting on a curve, and there were simply a lot of other fics that I liked better." But there have been occassional anomolies, like some of the discussion sparked from two writeoffs ago, where the final score didn't necessarily reflect the glowing praise / making it into other people's Top Five lists.

3956976
This.

This whole thread is a great big slice of

It's an intriguing debate but why you lot have guffed out so much hot air over it is baffling. I'm pretty sure there's more words here than in the entire contest [citation needed] .

I want to crank out twenty more Five Word reviews tonight, so I'll be brief with my thoughts:
People will vote the way they want to vote.
I don't believe there's a big syndicate conspiracy going on.
I like the voting system as is and don't believe change is necessary.
I'm fairly sure I already have nailed down my favourite fic of the contest, but I'm not going to be kicking up a fuss if it doesn't come out top at the end.
Crowd mentality is a useful tool, but it's nothing like a personal opinion, and it won't be changing mine.

Oh, and of course, this is fan-fiction. The rest of the world isn't taking you seriously, so why the hell are you, eh?

3957060
3956689
Speaking as a hardcore shipper here, anyone who writes a ship fic and doesn't know that it's a gamble doesn't understand shipping. It will probably influence some people, no matter how neutral they try to be, in either direction. That's the risk of non-canon shipping.

I'll also say that in the short story contests at least, it's the job of a good romance fic to sell you on a pairing. I'm pretty well known among shippers for my dislike of FlutterDash and RariJack, yet I can point to stories for each that are good. At the same time, I used shipping in my "There is Magic in Everything" entry, and Present Perfect called me out on not selling it, and that's legit. People writing romance for the contest here need to keep in mind they aren't preaching to the choir just as much as members of the choir need to realize that this ain't Sunday service.

It's similar to writing a villain redemption fic, or an unhappy ending: if you don't make people want to see the villain redeemed, it's not a good fic, and people will go into it with varying levels of wanting to see that character as good, so you'd better sell the idea. Many people are going to want a happy ending, so you need to make the bad ending feel like the only correct option for the fic.

These are all story options on hard mode. Tackle them at your own risk, you're never going to get everyone to be objective about them.

3957106

I would be thinking about why the voter thought my story was worth a 4, not that they actually think my story is worth a 7 but because it's a comedy or something they feel that they need to counteract all the 9's and 10's they assume I'm going to be getting. While I feel like the motives behind these practices are noble enough, I just don't feel like they are having the effect you guys think they are having.

As one of the people this is aimed at, I'd like to point out that I publicly posted exactly what score I gave every fic, and explained all of my top and bottom scores. And I would certainly update that if I changed my vote, I haven't been trying to hide anything about what I do in any of this.

This isn't any kind of behind the scenes sneaking around, I'm more open about my scores than most of the voters. It's about using my vote as a way to express my opinion of a theoretical story.

3957159

having a story end up with just a number without much explanation to it is a bit of a slap in the face

The reviews are the explanation. And far more important than the score.
Forget the score. Forget the votes! It's the reviews you need to be looking at!

I'm leaving for an out-of-town educators convention in half an hour, so I'm going to be brief here. I've been thinking about the voting, though, and I think I'd like to take another stab at putting my beef into words.

When I vote, I designate the story I thought was worst (or stories) as a zero, the story/stories I thought best as 10s, and then fill in the rest of the fics based on where they fall between those two extremes. Some voters vote on a curve, some use absolute standards for their votes, some probably have other systems. All of that means that a three from me might not mean the same thing as a three from TD, and neither of those are the same as a three from PP, and so on. It also means that some voters might end up having more "weight" than others.

None of those things bother me, because each of us is still rating stories according to what we thought, using whatever our standards are, were the best and worst in the group. We may be judging based on different criteria, and we may express those judgments in different ways, but we've all accurately communicated our relative opinions on a fic.

What I'm not convinced I can feel okay about is people voting contrary to their judgements. This is why bookplayer's "use a 3-10 scale, and knock a three down to zero if it's getting good reviews" strategy sat so poorly with me (though I would like to be fair and point out that it's much different, and much less objectionable, than the "give zeroes to stories that I thought got too much love" which I interpreted her original post to be): it means using your vote to communicate something you don't actually believe, that the zeroed story is worse than the three. This is why I'm still leery of Bad Horse's "knock a point off comedies to make up for other people's behavior toward them," even though I understand his logic; it means ranking comedies below other stories that you thought were as good or worse.

I'm not going to dive into the math, here--there are plenty of other people much more capable of doing that than I am. And I don't have a proposal to "fix" the system; even if I was sure it needed fixing, I have to agree that the vote results are far from the most important part of the write-off. But that doesn't mean the votes are totally meaningless, either, and I am fundamentally uncomfortable with people using them to do anything other than communicate what they, personally, thought of the stories they read.

3957227

I've seen that, and I greatly appreciate how transparent you are with your votes. That said, I assume this event follows the same trend as others that there are vastly more votes on a story than reviews? So even if I end up with a handful of satisfactory reviews, I'm still left with a number that for the staggeringly large majority of voters is unexplained. (Especially if, like Pav Fiera said, the tone of the reviews doesn't match up with the score.)

3957245

I'll repeat what I've said. If the reviews are what's so important, then why bother even having a score?

3957282
How else are you going to work out which fic won?

3957285

Ignoring the fact that I've seen several people argue that this event isn't really about the competition, if it really is about which fic wins or loses, then altering votes based on arbitrary judgements is even more unethical. It attributes to (and possibly guarantees) that certain types of stories will never "win" for completely superficial reasons that have nothing to do with the quality of the story itself. If this event truly is a contest, then that's an even bigger reason why the voting system needs to have a more rigorous standard to follow, and to do away with these "balancing against the popularity contest" practices.

3957133 3957159
Here are the results from last month. As you can see, each story is shown only with its average score, and the five with the highest standard deviations are recognized in the Controversial Stories section.

To me, it just feels like, for all the people in this event are emphasizing the reviews and peer evaluation and learning what aspects of my story-writing need the most work in an anonymous and therefore as unbiased as possible setting, having a story end up with just a number without much explanation to it is a bit of a slap in the face.

That's what the discussion thread is for. I can guarantee that at least three or four people have reviewed your story already, myself included.

3957305
It isn't "really" about fics winning or losing. You said that. I didn't.
That system is there because by nature this is a contest, so there needs to be a winner, which encourages participation and quality.
But the true benefit is the reviews, because that's how you're gonna improve as a writer.

That's it. It's basically as simple as that. No need to over-complicate it now is there?

3957282

If the reviews are what's so important, then why bother even having a score?

The way I see it, there are two elements to this whole writeoff thingamajig. Someone's spoken about how this is more a workshop than a contest, but I don't think that's strictly true. Based on what I've seen (though note that this is my first time, so I'm the exact opposite of an expert on this topic) the writeoff is a mix of both. The reviews reflect the workshop nature and, for a writer who wants to learn and grown and improve, they're what's important. But I'm sure there are a handful of people who just enter 'cos they want a chance to win something, and I'm equally sure there's a large group of people who are a mix of the two. If you ask me, the votes are there to make this a contest, whether all participants treat it like one or not. Maybe it's not a fair contest because of people voting subjectively or voting in different ways to each other, but I think it's a lot more fun to have a contest with votes from everyone involved than just from a handful of judges. It's quite fun to know that your vote counts, even only just a little bit! At the end of the day, I think this is really just about having a bit of fun, and that's more important than who won or what position you actually got.

Did I really just advocate the "it's not the winning that counts; it's the taking part" attitude to competitions? Wow, I've grown uncompetitive lately.

3957106

Then there's the practice of making sure the votes are weighted as correctly as possible by making sure you give at least one 0 and one 10. This strikes me as an incredibly poisonous way of approaching the system, because it guarantees that there will be at least one author who thinks his story is terrible in every conceivable way and one who thinks his story is a literary equivalent of the coming of Christ.

You're still not getting my point. These scores make no sense outside of the event they were written in. They're not supposed to. Getting a 9.7 doesn't mean a story is amazing. All it means is that relative to the other stories in the event it was submitted into, it was generally well-received. You cannot compare this score to a fic that got a 8.2 in a different event. The point of scoring isn't to determine a fic's quality. It's to compare how well-received it was in relation to other fics. It's been mentioned in other threads and by Roger that the 0-10 scale isn't absolute value.

I realize that there are different ways than this that you guys are basing your votes on, but I'm saying this not as someone who is casting these votes, but as someone who is going to be receiving them, and how I am going to interpret what I see.

Here's how to interpret your what your score means to the quality of your fic:

You don't.

It's a relative value that has nothing to do with how absolutely well or poorly you wrote. You can't say "I'm shooting for a 7.5 or higher this round!" because 7.5 means nothing outside of it's location in the range of scores that the event's stories have received. And even the ranking of the story compared to the others is ultimately a subjective assessment of value. It's up to you to decide if you think the votes the writeoff community have given you are worth anything.

I just feel like it's a bit hypocritical for people to be talking about the weight of votes and how they need to account for certain stories being inflated unjustly while at the same time saying that the votes don't really matter and the focus should be on the reviews anyway.

Look at it this way: we don't care about the votes, which is why we allow ourselves to modify them in such a way that may result in a ranking that most accurately reflects our beliefs as a group about which fics are subjectively better and which fics are subjectively worse. The fact that you've brought up the idea of voting on some kind of absolute scale to determine a fic's worth simply means that you value the integrity of the vote more than we do. But, as I've mentioned, this is borne of a misunderstanding of the purpose of the vote.

3957282

I'll repeat what I've said. If the reviews are what's so important, then why bother even having a score?

Because seeing how well a fic was generally received by the audience (including the non-reviewing audience) is valuable information. Of course it's secondary to the reviews, but nothing in the past has shown voting having an effect on reviews. And that's all that really matters, since the reviews are generally considered the most important part of the event.

3957227

non-canon shipping

Which limits you to all of 5 couples... the Cakes, Rarity's parents, Twilight's parents, Cranky/Matilda, and Shining/Cadence. Yes, it's a shipping story's job to sell you on the ship. I'm not talking about that. I'm talking about people who downvote Rarilight because it's Rarilight. I have seen at least one comment during this write-off to that effect. It spoke of nothing about how well the story did with the ship. It spoke only to thinking those characters don't belong together, period, in the commenter's headcanon.

3957282
Well, adding to my previous comment, part of the air-quotes "problem" is that not everyone posts reviews. So in some cases there may be sampling bias: the people who posted comments liked it, but the people who did not post comments did not like it. Sadly, since they didn't comment, we won't know why they didn't like it. I would never propose that we make commenting mandatory (*grumble 93 fics grumble*) but it's definitely why comments in this system are so valuable to the authors.

Another issue is me. I am the problem. So, I didn't review All In, but for those who've seen my reviews in the past, I generally go one+ paragraph of stream-of-consciousness, and one paragraph of final thoughts. I don't traditionally post my final scores (though I do post my Top Five picks at the very end). Now, I wouldn't call me a hugboxer per se, especially since I grade on a curve, but when I'm pointing out issues, I may not be that clear about if we're talking 5/10 bad or 1/10 bad. Since I'm critiquing but framing them with a sort of neutral-to-positive tone, it's not always clear what score I gave a particular review. Why don't I post scores, then? Ego. Doing reviews for the Writeoffs is essentially how I got discovered/picked by Seattle_Lite to join SA, so for some reason or another, people seem to listen to what I have to say. Bwahaha, the fools. So I suppose on some subconsious level, I've always worried that me posting a review on Monday will color how other people interpret the fic—exactly the issue we've been discussing here. If the review, then, contains a detailed critique but is more score-ambiguous, that at least seems like less of a spoiler effect? Somehow? IDK. But then, to the points here, the author loses some potential feedback.

What if, similar to the new system where you can see who correctly guessed that you wrote your fic, we add a system to show your histogram of scores after the results are released? So, you have not just the average score, but you can see how polarizing the fic was? I'm thinking just scores, not which-voter-gave-which-score, because that could lead to uncomfortable situations. Even then, on the one hand, while it could be nice to see more data on my fic, I don't know how useful that actually is. "Okay, I got a whole bunch of 7's and 8's, and a single 2. Why did I get this 2?" You'd never know the answer to that. Not very helpful.

3957326

Um, no, you were the one who asked how we can determine which fic wins without a score. I never even mentioned the fact that this event would have a "winner" until you brought that up. Every concern I had about the current voting system was about how an author would use it as a reflection of their skills as an author. Can we make an agreement to not put words into each other's mouths?

3957346

I completely agree. Some people view this event as a contest, some as a workshop, but it's probably true that most people view it as both. In that case, I'm not trying to force some kind of separation between reviews and votes to those regards, but rather make it so the votes are still meaningful to people who are just trying to get some honest feedback. As it currently stands, the voting system seems rather arbitrary to people who want critiques, and personally it seems even more so now that I know there are people who are basing their votes in part on some random metrics that are neither necessarily warranted nor strictly helpful to the author.

3957356

I'm getting the point just fine. You're saying that the 0-10 scale should be a measure of how stories measure up in comparison to other stories in the same event. Only that makes no sense, partly because a 0-10 scale isn't a good metric for that sort of measurement, and partly because that's not how a lot of people (especially newcomers like me) are going to interpret it. The very nature of a 0-10 scale implies that a score a story receives is indicative of its level of quality. Anyone who went to a school with a conventional grading system (as far as USA is concerned, not sure how it is in other countries) will already know and be expecting this - 0% is utter failure while 100% is utter perfection. If the score is to be a reflection of how a given story is in relation to others, then the system should reflect that by having a system where people rank stories from first to last, not by giving a numerical score.

As it is, having the system intuitively suggest one thing while a segment of the more prominent reviewers are suggesting something else entirely is just confusing. Like I said before, just because you are basing how you vote using a particular system, that doesn't mean the people you are voting on are going to see it that way. You may give a story a 10 because even though it may have had some issues, it was your personal favorite. The author will see that 10 and think you thought their story was perfect in every way, with little to nothing needing improvement. You see the difference here?

Personally I think this attitude towards the scoring system is commendable. I just also think it's not very realistic.

3957400

I don't like the histogram idea much for a lot of the same reasons as the single score implementation. All it is is a bunch of data without much explanation. I think it would be better if, like my earlier suggestion, the voting was able to be accompanied by commendations on specific aspects of the story, like if a voter thought the story overall was a 6 but thought the characterization was particularly good. Another idea might be to attach optional comments to a particular vote that are revealed at the end of the event. Just something to give these arbitrary numbers more usefulness and meaning.

3957454

but rather make it so the votes are still meaningful to people who are just trying to get some honest feedback.

The issue I have with this is that I don't think this is the purpose of the votes, and I don't think making it their purpose will actually do a lot of good. They're not meant to be meaningful feedback, and I can't really see any way they could be. Not even the way I'm voting (assigning a percentage of my score to various categories, weighted in a way that I think is fair and will change from contest to contest until I find a weighting I really think reflects the importance of those categories) is particularly meaningful feedback: one entry might get a five from me for being terribly structured and unrelated to the prompt, but otherwise thoroughly competently-written and enjoyable; another might get a five for being poorly-written and just something I didn't enjoy, while following a structure that is a work of genius with the narrative voice and subtleties of Terry Pratchett[1]; and yet another might just be middling on everything.

And while my system may suffer from this the most on account of its far-too-rigid distribution of points across categories, at least I try to explain it in a review. Not every voter has the time or patience to do that, and frankly I don't expect them to. If I were scored a five for an entry I wrote, it would be useless feedback—it would be telling me that either some bits are good and some bits are bad (but not which bits!), or that the whole thing was just mediocre, and I'd have no way of knowing. What it does tell me is that, on average, people enjoyed it less than an entry which got a seven, and more than an entry which got a three. And maybe it's not useful feedback, but it's simply not meant to be.

[1] That said, it would be very hard to write something I didn't enjoy in the narrative style of Pratchett. But hypothetically, I guess it's possible.

3957454

As it currently stands, the voting system seems rather arbitrary to people who want critiques, and personally it seems even more so now that I know there are people who are basing their votes in part on some random metrics that are neither necessarily warranted nor strictly helpful to the author.

I don't like the histogram idea much for a lot of the same reasons as the single score implementation. All it is is a bunch of data without much explanation. I think it would be better if, like my earlier suggestion, the voting was able to be accompanied by commendations on specific aspects of the story, like if a voter thought the story overall was a 6 but thought the characterization was particularly good. Another idea might be to attach optional comments to a particular vote that are revealed at the end of the event. Just something to give these arbitrary numbers more usefulness and meaning.

You're looking at the wrong part of the process. As I said earlier, reviews are posted in the discussion thread. You don't use a screwdriver to hammer in nails, and you don't use the fic voting to deliver critiques.

3957514

And maybe it's not useful feedback, but it's simply not meant to be.

That's sort of my point though. There is such a strong divide between people who think the votes are another important way to determine a story's level of quality and people who think the only reason they exist is because a contest by definition needs a "winner". The problem is that, because of this divide, I feel like there is a very real chance that stories aren't getting judged fairly. I mean, I get that everyone will determine a story's score differently, but that's not the same thing as everyone evaluating scores based on completely different and incomparable structures.

3957524

That's the thing though. The only reason the votes can't be used in the critiquing process is because of how they are currently handled, and the fact that enough people have an issue with how the votes are handled is the whole reason why this thread exists in the first place. The prevalent opinion on this forum seems to be that the votes shouldn't be taken seriously when looking for feedback. And the only thing I can ask in response to that is, "Why can't they?" I mean, sure, the votes can't and shouldn't be used as a replacement for in-depth reviews, but there's no law saying that they should be utterly worthless in that regard.

3957454

Only that makes no sense, partly because a 0-10 scale isn't a good metric for that sort of measurement, and partly because that's not how a lot of people (especially newcomers like me) are going to interpret it.

It is entirely within the realm of expectations for newcomers not to realize how we use our scoring metric. That is why I am telling you about it now.

The very nature of a 0-10 scale implies that a score a story receives is indicative of its level of quality.

First off, that's subjective. Secondly, we've already told you that this isn't the case. Thirdly, it's actually something that we, as a group, have talked about before. Click here for Roger's word about it.

The author will see that 10 and think you thought their story was perfect in every way, with little to nothing needing improvement. You see the difference here?

This is an artifact of misunderstanding/ignorance of this group's intentions and tendencies, not a malicious or flawed system of doing things.

Personally I think this attitude towards the scoring system is commendable. I just also think it's not very realistic.

As far as I am aware, this is the attitude we all have. That scores are relative. No one in my experience has ever used scores to determine a fic's absolute quality in any of the Writeoffs I've participated in. To be frank, this isn't a problem with the group, this is an example of culture shock on the behalf of the newcomer, which is unavoidable at least to some degree.

I'm honestly not seeing the problem anymore. The only thing further thing we can do is to make it more clear to newcomers that scoring is not an absolute assessment of each fic's quality. Is that what you're suggesting?

3957543 The main reason I don't think votes can be used seriously for feedback is because there is very little way of knowing what was going on inside a particular voter's head—it's why I'm very glad to have seen the logic behind the voting of bookplayer and TD, as well as many of the other reviewers, and why I try to be transparent about mine. But if you don't read the threads and look through the reviews, you're not going to understand where those numbers are coming from; on the flipside, if you do read through them all, you're already looking through much more valuable feedback in the reviews themselves.

Add to this the fact that many voters don't include their reasoning and their own particular system, and the fact there's no way to tell who the majority of votes come from anyway even if we do go for the histogram approach, and you end up in a pretty rough position if you try to claim that the votes could ever be used as better feedback than "on average, this was liked better than these entries and worse than these entries." I can't see any way that they can be anything more than an indicator of relative value. If someone proposes a system that would make it so... well, you'll have convinced me otherwise. I've yet to be convinced by any suggestions, though.

3957561
3957572

I feel like I could argue this practice for days, but it will boil down to a difference in opinion, and since it's essentially an argument of the newcomers vs the people who have been in this event practically since day one, further discussion is ultimately pointless. It's been made pretty clear that you guys have an established stance on the voting practices and aren't really willing to consider dissenting opinions about it.

I mean, not that I'm blaming you guys for it. It's your group, and ultimately you should have the final say on whatever policies are and aren't implemented.