Site Post » Possibly moving to a Newgrounds-style approval system · 3:14am Sep 24th, 2012
Please actually read the entire post before commenting! I have done my best to alleviate most potential concerns about the system, and comments that are just repeats of problems brought up in the post may be ignored or deleted.
knighty brought up an interesting idea tonight. We've had a problem for a very long time with how our moderation system works (click here if you aren't sure how it works now), which is that if none of the moderators feel like approving stories, nothing gets approved! The reason that moderators are unwilling to approve stories is that it's massively boring, and massively unrewarding work. When we approve stories, we only occasionally get a thanks, and when we fail stories, we're pretty likely to get a scathing message asking how we could possibly fail their stoiy. This leads to situations like the other day where our only active approver went out of town, and we ended up having 200 stories backed up in the queue to be approved!
So we started exploring other options. One option is to just bring on more moderators who are more willing to do this work. That presents its own set of problems. We have to train these new moderators on our rules, we have to work with them constantly to ensure they are not making errors, and we have to hope they have Skype so that we can communicate with them. Plus, we have to ensure that they're trustworthy and that they're going to be active and... it's a huge hassle, and all we get out of it is the hope that they might approve enough stories to be useful before they get bored. Another problem is that (and even we're guilty of this occasionally) they tend to make mistakes, which leads to junk stories that otherwise definitely violate our rules getting past moderation.
A second option is to just drop approval entirely; let ANY story get uploaded to the site, and have it hit the front page and be searchable immediately. This approach has a lot of obvious problems, and doesn't solve the primary one of administrative fatigue. While we wouldn't have to deal with story approval, there would be a very sharp increase in the amount of reports to deal with, and if those reports aren't resolved VERY quickly, users would start getting generally upset at the quality of fics that get submitted... trust me, you should see the sort of thing people submit when the queue is moderated.
The third option we're exploring (and I should stress that we are just exploring this right now) is a Newgrounds-style approval system. Instead of having moderators troll through an approval queue, we instead give the power to the users. We're not sure on the specifics, but it would probably a Steam Greenlight style question: "Do you think this story belongs on the site?" Users would then simply select Yes or No, in accordance with the rules. Now, of course, a system like this does not come without its own problems. knighty and I went back and forth on some of these and came up with some solutions (note that Newgrounds uses almost all of these safeguards as well).
First off, I'm sure most of you are screaming "But what about potential abuse?!" Yes, this system by itself is rife for abuse. However, there are some pretty simple things we can do to cut down massively on potential abuse. The first thing is that only accounts older than a week would be allowed to vote on stories for approval. We could always adjust this number if need be, but this seems like a pretty fair number to us. This also prevents people from just spamming a bunch of accounts to upvote their own or a buddy's story. Secondly, we will of course be monitoring this system to ensure there are no unexpected abuse cases that come up - and will be acting quickly to fix them. If we notice you doing ANYTHING to abuse this system, expect some harsh consequences!
The second concern we had was that of nobody actually participating in this system. The average view count of a story on the front page, drawing ONLY from front-page views, is actually quite low. The average story barely gets 20 views - many stories do not even get 10. If we simply released this system out into the wild, we'd almost certainly be in an even worse situation than we are now. That being said, we decided to implement some sort of rewards system for users who participate actively in this system.
Users who participate in approving stories might be rewarded a new icon for their profile, which would show up on comments and on their User Bar at the top of the page. We might hand out little awards to put on your user page, or maybe even custom titles... username coloring, stuff like that. The one we're by far most likely to do is the user icon, since it's quite simple and unobtrusive, but we're not open to expanding on the reward system is the system sees inactivity. We really want people to USE this system if we implement it! EDIT: Of course, you would only be rewarded for stories in which you voted in line with the majority decision. Randomly spamming votes will get you nothing (and may even get you banned).
In addition, any fics that are failed by users may be flagged for administrative review, to ensure that there are no "downvote storms" on stories which do, in fact, belong on the site.
The basic goals of this change, therefore, are:
-- Reduce administrative workload and fatigue
-- Give more power to the users as to what they think is appropriate content on the site
-- Reward users more actively for participating in the community while improving the overall quality of work on the site-- Get knighty to do some damn work
I'd love to hear what you guys think of this. I should stress that we are NOT moving to this system yet! This was just an idea we banged around for a bit. Even if we DO implement this system, if it ends up not working out, we will just revert back to the old system. We wanted to give you all a heads up before we did this though, as it would be a fairly radical shakeup of the way the site works now.
knighty and I are gonna go play Borderlands 2, but we'll be back to look over your comments later. Have a nice day.
I approve this approval system
I approve this approval of the approval system.
that's a pretty good way to trick the userbase into doing your job for you poletron you lazy ass
(USER WAS BANNED FOR THIS POST)
Sounds like something worth trying, at the very least.
378807
cosmo please go
378807
How polite.
I agree with the Newgrounds-style system. IP checks to reduce sockpuppets and rewards for showing initiative and helping approve stories sounds like a good idea.
This actually sounds like a much better system, especially for when the queue is so backed up. Plus, I personally love the idea of actually getting rewarded for reading all the stories that hit front page.
Sounds reasonable. If you coupled this with a way to improve freshly posted story views, it could work better. I always found the front page to be unappealing to scroll through. You could have a small box next to the featured box - a 'New Box' so to speak. People could be encouraged to look at new things for the sake of keeping things running. I don't think people would mind such a feature/responsibility.
I had what might have been a good idea, but I'm too drunk to remember.
Though this system still has a chance of abuse, and some amazing stories by new authors could be overlooked in favor of a more known name, also, if you get enough yes votes it would be nice to inform the yessers that the story got through, thus assuring at least a healthy viewer base.
I think this is a fantastic idea. I'd like to see how it works out.
378818 Like you needed an excuse to read all of those stories.
t2.gstatic.com/images?q=tbn:ANd9GcQaZz51bC1VU_EUbVPgs8v9KGxHjyaSiemO3NYGwicb8baOxNoPFA
That sounds like a pretty good idea for a system. The moderators work really hard to approve stories, so anything to help them out I'd be okay with. If you were to try a trial version of it then see what happens from there, then all for the better.
I'd be okay with this, but, maybe upgrade how old the user must be. We don't want some half-assed guy clicking 'no' to a story. Maybe 3 - 4 weeks seems reasonable. Hell, go 10 weeks. Make them wait and earn the right.
Anyways, we'll see how this flip-flops.
378823
That's a good point, though that might get spammy for approvers... We'll consider that.
you have just offered us what Steam gave TF2 players. Hats. in this case they are metaphorical hats, but hats nonetheless. But you know what? I WANT SOME HATS!!!!
I would gladly participate in this kind of system, but there are some changes that I would ask. maybe instead of 1 week, it be 2? just asking, because 1 week seems too short. as long as you guys/girls would be able to control the horrors of the internet user, then I see no problems with this.
I'm certainly up for anything relieving the stress of our crabby administrators (kidding! mostly...)
The only question I have is, would the work you guys would have monitoring users for abuse of the system wind up being just as much, if not more work than the current load?
It seems a reasonable approach. For keeping down the possibility of abuse, you could potentially (depending on what kind of response you can eke out of the user base) try only presenting a randomly-selected subset of pending stories to any given user. That would make it harder for any person or group of people to game the system.
It blew my mind when I found out you guys take the time and effort to pre-screen every story submitted to the site. Anything that makes that job easier sounds good. I'd definitely like to see it done on a trial basis, just to see how it goes.
I could get behind this. Are you guys thinking of copying any of the other NG-style rewards for the fics in addition to the voters? 'Daily Best', more long-term ranking displays, etc?
And just for the record, is this also a move for users to suss out quality issues, as well as rules issues like the moderators currently only do? (Can we get rid of the bazillion 'I died and went to Equestria' fics this way, in other words?)
I say this is a great idea.
Could one of the rewards for participating be, say, being allowed to have your own custom banner for one of your stories? Like Harmony or Antipodes?
378835
Yeah, we might end up making it 2, but that's the sort of thing we can always adjust later. Some people are asking for 2 - maybe.
378838
The idea is that it won't, but it certainly could end up being that way. But think of it this way - if it is, we have the same amount of work we do now, and users get to have more power. Sounds like a win/neutral to me. That's a thing probably.
How about we take all the crappy authors...
... And put them in a Recycle Bin...
Just kidding.
Don't know why I wanted to say that... Hrm... Meh.
I already gave me two scents.
378843 I also believe rewards are in order.
A few concerns I've come up with so far:
1) What's stopping someone from blindly upvoting everything just for userpage shinies?
2) I might be misremembering, but don't even established/popular authors get their fics disapproved sometimes? That would be a thing of the past with this system.
3) Unpopular premises (uncommon ships, OC fics, anything human, etc.) could well go unread, and depend on point 1 up there to ever see the light of day. And if there's ever a choice between an unpopular premise, a Luna Is Bad At Modern Things, and an LOLsometa fic then well, sorry little fic. And yes I know meta fics are against the rules, but you and I both know that thing would be upvoted to hell.
4) What's the point of approving fics at all if everyone can see unapproved fics?
378839
Not a bad idea, something to consider... people might still just sit there until they get their "chosen" story though. We'd also maybe like users to choose stories they are interested in, and a random selection is unlikely to facilitate that.
378841
Not right now, but we'll see where it goes. We do already have the Featured Box, after all.
378843
If it's Pink Floyd themed I approve of it
The idea does indeed have its merits. But I have to say, do you have any ideas on the table for appeals to disapproving stories?
Also, can you go into more detail, if you can, on the user ratings themselves? Is it the strories drop below a certain approve/disprove ratio or is there a finite threshold of downvotes?
I approve of the new system, even though it still sounds like its in the conceptional stage and has many kinks to work out.
However my big concern is the large one-shots that got in excess of 10k words, and exactly how long it would take for a fic to get approval onto the site. If I can make a recommendation, perhaps keep the ten-minute queue updater thingy.
378844 Very true. It sounds like a very good idea to me, was just a little worried about the backfire potential.
Oh, just thought of something.
You could add a limit for the story to reach in order to be posted.
Let's say it has to reach like, 25 up votes or so. If it reaches 25 down votes, then it's immediately disapproved.
Seems legit.... and I think it would be interesting to see how it works out....
So what happens if someone gets a bunch of people (not bots, literally other people) to upvote his fic (with the possibility that it may be crud), and people see that it actually is crud and downvote it? Wouldn't that technically mean you are voting against the majority? How would you protect against that?
I've been wanting a system nearly identical to Newgrounds here for a long time. They do indeed give rewards for voting, and those who vote more get more voting power.
Now, this would also mean you have 2 separate queues, so it might be rough at first (your front page crawl as usual once a fic is approved, as well as the approval queue). You should also give some sort of boost to the fic itself upon approval if you go to this system. (IE any thumbs the fic gets in approval should transfer over to it's front page number). Obviously you still only get one vote per fic but it can influence the desire to read a fic or not when it hits the page. Overall, this could be a very very good thing, and leave the mods to deal with fixing issues rather than reading absolutely everything submitted, like mods should be doing.
I can sense a potential disturbance in the force from this sort of system, in that it relies on a certain measure of user activism and awareness. Neither of which you're guaranteed, even with a rewards system.
Picture a scenario in which a bunch of users who are only in it for the icons and such simply go in and maybe just sort of skim a story (or, heck, who guarantees they read it at all?), then vote for an approve/disapprove because, darn it -- rewards. Move to the next story to repeat.
The problem is that you can't guarantee any sort of quality in approval from the relatively few users who'd care enough to do the job in the first place. Because even with a vague reward system in place, it's not particularly rewarding to read a terrible story. Moreover, you would probably have to consider that, even with certain baseline rules in place, people would inevitably vote for rejection on terrible stories (or, at least, stories they believe to be terrible). Whether or not they technically fit the rules.
Frankly, what I'd imagine happening with the proposed system is that the mods would go from policing stories to having to police the users approving the stories. And that's assuming there're enough users who participate in the first place. No guarantee of that, really.
Invoking more moderation staff might seem like a bit of a hassle, but I guess you could view it as an investment. Once you've actually found a trustworthy user with whom contact can be made, you've got another pair of hands helping to approve stories for the duration with a relatively low amount of added input. Compare to an ongoing hassle of worrying about users who might be ruining the "Greenlight" system.
I've uploaded several stories and never felt that the delay between 'hitting submit' and seeing my story appear (usually a few hours, never more than 2 days) was onerous, so if the only problem with the current system is that people don't like the delay, I don't have any problem leaving it as is.
Then again, I'm not a mod and don't know what kind of fatigue they're under, so if this makes the site easier to run on their end, no objections. :-)
Oh, also, make sure to run a 'beta' system first. To see how it works. It always helps when Users give feedback first before rushing into the idea.
well, certainly a good idea, but what if some people get a lot of stuff and people start getting all jelly and just stop's helping out entirely. But that wouldn't really happen a lot sooo, why the heck not. I approve of this approval system like others aproved of the aproval system before I approved.
378833 You'd have to be able to toggle whether you wanted to get notifications for all approved stories you voted yes on, of course.
LOL DELETED TEXT.
I'm not so sure about the user-vote system. I think participation will be the biggest issue with this idea, as well as time frame. Site traffic and how long a submission is up for voting will largely affect what goes up on the front page.
Another problem is a very specific participation issue - it would be difficult to ensure people are actually reading the thing before they vote on it. They could click the link and it counts as words read, but there's no way to truly enforce any requirement to read before voting. Also, it's easier for a small group to follow the rules than a massive group who is more often than not keeping an eye out for quality or their favorite authors than the rules themselves.
A long term effect of this going into place could mean a downturn in perception of quality in the site's work. Take for example, EQD - there is already a stigma that because a small elite group screens stories for approval, the volume of Fimfiction's approved submissions unintentionally misrepresents the quality of the fiction. Now imagine a popular-vote system in which untrained blokes like me suddenly have a say in what goes up. I am considered to be a novice writer at best and a sloppy hack at worst. Hundreds of site users just like me would be suddenly empowered to make decisions that we aren't fully trained to make. Don't hand the keys over to hundreds of folks like me who don't really have an eye for this stuff.
Finally, this approval system would be a subtle reflection of the front-page-to-featured/popular system. Stories will show up on one queue, and people read them and the turnout is reflected in what comes to the forefront. Same principle, really, or similar enough.
That is a GREAT idea!
It will give me something to do when I'm bored!
Because it IS possible for me to get bored.
I think you should smooth this out for a bit then start using it.
378849
1) We'll implement some sort of penalties if you vote "incorrectly" too many times in too short a span. As a very rough example (not what we'll actually do) if you vote wrong 50% of the time, you might get your privilege taken away. Also please check my edit on the blog post about halfway through.
2) Maybe... we'll keep an eye on that. It's certainly something we already struggle with, but there's not much to be done about it.
3) Also not much we can do about this... and frankly we won't be SUPER bothered if people downvote stuff that's REALLY bad... again, we are going to be checking each fic that's failed.
4) Not everybody will bother with this system, in fact I bet the vast majority won't even participate or care.
I'm really going off now, so I will come back and read the comments later. Bye bye!
378807
Uh oh, Ponychan is leaking. We're doomed.
This seems like it could be the greatest thing to happen to the site in my... *goes to check* Five months of contributing and untold millennia of lurking.
It could fail, sure. It's not foolproof, by any means. But I definitely think that we should at least try it out, because it hugely benefits involved users like me.
Firstly, we can get rid of the shit fics that are TECHNICALLY allowed but still are really awful and do not contribute to the site in any way. (See: Anything submitted by this guy). Wanderer D, alexstraza et al. get to stop weeping about having to approve awful, awful stories.
Also, it provides incentive for people to read more up and coming authors instead of just staying within the bounds of well-established authors. This is important! Plus people who already read a lot of front-page stories will get cool perks for doing the good work that they already do.
And of course, there's the perks for you guys. I love our moderators, and seriously, you guys definitely deserve a break. Doing all this for us
because you love attentionout of the kindness of your hearts, it's really great. Thank you.378807 Can we please, please have an archive of all the things Cosmo has gotten banned for?
378871
It was a joke. Look at his userpage, he's not banned.
Youmustbenewhere.jpg
378862
It could be because of the larger user base on NG, but this has never been a problem there.
But doesn't also face the problem of having revolutionary ideas fail due to the authors who came up with them are not strong writers?
Aside from what's been mentioned already, I have a question: Exactly how many votes would be required for approval, and what ratio of upvotes to downvotes would get a story approved? Requiring a large number of votes could make it too difficult for some stories to get enough thumbs up for approval, and if the number is too small it could get rejected just because a handful of people who dislike the premise happened to be the ones who voted first.
Another suggestion is that maybe each person could just choose a few tags, and then only stories with those tags would end up being posted on their page for approval. That way if somebody dislikes shipfics or grimdark on principle they won't have to see them.
378885 I'm all kinds of stupid tonight.