Hey, so I've heard that the group has it so that every editor has to read the entirety of a longfic before it's considered, and as a result the RCL is almost overwhelmingly in favour of short fics: Of 87 slice of life stories in here, for instance, one is over a hundred thousand words, and only three more break 20,000: Four out of five pages of results don't seem to break 10,000 words.

So it seems that the group, then, optimizes towards reading shorter stories that all the editors can burn through and agree on. Unfortunately, that's not interesting to me as an audience: Odds are far higher for me having read those stories myself, in the first place, than the highly crafted but much more time-consuming long-form stuff.

So here's my request: Could the group not require all editors to read all of a story, but only require that one complete it and the rest only need to read enough of it to form a consensus of its quality? This seems likely to produce more long-form stories as a result without drastically improving workload over all readers.

I know it's sub-optimal, but it strikes me as a means of getting a lot more variety out of the posted submissions.

5845111

Alternatively, perhaps one week out of every month could be set aside for a longer work that's granted a month's duration to read through, in parallel with the shorter fics. It would still skew things in favour of said shorter fics, and thought it's divided up among four weeks, it would nevertheless require a slight increase in reading material. However, it takes the pressure off trying to cram an epic in a week, would prevent the vagueness and patchiness of relying on one user and hoping the others can glean enough in a skim-read, and would at least guarantee a regular supply of big stories.

5845302

Same problem of increasing readers' workload non-trivially.

5845111
5845302
Longer stories are a lot more work to read, and often require you to read through the whole thing to know it is quality. In particular, longer works need to pay off, but it is hard to know if a work actually pays off without reading it through. Long stories with disappointing conclusions are disappointing.

When I used to do reviews for the Royal Guard, longer stories were a major sticking point for this very reason. But the thing is, it is just easier to work on reviewing shorter works, and given that only a small fraction of works on the site are in fact longer, it makes sense that most works in the RCL would be shorter works.

Indeed, only 2,678 stories on the site are over 100,000 words (typical paperback novel), and 8,537 are over 40,000.

This is out of a total of 107,331 stories on the site.

Assuming story quality is distributed evenly over the length distribution of the site, we'd expect only about 8% of RCL stories to be novel-length (40k+ words), and 2% to be more than 100,000.

5845367

Right, but the whole point of the RCL isn't to review most stories, is it? Longer stories might be underrepresented by total quantity, and they take more effort to read but they also represent a lot more work to write as well, so it seems disheartening that it's less likely to be recognized under the current format. It doesn't help that you can only nominate an author for the RCL once, so it's unlikely for the story that gets picked to be the story that had the most time or effort invested into it because of that.

5845384
Just because something has more work put into it doesn't make it better.

I'll also note that you picked the wrong category. Slice of Life stories tend to be short by their very nature. Adventure stories are much more likely to be long, and a much higher fraction of them are long - 12 of the 30 adventure stories they've reviewed are novel length (40k+ words). 5 of 30 dark ones are. 2 of 7 mystery ones are.

They've reviewed 175 stories in total. Assuming random distribution of lengths, we'd expect about 4 100k word stories and about 14 40k word stories.

They seem to be marginally above those numbers, which would suggest that there is no net length bias in their reviewing.

Right and fair; I substituted Slice of Life as a good benchmark because it held a nearly a third of the total stories in here, which made it the easiest to sort by views for what I thought would be an indicative sample size.

EDIT also:

Just because something has more work put into it doesn't make it better.

But it also doesn't make it less deserving, and the current system is biased against it, by its construction. Not so that longfics can't get in, just that they have more of an uphill climb to reach a consensus regardless of quality. That remains the issue.

5845416
It isn't, though; as I noted, assuming a random distribution of quality and length, we'd expect about 4 100k+ word stories and about 14 40k+ word stories out of the 175 stories they've reviewed.

They've reviewed more than 4 100k word stories and 14 40k+ word stories, so there doesn't seem to be a significant bias against them.

5845416
Just from a quick count from the categories:

40k-100k:
Her Soliders, We
At the Mountains of Discord
Alarm Clock
Apotheosis
The Celestia Code
Eclipse
Equestria from Dust

100k+:
Flash Fog
Integration
Outside the Reaching Sky
Friendship is Optimal: Always Say No
The Colour You Bleed
The Mare in the High Castle
Stardust

That's 7 40k-100k word stories and 7 100k+ word stories. That's not a statistically significant deviation from what we'd expect.

5845359

Then extend it to once every eight weeks, or ten. If you want a fair appraisal of long fics, then the policy of leaving it up to one reviewer to actually read it is counterproductive. You're going to get an increased reading workload either way; the question is how to distribute it so that it's easier on the reviewers, without making it so rare that we needn't have bothered introducing the policy in the first place.

5845367

Assuming story quality is distributed evenly over the length distribution of the site,

Which isn't necessarily wrong, but it seems to me that the length of a fic could easily have a non-negligible influence on the overall distribution of quality among stories. It's insanely easy to churn out a bad fic of 1,000 words, since it takes barely any effort to produce that much writing. What could push the quality ratio in favour of the novel is the requirement for stronger motivation and commitment, both to read and to write, which could plausibly translate into more bang for the buck in terms of both author and audience investment. If that's the case (I grant this is untested and therefore hypothetical), following the ratio of long-fics-to-short-fics would actually be an unfair skewing and detrimental to this group's purpose.

If only there was a way to know the ratios beforehand...

On an unrelated note, since we're both interested in accuracy, we should really exclude any story that isn't complete, which yields these figures instead:

1,261 stories are over 100,000 words long and complete (2.2641% to 4 decimal places)

3,826 stories are over 40,000 words long and complete (6.8694% to 4 decimal places)

55,696 stories are complete (100.0000%)

5845465

Which isn't necessarily wrong, but it seems to me that the length of a fic could easily have a non-negligible influence on the overall distribution of quality among stories. It's insanely easy to churn out a bad fic of 1,000 words, since it takes barely any effort to produce that much writing. What could push the quality ratio in favour of the novel is the requirement for stronger motivation and commitment, both to read and to write, which could plausibly translate into more bang for the buck in terms of both author and audience investment. If that's the case (I grant this is untested and therefore hypothetical), following the ratio of long-fics-to-short-fics would actually be an unfair skewing and detrimental to this group's purpose.

Writing 40,000 words is only more time consuming than writing 1,000 words. There are people who, instead of producing a large number of short works, instead produce a single extremely long work, at times updating it every day or even multiple times a day with minimal editing.

Indeed, it is much easier to polish something extremely short than it is to polish something extremely long, because it is a lot less work to do so and there's a lot less parts to fit together. So in some ways, this actually makes it more likely an extremely short work will be of high quality - because it is easier to polish, it is easier to bring up to a consistently high level of quality.

Of course, I may be biased; my own RCL-inducted story (The Collected Poems of Maud Pie) is only 1,017 words long. Statistics, another RCL inductee, is 1,042 words long. Indeed, doing another count, there are 14 stories under 2k words that they've reviewed.

I do suspect that very short stories (under 2k words) are on average of lower quality than things above that length, I also think that's mostly because very short stories are A) harder to write than longer things (in terms of difficulty level) and B) the 1k word limit artificially skews lengths at the very bottom.

5845111
I see the value of spotlighting longer stories, but I think it's far, far outweighed by the review quality RCL brings to the table. It's also key that authors are rarely featured more than once; I look to RCL reviews to find high quality authors just as much as high quality stories. Being able to write an excellent short story and an excellent long story aren't the same, but they aren't mutually exclusive either. I'm far more likely to pick up a novel-length work from an author I've read and enjoyed before.

I don't want to draw out this debate any more than it already has been, though. We're speaking for a handful of reviewers with their own obligations, styles, and lives. How they do what they do is up to them; I'm content that they're willing and able to keep doing it.

5845485

Fair enough. I don't really have an answer to that. My own case was speculative at best, and it's really up to the reviewers' inclination first and foremost.

5846382

How they do what they do is up to them; I'm content that they're willing and able to keep doing it.

Oh, absolutely. I don't want to come across as telling them what to do. After all, I don't know what work goes on behind the scenes.

The funny thing about all this is, I'm currently reading not one, but two stories with six-figure wordcounts for the RCL.

So, I think there are two things we need to consider here: first, are longfics (which I'll broadly define as "over 100,000 words") better on average than shorter stories? And second, does the RCL consider enough longfics, based on both expected quality compared to shortfics (under 100,00k words) and in terms of number of longfics compared to shortfics which the fandom has produced?

Let's start with the relative quality question. I've been doing reviews in this fandom for quite a while, and according to FiMFic, I've reviewed 56 stories over 100k words (there might be a few missing stories I reviewed which aren't on FiMFic, but it's probably just a few, if any). Glancing through them, it looks like I've got 27 stories which I rated one or two stars (although my ratings for my blog don't directly correlate to the RCL's acceptance criteria, it'd be pretty unusual for me to rate something that I rated that low on my scale to still muster a pass from me in the RCL), and 10 that got four or five stars (again, not a perfect proxy, but a pretty good indicator that I'd approve it).

So based on that, it looks like about 27% of the longfics I read are ones which, at least in terms of quality, I'd be likely to judge RCL-quality. That's not too far from my overall ratio of 4-5 star fics to total fics reviewed, which suggests one of two things to me: either there's not a strong correlation between length and quality, or people tend to be worse at recommending (what I would judge to be) good longfics to me than they are at recommending shorter fics. I feel the former is most likely. As a proxy, then, I think that it's pretty reasonable for the RCL to feature longfics at about the same rate--relative to our number of features--as the fandom produces them.

And 5845452 has already made the point that we do feature longfics at about that clip.

So, I'm not worried that we're under-featuring longer stories, but that doesn't mean that concerns about review time aren't valid. Although not every one of us reads every feature, it's not uncommon for all of us to have to weigh in on a fic to settle on a consensus, and getting five people through a couple hundred thousand words is a lot more time-consuming than getting us all through just a few k. And we do take steps to expedite; sometimes we'll discuss conditional approval or disapproval as we read, so that if it's obvious that we aren't going to reach a "feature it!" consensus, we don't all have to read the whole thing, for example. But like was said upthread, you really shouldn't judge longfics by their first quarter; sometimes it turns out to be indicative, but I'm sure we've all seen very engaging stories go completely to pot. True, we might hope that whichever one of us recommended the story wouldn't do so if it had a ruinously lackluster back half... but then, if we trusted each other not to make those kinds of mistakes, we wouldn't need a second/third/fourth/fifth set of eyes on each fic to begin with, would we?

Now, with all of that said... if you feel like we've been under-representing the longfics that you'd prefer to see us interview authors about, our Story Recommendation Thread would be a great place to put forth some things you think we should read! We are but five readers, after all, and while we all try to do our share of exploratory reading, there's no substitute for a good recommendation. And if you're looking for us to suggest stories that you haven't already read? Well, even an informal recommendation of "this story looked interesting, but I haven't read it myself" to one of us couldn't hurt.

Thank you for bringing our attention to the desire for more longfic-based interviews! As I said above, there are two such that are under consideration at the moment, and while it's obviously still an open question whether both, either, or neither will pass our collective muster, it's good to know that we aren't wasting our time looking at them. And regardless, we'll continue to keep our eyes peeled for more.

5846950

Well, those two points you made certainly satisfy my main questions, at least unless someone comes up with more rigorous statistics, though I'm not holding my breath. A thorough and exacting reply. I'm convinced, at least.

Thank you for bringing our attention to the desire for more longfic-based interviews!

To be fair, there are exactly two users* on this thread who argued for said position, and having seen your and Titanium Dragon's counterarguments, I'm changing my position accordingly.

* Both, oddly enough, with Numbers in their usernames. What a conspicuously apt coincidence, considering the mathematics involved in this discussion.

Right, then.

Um... carry on, chaps!

5847197

To be fair, there are exactly two users* on this thread who argued for said position

True enough, but I'm interpreting the 11-as-I-type-this upvotes on MrNumbers' original post as meaning "we like longfics too, even if we don't necessarily have anything else to add."

That, or we've got nearly a dozen people 'round these parts who just really like to click those little green thumbs...

5847281

They are so addictively clickable. For other users, maybe. It's like cats and laser pointers, but for less intelligent human beings.

Excuse me, you've got a little thumb... right there... one sec...

Anyway, such compulsive mob mentality is so ridiculous and quaint, am I right?

Thanks for speaking up, all. It's great to get this sort of feedback; we hear surprisingly little from our readers about what they want.

I'm coming in late on the thread, so I don't have much of use to add beyond thirding that the statistics above are pretty compelling. (I wasn't aware of them myself!) However, I did want to be clear on one thing, just to slightly better address the original concern:

5845111

Hey, so I've heard that the group has it so that every editor has to read the entirety of a longfic before it's considered …

This isn't quite true, and while the difference is slight, it's important when you talk about whether or not it creates a bias. The actual process is:

1) Any curator can unilaterally nominate any work they find exemplary (regardless of length).
2a) Once nominated, it joins a queue of fics under consideration, and that is the stage at which all curators must read a fic.
2b) Even then, "all" is a little fiddly around the edges. What we're looking for is whether or not our collective votes on its quality push it past a scoring threshold — if it's getting good scores consistently, it can be approved with as few as the nominator plus two other readers. Typically we make a practice of all of us reading all the short ones, and we all try to read everything (they're good stories, brent!) … however, once a story has gone past the threshold, we would rather not delay its feature.
3) We read and discuss the nomination (asynchronously — there's no set order to the queue).

The difference here is that only one person has to like it to get a story in the queue. Now, once it's been nominated, it waits for a full up or down vote — but that's just a matter of delay, not consideration. It does takes longer for longfics to work their way through our process, but we have never, never dropped a fic for length reasons. The one time we explicitly picked a short fic over a longer one, what happened is that two different curators were planning to nominate different stories by the same author, and after some discussion of their relative merits we went ahead with the short one. That's an edge case, and I made sure in the feature to give a shout-out to the longer works as well.

Granted, there is an argument to be made that if we're not reading as many longer fics. they're not making it to our queue in the first place. As you note (5845359), though, changing that is a non-trivial amount of extra work … and by "non-trivial" I mean, each additional 200,000-word fic that one of us vets is (at average reading speeds) literally volunteering a full eight-hour workday to the reading list. If every 100k+ fic was exactly 100,000 words (which is silly, but at least gives us a lower bound), then distributing the work so there was one pair of eyes on all of FIMFiction's complete-as-of-this-post longfics would require the five of us to put in 126 full eight-hour days, each. (But hey, fundraise a hundred thousand bucks to cover the expenses of us quitting our day jobs and I think we'd be willing to try…)

The point is, the time commitment is the killer, as it always is with these things. I'm totally with you on that. It just seems like the sort of optimizations we're talking about are already mostly baked into the system — it pretty much either boils down to read more, read differently (dropping 50 shortfics to cram an extra longfic in somewhere?), or read more efficiently (e.g. following strong outside recommendations).

Still, it's a tricky balancing act and I appreciate you speaking up.

5851853

This is a fascinating comment, like a helpful behind-the-scenes DVD featurette. Thanks for taking the time to give us the sneak-peek. It's always interesting to see how the process works, and I'm entirely sympathetic to the need for a careful balancing act. (I couldn't imagine doing this sort of thing myself, to be frank!)