Site Post » New Backend For Unread Chapters · 5:35pm Aug 12th, 2012
Please read bottom of post
I've redone the backend for unread chapters now because it was causing us awful performance issues. On our end this should be a lot faster. However, I haven't copied across the old "read" table to the "unread" one, so right now you'll have 0 unread chapters. I'll try and copy this across later. This is copied across now.
Anyway, there are more areas now where this data could "unsync". If you get weird issues where the number at the top is MORE than you actually have unread, please please let me know, and we should be able to find if there's anywhere I'm forgetting to update this value.
For future reference, this new system will nuke any unread chapters you have after 2 weeks. I think this is a fair length of time to give people to read chapters, and this ensure the system does not become explosive in content. Keeps it growing at a rate relative to user count, rather than time.
Ta.
edit: Just remembered, right now when you add a favourite, all chapters will be unread on the new system. Will update this when fixed.
Should be fixed
edit2: Also wanted to say I'm caching a lot more stuff now too, including PMs at the top and the read it later list. You shouldn't see any difference other than faster loading times, but if you do please tell me.
Update
Since countless people misunderstood what I'm doing, I'm going to attempt to explain it better.
First things first, for typical use of the system you should not notice any difference at all. Right now, I have not populated the unread table fully, so that little number in the user toolbar starts at 0 for everyone right now. Anyone chapter updates from now on will update that, and when I repopulate the table properly you'll have your normal number back.
When I spoke about nuking unread chapters, I did not mean marking all chapters as read. Basically, if you went to view unread favourites in the past, it would show every single story on your favs with 1 or more unread chapter. Now, after 2 weeks, stories will stop appearing in that list and stop being counted in the unread chapters count in the user toolbar. However, their unread chapters are still unread! You can still view them in your favourites list. Initially I decided 2 weeks was enough time for an unread chapter to be an important event, worth appearing in the toolbar and in the unread list. I'm now going to try and do it based on count, so that number will never go over some certain value, like 200 or something.
Also, viewing unread stories in your read it later list is not doable anymore. This whole change is to fix performance issues that that kind of browsing was causing. That use of the read it later system is not intended so I have no interest in "fixing" it. Browsing for read stories probably will come back though.
I was just about to ask about this, but now you answered my question.
*hugs knighty*
I t will nuke my unread chaoters... but i have about 300!
maybe this us for the best?
Ps, Second!
Well, there goes my practice of reading half an incomplete fic and waiting for it to complete before finishing it. Two weeks does not permit that to occur. I suppose I'll have to throw such items into my 'read it later' list in order to interact appropriately with them.
282191
Some people have asked for a notification/email when a story completes. If I do that, it'll solve your problem.
282191 Ugh, I agree... I read anywhere between 3-12 fics at once, most of them backlogs, and this system is working against me. If you could extend that to two months, than I might be able to reign in the number from '312 unread' to a manageable '31-33 unread.'
Hm. Just had a quick glance through, and everything's still there, and the unread stuff is still marked unread.
So the only difference is that I now don't have a number to tell me I have 87 chapters to read? Well, okay then.
282197
No, you will, it's just that your unread chapters data isn't populated, which that number feeds off.
Well, that's interesting. I had 0 unread, I read the three chapters of "Enter: Fluttershy", favorited it, and now it claims I have 1 unread. And Unread Favorites just shows "Enter: Fluttershy" with all chapters read. Since this falls under "weird issues where the number at the top is MORE than you actually have unread", thought I'd comment on it...
http://www.fimfiction.net/story/43396/Enter%3A-Fluttershy
Nuke un-read chapters in two weeks? This could be a HUGE problem for me. I don't think it's even physically possible for me to read almost 2000 chapters in that short time to not have to worry about it. And after its nuked, then what? How am I going to be able to tell apart what I have and have not read so far? I'm not entirely sure I understand why this particular system feature was implemented and why it is necessary. Care to elaborate?
~Signed, InfiniteBrony
282199
D'oh, not checking whether chapters are published. Thanks!
(and fixed that result for you)
Well, this might cause a few problems, but will probably fix more than it creates, at least for me. I already fill my 'Read Later' tab to prevent copious amounts of updates, because once I get over 25 fics I try to read at once, that's too many updates at a time.
This should also weed out the stories that I don't actually want to read.
282194
You're right, that really would solve my problem! I'd be grateful if a story completed email were implemented.
282201
No problem. I'd glad it was something easy.
What? No. Just... no! That'll make the whole thing completely useless. If the system is keeping track of copious numbers of unread chapters, it's because people have copious numbers of unread chapters to keep track of!
282210
Yeah and if people aren't reading them they obviously don't care very much for them do they? Anyway, nothing has been delete or cast in stone yet. A system which limits you to a max count rather than date would be better, I'm trying to work out logistics for it right now.
Well tbhis is going to suck with ALL THE UNREAD CHAPTERS I HAVE.
282200 282210
I think the problem was that the data was always being fetched server side, which would cause a whole lot of queries that weren't really necessary. So, he made it so that your unread chapters no longer fill your unread updates list after two weeks. They are still marked as unread [strike]and the updates are still under that section in the dashboard[/strike], it's just that the server no longer has to work harder to maintain that list of unread updates. Like knighty said, they're not populating the unread updates anymore. I'm guessing that those are fetched every page load to provide you with the number next to the star, which could be pretty expensive for all of the users' updates.
At least, that's what I took from it.
282214:
Thank You, I now understand
282211 282210
What about people who go on holiday for 2 weeks and can't get internet there?
282214
This makes considerably more sense and incredibly less harmful to me. I sincerely hope that this is the case, instead of the way I (mis)interpreted it.
Thank you for helping to clarify this point.
~Signed, InfiniteBrony
282218
I refer you to: 282211
282195
This.
282214
Wait, so they will be here, but not on the number bar?
Or will it make you unfav so you can't have them?
I'm confused.
282224
Oh, okay then.
Thanks.
Now, I'd best start working on that backlog.
I like the whole nuking idea. I just want to clarify one thing: will we be getting our old unread chapters back, and then have the two week limit start then? Or will the chapters which have a date older than two weeks get nuked right away?
282226
[strike]They'll be on the dashboard page, like they were before.[/strike] You'll still have the fav, it just will stop pulling the unread update data after two weeks. That means the number beside the star goes down and the update can't be directly accessed from the star (which is an unread updates link).
Further updates will put the story back in, but I'm not sure if it will skip the ones that are old. I think it may.
282211 Alrighty, then! That's a big lump o' worry offa my back! Time to get back to reading, though. I'm only on Chapter 7 of Cerulean1313's fic 'Unplanned Parenthood'.
Also, is there any way we can get a system for showing like fics again? I don't like getting most of my fics through random searches and the Featured Box. It makes me feel like I'm not reading the right things, or something. Disregard this, if you want. I'm feeling dizzy.
282214 Wait wait wait. I'm confused. So all that's changed is that the number that appears here i.imgur.com/RGmh4.png will go down after a chapter is two weeks old?
And 'unread' chapters such as these i.imgur.com/LD5dc.png -won't- be changed to 'read' after two weeks?
282232
So... if a fic updates, and 2 weeks later, you haven't read it, you can only find it on dashboard instead of under favs?
Or do they just stop been counted as waiting unread?
282211
Bzzzt. Sorry, wrong answer! Assuming things about your userbase is never a good thing, especially when you assume something negative, i.e. that people "don't care" about the unread fic updates they have simply because they don't get to them fast enough.
On the other hand, as long as the dashboard maintains unread updates and the chapters themselves remain marked unread, then there's no real issue, really. I personally have my Dashboard bookmarked for this site, not the front page, so I see any unread updates I may have.
If this helps site performance, however, this may just be a pill we have to swallow. I'm curious as to how much caching occurs in the backend that could potentially help some of these issues. I would love to get my grubby paws on the code to help the site out!
+1 from me for the "notify me when a story in my RiL list completes," by the way. That sounds like a great feature!
Re: The other bit of your comment — how about a system that just makes the favorites thing light up a different color when there are updates, instead of a number or a ceiling to the number of updates? So, it'd basically be a "hasUnreadChapters" boolean. Just a random suggestion.
282236
Switch the images, and yes, that's exactly it.
282237
They will still be under favs, sorted by update date. The system just stops counting them.
282236
When you go to view unread stories they also won't display there. But yes, those icons will ALWAYS show whether you've actually read them.
Regarding the dashboard, I forgot to update the query there. Fixed now, so they'll be empty.
Maybe I should have repopulated that list before rolling this out because people are incapable of reading a simple blog post.
282239
Yeah, my copy-pasting skills failed me there.
Ok. I'm fine with that change then. If it means less 502/504ing, definitely.
282242
So it's just, less numbers?
Why couldn't he have just said that?
Nuked made it sound much, much worse.
282211 A system with a limited count would be a lot better.
Just because I haven't read a story for two weeks doesn't mean I don't care for it anymore.
282246
Basically less time [strike]and bandwidth[/strike] spent collecting fic data to count unread chapters, which is what the system does to display the number, if I understand correctly. And I have no idea about nuked. Sounded pretty great, though.
282248
Nothing to do with bandwidth. Also, disk space used actually goes up with this (better performance all too often increases memory usage).
282249
My bad. I'm not too versed in running servers, I just know enough about computers to make a pretty good guess.
I'm fine with this as long as the two week pruning for unread chapters doesn't start immediately, it would be nice to have a small window to get the bulk of my older unread stories into the RIL list.
On an slightly unrelated note:
Where's the option to search for read/unread fics? I don't think that one caused much server strain, did it?
I used it to search through my read-later list (with a little URL hacking) for fics I started reading and wanted to finish but I didn't consider them good enough to add to favourites.
I have 102 unread chapters to get through quickly then, or else I'll never get found to reading them.
Are you planning on doing something similar with the Read Later list too then? I currently have 110 stories in that list.
For those who deeplink their URLs, this change removes &unread and thus prevents &read_it_later to be used akin to the old &tracking system.
Here's an interesting little glitch. Unfavorite one of your current favorites, then favorite it again. Though the chapters still have checkmarks indicating that they are read, it'll add them all to the unread count at the top until you unmark the chapters as read, then mark them again (or simply view the chapters in question).
282261
Should be fixed.
My unread updates count is mysteriously climbing for no apparent reason as of right now. It started at one a while back, now it's at 3. When I click the link there's nothing there.
EDIT: All fixed now.
282271
Yeah, that took care of it.
So is this why I can no longer search for stories I have read a chapter of? I found it convenient for stories that weren't of favorite status but I still wished to read (search by update date and read).
I'm not even going to ask how long it took to code all of this. Hell, it took me nearly two hours to set up a bare-bones web page in HTML for computer science.
On the other hand, maybe that's just my near-complete illiteracy with computer coding. Still seems like an obscene amount of work.
OK, so my English isn't the best, and I'm having some troubles understanding this, so can someone just clarify this for me: I currently have 604 unread chapters in my favorites and 143 stories in my "read later" list(blame FOE and Project Horizons...), will these stories remain there, even in two weeks?
282294
The stories aren't disappearing, no