Reset Password
Resend Verification Email
0
157
101
Joined June 2011
New Backend For Unread Chapters · 5:35pm
Please read bottom of post
I've redone the backend for unread chapters now because it was causing us awful performance issues. On our end this should be a lot faster. However, I haven't copied across the old "read" table to the "unread" one, so right now you'll have 0 unread chapters. I'll try and copy this across later. This is copied across now.
Anyway, there are more areas now where this data could "unsync". If you get weird issues where the number at the top is MORE than you actually have unread, please please let me know, and we should be able to find if there's anywhere I'm forgetting to update this value.
For future reference, this new system will nuke any unread chapters you have after 2 weeks. I think this is a fair length of time to give people to read chapters, and this ensure the system does not become explosive in content. Keeps it growing at a rate relative to user count, rather than time.
Ta.
edit: Just remembered, right now when you add a favourite, all chapters will be unread on the new system. Will update this when fixed.
Should be fixed
edit2: Also wanted to say I'm caching a lot more stuff now too, including PMs at the top and the read it later list. You shouldn't see any difference other than faster loading times, but if you do please tell me.
Update
Since countless people misunderstood what I'm doing, I'm going to attempt to explain it better.
First things first, for typical use of the system you should not notice any difference at all. Right now, I have not populated the unread table fully, so that little number in the user toolbar starts at 0 for everyone right now. Anyone chapter updates from now on will update that, and when I repopulate the table properly you'll have your normal number back.
When I spoke about nuking unread chapters, I did not mean marking all chapters as read. Basically, if you went to view unread favourites in the past, it would show every single story on your favs with 1 or more unread chapter. Now, after 2 weeks, stories will stop appearing in that list and stop being counted in the unread chapters count in the user toolbar. However, their unread chapters are still unread! You can still view them in your favourites list. Initially I decided 2 weeks was enough time for an unread chapter to be an important event, worth appearing in the toolbar and in the unread list. I'm now going to try and do it based on count, so that number will never go over some certain value, like 200 or something.
Also, viewing unread stories in your read it later list is not doable anymore. This whole change is to fix performance issues that that kind of browsing was causing. That use of the read it later system is not intended so I have no interest in "fixing" it. Browsing for read stories probably will come back though.
Cancel
Send
Comments ( 190 )
I was just about to ask about this, but now you answered my question.
*hugs knighty*
I t will nuke my unread chaoters... but i have about 300!
maybe this us for the best?
Ps, Second!
Well, there goes my practice of reading half an incomplete fic and waiting for it to complete before finishing it. Two weeks does not permit that to occur. I suppose I'll have to throw such items into my 'read it later' list in order to interact appropriately with them.
>>282191
Some people have asked for a notification/email when a story completes. If I do that, it'll solve your problem.
>>282191 Ugh, I agree... I read anywhere between 3-12 fics at once, most of them backlogs, and this system is working against me. If you could extend that to two months, than I might be able to reign in the number from '312 unread' to a manageable '31-33 unread.'
Hm. Just had a quick glance through, and everything's still there, and the unread stuff is still marked unread.
So the only difference is that I now don't have a number to tell me I have 87 chapters to read? Well, okay then.
>>282197
No, you will, it's just that your unread chapters data isn't populated, which that number feeds off.
Well, that's interesting. I had 0 unread, I read the three chapters of "Enter: Fluttershy", favorited it, and now it claims I have 1 unread. And Unread Favorites just shows "Enter: Fluttershy" with all chapters read. Since this falls under "weird issues where the number at the top is MORE than you actually have unread", thought I'd comment on it...
http://www.fimfiction.net/story/43396/Enter%3A-Fluttershy
Nuke un-read chapters in two weeks? This could be a HUGE problem for me. I don't think it's even physically possible for me to read almost 2000 chapters in that short time to not have to worry about it. And after its nuked, then what? How am I going to be able to tell apart what I have and have not read so far? I'm not entirely sure I understand why this particular system feature was implemented and why it is necessary. Care to elaborate?
~Signed, InfiniteBrony
>>282199
D'oh, not checking whether chapters are published. Thanks!
(and fixed that result for you)
Well, this might cause a few problems, but will probably fix more than it creates, at least for me. I already fill my 'Read Later' tab to prevent copious amounts of updates, because once I get over 25 fics I try to read at once, that's too many updates at a time.
This should also weed out the stories that I don't actually want to read.
>>282194
You're right, that really would solve my problem! I'd be grateful if a story completed email were implemented.
>>282201
No problem. I'd glad it was something easy.
What? No. Just... no! That'll make the whole thing completely useless. If the system is keeping track of copious numbers of unread chapters, it's because people have copious numbers of unread chapters to keep track of!
>>282210
Yeah and if people aren't reading them they obviously don't care very much for them do they? Anyway, nothing has been delete or cast in stone yet. A system which limits you to a max count rather than date would be better, I'm trying to work out logistics for it right now.
Well tbhis is going to suck with ALL THE UNREAD CHAPTERS I HAVE.
>>282200 >>282210
I think the problem was that the data was always being fetched server side, which would cause a whole lot of queries that weren't really necessary. So, he made it so that your unread chapters no longer fill your unread updates list after two weeks. They are still marked as unread [strike]and the updates are still under that section in the dashboard[/strike], it's just that the server no longer has to work harder to maintain that list of unread updates. Like knighty said, they're not populating the unread updates anymore. I'm guessing that those are fetched every page load to provide you with the number next to the star, which could be pretty expensive for all of the users' updates.
At least, that's what I took from it.
>>282214:
Thank You, I now understand
>>282211 >>282210
What about people who go on holiday for 2 weeks and can't get internet there?
>>282214
This makes considerably more sense and incredibly less harmful to me. I sincerely hope that this is the case, instead of the way I (mis)interpreted it.
Thank you for helping to clarify this point.
~Signed, InfiniteBrony
>>282218
I refer you to: >>282211
>>282195
This.
>>282214
Wait, so they will be here, but not on the number bar?
Or will it make you unfav so you can't have them?
I'm confused.
>>282224
Oh, okay then.
Thanks.
Now, I'd best start working on that backlog.
I like the whole nuking idea. I just want to clarify one thing: will we be getting our old unread chapters back, and then have the two week limit start then? Or will the chapters which have a date older than two weeks get nuked right away?
>>282226
[strike]They'll be on the dashboard page, like they were before.[/strike] You'll still have the fav, it just will stop pulling the unread update data after two weeks. That means the number beside the star goes down and the update can't be directly accessed from the star (which is an unread updates link).
Further updates will put the story back in, but I'm not sure if it will skip the ones that are old. I think it may.
>>282211 Alrighty, then! That's a big lump o' worry offa my back! Time to get back to reading, though. I'm only on Chapter 7 of Cerulean1313's fic 'Unplanned Parenthood'.
Also, is there any way we can get a system for showing like fics again? I don't like getting most of my fics through random searches and the Featured Box. It makes me feel like I'm not reading the right things, or something. Disregard this, if you want. I'm feeling dizzy.
>>282214 Wait wait wait. I'm confused. So all that's changed is that the number that appears here
And 'unread' chapters such as these
>>282232
So... if a fic updates, and 2 weeks later, you haven't read it, you can only find it on dashboard instead of under favs?
Or do they just stop been counted as waiting unread?
>>282211
Bzzzt. Sorry, wrong answer! Assuming things about your userbase is never a good thing, especially when you assume something negative, i.e. that people "don't care" about the unread fic updates they have simply because they don't get to them fast enough.
On the other hand, as long as the dashboard maintains unread updates and the chapters themselves remain marked unread, then there's no real issue, really. I personally have my Dashboard bookmarked for this site, not the front page, so I see any unread updates I may have.
If this helps site performance, however, this may just be a pill we have to swallow. I'm curious as to how much caching occurs in the backend that could potentially help some of these issues. I would love to get my grubby paws on the code to help the site out!
+1 from me for the "notify me when a story in my RiL list completes," by the way. That sounds like a great feature!
Re: The other bit of your comment — how about a system that just makes the favorites thing light up a different color when there are updates, instead of a number or a ceiling to the number of updates? So, it'd basically be a "hasUnreadChapters" boolean. Just a random suggestion.
>>282236
Switch the images, and yes, that's exactly it.
>>282237
They will still be under favs, sorted by update date. The system just stops counting them.
>>282236
When you go to view unread stories they also won't display there. But yes, those icons will ALWAYS show whether you've actually read them.
Regarding the dashboard, I forgot to update the query there. Fixed now, so they'll be empty.
Maybe I should have repopulated that list before rolling this out because people are incapable of reading a simple blog post.
>>282239
Yeah, my copy-pasting skills failed me there.
Ok. I'm fine with that change then. If it means less 502/504ing, definitely.
>>282242
So it's just, less numbers?
Why couldn't he have just said that?
Nuked made it sound much, much worse.
>>282211 A system with a limited count would be a lot better.
Just because I haven't read a story for two weeks doesn't mean I don't care for it anymore.
>>282246
Basically less time [strike]and bandwidth[/strike] spent collecting fic data to count unread chapters, which is what the system does to display the number, if I understand correctly. And I have no idea about nuked. Sounded pretty great, though.
>>282248
Nothing to do with bandwidth. Also, disk space used actually goes up with this (better performance all too often increases memory usage).
>>282249
My bad. I'm not too versed in running servers, I just know enough about computers to make a pretty good guess.
I'm fine with this as long as the two week pruning for unread chapters doesn't start immediately, it would be nice to have a small window to get the bulk of my older unread stories into the RIL list.
On an slightly unrelated note:
Where's the option to search for read/unread fics? I don't think that one caused much server strain, did it?
I used it to search through my read-later list (with a little URL hacking) for fics I started reading and wanted to finish but I didn't consider them good enough to add to favourites.
I have 102 unread chapters to get through quickly then, or else I'll never get found to reading them.
Are you planning on doing something similar with the Read Later list too then? I currently have 110 stories in that list.
For those who deeplink their URLs, this change removes &unread and thus prevents &read_it_later to be used akin to the old &tracking system.
Here's an interesting little glitch. Unfavorite one of your current favorites, then favorite it again. Though the chapters still have checkmarks indicating that they are read, it'll add them all to the unread count at the top until you unmark the chapters as read, then mark them again (or simply view the chapters in question).
>>282261
Should be fixed.
My unread updates count is mysteriously climbing for no apparent reason as of right now. It started at one a while back, now it's at 3. When I click the link there's nothing there.
EDIT: All fixed now.
>>282271
Yeah, that took care of it.
So is this why I can no longer search for stories I have read a chapter of? I found it convenient for stories that weren't of favorite status but I still wished to read (search by update date and read).
I'm not even going to ask how long it took to code all of this. Hell, it took me nearly two hours to set up a bare-bones web page in HTML for computer science.
On the other hand, maybe that's just my near-complete illiteracy with computer coding. Still seems like an obscene amount of work.
OK, so my English isn't the best, and I'm having some troubles understanding this, so can someone just clarify this for me: I currently have 604 unread chapters in my favorites and 143 stories in my "read later" list(blame FOE and Project Horizons...), will these stories remain there, even in two weeks?
>>282294
The stories aren't disappearing, no
Heh. Two MONTHS isn't enough for me to get to unread chapters. As long as they're still there and accessible, I guess it's fine (as I understand it, you're just getting rid of the constant notifications for everything), I'm just going to forget that I have to read them at all eventually.
Could somebody explain this to me?
My Favorites thing doesn't have a number on it anymore - it just says "Favorites"
>>282330:
Don't worry, it'll go back to the number of un read stories later when he fixes it
>>282322:
I second this
>>282334 Oh, that's a relief! Thanks!
My favs are all gone!
I'm a bit disappointed, as it should be fixable by a better database schema instead of reverting to nuking or hard limits, but I understand the need for a quick temporary fix.
Hopefully in the future someone can work out a better schema that can properly handle these cases. I'm not familiar with how Knighty has it set up now so I can't say how much work it would be. An upgrade like that could be pretty major.
>>282359
They're not limits that should affect a normal user. Much like limiting comments to no more than 1 every 30 seconds.
>>282376
So... the number will be there, but it will only go upto 200 (Or whatever number)
So if you had say... 5 more than the limit, and you read 5, it would still be at the limit because it only shows that many so they fall into it.
For example: 100 updates visable, 105 updates there, read 3 and it stays at 100 because it's still over.
>>282384
No.
If I could know what number it was over 100, why would I not be displaying it?
>Also, viewing unread stories in your read it later list is not doable anymore.
Well, that's annoying. Now I'm going to have to move them all to the favourites list, even if they aren't really a "fav", and I'm just reading them...
I also don't see how it's going to give better performance. The RiL list only searched by unread when I (manually) asked it to. The Favourites list has to update all the time (for the counter)
Surely having everything in one big list, and constantly querying that list is qoing to be worse than having it split in two, one of which is only checked rarely.
>>282387
That.
>>282391
Yeah and for some users those queries cost up to FIVE SECONDS to do. I'd ask you to not comment on how you think the backend is affected by this without knowing anything about how it works.
These changes, there are so many.
Hold on, this is an actual question here Knighty. What happened to the words read? Did.....you take that out on purpose? Or like, is it just a malfunction. I'm a little concerned because there are readers here who have more than 10+ million words read and I'm not sure they'll be too happy with losing that displayer.
>>282394
My issue is then what is the point of the read it later list. I have a lot of stories I have not read it but intend to read later. If you do this, then I won't be able to find stories in my read it later list, since I only put stories I'm interested into my favourites list while the read it later stories are tentative at best.
All this is fine by me. In the long run, I don't read much on here anyway. I just write.
>>282394
*shrug*
Okay.
Is it possible to get stuff moved from the RiL list to favs automatically, or should we do it manually?
I got on 3 minutes ago, and all I see is "1 unread chapter". I slapped myself so hard on the forehead that my couch flipped over with me. I had thought that I pressed Mark All Read again XD
Without Mature: ~384
With Mature: ~672
I find at least one big problem here:
Cutting off favorites appearing after two weeks + not being able to search for unread favorites anymore = Long treks through my favorites list whenever it takes me a while to get to a chapter of a fic. Which actually happens quite often to me. I feel that you're assuming things about your userbase that aren't necessarily correct, and making the user experience worse for some of us.
Also, I liked the estimated words read count. It was fun seeing my count go up as I read fics. So what if some people artificially increase them? It's not like it'll hurt anyone.
>>282295
Mr. knighty, sir? Like >>282340 all my favorites are gone
.. Can I have them back, pleeeeeaseee? 
So, the reason that all of my read favorites are mashed in with the unread ones is because of not having the unread table populated, right? Because currently my favorites and all favorites buttons are identical.
>>282405
This is his site and all, but he's seriously being unreasonable for this. (I have a few other words in mind but my manners, internet included, aren't going to allow me to voice or write them.)
Is he going to at least ask us if we'd prefer these changes? I did like the total words read, because it gave me a feel of how much I've read since joining this fandom. But now with all of these new changes, I just might give Equestria Daily a visit. And I dislike Equestria Daily.
...
Can I have clarification on what this means? Because I don't think I'm getting it.
I liked the Words Read. Every time I'd go on someone's page, it gave me a general idea on how experienced they are in this fandom's fanfic community. Besides, I was 2,500,000 off from getting to the 10,000,000 benchmark.
This complicates things a lot. I favorite stories that are one shot and the like, and I had a few unread chapter that are months old, now I have search them again...
knighty...
ye?
you broke the game!
>>282415
Yeah, this happens with every update.
It's his site, if he doesn't want it, and sees no reason to keep it, it's gone.
>>282376
Based on what data? Obviously if it's a problem, then a lot of "average users" are going to be losing a lot of entries in their list, or it wouldn't be saving any server load, and once those entries fall off, most users will never find them again (unless they make room in their list AND the story gets updated, I presume). Searching through dozens of pages of old favorites for stories that have fallen off the list just isn't something people are going to naturally do.
If you want to get technical, it's possible to say that a lot of people's favorites they don't immediately jump on actually belong in "Read it Later" instead. Maybe stories that fall off the updates list could fall into Read It later or a new list linked at the bottom of page twenty of the updates list? But then, why do that at all when you could make it seamless with the existing list? No one would care at all if you just cached entries past two hundred and didn't count them.
Wait, did I say page twenty? I was thinking of stories, but you said updates. If something is hanging around my unread list, it probably has five to ten updates waiting, so actually it could be as little as two to five pages.
>>282415
Yeah, estimated words read is fucking integral to site functionality, and didn't have any side effects like adding an e-peen meter that made people just click on chapters to increase it. I admit it, I'm an incompetent admin, you got me!
>>282435
They don't fucking fall off. Holy shit can people please take 5 fucking minutes to read a simple blog post before saying stupid crap? The only thing that happens is that they stop showing up when searching for unread chapters. THAT'S IT. Why would I added a new list to do exactly what the full favourite list does?
>>282438
Except that some of us use that count for other things than e-peen. Some of us actually liked seeing how far we'd come. If you're so worried about it causing problems, why not make it only viewable by the user, so that those who get some use out of it can still get some use out of it?
Must the solution to "someone might abuse this for some marginal thing" always be "cut the content even if it's useful for some people?"
-Stars aren't representative! -> Replace with Likes/Dislikes that are even less representative.
-People aren't using favorites and tracking like they should. -> Make them only one list.
-Some idiots artificially inflate word counts -> Remove it for everypony.
This is why we can't have nice things.
Somepony's getting angry.
Must feedback about site changes be retaliated against with swearing and exaggerated hypotheticals? We're all just users that are curious and care about the site. Much as you are, knighty. You put countless hours of effort into the site to keep it running, and we're all grateful for it. We're just trying to understand the thought process behind some things, and it's off-putting to read replies like >>282442 and >>282438 to posts where people are just trying to understand things.
Yes, estimated words read is a metric that means nothing in the grand scheme of things. Yes, people can artificially inflate it. Why get so defensive over it, though? I didn't think it was a big problem, really. Was it causing performance issues? If so, than that's really all that needs to be said. "I didn't care for it," however, is hardly a reason to take features away from users, and even less so a reason to bite other users' heads off.
>>282443
Everything I do for the site is because I've seen issues happening. Not to mention the first item you list, likes/dislikes is a well understood problem and has been done be many sites including Youtube.
It's like some people forget I'm a user on this site too. And I have been for over a year, and I watch everything that happens every day.
>>282448
On some peoples profiles, yes it was causing issues.
Uh... I think I'm experiencing a bug. My favorite list shows one story only. If I go to "View all favorites" it show that only one story, But if I manually head to one of my favorited stories it shows as already favorited (with the Notify updates checkmark).
I'm speaking of stories I read this week, and favorited 5 or 6 days ago...
Is this expected functionality mr knigthly? I read the original post and all comments, and this doesn't seem right. If it is, sorry to bother.
>>282452
Ah, well that's reasonable enough then. It's just that sarcastic quips like >>282438 make it come off as "some people misbehaved so now no-one gets it."
It says I've got 12 unread chapters. Anyone got a fix?
Before update: 192 unread chapters
After update: 306 unread chapters
I'm also so confuzzled....
I freaked when I saw 85 unread chapters. I need help
I could use that I have 12 new updates on stories xDD
i have 11 or so, but it says 43
>>282416
I second the unread stories thing. My whole reason for using read later is so I can use it to have a list of unread stories I intend to read when I have the time. I thought that was the whole point of it.
Also, on an semi-related note, my unread chapter count says 79 when it should only be at 24(I have a long story that I have half read, but I don't have the time to catch up with it at the moment).
I think most of all unread chapters I have will go down. I have fictions that I shuld have started reading mounths ago.
*Sigh*
It is never easy.
I think I was around 19?? unread chapters. It says 1897 now. It is imposibel for me to read that much on multichapter stories, why must I find so many fictions that I want to read.
I haven't read any of the comments, so this has probably been dealt with already (I apologize in advance for being lazy and asking questions that have likely already been answered
). Anywho, where'd the words read go? I thought it was a nice feature. Were they causing problems for the site? 
Truth to be told i don't need a number of how many unread chapters there are, all what i really need is a way of telling if there are any new chapters since the last time i clicked on it. I don't care if there is 30 or 300 unread chapters, but when the 30 change to 31 i care.
Shouldn't that be easier for the server?
So now I have to read all of my updates within two weeks, or risk forgetting about them (even though they're on my favorites list).
Another excellent site optimization with no downside. As someone who reads quite a lot on this site the proposed change to a 200 unread favorite limit seems more than fair. I honestly have to wonder about a users reading habits if they can rack up such a large backlog after moving them from the provided Read It Later section.
Someone mentioned earlier that they have 2000 unread chapters. I have to wonder how that is possible unless people are just reading the first chapter of a story, adding it to the favorites list, then forgetting out it.
cool thanks Knighty!
>>282438
.....Yeah see that was uncalled for. I mean, sarcasm and use of profanity, really? Is that how you really are? I respect you as an admin and fellow user, but that was seriously a little low and childish. I could practically feel the venom in those typed words. But regardless, to some people, myself included, it does give some feel of accomplishment. This isn't a classroom, so one bad apple shouldn't ruin the bunch. Some of us had a sort of a complex process of joining bronydom, and the word count actually helped me to understand how I've changed from the guy who thought of bronies as weird to the guy who started writing fanfiction for the thing. I think I speak for Americans here when I say I want a democracy (Polls maybe?), not a constitutional monarchy or a government system like that of King Henry VIII, in which case you are the famous man, himself.
That aside, is there some sort of family or personal issues going on? I'm seriously concerned because in the weeks that I've used this, you've been a pretty cool-headed guy. That one incident with the insult to Mic aside. (Honestly I could care less) But it feels like you're down in the dumps. You alright, man?