knighty 4,166 followers

knighty is the creator and lead developer of Fimfiction. It's probably best if most requests for help go to other staff members

News Archive

  • 15 weeks
    New BBCode Tags

    Hey guys,

    One of the features in this new update was reader-side paragraph formatting. This helps improve consistency for readers across the site, especially for those of us who can’t stand reading indented text on a computer screen.

    However, one thing that wasn’t accounted for was the legitimate need for specific indenting of passages and for certain blocks of text to have no paragraph formatting. Some examples would be lyrics and poetry.

    Taking this into account, we have come up with a couple of new tags that remedy this situation which are documented below (copied directly from the bbcode guide)


    [indent] Indent

    The indent tag can be used to, unsurprisingly, indent portions of your text.

    [indent]The indent tag can be used to, unsurprisingly, indent portions of your text.[/indent]

    It also support levels of indenting

    Read More

    168 comments · 4,005 views
  • 15 weeks
    Fimfiction 4.0

    It’s been a very very long time coming, but we’ve finally updated the site again. this is by far the biggest update we have ever done. There is a cavalcade of new features but the biggest changes are under the hood and affect how easy it is to extend the site and performance. A change log of everything I can remember can be found below.

    There are bound to be unforeseen bugs. If you come across anything major please let us know in the comments (or preferably in the #site-help-and-dev discord channel).

    Miscellaneous / Site Wide

    • Dropped support entirely for pre-IE11
    • Updated inline searching across the site to order much better. Eg. Typing "Ra" into the tag selector actually shows Rainbow Dash first. On shorter lists like bookshelves, we use a different algorithm that lets you type things like "ril" and it’ll prioritise a shelf called "read it Later".

    Read More

    1,362 comments · 17,940 views
  • 16 weeks
    💩

    So, emojis are now supported all over the site. Go have fun and stuff.

    oh god what have we done

    192 comments · 3,577 views
  • 26 weeks
    TLS for all users

    We have implemented TLS site-wide as an unconditional redirect. (http -> https) This improves security site-wide for all users, and shouldn't have any negative effects, performance or otherwise.

    90 comments · 2,896 views
  • 36 weeks
    New Character Tags

    I have added a total of 70 new character tags to the site today. They can be found below:

    Read More

    337 comments · 9,749 views
  • 82 weeks
    Minor (nah, they're totally major) Updates

    Over the past 2 weeks I've been rewriting our entire ajax routing (or lack thereof) structure. That's not very important to anyone using the site, since in theory absolutely nothing should change for you with this update but as with any significant rewrite of code, there are bound to be bugs that were introduced. I've tested as thoroughly as I can so hopefully there will be little in the way of issues but if you get any, let us know here so we can fix them. The only change in theory should be better error reporting to you and more endpoints that are signed and therefore a bit safer/less prone to abuse.

    I've also added a couple of minor fixes. For one, comment jumping should work better on stories and user pages now. There was also another issue I fixed but....it'll be a surprise because I can't even remember what it was myself!

    Read More

    330 comments · 5,828 views
  • 84 weeks
    New tag information page

    I have added a tag information page (also accessible in the FAQ dropdown) which provides guidelines for what the various rating and category tags should be used for. Please note that this is not intended to change how these tags are used, this is just documented already-enforced rules.

    This should help answer questions like "does my story need to be tagged mature?", explain how teen+sex works, etc. Users have been requesting more details about what each tag is intended for for a while now, this finally provides that information in a (hopefully) easy to understand way.

    I would also like to remind everyone that if you encounter a story with mature content that is not marked mature, you should report it. This is always a rules violation, and is enforced strictly. We want to make sure that users that have mature filtered out are not exposed to such content.

    82 comments · 2,321 views
  • 85 weeks
    Imgur problems

    Imgur has blacklisted us for image embeds, stating that they do not allow images on their service to be used as content on other websites. Their ToS seems to confirm this, stating:

    Also, don't use Imgur to host image libraries you link to from elsewhere, content for your website, advertising, avatars, or anything else that turns us into your content delivery network.

    This apparently applies to users uploading images to post on other websites (including this one, and I would have to assume most others), which we were not previously aware of. Unfortunately I do not have a recommendation of an alternative site for users to host images they need to embed on our site or elsewhere.

    Read More

    221 comments · 8,292 views
  • 92 weeks
    Minor rule clarifications and additions

    A few minor adjustments and additions have been made to the general rules.

    Read More

    346 comments · 9,010 views
  • 94 weeks
    Adjustments to chapter formatting controls

    I have made some minor improvements to chapter formatting controls.

    1. "Flash of unstyled content" should occur less often when loading chapters, especially on mobile. This should also help chapter load times on slower devices.
    2. Four new themes have been added (if someone has any particular wants for a colorscheme we're missing, please leave a comment)
    3. Authors notes box and chapter selector are now dark on dark themes.

    I plan to do some additional work to solve the white background that appears above the ratings box on themes that don't already use a white background, as well as some work to have a dark version of that bottom-of-chapter ratings box for dark reader themes, so there isn't a big light box right below the chapter content, as I've noticed this can be annoying while scrolling, especially on mobile. Both of these things are slightly larger scope projects, however, and I wanted to get these three improvements out now.

    51 comments · 1,943 views
Jul
10th
2017

Site Update » Fimfiction API · 12:18pm July 10th

If you're not a developer you can probably ignore this post.

It's been like 6 years, but hey, things take time. The API is currently very WIP still but it's ready for people to get working on in our development chat room.

API documentation can be found at https://www.fimfiction.net/developers/api/v2/docs and you should join the Discord Chat and PM me to add you to the private API channel and I can help you get started. The functionality is very limited right now but I'm dedicating all my time to it at the moment and would love to have people add their input to the process.

Report knighty · 4,237 views · #api
Comments ( 59 )

Gosh darn it, all the cool stuff happening while I'm on holiday

Can't wait to give it a try when I get back :twistnerd:

I don't even know what an API is. :derpytongue2:

PresentPerfect
Author Interviewer

4597534
I think it's actually spelled "apple". :ajsmug:

Don't replicate existing functionality of the website.

Well, that basically kills off my interest. My main use for an API would be re-implementing the browse functionality so I can implement a "hide stuff I've already read" filter while still guaranteeing a minimum pagination increment.

(Something which can only be implemented in a userscript by sending new requests until enough unread stories have been accumulated... which would violate the "one request per click" rule unless I violated the spirit of it by having the script ask me to click a button five or ten times to queue up "technically allowed XHRs" before it goes to work.)

(Another interesting but disallowed tweak would be to implement infinite scroll using history.replaceState to make the approximate scroll position bookmarkable in a site-friendly way.)

knighty
Site Owner

4597541
Doing that would also require potential request spam so I'm not sure how the API would help there specifically? The rule regarding replicating existing functionality is really there to prevent people leeching off Fimfiction as a backend. I want to make it clear that that's not okay upfront so we don't get a situation like Twitter had where they had to screw over all their app developers because for years they let them use Twitter without people ever needing to visit the site.

No API for messages? You are dead to me.

I jest. It looks like a good start.

4597543

Doing that would also require potential request spam so I'm not sure how the API would help there specifically?

It wouldn't be a perfect fix unless you chose to expose read/unread information as a filterable attribute, but it'd be a great improvement if I could combine the higher pagination limit of List View with the presence of some kind of read/unread indicator from Full View. (Actual read/unread checkboxes or bookshelf membership would both work acceptably.)

(Heck, I could implement something I'd consider satisfactory without an API if the pagination limit were raised on Full View or the aforementioned cues were present in the compact view somehow.)

If nothing else, it'd be much more comfortable to reduce, by a factor of 5, the amount of time I spend blocked on the various latencies involved in requesting the next page of results. (Acquiring the button with the mouse and clicking it, waiting for the network round-trip, waiting for my sluggish browser to re-render the base portions of the site template, and re-acquiring the position of the entries so I can begin to do my skim-scroll reading again.)

That's why bookmarkable infinite scrolling also occurred to me as a desirable concept. Regardless of how perfect the solution is, the base problem is the mental drain of scrolling past literally hundreds of top-rated stories I've already read to find the handful sprinkled among them which I haven't yet read. Anything which can reduce that is welcome and will be appreciated.

I want to make it clear that that's not okay upfront so we don't get a situation like Twitter had where they had to screw over all their app developers because for years they let them use Twitter without people ever needing to visit the site.

A reasonable concern and one I don't mind. My biggest reason for hating Twitter for doing that doesn't really apply here.

(I don't use Twitter beyond responding to others specifically because there's no RSS feed and I don't want my e-mail inbox to devolve into "RSS without the organization". Fimfic's update notifications are perfectly fine.)

This is relevant to my interests. I must research it further.

knighty
Site Owner

4597572
The problem is the data doesn't exist to do that filtering and it's kind of difficult for it ever to exist based on how Elastic works here. It's not impossible but it's prohibitively difficult to make it a "let's make this one day" kind of project. There would have to be some kind of cached data for stories you've fully read in elastic but then whenever a new chapter is added, every single person who's ever read a chapter on that story needs updating and it just becomes a nightmare.

In the Quick Start Guide leads the endpoint link to a 404. might want to fix that

4597588

I can understand and accept that with regards to actually filtering by that data, but what about some kind of view that combines the increased pagination limit of List View chapter-level read/unread flags or bookshelf membership data that's already displayed in Full View?

I can already craft a terms-compliant userscript if I'm willing to accept "all results hidden" as a risk and that would be far more comfortable if I didn't have to click "next" so many times.

(And it doesn't strictly have to be read/unread status either. Whether my script is hiding based on the absence of unchecked read/unread checkboxes or the status of certain bookshelf toggle buttons, either would be a great help when I'm looking for something new to read. In fact, being able to hide based on bookshelves would work better, since it'd be more flexible, based on how I file away "I'm interested, but not right now" stories.)

knighty
Site Owner

4597645
That's a half solution though. If I send down 100 results based on the fast that only 10 will show that's extremely wasteful on resources. I'm absolutely not doing that. The reason I can show many more cards is because they don't select chapters. When stories can have up to 1,000 chapters, searching by longest on full view is already slowish. Its just not going to happen.

4597652

What about bookshelf membership? That's story-scoped, so it shouldn't have to select chapters, and I can still use that to build a userscript which hides "stuff I've read" based on how I assign it to bookshelves.

It'd even be more useful, because I could toggle between interpretations like "hide stuff I've shown interest in" (present in any shelf) and "hide only the stuff I've fully read or plan to be caught up on soon" (by personal policy, stuff in my tracking list).

knighty
Site Owner

4597666
If you want to read stuff you haven't read in bookshelves...use the unread chapters functionality?

4597671

You misunderstand.

I'm saying that the base functionality I desire is "Hide results that I've put in any bookshelf" and it'd be a nice secondary feature to some times be able to say "On second though, I don't mind if stuff I bookmarked long ago shows up in this 'filter for tagX and sort by rating' query as long as it's my "Read it Sooner" or "Read it Later" shelves. It'll save me having to make two or three queries."

EDIT: I clarified the "bonus feature" part with the underlined text.

*sees new blog from knighty*

If you're not a developer you can probably ignore this post.

thumbs.dreamstime.com/z/school-bullies-lonely-kid-5926115.jpg

Hmm. API...API...Armor Piercing Incendiary?

4597652
Except that if you don't implement the suggestion you are sending those 100 responses ANYWAY! So, it is a no-change situation. In one, 100 results are displayed in which the user ignores 90, and the other only 10 ten are displayed, but in both cases all 100 responses are sent (but one is much more user friendly).

Also, as is the case for ANY situation in which you choose to implement a feature for which no data currently exists (i.e., a flag of some kind), it doesn't matter. The information -- going forward -- will slowly accumulate. I can live with not being able to sort/filter old stories knowing that going forward I will be able to do so, even if it is limited to only NEW stories or to stories I read only after that implementation. Hey, if nothing else, I can quickly reload an old story, thus setting its relevant flag as needed.

Meanwhile, you're doing a great job with the site. Thanks for all your hard work. I really have no complaints about any of it current features.

Learn how to work with the Fimfiction API by reading the endpoints documentation.

(from the API Documentation page)

The endpoints -insert whatever word would be used for a word with a url linked with it- on the page leads to our beloved Derpy and the infamous 404 page. Just wanted to give you a heads up!

4597671
I know this is a bit off-topic but I think this site needs a really good app for smartphones and tablets. It's kinda a pain in the ass the way it is right now.

4597960

Not really off-topic at all. To my understanding, this API is for app development, unless I somehow completely missed the point of it. I look forward to an eventual app!

knighty
Site Owner

4597960
Not likely to happen. I have worked on an app twice and decided it's not worth it currently. I think the mobile site is in a good spot where it is right now and I say that as someone who mainly uses Fimfic nowadays on a phone. I have a long term game plan there.

4597970
I agree, once you get Use to it reading on the phone via a browser is really good. I read almost everything this was and find the phone version quite pleasant.

4597543

The rule regarding replicating existing functionality is really there to prevent people leeching off Fimfiction as a backend.

If the worry is that users skipping the site would deprive FimFiction of revenue to operate, have you considered using per-call payments? Something like 21.co's Bitcoin Payable API, where each HTTP request has to have an open bitcoin payment channel and pay a certain number of Satoshis per API request?

Or to require app developers to have their users bring their own per-user API keys...and only grant per-user API keys to people who fund you on Patreon, to compensate for the lost ad revenue?

like I don't ignore your posts anyway

... wait shit

Having been tinkering off-and-on for the past year on an app that syncs my bookshelves with corresponding Calibre metadata, I am thrilled to see this finally become a thing. I'll be happily converting all my site-scraping code to proper API usage ASAP.

Looks overcomplicated, honestly. But I don't really care for that sort of paranoid, micromanagey stuff like invite only API keys, so take that for what you will.

knighty
Site Owner

4598351
Not sure how any of this is over complicated... The invite only is just while it's still work on progress

knighty
Site Owner

4598193
Would you like access? Like me on discord if you do

Would you be upset if I scraped the entire site periodically, saving any new or altered stories and chapters as an archive?

knighty
Site Owner

4598443
Kind of, yes. Considering somebody already does that, meaning it'd just be double the load for zero benefit.

4598494
Are you talking about the user Fimfarchive? If so, would you happen to know the methods Fimfarchive uses? Seeing that s/he probably does not use the API yet, the archives are probably collected in a very roundabout way.

That being said, I have a personal suggestion that will probably be disapproved of, which is that I feel that deletion of anything should be disallowed. Instead, there should be a system somewhat like Github with version control, where previous versions of chapters can be found by going into earlier "commits." Or maybe something like the Deviantart stash, where stories the author wishes to hide can be placed in and accessed by those who really want to, but cannot be truly deleted. This altogether should negate the need for some users to put a huge load on the site by archiving everything.

4598367
I guess I'm just complaining about OAuth and all that token stuff I mean. It's standardized, so I suppose you can mostly ignore it behind a client API, but I think it requires every client application developer to also maintain an active web server. I'd just use a digital signature to verify logins and/or clients, so instead of like...

client -> user | client id plus scopes, plus URL for my server that I totally have
user -> fim | all that stuff
user <- fim | request for approval
user -> fim | okay
user <- fim | go to this URL
user -> somewhere | a token to get a token, sent to aforementioned URL
somewhere -> client | here's the token, which I totally didn't steal

and from then on:

client -> fim | that token-getting token plus client secret that can't be secret unless client is closed source
client <- fim | the real session token, logged in as that user with allowed scopes

I'd have something like...

client -> user | client public key plus scopes, and no URL
user -> fim | all that stuff
user <- fim | request for approval
user -> fim | okay

and from then on:

client -> fim | signature of client key
client <- fim | session token logged in as that user with allowed scopes

Since public keys don't have to be secret, It avoids needing that secret token-getting token and a dedicated, secure web server. It also doesn't (technically) force all clients to be closed source.

The rest of the API seems pretty chill. Though (after complaining about overcomplication) I'd ask for support for sending actual patches, as in... the client remembers the previous chapter contents, and only sends a diff from that, instead of the whole new 40K chapter. That actually gets annoying when you just uploaded your chapter, then you see a one-word typo and you gotta wait 20s to upload the whole chapter again. It's a minor complaint though.

knighty
Site Owner

4598684
Frankly I have no idea what system you just described there. There's a reason OAuth is well used, because it's battle tested and secure. Adhoc solutions wouldn't add anything other than potential concerns. OAuth 2 is ludicrously simple to set up, and if you don't want to do anything with users, you just use client credentials which does not require a secure server, just ability to make SSL requests.

4598692
Yes, I'm aware that if you just want to read stories and not change anything, you don't need to authenticate.

Anyway feel free to ignore me. I just won't be able to write clients for it any time soon. (Trust me that's not a big loss.)

4598494
Are you talking about fimfetch, or somebody else?

Thanks for this post. I might want to check it out later this year or next year when I have a better understanding of app development and Xcode. Seriously, FimFiction needs its own iOS and Android app and I plan to make one.

4598372
Thanks. I'll look into downloading Discord and setting up an account in the next couple days. Not sure how often I'll be able to participate (thin walls and awkward working hours make voice chat a dubious prospect), but I'll give it a go.

Yay! I'm excited to try this out!

knighty
Site Owner

4599150
It's just text chat

maybe this could mean the fimfiction app can soon become a reality? sure a browser on the phone is good, but a dedicated app is usually better :)

Comment posted by Emtu deleted July 14th

4598609

For each release I currently traverse all story IDs using the old API. I do this over the course of an entire month. When a new or updated story is detected its contents is fetched from the story downloader. The stories are then converted to EPUB and passed through Calibre to ensure they work on most e-readers.

The new API will in time make it possible to include more meta data. Also, to be kinder to the server. :rainbowwild:

4602242
Is there a way to customize the epubs on download? I find that in my epub readers (I've tried several) there are no gaps between paragraphs or indentations, thus it is difficult to differentiate where a paragraph starts and ends.

Login or register to comment