• Member Since 15th Jun, 2013
  • offline last seen Tuesday

Fimfarchive


Third-party archival project, do not contact Fimfiction staff for support.

More Blog Posts47

Sep
1st
2016

Fimfarchive 20160901 released! · 6:18pm Sep 1st, 2016

Fourteenth release with 140713 stories.


The complete archive.
Name: fimfarchive-20160901.zip
Size: 4.3 GB
Torrent: Takiri
Direct: Sweden
Magnet: magnet:?xt=urn:btih:25c7ede4f78672f627a697b14e0c34c3f77cbecb
MD5: f4ddc0e68558d1c0bbe6f8b728a31917

The xdelta3 patch.
Name: fimfarchive-20160525-to-20160901.xdelta3
Size: 0.4 GB
Torrent: Takiri
Direct: Sweden
Magnet: magnet:?xt=urn:btih:fd4ab37e04bf28b889132e72927d6620b53a3866
MD5: d593bdfd6b58470bccd10d47773ee72d


I will be replying to comments on Saturday.

Report Fimfarchive · 758 views ·
Comments ( 8 )

Thanks for these! :)

I just wish there was a way of backing up our fave lists.

Thanks!

/me starts on parsing the archive into the database...

Thank you for providing this invaluable service.

4185626
Download and install Calibre.

Download and install the plugin FanFicFare.

In FanFicFare's configuration dialog, go to personal.ini -> Edit personal.ini.

Ensure that "is_adult:true" is uncommented (assuming you don't want Mature stories omitted).

If the list you want to save isn't public (and you don't want to make it public, and don't mind the stories being marked as read for you), create a "[www.fimfiction.net]" heading with your username and password, using the syntax shown in the commented examples.

Save the changes.

Back here on Fimfiction, identify the ID of the list you want to save.

View the list, in List View mode with the Show Mature setting on (again, assuming that's how you want it).

Identify the number of pages.

Copy the URL "https://www.fimfiction.net/bookshelf/list ID/?view_mode=1&page=last page", with the relevant numbers filled in (e.g. https://www.fimfiction.net/bookshelf/139184/?view_mode=1&page=20). (Edit: URL format updated to work in 2018.)

(I say to start with the last page because I'm assuming that you're saving your own Favorites list, and so it's more likely for a story to disappear mid-process than for one to be added to the list. If you're saving a list belonging to someone else, you may want to start at page 1 and move forwards instead. You could also check the number of stories on the list before starting and after finishing to confirm that it hasn't changed.)

Back in Calibre, under FanFicFare's main menu, select "Get Story URLs from Web Page". The URL should autofill from the clipboard, so you should just need to press Enter.

Copy the story URLs from the resulting window. (You may need to wait a few seconds for it to appear; the status bar should read "Fetching Story URLs from Page...".) Paste them into a text editor.

(You may see a bunch of URLs to random chapters of stories rather than those stories' main URLs; this doesn't matter, as you'll get the full stories anyway.)

Repeat with the next page (or previous page, as applicable), and so on til you finish.

When you've finished, you should have one big list of the URLs to all the stories. Copy them to the Clipboard, and use FanFicFare's "Download from URLs" option to download them in one big batch.

When the stories have downloaded, you'll get a small prompt in the lower right of your Calibre window, saying "FanFicFare download complete" and asking "Proceed with updating your library?". If it mentions "bad updates", you'll probably want to select "View log" to see which stories didn't download for whatever reason (they'll be at the bottom of the log), before selecting "Yes" to update the library (assuming that at least some of the updates are good).

So, basically it can be done; it's just an annoying amount of work. Did I cover everything?

4186922
Thanks for this

I loaded this data up in Tableau, and wrote a blog post about it:

http://www.fimfiction.net/blog/669689/analyzing-fimfiction-data-with-tableau-and-postgres-part-1

Look at the Tableau visualization here:

https://public.tableau.com/profile/publish/fimfarchive/FimfarchiveAnalysis1

I'm happy to share the code and/or schema if anyone wants it.

And thanks for providing this data - it's been great to experiment with.

4185626
4185710
4186010
4186064

You're welcome! I'll make sure to keep rolling out releases. :twilightblush:

4186922

Annoying amount of work, but definitely worth it. :rainbowwild:

4188704

Sweet! It's nice to see that some stats have been created already! :twilightsmile:

Login or register to comment