Fimfarchive 20221201 released!

Thirty-eighth release with 207422 stories.

---

The complete archive.
Name: fimfarchive-20221201.zip
Size: 7.3 GB
Torrent: Official (SE)
Direct: Official (SE)
Magnet: magnet:?xt=urn:btih:8fed7a02929f7c06057e7d18cb96e0ce3927ce35
SHA1: 3b67ed1defb4b71278eef2c390337ca866a9bace

The xdelta3 patch.
Name: fimfarchive-20220901-to-20221201.xdelta3
Size: 0.4 GB
Torrent: Official (SE)
Direct: Official (SE)
Magnet: magnet:?xt=urn:btih:41971efa246cd4b4b0a0f5dc6ef5881f02fd99aa
SHA1: 851240790fa97aab27540995bd6a742b0a27accf

---

This is not an official Fimfiction project!

Do not contact Fimfiction staff for support regarding this archive.

Post questions as a comment or private message to this account instead!

---

Thanks to hawthornbunny for looking through the archive prior to this release.

I wish you much snow this season!

Comments

[Odd_Sarge]

Thanks again!

[HeroicNonsense]

Edit: A suggestion while I'm thinking about it, though I don't know if it's feasible. Split the archive into multiple zips for ease of searching. I have a beefy gaming laptop and the index file takes at least 10 minutes to load, assuming it doesn't crash.

Glorious. Can anyone advise on the best way to search the archive with multiple tags? There is a story that I know existed, am certain is no longer here, and I'm fairly sure would have certain common tags & characters.

[alkarian]

5701138
You're doing it wrong if you're trying to open index file directly. General purpose text editors are not designed to handle that much text in single file. Which is why index is formatted in such a way that single entry takes a single line. You can use tools for line-oriented text search like ripgrep to find entries that have i.e. "Trixie", in fraction of a second.

[HeroicNonsense]

5701401
That makes sense. I'm looking around for browser-based .json viewers now, though none are able to handle a file of this size (so far).

[Midnight Sapphire]

5701466
Here is a spreadsheet of the latest release
https://drive.google.com/file/d/11PksBz3lCFNO6k3ZT3aU_bBKKW700bP1/view?usp=share_link

[Midnight Sapphire]

5701466
i fixed the link should be downladable now

[HeroicNonsense]

5701615
My man. Just got home and saw this. Filtering the tags column takes ~5 minutes per word (lol), but it is working. Hopefully the story did exist here at some point. I suddenly had the dreadful revelation that I might have read it somewhere else, though I don't know of any other repositories like this fimfic. Regardless, thank you for this.

[Midnight Sapphire]

5701629
You're welcome! I am trying to speed it up by creating an sqlite database with an index should not take long to search.

[Midnight Sapphire]

5701629
If you know the title or author I can search my database I have different repos.
ffn and ao3

[HeroicNonsense]

5701640
I wish. All I have to go on is a moderate remembrance of the plot and main characters. That said, your .csv helped immensely. I was able to filter the rating column and keyword search from there. No dice, so I'm now certain I must have stumbled upon another site story collection at some point.

[axxuy]

Just the patch is almost half a gig. That's reassuring that the amount of new stories is still going strong.

[Midnight Sapphire]

5701646
you can try
https://fimfetch.net/

[uis]

5701646
I have tool, that can use full-text search entire archive and search by tags. If this would be helpful, write to Derped Green at fimfiction discord server

[HeroicNonsense]

5702679
I may just take you up on that. Cheers!

[Lahirien]

[justtocomment]

Thank you for this.

What does xdelta3 mean?

[SilverComet]

5709424
The "xdelta" is a (smaller) file with only the "changes" done since the last iteration of the "whole" archive.
It is a "patch" that does NOT work by itself alone.
You can not use it if this it the FIRST time you download the Fimfiction Archive.
-> You can also always ignore the "xdelta" file later [if you do not mind having to download the whole 7+ GB all over again every time the whole archive updates].

- - - -
{ This is what I did, it worked }

. Months ago, I downloaded the file "fimfarchive-20220901.zip" (the 7+ GB version from September 2022) using bittorrent, leaving it in a special folder for "temporary storage" (different from the "active" folder where I decompressed its contents to actually use this archive).

. [When this new version was available] I downloaded the file "fimfarchive-20220901-to-20221201.xdelta3" ( only 400+ MB ) & placed it in the same special folder the "fimfarchive-20220901.zip" was stored.

. Then I downloaded the executable that "does the magic" (with a very simple "Graphic User Interface") as a zip file; from Josh McDonald (KaioShin)´s page on romhacking.net [this is the original source]:
"xdeltaUI.zip" 31.05.09 v1.1 (251 KB)
https://www.romhacking.net/download/utilities/598/
-> It asked me for a "Human Verification" step, showing an 8_digit (only numbers and CAPSLETTERS) captcha that you have to type correctly where it says "Download Password" (as you type it, it will be hidden as ******** ), then clicking the "I AM HUMAN" button.

. After that I decompressed it and moved BOTH "xdelta.exe" (223 KB) & "xdeltaUI.exe" (79 KB) [I deleted the "readme.txt"] to the same folder with the previous version of the Fimfiction archive & its new Xdelta3 "patch".

. With all 4 files in the same folder, I opened patching program (the "xdeltaUI.exe" file with a red X on the left side of the icon).
-> Note: (As it opens by default in the first left tab "Apply Patch"), where it says "Patch" a click on the "Open" first top button lets you navigate to the folder; but to be able to actually select the patch_file, you have to change where it says "xdelta patch (*.xdelta)" to "All files (*.*)" first, and then click on "fimfarchive-20220901-to-20221201.xdelta3" ]

. Next where it says "Source File", a click on the second central "Open..." button should open in that same folder, to select the previous "whole" version of the Fimfiction Archive (in this case it was "fimfarchive-20220901.zip" ).

. Then going to "Output File" & clicking in the empty "..." third bottom button lets you select the folder & name of the merged, "updated" new version of the Fimfiction archive.
[OPTIONAL] -> To be 100% sure that the patching worked correctly, it is a good idea to name this OutputFile with the exact same name than the one that is used for the bittorrent file of the new update whole archive (in this case is "fimfarchive-20221201.zip" ); so later you can use a bittorrent program that lets you check if "files already exist" (when you create the "task" for downloading it), so it checks its "integrity" [not needing to download anything if it is OK].

The "Patch" button will start a VERY INTENSIVE work task for the hard drive & CPU (it takes a while because it has to explore, modify & write a new 7+ GB file), with no visible "completion bar" to fill.
This has to finish (and the program be closed) before trying to decompress the new updated FimFic archive [since I had the previous version file just in case, I deleted the "active" folder with its contents, to avoid having duplicates or errors & to free up space].

This was how I used it to save me time. You do not actually need to do any of this if you can wait to download it in full.

[Prince Polaris]

Just to make sure, when I download the most recent fimarchive, I don't need the older ones anymore, right?

[Midnight Sapphire]

5710069

There have been stories removed in the past so take with that what you will

[Prince Polaris]

5710658

Damn, I already deleted two older iterations