Fimfiction author word clouds · 1:53am Jan 15th, 2017
A couple of days ago I was struck by my coding muse, and thus have been hammering together a little project out of Python and Javascript. If you haven't already guessed from the title, it's this:
Word frequency clouds! This above is a collection of the words that I use most frequently in the stories that I've published. There are a few caveats:
▶ A "word" is defined to be any contiguous sequence of non-whitespace characters, after disregarding common punctuation such as periods and quotes.
▶ "Frequency" is the number of times the word appears in all the story text published by a given author.
▶ Common words such as "the", "and", etc. are disregarded. I'm using the top 5000 words from the google-10000-english list, which isn't exactly literary but seems to do the job. A number of common contractions such as "I'm", "it's", etc. are also excluded.
▶ Two-word phrases (eg."Rainbow Dash") are also included in the results alongside the single word data.
▶ Any word which ends in the common possessive suffix "'s" is considered to be the same as the word without the suffix; ie. "Applejack's" is the same as "Applejack" and will be counted as an instance of "Applejack".
▶ Words with a frequency of less than 10 are not shown in the word cloud.
▶ The word cloud contains a maximum of 300 words, because any more causes my laptop to catch fire.
The hardest part was the cloud generation algorithm, which I initially tried to do in Python before realizing that that would just be a total nightmare. Therefore, I switched to Javascript, which is like a nightmare that you can inspect in-browser. The generation algorithm is fabulously inefficient and takes a few minutes for each cloud. The word colors are based on the word's hash and have no significance.
My word cloud is probably not terribly revealing, since most of my stories are short one-shots and I haven't written many, so there's not a lot of data to go on. Twilight's prominence does surprise me a little, since she hasn't really been a main character in any of my fics (whereas Rainbow Dash has, in two), but she does show up in a lot of them. Also, there's a lot of yelling and blinking in my stories.
Twilight prominence is quite common for many authors, which may well tie in with her being generally more prominent in stories in general, as noted by Colin Wallace in the Fimfiction analysis that he conducted last year.
Anyway, let's see some more clouds! Here's a few from various selected authors.
Loganberry (24 stories)
Logan is one of the authors who doesn't lead with Twilight, although she's still a respectable third. No surprises for the two most prominent here, with Rainbow just a couple of smidges ahead of Fluttershy. There's also a tiny Luna, not that I'm suggesting any kind of princess bias at all.
FanOfMostEverything (56 stories)
The king of card games is the most prolific author in this little selection, with 56 stories. Another Twilight-er, but I suspect this is also probably the Ditzy-est cloud that you'll get on Fimfiction. Possibly these results might be skewed slightly by the Group Precipitation fic, which is a collaboration between lots of different authors.
horizon (27 stories)
Another big ol' Twilight, but also a large Sunbutt as well. And apparently more muzzles than Applejacks. A lot of these words feel quite sensual. Twilicorn is best princess.
The Albinocorn (14 stories)
Are you even surprised?
Those look really cool!
Ooh! Can you do me, Hawthorn?
I may have the highest story count, but Albinocorn has me beat on words by almost a hundred thousand. And what appears to be twenty thousand Sunsets.
In any case, it's very interesting indeed to see how these clouds look, especially my own. It confirms some suspicions I had about my writing tendencies. And on a less egotistical note, that Twilight bias is a fascinating phenomenon indeed. Thanks for showing us these.
(I can only imagine how long it would take to process SS&E's combined works...)
Ooh, interesting!
On the princess point, the only princess (other than Twi) I've written about in any significant way is Cadance, in Shining Armor's Amour's Armour. Many of the "Celestia" mentions aren't quite what they seem: they're from This Fragment of Life and refer to a rock Twilight fell in love with and named after her mentor. The only mention of Luna I can remember offhand is a single reference in the second chapter of Friendship is Poetry.
I'm pretty sure my cloud is skewed by Where They Understand You. For a start, it's 18,000 words long, and nothing else I've published here is even a quarter that length. And since that fic is set entirely before S1, Twilight doesn't feature in it at all. It's a FlutterDash friendshipping story, so both the pegasi get a lot of mentions. It's also unusual in that my usual semi-random varying between the various names for RD doesn't apply, and the choice is in many places actually significant to the story.
4383130 Thanks! Yeah, I'm quite pleased by how they turned out. ^^
4383135
Will do! Another batch is on the way.
4383157
Sunset finds the Mirror Pool
"Sun!" "Sun!" "Sun!" "Sun!" "Sun!" "Sun!" "Sun!" "Sun!" "Sun!" "Sun!" "Sun!"
He's on my list of authors to analyze, so you don't have to imagine; I'll time it! Generally, though, the frequency analysis is the quick part (at most it takes around a minute). It's turning the frequency data into a cloud that's the slow part because I'm just doing brute-force collision detection and haven't optimized it.
4383163
oops forgot to hit reply
Luna is mentioned 10 times, the minimum for her to appear in the cloud. She most recently made a celebrity cameo appearance in More Than Alive. :)
In the interests of science, I checked this, and you're correct. Here's your cloud with WTUY excluded:
i63.tinypic.com/azgnrs.jpg
But otherwise, you're the only author I've looked at so far who doesn't have Twilight in first or second place.
4383691 Yeah, it kinda looks like something I had to do way back in 6th grade. We'd type in words and then it would create images with them like dogs. Though I can't remember what it was called.
4383719 Heh, yay for me forgetting a detail of a story I published three weeks ago!
The second cloud is interesting; thanks. The 20 Percent Solution is a Pinkie story and is considerably longer than most of my others, so again I'm not surprised that Pinkie is so prominent. The relative lack of saidisms is probably partly because I tend to write a lot of my dialogue without using them at all, along the lines of:
That sort of thing. (That dialogue was written just now, by the way, so don't go looking for it in a story!)
oh god, I just love that SUNSET XD
I'm horribly wondering what mine looks like now. :3
4384730
I kinda like that it also sorta coincidentally looks like a sunset
I'll have one for you later today.
4384828
:D