Curious Crepuscular Chemistry · 12:07am Apr 7th, 2017
The kind and generous hawthornbunny went to the trouble of word-clouding my stories. This has led me to a most unusual discovery.
First off, here's the word cloud for all of my stories together:
I've written 372,000 words, and Twilight's Secret Journal accounts for 216,000 of them. Yes, you read that correctly: TSJ is much larger than all twenty-four of my other stories combined. It accounts for about 60% of the horse words I write. This is why it is a novel.
Because of this, seeing a giant Twilight in the center of the tag cloud isn't exactly a surprise. TSJ is written from Twilight Sparkle's perspective, after all. Almost every word of it is in her hoofwriting. The entire story revolves around her perception, her mental state, and her personal struggles. Every single page of Twilight's Secret Journal is jam-packed with Twilight, Twilight, Twilight. Clearly, this is why Twilight dominates my writing.
Clearly this is why. Clearly.
But, just for curiosity's sake, let's just take a look at what happens when we remove those 216,000 words from the cloud:
WAT 
My other stories contain even more Twilight Sparkle than Twilight's Secret Journal...! 
I'm not sure how this is even possible. At first I was thinking it was because Twilight's name doesn't come up as frequently as you might expect, since it's her writing in first-pony perspective. So most of the "Twilight" becomes "I" or "me" instead. But even then, everypony in the story is constantly using her name in dialogue, and she talks about herself most of the time too. In order to out-Twilight TSJ, I would need to pump out an incredible proportion of Twilight-text into almost all of my other stories.
It turns out I do. 
I had no idea I used so much Twilight Sparkle in my writing, but I guess it makes sense. She's smart, and her presence can help explain things that would confuse other ponies. She's fun to write. I think a lot like her (even though I identify more with Ponkus). 
I end up sticking Twi into stories as a reference even when the story isn't about her. My second and third largest-stories (the novella Broken Symmetry and the well-reviewed The Price of a Smile) are extremely Twicentric, as are most of my other long stories.
Out of the dozen or so stories I'm idly writing, most of them are centered on Twilight too. I never noticed this. Seriously.
I guess I write a ton of Twilight stuff. Huh.
That's awfully weird, but... I think I'm okay with this. 
What site was used to create those word-clouds?
<-- an easily impressed visual learner
Could it be partially due to name abbreviations not being counted for other characters, while Twilight's name usually has her first name in its entirety at least?
4486875
I mentioned, above. It was a gift from hawthornbunny (there's a link to him). I don't know if he used a website.
4486890
I seldom use abbreviations. It's because I write Twilight into everything.
I am not sure but it is possible that in the Journal you get less uses of Twilight than you might think since it is in her perspective and she probably says "I" reducing the number.
I do notice that Pinkie goes way down though when you remove that story. This must be fixed write more Pinkie!.
Heh, you write Twilight pretty well so it's no surprise that you do it often.
That's pretty cool.
I like you can read "TWILIGHT (um) Celestia" in the first one :V
Ponkus?
also, twilight was supposed to be the main character of mlp in general when it began. its hardly surprising she's the main character in a lot of fanfics.
4486892
Welp. There goes that theory .
Perhaps more interesting is how Twilight's name is used. In the top word-cloud, "Twilight Sparkle" is four words below "Twilight."
"Twi" is right above the g in "Twilight."
You might not have even seen "Princess Twilight" in the first one. It's absolutely tiny.
There seems to be a similar pattern in the second one. "Princess Twilight" is tiny (draw a line down from the space between the g and h in "Twilight"), "Twi" is also small (underneath the L in Applejack, two below "Twilight"), and Twilight Sparkle is fairly large above "Twilight."
hax.
You know, it could just be because Twilight is almost universally used as a plot device. Think about it. Whether it's a friendship problem, monster attack, or magic problem, guess who leads the resolution? Twilight.
My theory? You want a smart, sexy self made book mare to own you. Just a theory. Nerdy doms are hot.
Would Twilight be the most commonly used word in the entire fimfiction corpus? What would the data look like normalized to the word usage of the typical fimfiction user?
4487381
That's a great question! Maybe hawthornbunny could help, but you'd need to get the unigram and bigram counts from all the stories.
Hmm. I'll put a bug in Knighty's ear about it.
4487382 I've seen some people do analysis of fimfiction word frequencies before (e.g. https://www.fimfiction.net/user/Wallacoloo/blog). His latest blog post has a link to a file with fimfic unigram frequencies. Not sure if a bigram file exists but it could probably be obtained from mining some of the fimfic archives that he mentions.
4486875
These word clouds are generated using a couple of Python tools that I wrote to scrape story data from Fimfiction and count word frequencies, and rendered as SVG in a browser using some horribly inefficient Javascript.
There are better cloud generators out there (especially this one! It's amazing! How the hell is it so fast), but I wanted to try making my own. The upshot is that I can point my script at any author and get all their word frequencies in one go. 
4487381
4487382
This would literally set my computer on fire
4487390
Wallacoloo's statistical analysis (linked in his first blog) is excellent (as far as I can tell, anyway) and was one of my inspirations.
He does mention the phenomenon of Twilight's prevalence:
I've not yet dared to try tackling the entirety of the Fimfiction corpus myself.
4488817 Well, Wallacoloo has already mined the word frequencies from the Fimfic corpus (https://github.com/Wallacoloo/fimfic-stats-build/blob/master/build/stats/word_frequencies.json), so all you'd need to do is divide the frequencies generated by your script by Wallacoloo's. But, I would agree that looking at bigrams would be quite computationally intensive.
4487324
FTFY
4497480
http://lmgtfy.com/?q=ftfy
...and that's not the cawk I'm talkin' about