Rumble left Ponyville when he was old enough to. First place he went: Cloudsdale, the place any right minded pegasus would go. Rumble was tired of the ground, and was ready for things to start looking up, so to speak. The past is in the past, and he only wants the future.
However, after a few months of living there, he finds something behind his apartment building that makes him think he didn't put quite enough distance between himself and Ponyville.