Because I know you, Twilight. I know that when you have concerns, they consume you. So first I will tell you, as simply as possible: no, alicorns are not immortal, in the purest sense of the word.
Celestia's reassurance should be a relief, but true to form, Twilight can't stop thinking.
Contains discussion of death.