Russian names have several variants depending on the nature of the relationship between two parties involved in a conversation, formality of the setting, etc. A lady officially named Ekaterina may be called Katya among her friends but Katenka by, say, colleagues. I understand shorter names may prove useful in case of children since they are easier to pronounce/recognize for them, but this is not always necessarily the case either. For instance, an Ivan is frequently referred to as Vanya in a sympathetic manner, which is in fact longer than the original name.
So, I'm wondering, why bother? Why do such variants exist and how did they come to existence?