What is the reason in terms of the history of the language and is there the exact same difference in other Slavic or non-Slavic languages ?
1 Answer
Animacy as a syntactic feature appeared in Proto-Slavic not long before its split, and it took some time for it to spread to all semantically animate nouns (meaning nouns answering the question "who" rather than "what").
The usual explanation is that in most nouns — basically, all nouns except those with historical stems in -a, -ia, -u which gave modern Russian words жена, ладья, буква, respectively — the nominative and the accusative coincided in the singular, which gave rise to ambiguities similar to мать любит дочь in modern Russian.
To keep free word order while being able to resolve these ambiguities, some nouns started to take genitive forms in what syntactically is accusative.
In Russian, initially, it only worked for titles of nobility, ranks in the army, government positions etc., but it had since been spreading to a wider class of nouns, and by the XVII century most titles, personal names, animal names etc. started to take the genitive form in the accusative.
You can still see traces of the old usage in words and phrases like замуж, на конь, идти в гости, кандидат в президенты etc.
In the plural, the accusative and the nominative did not coincide for nouns with the historical -о stem (that is most masculine nouns). It used to be херувими (nom.) vs херувимы (acc.), as seen in the well-known line from the Cherubikon. But this distinction almost disappeared by the XV century.
In the singular words with the -а stem, though, the distinction between the accusative, genitive and nominative forms has been enough to retain them as separate.
Is there the exact same difference in other Slavic or non-Slavic languages ?
Other Slavic languages treat even wider classes of nouns as animate: names of mushrooms, dances, some plants etc.
Some Slavic languages also distinguish between the dative forms of animate and inanimate nouns (which end in -ovi and -u, respectively, modulo local sound changes), although it's not a one-to-one correspondence to the difference in accusative forms.
In general, this distinction is a manifest of differential object marking, a feature which is present in many languages.
Russian has more examples of it, like use of genitive vs. accusative depending on definiteness (съесть хлеб / съесть хлеба), negation (пить воду / не пить воды) etc.
Other languages, like Spanish, can treat objects differently depending on their animacy as well. From the linked Wikipedia article: Pedro besó a Lucía / Pedro besó el retrato.