"Я мог бы покляться" appears in many translations of English books. However, it sounds bookish and not quite idiomatic.
Let me turn the tables around a bit and ask: why does English use the conditional mood in "I could've sworn"? Is it to say "I could have sworn but I didn't because I knew I was wrong?" Probably not. The use of conditional makes the language milder e.g. "Could you help me?" vs "Can you help me?", "I would think" vs "I think", "I would like" vs "I want", etc. So what you are really saying is "I swear", just using more "polite" language.
Similar use of conditional exists in Russian: "Не мог бы ты помочь мне?" vs "Ты не поможешь мне?", "Я бы так не думал" vs "Я так не думаю", "Шёл бы ты ..." vs "Иди ты ..." but, as I said, it sounds bookish and "overly polite" which is why when combined with the common insult "Иди ты", it creates a comical effect (well, used to; now the effect has nearly worn off, leaving the two almost identical).
What "I could've sworn" is, it is largely an attempt to rationalize one's surprise reaction ("I was sure until now that...") and there are countless ways you can do that:
- Странно! Я был полностью уверен, что ...
- Удивительно! Куда она могла подеваться?
- Опаньки! А я-то думал, что ...
Or how Mr. Скуперфильд would put it,
- Провалиться мне на этом самом месте!