Слышал in the second sentence is somewhat idiomatic.
If you replace it with услышал, this sentence now would mean "that's how I heard that", not "that's what I heard".
This would assume that hearing or listening was already the topic of the conversation, and the speaker elaborates on what exactly did he hear, not whether or not he heard something.
Let me provide a couple of examples:
Ты видел, как Джон выходит из дому? // Did you see John leaving the house?
Seeing or watching John has not been the topic of the conversation yet, it's something that is being introduced as a topic.
Ты увидел, как Джон выходит из дому? // Did you manage to see John leaving the house?
This is something you would ask if you somehow knew that your party saw John, or was watching his doorstep, and you want to elaborate on what exactly did they see or how exactly your party saw John leaving his house.
You don't ask this out of blue. If you do, your party would probably say: "no, why, did I give you a reason to think I was stalking John"?
If you squint hard enough, this kinda falls in line with the usual distinction between imperfective verbs describing state and perfective ones describing state transition.
The first example describes the state of "seeing John", and the second one describes the state transition of "not having seen John" to "having seen John" just that one time (which is already in context somehow).
However, it's a bit of a stretch, so it probably better to write it off as idiomatic usage.
So in your original examples,
Поттеры, всё верно, именно так я и услышал
would mean that the speaker confirms that he heard the word "Potters" uttered that one time they were discussing.
Поттеры, всё верно, именно так я и слышал
would mean that the speaker is familiar with some rumor concerning the Potters, and not talking about one instance of hearing something.
он случайно услышал обрывки их разговора
Here, the author describes one particular case of the character having heard something, which is in context (на обратном пути, сжимая в руке пончик).
Hence the perfective услышал.