Стать isn't really important here. This is just one particular modality of a catch-all Russian construction that talks about "trouble with" something or other. Which has two fairly self-contained elements, a predicate specifying how bad the trouble is (here, стало хуже), and the "c + noun" part where the noun identifies the source of the trouble — in this case, память.
The "how bad" part is very, very flexible.
В городе неблагополучно с преступностью.
Да у него с головой плохо. [~"That one's crazy in the head".]
С котами была сплошная беда. [~"The cats were no end of trouble."]
Note that a form of быть appears in the last example — well, really, it's been there all along, just not apparently because of present-tense copula dropping.
What I'm trying to say here is, it's essentially just an accident that you can also say память у тебя стала хуже. Because you definitely can't say
*в городе неблагополучна преступность or
*у него плохая голова. It's just that in your example, there are two elaborations on the basic construction: the comparative хуже and the modality introduced by using стать instead of быть. Which, understandably, makes you wonder why go through all the trouble of making it impersonal and oblique when you could just make it a plain subject+verb. Whereas in fact this is a syntactic set piece where the impersonality and obliqueness are there to begin with. And it doesn't always convert this easily into subject+verb.
Admittedly there's a very small difference between saying "your memory's gotten worse" and "this memory thing of yours has gotten worse", but that difference is there.