It means that the poet lives at the floor that is very close to the roof, for instance in the attic.
Под крышей would mean "in any place under the roof", in other words - somewhere in the house. When someone uses just "под крышей" it is synonym for "in the house". Another example is "Иметь крышу над головой" - literally "someone lives in a place where there is a roof".
Под самой крышей means "under the roof, as close as possible".
Чуть ли не под самой крышей means "almost as close to the roof as possible", so it can be not the attic itself, but at the last floor.
Based on the passage you've shown, the author depicts a typical "romantic poet" figure, that has no money to pay for a better place and lives in a cheapest flat (or even single room) possible. At least in old books and fairy tales the romantic heroes live in such places (when elevators were not invented yet and it was hard to get to the top of the house).