Before you start, I want you to imagine a society of short people and how they would operate.
Do you think that little people would live in the same as you do? The very same house where everything you own is out of reach?
No, the little person would live in a home whose convenience is the same as an average sized person. The same goes for giants, parapelegics, etc.
So when building your fantasy town, imagine what a society of cats would need, what do they do, what are their general nature?
Cats are very agile creatures that can leap bounds! So imagine out of reach places that only cats can reach. Cats like to bury their droppings, hence why they have litter boxes; so consider where and how they poop, especially with that pesky tail in the way. Tails, cats have them and it's their way to control their center of gravity when moving, and a hiderance for when they where pants; so would they be a society that wear pants?