So I have 3 images here where I redlined your work with a different point in each.
First one is me trying to determine where your horizon line is and therefore where your two vanishing points lie as others point out, because the perspective is very skewed. So this is why, they're not really aligned or planned in any consistent way. Setting the environment up in a simple grid like this is going to take a lot of guesswork out of the process, speeding up your output significantly.
Second one is aligning the vanishing points to try and get the same general perspective, as well as a dmonstration of how I personally do line work for buildings like this. I've kept your overall design and added in details and textures as suggestions. I don't really completely fill out any entire surface area of a single texture, I just put in enough lines to describe the material and let the brain assume the rest. Especially when you have an entire townscape illustration, you want to do a more 'lost and found' method so no single building is demanding more attention than another. They need to blend in to an entire scene, and allow presence to be dominated by characters (if any.)
The details I've added might seem a little random because I wanted to add in various ideas for your consideration. A huge element missing from your architecture is frames. Door frames, window frames. The ones that were drawn in didn't have a whole lot of consideration for their material type either. Wood, stone, whether the frame is flush or pushes out or in. There's hundreds of different methods and tiny aesthetic/ support details like this can add up to become cultural identifiers when you're dealing with multiple story factions.
Notice that the lines are heaviest in natural crevices and where cast shadows fall. Line art can not only describe materials, but light and shadow too. It's why I will rarely have a straight solid line where the exterior wall of a building meets the ground, I want them to 'marry' each other, especially if it's brightly lit. One exception is when it's a different material or I'm describing an object intersecting the ground and wall.
I added a pipe because I have no idea what your setting/ technology is, it's just one of those small 'people live here' elements that's commonly forgotten about.
I added a little porch and sign on the side as well, I found that adding some buildings with an extension on the side can drastically improve a city scene by making it feel a lot more dynamic. Be prepared to draw some that diverge from the box archetype (though you have some towers and a windmill there, which shows you're already thinking about that. Just remember to build outwards as well.) I'm assuming it's a town square too, so if some are shops or restaurants you can think about what they might sell (and thus if their 'store' extends a bit outside, like a cafe may have tables outside the door. If it's residential, carry on.
That brings me to the last image, where I've red-lined missing 3D planes and purple-lined missing support structures, notably frames around doors and windows. Keep in mind, the latter are both often decorational as well as supportive! Look on google for thousands of unique ideas. For the former, be very careful about drawing objects so flatly. Very few materials are as thin as paper, which can't realistically be drawn with a side edge. (Though I think I got too lazy to change the fountain colour, I just wanted to make some lines on it a bit more clearer about it's dimensions.)
Here's a pretty straightforward video explaining 2 point perspective drawing with a single house if you're not familiar with how to use it.
