As @tired_programmer and @SleepingPoppy mentioned, gather up evidence and figure out if he's done this to other students as well. It's great that your mother and other instructos are on your side, and it's less of a "he said, I said" situation.
I have had teachers who tried to pull this on me and several students throughout grade school and high school, and often times, it worked in our favor when we as students showed the overwhelming evidence. I'm glad to say I haven't encountered that in college currently, but I know it still happens in higher levels.
Exams, interactions, anything inappropriate he's said -- that needs to be documented as evidence and addressed.
And if some of the other targeted students aren't comfortable going to an authority, you can find a way to get their voices out there without them feeling exposed. Using something like an "anonymous caller" type of unit.
But if you do have to keep going to his classes (as in it's mandatory, and there's no way to re-take that class over the summer), then keep your interactions (from your end) towards him neutral at best. He certainly doesn't deserve your respect and you are well within your right to be anger, but I have seen instructors try and use their students' justified anger in an attempt to recreate "they said, I said" situations.
Let his behavior show even worse in comparison to how you respond civil but firm to him. It can gradually show outsiders that something's wrong with him more than you.