If the character designs and personalities are different enough, you don`t need to differentiate the characters.
Another factor is how connected is this multiverse.
If members of universe A will never meet universe B, then the stories are self contained and readers don`t need to have all characters from all stories memorized to follow what is going on.
If the universes are connected and the chances of crossovers get higher, then a guide may be helpful.
Anyway, a character guide can be a fun thing to do and your readers will probably enjoy this trivia if they are engaged enough with the universe. Of course, if there are too many spoilers with some characters, it can be trickier to make the character chart for them without revealing too much.