Well its hard to give you a formula, but the closest I can get is a little thing I unconsciously do.
Normally I put a part of my personality on my important characters, but just a part, like a character might have a huge problem that I once had on my life, or a particular taste or something, and then I work from that, I start to find other characteristics that could work and build them from there, then I just keep spicing them up as I come up with the story, but the most important thing about the character is usually the one that's linked to me personally, so its hard to forget. Hope it helps!
I agree with @SleepingPoppy
you can use irl people you know and borrow some of their ticks and quirks
but as everyone said above ^^^ background and goals helps a LOT
but I wouldn't agree on strictly using motivation/background
the best way I think you could make a character different from one another is also just, as silly and simple as it sounds. is to use completely different personality traits? well also a different variation of ideals.
like, here's an example:
scientist bob and scientist willy are both scientists, both of them are from rich families, both of them are in the search of knowledge and help make a better world, therefore they are scientists in that regard.
Both of them are working towards a goal to cure cancer, they both hate to see people suffer.
Bob, however, is more outgoing, and charming, whoever he talks to make that person feel like the most important and meaningful person in the world, which lines up with his character. he cares about people.
Willy, however, is the opposite, he is aloof, less talkative only using grunts and hand waves to talk to people, Willy wants to devote all his time and effort to curing cancer for the betterment of humanity, even though he comes off as a jerk, he has good intentions.
Bob wants to cure cancer by using the questionable act of human augmentation through bioengineering life, essentially warping the human form into something completely different, while Willy wants to devote his time into a more ethical method, but that would take years or even decades, and even with that, it's not guaranteed. while Bob's method is faster, has instant results, yet is ethically questionable, as these people aren't even the same person anymore. these characters have the same background and goal, but are completely different people.
now see what happens if both characters have been up to date with the other's works, and you put them in the same room, they meet for the first time, and seeing their methods, both of them have very strong opinions of one another.
iterating on that last point ^^^ doesn't matter if you have 8 vastly different characters, none of it matters if none of them bounces off of one another in any meaningful way, they are just there, instead of existing with each other, that's kind of the point of having vastly different characters, is that bounce. and even with that, there's nothing wrong with having less foil and making some characters similar, hell, it gives a reason for one character to relate to or even be friends with another character. as long as the personality isn't like the same (unless they're actually clones grown in a vat, or a hive mind) and it doesn't mesh so hard that they are essentially a copy paste of another character (in your world).
like, using an opposite example
a cop from a rich family, white knight, wants to enforce the law and safety of others, and a criminal that is an anti-hero, from a broke af family, is straight up an anarchist. one has a high and mighty personality, the other is more trivial, and shy. one is a white supremacist, the other is a social justice warrior, and lastly, one loves jewlery and is greedy, the other only want the essentials.
they both meet and realize their immense passion for star wars, like that passion is so freakishly immense, so much that they even have the same underwear of luke skywalker, now they don't wanna shoot each other, because apparently in this world, legitimate star war fan bois are rare these days. and they finally meet a person they can share their passion with.
boom, a bounce, two unlikely characters possibly become friends
and two characters who would on the surface seem like they'd be friends, end up being unlikely enemies.
I put my characters into scenarios and see what sticks:
Character A is walking down the street and spots a (blank), how do they react?
And then base don those reactions, slap them into my mental movie theater and see how they react to, say, a vampire or a demon (because those are some creatures in my story)
Character A is walking down the street and spots a (demon!!), how do they react?
And then villains, I'll come up with a list of things I like and then write the opposite but I also base some of my villains on historical bad guys as well as bad guys from other media. In the end, it's all about how they interact with your plot and the overall story.
Also: give tropes a try. :> Read some, even if you're only getting ideas.
I think is where your taking the wrong turn.
If your characters dont want to be what you intended them to be then just let them be:3
Its very rarely that characters stick with the first traits I give them. You have to let it be natural. People are complicated. They dont fit into "character traits'. The thing you are fighting is probably making your characters more believable without you realizing it.
My main point is: dont force it
Edit: I also believe writing from experience is kinda BS:/ Just let your mind have fun with whatever it wants to!
Creativity isnt really a choice it just kinda happens. So have fun now, organize later. Who cares what other people will think of your writing? Not like anyone actually cares about it. People just want to be engaged with other people. Writing is just another way to engage with other people. That's all it is, a strange form of engagement.
Hi! Well, If you are making them different but at the end you don't follow the traits maybe it is because you are not thinking about what THEY would do or say in a situation, and instead you are making your charters do what YOU would do, or what is convenient to your story. Believable charters are consistent with their own ideas and values even if it is not the right thing to do, or what is convenient etc. Sometimes I had changed the whole chapter based on what my character would really do...like, go back to sleep instead of fighting XD LOL. It can help study types of personalities in psychology, that way you can know what behavior is consistent for certain type of people.
https://www.16personalities.com/personality-types
I definitely second what @JakiraJurosawa said.
Humans are always changing, always developing. They do things based on their own reasoning and feelings, experiences they have, and the people they encounter.
Just let things take their course. However, if they do change, let it also be within reason. Like, if they do something completely out of character, there has to be a good reason for that (or, I should say, what your character thinks is a good enough reason).
Example: A man based on honor and family who has never told a lie in his life. However, he lies to police about being a part of a murder because his accomplice threatened to harm his family if he snitched. In this case, the character has done something completely out of character and is still in line with his usual personality. It also conflicts with his usual traits and might've given him more growth.
He would do anything to protect his family, even lying to police if he thinks they'll be safe.
This conflicts with his personality based on honor because of the lying and doing something arguably dishonorable, but he put his family above that. So now we've got more to this guy -- he's willing to put personal pride and morals aside for others.
Right there, you've got a character who's strayed from the traits given to him and yet still A) kept his personality and B) developed as well.
Just let things take their course, and readers can figure things out on there own.
I love character development, I have a sort of weird process tho, I'll try my best to explain:
I start with a basic concept for the character. Like super basic- usually an archetype, cliche or something. I also decide if they overall play the good guy or bad guy role(even if the role has some gray areas)- and what their primary purpose in the story is. Ex: Lead Female Love interest.
Then I start thinking up things that you can write on paper- stuff you could put in a dating app for example, or bio, or whatever, Ex: intelligent?(book smart or street smart?), wealth/status/class?, goodlooking?, fashionable?, popular?, age, job, family, etc
Then I consider a basic personality- kind? Twisted? Helpful? Hardworking? lazy? etc.
From here on out I just like to go on long walks and think of different scenarios for my characters and how they would interact with eachother or to different events. After the walk I write everything down. A lot of it is dialogue driven, but not always. do this for a few weeks and the character is pretty solidly built in my mind. On these walks I try to establish as many different angles as I can, examples would be situations that develop the type of boyfriend/girlfriend they would be, the type of student/employee they'd be, the type of friend they'd be, the type of brother or sister, etc How they behave in tough situations? calm situations, by themselves, in crowds, etc. I usually think up specific scenarios that help me establish these qualities.
I'm no writer by the stretch of the imagination, but I try to think of a character backstory and my experiences as well as the experiences of others in my life. The chips fall into place when I do that and make it up as well as tweak it as I go along hoping I will be consistent. What's their character motivation? I want to do that with most villains because I do not want a villain that's evil for the sake of being evil.
"Edit: I also believe writing from experience is kinda BS:/"
What I say next is not meant to discount your own experience, but rather to offer an alternate point of view. Some people write, as the saying goes, "by the seat of their pants", letting their characters' personalities clash and then leaving the writer to work on the aftermath of the said clash. This ongoing conflict of personality and ideas is what creates a very organic story, and is very good for creating unpredictable plots.
However, I would argue that there's another subset of writers that very much need to plan first, sometimes(but very rarely) down to every line that each individual character will say in a conversation. I myself am such a person, although my outlines are very loose(I only outline the main plot and the main plot points, filling in the space in between with the freeform style I introduced earlier). This outlining-focused style benefits more from past experiences, since it's easier to draw from specific experiences and insert them into the story(in some for or another) without it feeling out of place. That said, this can also be done with the aforementioned freeform style, albeit, in my opinion, to lesser effect.
From how you explained your writing style, you seemed to prefer the former style of writing, which is fine. However, it's not for everyone(nor does it have to be).
Though I have a feeling you already know that.
I'm going to go against the grain a little and say that if you're having this much trouble with characters, it's because your plot isn't strong enough.
I'm a very plot-oriented writer. If I create a new character, it's only because I have a role I need them to fill. Even at the very start of a project, I'll decide what kind of story I want to tell first, and then decide from there what kind of character is needed for that story to happen. If you want to tell a story about characters fighting, you'll need to give them opposing goals and viewpoints. If they're going to fall in love, they need to have some things in common, some that get in the way. Only once you've determined their worth to the story can you flesh them out with more auxiliary traits. (IMO)
If you're writing your story and two characters are sounding too similar, it may be a sign that the second character isn't even necessary at all and should be cut. Ask yourself why that character HAS to exist. That might help lead you toward the characteristics that make them necessary.
Start with something simple, ask some questions to yourself and try to answer them with each character. I'd suggest doing questions that focus on the character's motivations.
What is the thing that the character values the most? What goal is the character trying to achieve, if any? How does this character want to live their life?
Also maybe some simpler questions like what would the character's daily life routines be, what would they do if they were in a certain situation, etc.
Use those answers as a foundation to build the character's personalities, actions, dialogue.
You really have to sit and work on empathy to get different kinds of characters going. I think it starts with people watching. As an exercise, pick a random person on a bus or something. Study them. Imagine you are them. What are you thinking? What was your morning like? Where are you going?
Go waaaaay out of your way and pick people as different from you and your experience of the world as you can.
If you don't take buses often, pick a random extra on a show. Pick characters. Pick celebrities. Just whoever until you have a wide database to draw off of for your characters.
In my experience, the key to creating distinct, different characters is their motivation.
Consider your plot. What drives your characters? Is it love, revenge, family, fame, the fear of being alone? Once you have a motivation, it's really a matter of finding traits that can complement each other, but that are not necessarily the same!
For example, I have a personal project I'm working where two character's motivations are the same. They both want to protect a secret. However, one is quiet, thoughtful, but concerned mostly with himself and his family. The other is loud, rude, but ultimately distances himself from those he cares about because he thinks he, himself, is responsible for their problems.
It's the difference between having layers; the first man is selfish, but appears otherwise. The second is abrasive, but actually cares a great deal.
If you find your characters would respond verbally to similar situations, I'd urge you to recall their motivations and write down a Q&A for them, essentially. Answer the questions as that character, note the moments they're too similar or static, and make them more distinct voices.
Or, people watch!
It's great fun, and it gives a huge number of different personalities for your writing arsenal. No conversation required!
Good luck!
I had pretty much the same issue, and what helped me most was writing fanfiction!
And I mean that very unironically.
Especially if you massively struggle with it right now, it's mostly easier to start with fanfiction than original characters. You don't need to worry about "coming up" with different/diverse characters, you already have a cast you can use with preset character traits, backgrounds, personalities, speaking patterns etc. It really helped me to detatch the characters from myself and how I would act on things to more think about "how would character XY handle this situation?"
It also makes it easier to check whether you're still in-character. Just pick your favourite show and write how character X and character Y need to go grocery shopping. Simple things like this, where you put yourself in their shoes and not the other way around.
You will spot certain characteristics that reoccur but that's okay, it's just practice (and a fun one, might I add)! Reading fanfics also might help to understand how other people perceive the very same characters and helps to reflect on your own portrayal, too.
This, obviously, is just for practicing and getting a feel for different characters. You also should maybe switch the POV between several characters every now and then to find their individual voices. And then, yeah, have fun with your own characters! There the others already gave plenty of advice.
(Also don't use the "I'm an introvert" excuse because it isn't - there are so many series, books, movies, etc. where you can start from! You only limit yourself in saying that.)