This is a really good question to ask! I think "make friends the same way you'd make friends with anyone!" isn't quite true, because the dynamics are often different when you're trying to connect with someone who has an audience.
Trying to Make Conversation can be off-putting because it's asking a creator to be more personal and open with you specifically. It can feel a little like that "pushing for friendship" thing. Instead, look at their tweets or posts as your chance to get to know them, and talk to them about those things, if they're things you're interested in!
But it's also important to remember that if they don't answer, or if they answer a question but don't really want to have a conversation, that's okay! It doesn't mean it "didn't work" or that you did something wrong -- Some creators aren't really looking for a conversation, or aren't looking for new friends, or don't want to do that over social media. ;u;
That's my experience, anyway!