I'm like you; I'm like a 3 tops, because in my own little family, I do throw around some sarcastic comments, but only because I know my partner gets it. Other than that, I'm really not a fan of it because I can never judge whether or not the person before me is mocking me—and there's little I hate more than being mocked.
Full disclosure, I didn't even understand sarcasm until I was well into my teens lol; it just always flew right over my head. Guess whatever spectrum I'm on had a hand in that. xD But this is why I can't use it in RL much with others because I don't ever want to make others feel the way I did. 
Ironically, I love sarcasm in fiction. Especially in my own characters. It's fun, but only because I know exactly what they mean. :'D