Robert gets picked on soooo much, but is written as a depressive character a lot. Ray's also written to be selfish more than Robert in a lot of the episodes, but when Robert is being selfish it's to an extreme (usually).
There's also a lot of shouting and fighting and bickering and not being nice to each other in that show - not unlike Roseanne. A lot of the sitcoms from that time period seemed to be built off of casual insults, or self-depricating humor. "How I met Your Mother" is a another, but it has moments when it's not like that (my favorite from that series is the slap count and how Barney never gets a bad photo of him taken)