Another reason: Life-source. Being overpowered probably is possible, but it comes with a price: it cuts down how long you live or it messes up your insides. So you choose to excel in one field and be weak/average in the other. To try and do everything at once could spell murder.
Or hell -- maybe the magic just drains physical strength. The more magic you use, be it all at once or for an extended period of time, the weaker your physical body becomes. Again, another give and take: to use the magic, you need to give something up. But this could make room to lose anything:
If not physical strength, then intelligence. If not intelligence, then social skills. Maybe you have to give up emotions.