I was a big fan of RPGWorld back in the days when it was uploading! I'm glad Ian JQ went on to do great stuff (Steven Universe and OK-KO!) but ah man... I would have liked closure, it really was a good comic.
But anyway, yes. You really can choose how deep you want to go with literal numbers. For example, one of the most popular Novels on Tapas "The Beginning After the End", which also has a comic adaptation, is set in a world where magic has a very RPG-like way of functioning, where people have elements, people with different magical elements have different capabilities that people know about, and magical power can be measured by the colour of the person's magical core they can sense within themselves.Adventurers in that story have classes with a specific hierarchy, so there are ways to "level up" without it ever getting into people popping up literal game UI to look at their numerical stats.
Try to think of real world equivalents. There are plenty of societies throughout history that have assigned castes or classes to people, and many jobs and militaries have a hierarchical structure, or in education you're given grades based on success and can reach named titles and get certificates for it. Example, a standard bachelor's degree technically allows a person to be called a "Bachelor of" or put BA/BSc after their name, but few people in our world do it because degrees are so commonplace it feels a bit silly.
There's also the concept of say Horoscopes. In our world, some people who believe in astrology assign personality traits and even elemental alignment based on when a person was born. ie. I'm a Sagittarius, which to astrologists means I'm curious and adventurous, inclined towards bows as my weapon of choice and associated with the element of fire. Imagine a world where something like that was real, like in the Elder Scrolls series, where being born in a specific month can make you literally run faster or absorb magical energy 
Have fun with it!