When you develop your own art style, and when you have a clear idea in your head what you want to draw.
Even the pros use references for complex stuff every once in a while. So don`t feel bad if you need references to know how to draw a piece of clothing, an animal, hands or whatever is complicating you.