I agree with what @TheLemmaLlama said about direction. Because of all the types of artists out there, I think it's the easiest to tell when a writer does not know what they want. ^^; Stories are meant to communicate something, whether it's a moral or a message or an emotion or just an experience. A great story knows what that is and conveys it well.
Personally, I usually have to have some kind of message in mind when I write a story...I need a guiding objective to prevent me from derailing the entire project into a showcase of whoever my favorite character happens to be. ⊙u⊙;
Besides, I never really get anywhere until I have something I want to say. For me, that's the difference between having a collection of cute character designs in a setting, and an actual STORY.
Take two recent fantasy novels I have in the works: One of them started out as just that, a collection of cute character designs in a setting. Only, at the time I came up with the idea, I had already matured enough as a writer to understand that I needed a goal to work towards if I wanted to have a story.
I decided on one pretty early on, and I keep it tacked up on top of the Word doc as a reminder: "I think that Monastery! is a story about how people can be saved by love and acceptance, at any age and at any time."
Seeing that every time I open it up to work really helps to keep me focused...it was my first time writing a coming-of-age/slice-of-life story in prose, and I honestly expected to make a mess of it. On the contrary, I'm happy to announce that I finished its first Volume last night. ^^
Now, the other story I've been working on recently is actually a reboot of a very, very, VERY old idea that I've failed at a few times already. I did a couple new character designs, fleshed out the worldbuilding, polished up the lore a little bit, and I was feeling very excited...only to suddenly realize that the story did not have a plot. ._.
Worse than that, it never did have one...I'd spent so much time working on this fantasy story as a kid, struggling to get it started without realizing there was no destination in sight. The characters had no development, their adventure had no real purpose...all I had was the vague idea of an epic fantasy and some cute character designs. I did have thousands of words' worth of lore, yes, but there was nothing I wanted to say.
...It was really demoralizing, realizing that. ^^; I felt like such an idiot for not seeing it sooner...I'm still a little mad at myself, tbh. But now I'm working on finding a message for this story, and for each of its characters, and I think that someday when I am actually able to write it, it'll turn out great.