The AI-drawing tool I wish I had

Could AI teach me to draw better?  — 

Image generators are an impressive improvement to stock image search and scrolling through Pinterest for inspiration. But they have not yet improved my illustration workflow. I wonder why.

Prompting is not most natural way to express visual ideas. When I try to communicate my idea for an illustration to someone, I use a sketch, not a description. It's really hard to express an illustration as a text. Even with the most elaborate prompt, the output is never even close to the glorious idea I have in mind. "No, the unicorn should be on the left not on the right!" You know what I mean. To be fair, I am not exactly great at translating my own ideas onto virtual paper either — those who can usually have decades of experience.

The AI-drawing tool I wish I had would take my sketches and lift my work closer these masters of the craft. When I see what AI has done for coding and text, I think it should be possible. What excites me about large language models isn’t that they can create whole applications or finished newspaper articles — it’s how they can anticipate and complete the code I’m about to write or help improve my writing, especially in a language that isn’t my native one. Couldn’t something similar work for illustration? Here are two examples to illustrate what I mean.

Auto-complete

While working on this illustration of a conductor, I start adding shadows. After a couple of them, the light source is clear, and the assistant begins proposing additional shadows. With a keystroke, I can either accept or ignore the suggestion.

I start to add shadows to my illustration. The AI gets what I am trying to do and proposes additional shadows. I can accept them or continue to give the AI more hints of what I am doing.

Correction

Hands are famously hard to get right—even for early image-generating models. But I’m hardly in a position to judge, as my own attempts often look worse. Ideally, an assistant could recognize the hand position I’m aiming for and suggest corrected lines that fit with the context. Again, I can accept or move on if the AI hasn’t quite grasped what I’m going for.

This interaction — drawing and refining with AI, rather than relying on cryptic commands — would make the tool feel much more satisfying. And instead of becoming an expert at prompting – a skill that may be pretty short-lived – I may even improve my drawing skills.

I sketch a hand in a specific pose. Again, the AI understands what I am trying to do and proposes more accurate linework.

Recently, Adobe introduced tools that seem to be moving in this direction: 3D-rotate and sketch-to-vector. And while they do something along the lines of what I'm describing, they are not nearly responsive enough yet. Still, they make me hopeful that someone out there is already building the AI drawing tool of my dreams.