Benj Edwards / Appel
On Wednesday, a San Francisco-based developer named Liu Liu released Draw Things: AI Generation, a free app available on the App Store that allows iPhone owners to use the popular Stable Diffusion AI image generator. Type a description and the app will generate an image in minutes. It’s a remarkable step in bringing image synthesis to a wider audience, with the added privacy of running it on your own hardware.
Stable Diffusion (SD), introduced in August, is an AI image generation model that creates new images based on text descriptions (called “prompts”). Typically, people use SD through the commercial DreamStudio service, on a remote cloud machine with leased compute time, or locally on a PC using a custom open source implementation. When used locally, SD requires a fairly powerful GPU to generate images quickly, but some developers have optimized the model to run on older GPUs with less VRAM (if you don’t mind waiting longer to see results).
In a similar vein, Liu Liu has managed to optimize Stable Diffusion for use on the iPhone, a somewhat difficult process described by the developer in a blog post. “The biggest challenge is getting the app to run on the 6GiB RAM iPhone devices,” writes Liu Liu. “6GiB sounds like a lot, but iOS will kill your app if you use more than 2.8 GiB on a 6GiB device and more than 2GiB on a 4GiB device.”
When Draw Things is first run, the app will download several necessary files, including the Stable Diffusion 1.4 model, to your iPhone. To use it, type a prompt at the top of the screen, then tap “Generate.” Between image generation, tap the number in the top center of the screen to randomize the seed, a number that partially guides the image generation.
Benj Edwards
On our iPhone 11 Pro, generating a 384×384 image took just over two minutes. It’s faster on an iPhone 14 Pro, according to Liu Liu, and generates an image in about a minute. Either way, SD is computationally intensive. After successive generations, our iPhone became remarkably warm to the touch.
It is worth noting that with Stable Diffusion, 384 × 384 images often generate relatively poor results with little detail, as the creators of SD trained the model using 512 × 512 images. When we tried to generate a 512×512 image on our iPhone 11 Pro, we got a warning and continued anyway, but the app crashed to a black screen.
In addition to the regular image generation tasks, Draw Things also supports inpainting, which allows you to replace part of an image with AI-generated images, and load additional image synthesis models, such as the unauthorized “Modern Disney Diffusion” model (which Disney – characters) and the anime-powered “Waifu Diffusion” model (we tested and it is possible to generate NSFW material using the app, so be warned). The inclusion of these capabilities means that Draw Things may not stay in the App Store for long if it becomes popular, as the content it generates may violate Apple’s terms of service.
Whatever the fate, Draw Things feels like an important proof-of-concept, showing that Stable Diffusion can run locally on the iPhone, even if it’s slow. If the app persists—perhaps with some filters or tweaks—Liu Liu describes room for potential future optimizations that could speed things up: “I probably left another 50% performance on the table.”
This isn’t the first time Stable Diffusion has run on an iPhone. In September, developer Matt Waller walked Stable Diffusion locally on his iPhone XS, but it didn’t provide an App Store app that would allow others to replicate the feat. We will likely see more local AI image synthesis on smartphones as the computing power of the devices continues to increase.
Go to discussion…
