AI tool automatically colors black and white photos

A Swedish machine learning researcher named Emil Wallner has released a free web tool called Palette.fm that automatically colors black and white photos using AI. After uploading a photo, users can choose a color filter or refine the colors with a written text description.

Palette.fm uses a deep learning model to classify images, which guides early guesses about the colors of objects in a photo or illustration. “I created a custom AI model that uses the image and text to generate a color,” Wallner wrote in a message to Ars. “One model creates the text and the other takes the image and text to generate the colorization.”

After you upload an image, the site’s streamlined interface provides an approximate caption (description) of what it thinks it sees in the image. If you don’t like any of the preset color filters, you can click the pencil icon to edit the caption itself, which guides the colorization model using a text prompt.

To test it, we took a photo of a small pumpkin and removed the color with Photoshop. Then we uploaded the black and white version and experimented with selecting the pre-made filters that Palette.fm provides. After finding a good filter, we edited the caption to refine the colors by describing the objects in the scene. For example, Palette.fm originally thought the pumpkin was a “claw” and didn’t recognize the sidewalk. But once we put those terms in the written prompt, the colors made more sense. We later refined the image further (not shown in the example below) by specifying “green leaves” in the background.

For now, Palette.fm is available as a free service, but Wallner plans to add a paid option. The site processes the images online, in the cloud. As for the privacy of the uploaded photos, the Palette.fm site states, “We do not store your images.” But as with any cloud service, take that with a grain of salt when it comes to private photos. Refreshingly, Palette.fm currently does not require any form of user account registration.

So far, Palette.fm has excited people on Hacker News who used the tool to colorize photos of beloved relatives, historical photos, and more. A variation of Wallner’s coloring technology is also available as a: bot on Twitter since late last year. Have fun coloring.