(This post was created by Bing GPT): Have you ever wished that your child’s drawings could come alive and move around on the screen? Well, now they can, thanks to a new tool developed by Meta AI Research, a division of Meta (formerly Facebook). In this blog post, I will introduce you to Animated Drawings, a web-based prototype that uses AI to automatically animate children’s drawings of human-like figures.
Animated Drawings is a first-of-its-kind project that aims to help creators and developers easily create their own drawing-to-animation experiences or products. It started in 2021 when researchers from Meta AI Research decided to apply recent advances in computer vision towards a fast and intuitive pipeline to animate the human-like figures found in character drawings. Drawing is a familiar and fun mode of expression, and the characters we draw as beginners are often wonderfully odd, abstract, and imaginative. Who hasn’t wished that those delightful characters might come alive, moving around on the page?
The researchers envisioned using object detection models, pose estimation models, and image processing-based segmentation methods to quickly create a digital version of the drawing. This could then be deformed and animated using traditional computer graphics techniques. By teaching AI to work effectively with children’s drawings, they hope this project will move us closer to building AI that can understand the world from a human point of view.
To use Animated Drawings, you just need to upload your child’s drawing to the prototype website at https://sketch.metademolab.com/. The drawing should contain a human-like figure (i.e., a character with two arms, two legs, a head, etc.). The AI system will then try to identify and isolate the figure from the rest of the drawing, pinpoint its joints, and generate an animation that makes it dance, skip, and jump. You can also download a video of your animated drawing to share with friends and family. And you can opt in to have your child’s drawing used to continue to teach the AI model.
Note from Barb: Of course I tried something other than a child’s stick figure. First, I generated an image in Microsoft Designer.
Then, I uploaded this image to Meta’s app. Here’s what happened, including a few sample animations at the end of the process.
The reaction to Animated Drawings has been overwhelmingly positive. Within the first few months, users had uploaded and provided consent to use over 1.6 million images. Interestingly, many of the users tried to use it for purposes other than the original intent (as you can see from my experiment). They uploaded pictures of company logos, stuffed animals, anime characters, pets, action figures, and many other things that they wanted to animate. While the prototype specified the necessity for human figures in the instructions, users uploaded quadrupeds, birds, fish, and many other forms.
This shows that there is a huge demand for creative tools that can bring drawings to life using AI. That’s why Meta AI Research has decided to open-source the code for Animated Drawings and release a dataset of nearly 180,000 drawings to help researchers and creators continue innovating in this space. You can find more details on their GitHub page.
If you want to try it out yourself or learn more about the research behind it, you can visit https://sketch.metademolab.com/ Have fun animating your child’s drawings!