Steele Blog

Make your own anime!

Machine learning, specifically GANs, have come a long way and we are at the point where the combination of many of these currently diparate networks could now be combined to make a fully end to end anime-generator.

• Mapping Head motions onto 2D anime characters:
  • Take Anime Talking Heads
    • Creator's Blog has more in-depth instructions.
    • Somewhat similar to the "virtual youtuber" craze
• Dubbing can be acheived pretty accurately using:
  • Anime Text To Speech
  • Alternative project that could be used offline
    • Additional instructions
• Body Mapping - either from human to anime or anime to model:
  • SPLIFY-X
  • Photo into 3d Model

You now have an anime head, a pretty damn accurate body post estimator (with hands!), and a voice to match the lips moving.

Remaining Needs:

• Environment/Setting for the characters
  • Needs a way to change scenes
  • Need a way to move items
  • Could probably use a video game engine like Unreal Engine or Unity Engine
• Needs a narrative:
  • GPT2 via Hugging Face
• Getting a texture on the body mesh that looks like a school uniform?