Geppetto – AI-Powered Lip Sync Animation

Automating Facial Expression and Speech Timing

Geppetto is an animation tool designed to simplify the creation of realistic lip-sync and facial movements within Unity. By utilizing advanced AI algorithms, the system processes audio or text files to automatically generate animations that align with the specific timing and tone of a character’s speech. This automation allows developers to bypass the manual labor of keyframing every mouth movement, focusing instead on broader creative aspects of the project.

The tool is versatile enough to support both 3D and 2D character models. While the AI handles the initial generation, the interface provides options to customize the results. Users can adjust mouth shapes and timing to ensure the animation matches the visual style of the character, whether they are working on a game, an animated film, or interactive media.

Integration Requirements and Blendshape Workflow

A fundamental technical requirement for using Geppetto is the presence of blendshapes. The animation system operates exclusively through blendshape manipulation, meaning character models must already have these shapes configured to respond to the tool. If a character lacks blendshapes, the developer recommends using external software such as Blender or Maya to create them before importing the model into Unity.

While the tool is optimized for humanoid characters, it remains compatible with other character models provided they utilize a blendshape-based setup. This flexibility ensures that unique or non-humanoid entities can still benefit from AI-driven lip-syncing as long as the underlying mesh data is prepared correctly.

Production Implementation and Pipeline Compatibility

The asset fits into a production workflow in two primary ways. First, it serves as a time-saving utility for pre-generating all lip animations from existing scripts or voiceover files. This is particularly useful for projects requiring extensive dialogue where manual animation would be prohibitive. Second, it can be paired with speech synthesis or text generation tools to create a more automated content pipeline.

Geppetto is compatible with Unity versions 2022 and later, though its original foundation was built on version 2021.3.8. It supports various render pipelines, including the Built-in, HDRP, URP, and Custom SRP. The package itself is lightweight, containing 22 assets within a unitypackage format, making it easy to integrate into existing project structures without significant bloat.

Processing via Internal API and Offline Playback

The animation generation process relies on an internal API, which necessitates an active internet connection during the creation phase. The AI analyzes the input data remotely to produce the high-quality movement data required for lifelike expressions. However, this online requirement is limited strictly to the generation stage.

Once the animations are generated and saved within the project, they are fully functional offline. This ensures that the final build of a game or application does not require a persistent connection for the characters to speak. Recent updates to the tool have also introduced unlimited usage during the production phase, removing previous constraints on how often developers can iterate on their character animations.

To assist with the initial setup, the developer provides example files and documentation through a dedicated repository. This allows users to examine functional implementations and understand how to properly map their character’s blendshapes to the AI-generated outputs.