Blendshape Lip-sync Animations for 2D Characters

SALSA and RandomEyes both work with 2D and 3D assets for lip-sync and eye-movement functionality. Typically, the 2D developer will leverage the asset's sprite animation technology and 3D devs will use the blendshape animation tech. Either of these techniques work and look great in both SALSA and RandomEyes. But what if you want your 2D character to leverage blendshape technology for smooth animation transitions? Unity maintains a pretty solid distinction between sprites and skinned mesh blendshapes, meaning sprites don't have blendshape functionality. However, what if we pair up a mesh mouth, complete with skinned mesh renderer and SALSA lip-sync blendshapes with the 2D sprite character? Best of both worlds? Yes!

The process is relatively easy and the lip-synchronization results are quite nice. In the video example below, we used Blender to create the mesh and the 3 recommended lipsync blendshapes for this demo. The original "girl" character's mouth-shape sprite images were loaded into Blender for reference and then traced, creating Blender shape keys (blendshapes). The resultant mesh is a completely flat representation of the sprite assets that is aligned on the character in the same manner as the original sprite images (as shown in the animated gif).

Finally, instead of using Salsa2D, we add Salsa3D to the mouth mesh object (with the SkinnedMeshRenderer and blendshapes). SALSA is then configured normally and the resultant animations are a smooth blend from shape to shape, in contrast to 2D-based sprite-swaps. Neither method is better or worse than the other, they are simply two unique stylizations of animation and one may be preferred over the other based purely upon aesthetic feel.

Please watch this short video for a side-by-side comparison of the two technologies.

This short video demonstrates a side-by-side comparison of the two technologies:

Also check out our 3 part video series on creating shape keys (blendshapes) in Blender!

Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~