Logo - Crazy Minnow Studio - game, asset, tool development

Welcome to Crazy Minnow Studio! We are a small indie software development team primarily focused on game development using the Unity engine. Our pipeline includes: games, game development tools and assets, and video tutorial production. Check out our Unity lip sync asset, SALSA Lip-Sync, available on the Unity Asset Store. Follow our blog for updates on our Unity asset and game development, as well as other happenings in indie game development.

Using 3D Blendshape Assets with 2D Characters for Lip-sync

Blendshape Lip-sync Animations for 2D Characters

SALSA and RandomEyes both work with 2D and 3D assets for lip-sync and eye-movement functionality. Typically, the 2D developer will leverage the asset's sprite animation technology and 3D devs will use the blendshape animation tech. Either of these techniques work and look great in both SALSA and RandomEyes. But what if you want your 2D character to leverage blendshape technology for smooth animation transitions? Unity maintains a pretty solid distinction between sprites and skinned mesh blendshapes, meaning sprites don't have blendshape functionality. However, what if we pair up a mesh mouth, complete with skinned mesh renderer and SALSA lip-sync blendshapes with the 2D sprite character? Best of both worlds? Yes!

Blender mouth shapes for SALSA lip-syncThe process is relatively easy and the lip-synchronization results are quite nice. In the video example below, we used Blender to create the mesh and the 3 recommended lipsync blendshapes for this demo. The original "girl" character's mouth-shape sprite images were loaded into Blender for reference and then traced, creating Blender shape keys (blendshapes). The resultant mesh is a completely flat representation of the sprite assets that is aligned on the character in the same manner as the original sprite images (as shown in the animated gif).

3D mesh blendshaps aligned on 2D characterFinally, instead of using Salsa2D, we add Salsa3D to the mouth mesh object (with the SkinnedMeshRenderer and blendshapes). SALSA is then configured normally and the resultant animations are a smooth blend from shape to shape, in contrast to 2D-based sprite-swaps. Neither method is better or worse than the other, they are simply two unique stylizations of animation and one may be preferred over the other based purely upon aesthetic feel.

Please watch this short video for a side-by-side comparison of the two technologies.

This short video demonstrates a side-by-side comparison of the two technologies:

Blendshapes for 2D lip-sync video link

Also check out our 3 part video series on creating shape keys (blendshapes) in Blender!

Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~

Categories: SALSA, Video

Comments: No comments yet