SALSA Overview

The SALSA lip-sync Unity asset provides fully automated lip sync approximation for 2D and 3D game characters. Unlike some lip-sync packages, effective SALSA animation does not rely on time-consuming, phoneme mapping (or key framing) to map expressions to audio files. Instead, it uses a combination of waveform analysis and four mouth shapes to produce high-quality, lipsync approximation, with minimal effort.

SALSA lip-sync comes with two components: Salsa2D for animation of sprites (2D characters) and Salsa3D for animation of BlendShapes (3D characters). Whether you're animating characters using Salsa2D or Salsa3D, you simply provide four approximate mouth positions as either sprite images (Salsa2D), or mesh BlendShapes (Salsa3D).

NOTE: With BlendShapes, sayRest is your base shape (all other shapes set to 0), so you only need to create saySmall, sayMedium, and sayLarge.

While your mouth expressions don't need to look exactly like the above graphics, it is recommended that you follow the size and scale references (the blue-dotted lines in the graphic above) as closely as possible for the best animation effect. After creating your expression images for 2D, or Blendshapes for 3D, bind the shapes and your audio file to the appropriate SALSA component in the inspector. During playback, SALSA will analyze your audio clip in real-time and automatically animate the appropriate expressions. For Salsa2D, the sprites will be swapped like a cartoon cell animation. For Salsa 3D, the BlendShapes will be blended to create smooth transitions.