Use SALSA and Photon Voice for networked, multi-player character lip-synchronization!

What's cooler than real-time character lip-sync animations? Networked voice transmissions with real-time lip-sync! This blog post will quickly outline how to leverage ExitGames' free Photon Voice product with SALSA's real-time lip-synchronization (using a Photon Voice demo scene [included in the product]). If you don't like to read, watch the video! NOTE: the video demonstrates setup using a previous version (v1.6) of Photon Voice. The steps are the same for Photon Voice v1.8 (the most current as of this post).

There are FOUR basic steps for implementing lip-sync using Photon Voice and SALSA Lip-Sync (using the Photon Voice demo scene) :

1. Import the Photon Voice and SALSA packages into your project.
   • Configure your Photon Server Settings (SHFT-CTRL-ALT P to bring up your settings window)
     • You'll need to enter your AppID and Voice AppID codes
     • Get your codes from Exit Games > https://www.photonengine.com/Account/SignIn
   • Load the demo scene (Assets > PUNVoice > DemoVoice > DemoVoice-Scene)
2. Prepare a SALSA-enabled character (we'll use boxhead, included with SALSA)
   • Drag one of the existing DemoVoice characters into the scene from the Resources folder (Assets > PUNVoice > DemoVoice > Resources) -- we used ZomBunny
   • Rename the root node of the character prefab (i.e. "boxhead")
   • Delete the (ZomBunny) character model
   • Drop the boxhead model into the scene, under the boxhead object
   • Add SALSA to the root "boxhead" object from the component menu (Component > Crazy Minnow Studio > SALSA Lip-Sync > Salsa3D)
   • Link the boxhead model to the SALSA component's Skinned Mesh Renderer slot
   • Auto-link the shapes (click the Auto-Link) button; boxhead utilizes the recommended shape names and supports auto-linking
   • Adjust the trigger values in the SALSA Speech Properties section (most microphones don't produce audio that is sufficiently dynamic for processing)
   • Drag the new SALSA-enabled boxhead character into the Photon Voice resources folder (Assets > PUNVoice > DemoVoice > Resources)
   • Delete the boxhead object from the scene
3. Add the SALSA-enabled boxhead to the CharacterInstantiation component/script on the PUN scene object.
   • Remove the existing objects (ZomBunny, ZomBear, etc.)
   • Add the number of boxheads for the number of players expected in the game. (i.e. 2 boxhead entries for two-players)
4. Build your game and play! (You'll need to run at least 2 copies to see the network-voice-based lip-sync in action.)

Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~