Use SALSA and Photon Voice for networked, multi-player character lip-synchronization!
~ Real-Time Lip-Sync with networked voice transmission ~
What's cooler than real-time character lip-sync animations? Networked voice transmissions with real-time lip-sync! This blog post will quickly outline how to leverage ExitGames' free Photon Voice product with SALSA's real-time lip-synchronization (using a Photon Voice demo scene [included in the product]). If you don't like to read, watch the video! NOTE: the video demonstrates setup using a previous version (v1.6) of Photon Voice. The steps are the same for Photon Voice v1.8 (the most current as of this post).
There are FOUR basic steps for implementing lip-sync using Photon Voice and SALSA Lip-Sync (using the Photon Voice demo scene) :
- Import the Photon Voice and SALSA packages into your project.
- Prepare a SALSA-enabled character (we'll use boxhead, included with SALSA)
- Drag one of the existing DemoVoice characters into the scene from the Resources folder (Assets > PUNVoice > DemoVoice > Resources) -- we used ZomBunny
- Rename the root node of the character prefab (i.e. "boxhead")
- Delete the (ZomBunny) character model
- Drop the boxhead model into the scene, under the boxhead object
- Add SALSA to the root "boxhead" object from the component menu (Component > Crazy Minnow Studio > SALSA Lip-Sync > Salsa3D)
- Link the boxhead model to the SALSA component's Skinned Mesh Renderer slot
- Auto-link the shapes (click the Auto-Link) button; boxhead utilizes the recommended shape names and supports auto-linking
- Adjust the trigger values in the SALSA Speech Properties section (most microphones don't produce audio that is sufficiently dynamic for processing)
- Drag the new SALSA-enabled boxhead character into the Photon Voice resources folder (Assets > PUNVoice > DemoVoice > Resources)
- Delete the boxhead object from the scene
- Add the SALSA-enabled boxhead to the CharacterInstantiation component/script on the PUN scene object.
- Remove the existing objects (ZomBunny, ZomBear, etc.)
- Add the number of boxheads for the number of players expected in the game. (i.e. 2 boxhead entries for two-players)
- Build your game and play! (You'll need to run at least 2 copies to see the network-voice-based lip-sync in action.)
Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~