Logo - Crazy Minnow Studio - game, asset, tool development

Welcome to Crazy Minnow Studio! We are a small indie software development team primarily focused on game development using the Unity engine. Our pipeline includes: games, game development tools and assets, and video tutorial production. Check out our Unity lip sync asset, SALSA Lip-Sync, available on the Unity Asset Store. Follow our blog for updates on our Unity asset and game development, as well as other happenings in indie game development.

Using SALSA Lip-Sync with Photon Voice

Use SALSA and Photon Voice for networked, multi-player character lip-synchronization!

Tested with Unity 5.5, SALSA 1.4.0, and Photon Voice 1.8 (2017-01-31):
~ Real-Time Lip-Sync with networked voice transmission ~

What's cooler than real-time character lip-sync animations? Networked voice transmissions with real-time lip-sync! This blog post will quickly outline how to leverage ExitGames' free Photon Voice product with SALSA's real-time lip-synchronization (using a Photon Voice demo scene [included in the product]). If you don't like to read, watch the video! NOTE: the video demonstrates setup using a previous version (v1.6) of Photon Voice. The steps are the same for Photon Voice v1.8 (the most current as of this post).

There are FOUR basic steps for implementing lip-sync using Photon Voice and SALSA Lip-Sync (using the Photon Voice demo scene) :

  1. Import the Photon Voice and SALSA packages into your project.
    • Configure your Photon Server Settings (SHFT-CTRL-ALT P to bring up your settings window)
      • You'll need to enter your AppID and Voice AppID codes
      • Get your codes from Exit Games > https://www.photonengine.com/Account/SignIn
    • Load the demo scene (Assets > PUNVoice > DemoVoice > DemoVoice-Scene)
  2. Prepare a SALSA-enabled character (we'll use boxhead, included with SALSA)
    • Drag one of the existing DemoVoice characters into the scene from the Resources folder (Assets > PUNVoice > DemoVoice > Resources) -- we used ZomBunny
    • Rename the root node of the character prefab (i.e. "boxhead")
    • Delete the (ZomBunny) character model
    • Drop the boxhead model into the scene, under the boxhead object
    • Add SALSA to the root "boxhead" object from the component menu (Component > Crazy Minnow Studio > SALSA Lip-Sync > Salsa3D)
    • Link the boxhead model to the SALSA component's Skinned Mesh Renderer slot
    • Auto-link the shapes (click the Auto-Link) button; boxhead utilizes the recommended shape names and supports auto-linking
    • Adjust the trigger values in the SALSA Speech Properties section (most microphones don't produce audio that is sufficiently dynamic for processing)
    • Drag the new SALSA-enabled boxhead character into the Photon Voice resources folder (Assets > PUNVoice > DemoVoice > Resources)
    • Delete the boxhead object from the scene
  3. Add the SALSA-enabled boxhead to the CharacterInstantiation component/script on the PUN scene object.
    • Remove the existing objects (ZomBunny, ZomBear, etc.)
    • Add the number of boxheads for the number of players expected in the game. (i.e. 2 boxhead entries for two-players)
  4. Build your game and play! (You'll need to run at least 2 copies to see the network-voice-based lip-sync in action.)
Buy SALSA on the Asset Store

Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~

Categories: SALSA, Tutorials, Video

Comments: No comments yet