This add-on is no longer supported, click here to switch to our UMA-DCS add-on.

UMA 2 - Unity Multipurpose Avatar is an amazing character generation system, and we're thrilled to announce the release of our workflow for adding SALSA lipsync and RandomEyes eye movement and blinking to your UMA 2 characters. SALSA with RandomEyes, for 3D characters, is a BlendShape-based system, while UMA 2 is a bone-based system.

The key features in UMA 2 that allows us to add support are the UMAExpressionSet and UMAExpressionPlayer components. The UMAExpressionPlayer adds adjustable value-based facial expressions that we can adjust using data from SALSA and RandomEyes. This workflow suppports SALSA lipsync, RandomEyes eye movement, blinking, and look target tracking with target affinity (works like an attention span), and facial expressions.

PLEASE NOTE: These instructions require you to download and install the appropriate 1-Click asset scripts in your Unity project. If you skip this step, you will not find the option for 1-Click in the menu.

*NOTE: RandomEyes 3D also normally supports unlimited custom BlendShapes used for facial expression; however, since UMA 2 characters have no BlendShapes, this feature is unavailable on UMA 2 characters. Instead, we provide a custom facial expression controller for the UMAExpressionPlayer.

---

11/26/2017 - v1.5 Final update. Since UMA has moved on to the DCS system, so have we.

---

Installation Instructions

1. Install SALSA with RandomEyes into your project.
   • Select [Window] -> [Asset Store]
     • Once the Asset Store window opens, select the download icon, and download and import [SALSA with RandomEyes].
2. Install UMA 2 into your project.
   • Select [Window] -> [Asset Store]
     • Once the Asset Store window opens, select the download icon, and download and import [UMA 2 - Unity Multipurpose Avatar].
3. Import the SALSA with RandomEyes UMA Character support package.
   • Select [Assets] -> [Import Package] -> [Custom Package...]
     • Browse to the [SALSA_3rdPartySupport_UMA2.beta1.unitypackage] file and [Open].

---

Quick Start Instructions

NOTE: Ensure you have downloaded and imported the 1-Click files into your Unity project.

(Using the included UMA character creator script / Runtime)

1. In a new scene, use the included 1-click setup from the GameObject menu to add all necessary objects and components.
   • [GameObject] -> [Crazy Minnow Studio] -> [UMA 2] -> [SALSA 1-Click UMA Setup]
2. Play the scene.

(Using a saved UMA.asset file / Design-Time)

1. In a new scene, use the included design-time setup from the GameObject menu to add all necessary objects and components.
   • [GameObject] -> [Crazy Minnow Studio] -> [UMA 2] -> [SALSA UMA Design-Time Setup]
2. On the SALSA_UMA2 object, add a [Character Created] event to the [UMADynamicAvatar] component,
           then drag SALSA_UMA2 to the Object field and select [CM_UmaSync] -> [CharacterCreated()].
3. Play the scene.

---

Implementation Instructions

(Runtime implementation)

1. Use, modify, or replace [CM_UmaBasic.cs] with your own character creation setup.
2. If you implement your own character creation script, use [CM_UmaBasic.cs] for reference, you will need to implement a few lines of code. Declare the following public string, delegate, and static event variables. These will be used to pass necessary data from the character creation script to CM_UmaSync.cs.

   
   // A string to hold a unique character name for this UMA.
   // CM_UmaSync.cs also has a public characterName string.
   // Each character creation script needs a unique characterName
   // and a corresponding CM_UmaSync.cs script with a matching
   // charcterName string variable.
   public string characterName = "UMA_SALSA_1";
   
   // Delegate to hold the UMAData object instance and characterName parameters
   public delegate void UMACreated(UMAData obj, string characterName); 
   
   // Event for CM_UmaSync.cs to subscribe to and receive the UMAData
   // object instance and characterName parameters
   public static event UMACreated OnUMACreated;
   

   After you initialize your UMADynamicAvatar, subscribe to the UMAData object instance OnCharacterCreated event. See this line of code in CM_UmaBasic.cs for an example.

   
   umaDynamicAvatar.umaData.OnCharacterCreated += UmaData_OnCharacterCreated;
   

   This will create an UmaData_OnCharacterCreated event method stub. Remove the Exception code if added automaticallly, and add a null check and event call for our OnUMACreated event. Now when our UMA gets created, this event will be caught and we'll fire our own OnUMACreated event and pass the UMAData object instance and characterName parameters to any subscribers (CM_UmaSync.cs).

   
   private void UmaData_OnCharacterCreated(UMAData obj)
   {			
   	// OnUMACreated will not be null 
   	// when it has subscribers (CM_UmaSync.cs)
   	if (OnUMACreated != null)
   	{
   		// Fire the event so that subscriber can 
   		// retreive the UMAData object and characterName
   		OnUMACreated(obj, characterName);
   	}
   }
   

   CM_UmaSync.cs can't subscribe to the OnCharacterCreated event directly because the UMAData object instance doesn't exist until runtime.

(Design-Time implementation)

1. Create your UMA and save it as an asset file.
   • [UMA] -> [Load and Save] -> [Save Selected Avatar(s) asset]
2. In a new scene, select 
   • [GameObject] -> [Crazy Minnow Studio] -> [UMA 2] -> [SALSA UMA Design-Time Setup]
   • This will create an UMA libraries GameObject called [UMA_Config], and an example UMA.asset character.
3. Delete the [SALSA_UMA2] GameObject, and drag your own UMA.asset file into the scene.
4. Add two components to your UMA.asset GameObject in the Hierarchy:
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaSync]
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaExpressions]
5. On the [UMADynamicAvatar] component, add a new [Character Created] event listener by clicking the [+] plus symbol then drag the UMA.asset GameObject from the hierarchy to the Object field in the new [Character Created] event listener. Then select [CM_UmaSync] -> [CharacterCreated()]

---

Using the new CM_UmaExpressions.cs script

RandomEyes3D provides a Custom Shapes system that allows any number of BlendShapes to be linked to the system and made accessible via the RandomEyes3D API. However, since UMA does not use BlendShapes, this script provides key RandomEyes3D Custom Shapes like behavior for the UMAExpressionPlayer facial expressions.

Instructions: 

1. Add CM_UmaExpression.cs alongside the CM_UmaSync.cs script.
2. From your custom script, include the CrazyMinnow.SALSA.UMA namespace.
3. Get reference to CM_UmaExpression.cs using standard Unity methods.
4. Call one of the two available public SetExpression methods.
   • To activate an expression for a specified duration:
   • umaExpressionHelper.SetExpression(CM_UmaExpressions.Expression expression, float blendSpeed, float rangeOfMotion, float duration);
     • ​Expression: umaExpressionHelper.Expression.leftMouthSmile_Frown
     • Blend Speed: How quickly the expression will reach it's specified amount e.g. 3.5f
     • Range Of Motion: From -1 to 1, with 0 being no expression
     • Duration: How long (in seconds) to hold the expression once the target amount has been reached
   • ​​To activate or deactivate an expression without any specified duration:
   • umaExpressionHelper.SetExpression(CM_UmaExpressions.Expression expression, float blendSpeed, float rangeOfMotion, bool isOn);
     • Expression: umaExpressionHelper.Expression.leftMouthSmile_Frown
     • BlendSpeed: How quickly the expression will reach it's specified amount e.g. 3.5f
     • Range Of Motion: From -1 to 1, with 0 being no expression
     • IsOn: Does not use a timer, the expression either blends to on or off.

See the example script below that demonstrates activating multiple expressions using a timed trigger.

using UnityEngine;
using CrazyMinnow.SALSA.UMA;

public class UmaExpression : MonoBehaviour
{
	public CM_UmaExpressions umaExpressions; // Reference to CM_UmaExpressions
	public CM_UmaExpressions.Expression[] expressions; // Array of expressions
	[Range(-1f, 1f)]
	public float rangeOfMotion = 1f; // Range of motion from -1 to 1
	public float blendSpeed = 3.5f; // How quickly the range of motion is reached
	public float duration = 2f; // How long the expression is held once the range of motion is reached
	public float triggerTime = 3f; // When to trigger this expresion (3 seconds after running the game)

	private bool triggered = false; // To ensure the trigger only occurs once

	/// <summary>
	/// Get reference to the CM_UmaExpressions script
	/// </summary>
	void Start()
	{
		if (!umaExpressions) umaExpressions = GetComponent<CM_UmaExpressions>();
	}

	/// <summary>
	/// Timer-based trigger
	/// </summary>
	void Update()
	{		
		if (Time.time >= triggerTime && !triggered)
		{
			triggered = true;
			Expression();
		}
	}

	/// <summary>
	/// Activates all of the expressions in the array using 
	/// the same blendSpeed, rangeOfMotion, and duration
	/// </summary>
	public void Expression()
	{
		for (int i=0; i<expressions.Length; i++)
		{
			umaExpressions.SetExpression(expressions[i], blendSpeed, rangeOfMotion, duration);
		}
	}
}

What [SALSA 1-Click UMA Setup] does

1. In instanciates an instance of the [UMA_Config] prefab into the scene. This is the UMA libraries.
2. It instanciates an empty GameObject [SALSA_UMA2] into the scene and adds the following components:
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaBasic] (UMA character creator)
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaSync] (Adds UMAExpressionSet, UMAExpressionPlayer, Salsa3D, and optionally RandomEyes3D to the UMA)
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaExpressions]
3. On the CM_UmaBasic component, it links the Generator, Slot Library, Overylay Library, Race Library, and the Anim Controller.
4. On the CM_UmaSync component, it links the mil.moves wav file from the Crazy Minnow Studio examples.

What [SALSA Design-Time UMA Setup] does

1. It instanciates an instance of the [UMA_Config] prefab into the scene.
2. It instanciates an empty GameObject [SALSA_UMA2] into the scene and adds the following components:
   • [UMADynamicAvatar]
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaSync] (Adds UMAExpressionSet, Salsa3D, and optionally RandomEyes3D to the UMA)
   • [Component] -> [Crazy Minnow Studio] -> [UMA 2] -> [CM_UmaExpressions]
3. It links the UMA.asset file to the UMA Recipe filed on the UMADynamicAvatar component.
       Assets/Crazy Minnow Studio/SALSA with RandomEyes/Third Party Support/UMA 2/Prefabs/UMA.asset
4. It links the Locomotion animation controller to the UMADynamicAvatar component.
       Assets/UMA/Example/Animators
5. It set the Load On Start property to true on the UMADynamicAvatar component.
6. On the CM_UmaSync component, it sets the Mode to DesignTime, links the UMADynamicAvatar, and adds an AudioClip.
7. After this:
   • You will need to add an event listenter to the UMA Dynamic Avatar Character Created (UMAData) event. Link this object to the event object field, then select CharacterCreated from CM_UmaSync.

---

---

Using SALSA and our live microphone input with UMA 2 characters

using UnityEngine;
using UnityEngine.Audio;
using CrazyMinnow.SALSA;

namespace CrazyMinnow.SALSA.UMA
{
	/// <summary>
	/// This example script demonstrates how to use SALSA and 
	/// our live micrphone workflow with UMA 2 characters.
	/// </summary>
	public class CM_UmaLiveMic : MonoBehaviour
	{
		// Create a new Group in the AudioMixer, set the Attenuation to -80
		public AudioMixerGroup audioGroup;
		// Mic input reference
		private CM_MicInput_Salsa3D mic; 

		/// <summary>
		/// Subscribe to the OnSalsaCompleted event in CM_UmaSync.cs
		/// </summary>
		void OnEnable()
		{
			CM_UmaSync.OnSalsaCompleted += CM_UmaSync_OnSalsaCompleted;
		}

		/// <summary>
		/// Unsubscribe from the OnSalsaCompleted event in CM_UmaSync.cs
		/// </summary>
		void OnDisable()
		{
			CM_UmaSync.OnSalsaCompleted -= CM_UmaSync_OnSalsaCompleted;
		}

		private void CM_UmaSync_OnSalsaCompleted(Salsa3D salsa3D)
		{
			// Redirect audio to the AudioMixerGroup so we can pass data
			salsa3D.GetComponent<AudioSource>().outputAudioMixerGroup = audioGroup;
			// Add the microphone input script
			mic = salsa3D.gameObject.AddComponent<CM_MicInput_Salsa3D>();
			// Unmute the mic to allow data to flow through the AudioSource
			mic.isMuted = false;
		}
	}
}

---

Using SALSA and RT-Voice runtime text-to-speech with UMA 2 characters

using UnityEngine;
using CrazyMinnow.SALSA;
using Crosstales.RTVoice;

namespace CrazyMinnow.SALSA.UMA
{
	/// <summary>
	/// This example script demonstrates how to use SALSA and the the RT-Voice 
	/// runtime text-to-speech asset with UMA 2 characters.
	/// 
	/// Make sure the RT-Voice Speaker component has been added to your scene.
	/// </summary>
	public class CM_UmaTTS : MonoBehaviour
	{
		// Text to pass to RT-Voice
		public string ttsText;
		// AudioSource reference
		private AudioSource audioSource;

		/// <summary>
		/// Subscribe to the OnSalsaCompleted event in CM_UmaSync.cs
		/// </summary>
		void OnEnable()
		{
			CM_UmaSync.OnSalsaCompleted += CM_UmaSync_OnSalsaCompleted;
		}

		/// <summary>
		/// Unsubscribe from the OnSalsaCompleted event in CM_UmaSync.cs
		/// </summary>
		void OnDisable()
		{
			CM_UmaSync.OnSalsaCompleted -= CM_UmaSync_OnSalsaCompleted;
		}

		private void CM_UmaSync_OnSalsaCompleted(Salsa3D salsa3D)
		{
			// Get a reference to the SALSA AudioSource
			audioSource = salsa3D.gameObject.GetComponent<AudioSource>();
			// Send text to and our AudioSource reference to RT-Voice
			Speaker.Speak(ttsText, audioSource, Speaker.VoicesForCulture("en")[0], true);
		}
	}
}

---

NOTE: While every attempt has been made to ensure the safe content and operation of these files, they are provided as-is, without warranty or guarantee of any kind. By downloading and using these files you are accepting any and all risks associated and release Crazy Minnow Studio, LLC of any and all liability.

Download Files

Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~

Buy SALSA on the Asset Store

---