After the launch of the IBM Watson Unity SDK, we were curious about how the text-to-speech service might work with SALSA to deliver real-time text-to-speech-based lip-sync. This can already be achieved with the wonderful RT-Voice asset from Crosstales, but we take pride in SALSA working with as many solutions as possible to make our customers options as broad as possible. After signing up for a free account and having a look at the documentation, we found the IBM Watson text-to-speech service simple to use and a great pairing with SALSA lipsync.

NOTE: This is a simple demonstration script. Implementation of Watson or the Unity Watson SDK is beyond the support scope of Crazy Minnow Studio and/or SALSA Lip-Sync. It is a proof-of-concept and not a full implementation of the Watson API. Implementation support questions should be directed to IBM's support tiers. Crazy Minnow Studio does not offer support for Watson implementation of any kind. Please also note, the Watson API support team does not support Watson on a Web-Player or WebGL platform.

SALSA requires no special handling to work with any service that produces output via a standard Unity AudioSource. Likewise, if your implementation of Watson text-to-speech utilizes a standard Unity AudioSource, SALSA should work without issue. Simply output the Watson-generated AudioClip to the same AudioSource component that SALSA references and you're good to go. I've included a very basic script below that demonstrates the process.

05/07/2019 - Watson for Unity has been deprecated.

Code Example

NOTE: While every attempt has been made to ensure the safe content and operation of these files, they are provided as-is, without warranty or guarantee of any kind. By downloading and using these files you are accepting any and all risks associated and release Crazy Minnow Studio, LLC of any and all liability.

using System.Collections;
using System.Collections.Generic;
using UnityEngine;
using IBM.Watson.DeveloperCloud.Services.TextToSpeech.v1;
using IBM.Watson.DeveloperCloud.Utilities;
using IBM.Watson.DeveloperCloud.Connection;
using IBM.Watson.DeveloperCloud.Logging;

public class WatsonTTS : MonoBehaviour
{
    public string url; // Your IBM Watson URL
    public string user; // Your IBM Watson username
    public string pass; // Your IBM Watson password
    public string text = "Hello SALSA, I'm Watson, your IBM services representative, how can I help you?";
    public bool play;

    private Credentials credentials;
    private TextToSpeech textToSpeech;
    private AudioSource audioSrc;

    void Start ()
    {
        credentials = new Credentials(user, pass, url);
        textToSpeech = new TextToSpeech(credentials);
        audioSrc = GetComponent<AudioSource>(); // Get the SALSA AudioSource from this GameObject
    }

    private void Update()
    {
        if (play)
        {
            play = false;
            GetTTS();
        }
    }

    private void GetTTS()
    {
        textToSpeech.Voice = VoiceType.en_US_Michael;
        textToSpeech.ToSpeech(OnSuccess, OnFail, text, false);
    }

    private void OnSuccess(AudioClip clip, Dictionary<string, object> customData)
    {
        if (Application.isPlaying && clip != null && audioSrc != null)
        {
            audioSrc.spatialBlend = 0.0f;
            audioSrc.clip = clip;
            audioSrc.Play();
        }
    }

    private void OnFail(RESTConnector.Error error, Dictionary<string, object> customData)
    {
        Log.Error("ExampleTextToSpeech.OnFail()", "Error received: {0}", error.ToString());
    }
}

NOTE: While every attempt has been made to ensure the safe content and operation of these files, they are provided as-is, without warranty or guarantee of any kind. By downloading and using these files you are accepting any and all risks associated and release Crazy Minnow Studio, LLC of any and all liability.