Unity Timeline for SALSA AudioClips, Emotes, and SalsaTextSync
NOTE: The following information is more or less accurate for SALSA LipSync v2 -- some interpretation may be required until we can get the documentation properly updated.
Tested with Unity versions 2017.1 - 2019.1 (release)
PLEASE NOTE: TimelineSalsaCore is currently available for SALSA LipSync Suite v2. The example scene uses a couple of audio files from the EXAMPLES-Core package which has not yet been released. For testing, you may substitute your own audio if you wish. The majority of the information in this legacy blog post is applicable and you should be able to get by until we can get proper documentation updated for the new add-on.
Timeline functionality for SALSA Lip-Sync is now called "Unity Timeline SALSA Core" and contains functionality for sequencing AudioClips on a SALSA-enabled character as well as a new option to sequence character emotes, leveraging RandomEyes3D configuration of custom-shape groups. In addition to the SALSA Core functionality, we also provide a separate Timeline implementation for our free SALSA TextSync add-on. For implementation instructions, check out the videos below. And download the files in the SALSA Downloads area.
UPDATE: v2.0.0 (2019-06-20): TimelineSalsaCore released for SALSA LipSync Suite v2.
The EmoteClip action: When used on an Emote Track allows configuration and timing of EmoteR custom-shape groups. Simply add an Emote Track to the Timeline and add Emote Clips to the track. Select the emote clip in the Timeline and configure the name of the EmoteR emote to drive (case-sensitive).
Ensure your EmoteR component is linked to the track binding and your emotes are properly configured. NOTE: the same EmoteR component instance can be linked to multiple Emote Track bindings to create NLE-styled overlaps/blends of emotes.
Ensure the EmoteR emotes are configured and the emote names match the names configured on the Timeline Emote Clips.
[image coming soon...]
The AudioClip action: When used on a Salsa Audio Track allows configuration and timing of AudioClips to drive SALSA's lipsync approximation algorithms. The Salsa Audio Track is configured to point to the AudioSource used by SALSA. Unity maintains control focus for AudioClips dragged onto a Timeline track, so it is necessary to configure the specific AudioClip in the Inspector. Once an AudioClip is configured for the track clip, the clip container in the track will dynamically adjust its length to be the duration of the AudioClip. Additionally, the clip container's name will also be dynamically changed to that of the AudioClip name.
The TextSync action: When used on a Salsa Text Sync Track allows for configuration and timing of TextSync text on a Timeline track to drive SALSA's implementation of text-to-lipsync via SalsaTextSync. It works similarly to the Audio Track and dynamically adjusts its length to be representative of the SalsaTextSync processing of text input. Additionally, it dynamically changes the clip container's title to be that of the configured text string.
The following videos were created for SALSA v1.x; however, the idea behind their implementation is nearly identical and they should be a good place to start until we can get new videos in place.