EmoteR In-Depth

Watch our video tutorial series for SALSA LipSync Suite v2.

EmoteR (Emote Randomizer) is a component included in the SALSA LipSync Suite 2.0. EmoteR allows the configuration of 'emotes' that can be triggered randomly, used with SALSA emphasis triggers, or easily create cyclically repeating animations (repeaters). Additionally, emotes can be fired manually via a programmatic interface to the component.

NOTE: Please read the Interface documentation if you have not already done so.

Where SALSA and Eyes have a single instance of their components on an object, there can be any number of EmoteR components on a GameObject as design dictates. Reasons for doing so are varied and can simply come down to organization. There is no real performance penalty other than the added LateUpdate() call in the script hierarchy. There are some strategic possibilities to doing so, which may be interesting to some designers.

For example:

  1. Additional EmoteR's may be implemented where random, fractional, or hold time settings differences are desired.

  2. Creating separate EmoteR instances with themed configurations for SALSA emphasis triggers. This would allow an interesting transition between emotes for emphasis based on the mood of the character model -- i.e. mad, happy, sullen, etc.

Much of EmoteR's power can be leveraged without writing any code. This, of course, depends on the requirements and situations where EmoteR is being used. For more advanced requirements, please visit the API documentation. For an in-depth understanding of how EmoteR works, read on.

SECTIONS

The EmoteR component is divided into three sections; References, Settings, and Configuration. In addition, since EmoteR can have multiple instances on an object, there is a Name field at the top of the component's inspector. This is entirely for organizational reference and is not used during any processing. For example, it might be desirable to configure more than one EmoteR for SALSA emphasis linking -- one for a happy demeanor and one for an angry demeanor. Labeling one emoter as "happy" and one as "angry" gives the developer an easy way to distinguish between the two when switching the EmoteR SALSA is linked to.

There is one instance where naming the EmoteR instance is invaluable when there are multiple EmoteRs on a character object. SALSA-linked emphasis emotes are configured in SALSA's references and when multiple EmoteRs are available, the SALSA interface provides a handy drop-down selector to chose the appropriate EmoteR instance. The drop-down links use the EmoteR instance name to display the available choices.

References:

This section is used to configure the external reference links EmoteR uses for operation. Currently EmoteR only requires a reference link to a QueueProcessor.

  • A QueueProcessor is required for EmoteR operation. The queue is the brains behind handling animation calculations and conflicts. SALSA, EmoteR, and Eyes all use a QueueProcessor. A single QueueProcessor can be used for any number of SALSA, EmoteR, and/or Eyes instances. Or, you may also implement multiple queues if you like. We currently recommend one QueueProcessor per character model. This is the most performant configuration and there is no advantage to having a single processor for the entire scene.
  • There are no settings necessary for queue operation. Simply link a QueueProcessor to EmoteR and configuration for the queue is complete.

NOTE: If EmoteR does not detect a QueueProcessor on the GameObject, you may click the "Add QueueProcessor" button to automatically create a GameObject with a QueueProcessor component. If EmoteR detects a QueueProcessor on the GameObject, you may click the "Link Existing QueueProcessor" button to easily configure the link for SALSA.

Controller Null References

As of version 2.5.0+, EmoteR will confirm a component controller is valid prior to attempting registration with the QueueProcessor. Please refer to the SALSA Null References documentation for more information on controller null-ref control.

Settings

The settings section currently does not have any configuration requirements outside of the defaults and therefore will not exhibit the gold, "config-required" display. It is divided into three sections (Random Pool Settings, Common Settings, and SALSA-linked Settings). The minimal Settings view looks like the following.

minimal-settings
(minimal EmoteR Settings)

max-settings
(all available EmoteR Settings)

Random Emote Settings

This section applies to emotes configured to populate the random emotes pool. The random pool can be enabled or disabled with Use Random Emotes. This can be programmatically achieved as well. If an emote is configured to be added to the pool, this setting must still be enabled for emote triggering. Once enabled, Random Chance specifies the percentage probability [0 .. 1] that one or more random-based emotes will be triggered. 0.5 = 50%, 1.0 = 100%, and so forth. If Per Emote Chance is enabled, each emote in the pool individually applies the Random Chance to determine if it triggers. This can be further modified by selecting the number of Emotes Per Trigger Cycle. For example, assume 5 emotes are configured to use the random pool and Emotes Per Trigger Cycle is configured for 3 of 5. Each cycle, EmoteR will randomly select 3 of the 5 available emotes in the pool and then apply the Random Chance on the group of 3 (or individually on the 3 selected if Per Emote Chance is enabled). * Finally, Time between random emotes configures a random time between From and To (computed each cycle) to trigger random emotes, based on the above settings. This timer does not take into account the length of animation timings configured on each emote. It is therefore possible to interrupt emotes during their animations. NOTE: It is possible, considering Random Chance settings that zero emotes in the random pool will fire on a given cycle. And, if the Random Chance is set to 0.0, emotes will never fire.

Common Settings

  • Hold Time Duration Variation applies only to RoundTrip handled emotes. It is a random time adjustment similar to the Time between random emotes. However, this time variation is applied only to the hold time of the emote. If this is enabled, it will override the Hold time of each emote triggered, varying the length of time the emote stays active, prior to beginning the Off cycle of its animation timings.
  • Fractional Variation applies a variable amount of max extent to any emote. This applies to the On timing cycle only and provides variation to the emotes max animation extent, giving a good dynamic look if desired. Use the Fractional Bias adjustment to set a minimum value to which the Fractional Variation is added. This guarantees the amount of animation that is visible.
    emoter-common-settings

SALSA-linked Settings (Emphasizers)

This applies to emotes configured to respond to SALSA audio analysis triggers. When SALSA is linked with EmoteR, it will send notifications to EmoteR when its emphasizer trigger threshold is exceeded, along with the amount of excess achieved. For example, if the SALSA emphasis trigger is set for 0.5 and the audio analysis value is 0.75, the excess value is passed to EmoteR and that value is used to increase the EmphasisChance and the fractional extent of the emote.

emoter-salsa-linked

What this ultimately means is the more the audio analysis value exceeds the trigger, the higher the chance of ultimately triggering an emphasis emote and the larger the animation extent (bigger animation). The OneClick solutions we offer configure emphasis emotes on character models. This setting is purely a personal taste and you are free to change the trigger and chance settings to your preference. However, we find that it looks more natural to set the trigger to 0.0f and the chance to 100%, meaning it will always fire the emphasis configuration. These settings correspond to very active emphasis emotes at largely varying extents. Lower analysis values equate to subtle emotes and larger analysis values equate to more exaggerated emotes. It works quite well.

Additionally, as of EmoteR v2.5.0+, there is a new option in the emphasis configuration. Individual emphasis emotes can now be configured as always fired and are no longer random emphasis emotes. If there are one or more emphasis emotes not configured to always fire, they are considered random emphasis emotes. The number of random emphasis emotes triggered on each SALSA tick, can be configured in this section as one or more, up to the maximum number of random emphasis emotes, per tick cycle.

For more information, please see Part 8 of our video tutorial series.

Emote Configuration

The configuration section is nearly identical to SALSA's. It is where all emotes are defined. EmoteR uses the same basic building blocks for animation configuration and, as such, all emote components are identical in configuration to SALSA's; however, the inspector reveals some settings to the designer that are not available to SALSA, namely the "Hold" animation timing. Additionally, the emote itself has some different options since it is not driven in the same way a SALSA viseme is triggered.

  • Under the section header is the display menu bar. This offers a consolidated settings Collection Display Mode as well as Expand shortcuts.

Collection Display Mode

The collection display works similarly to SALSA's trigger display mode, but consolidates the view to present the collection membership of each emote (Random, Emphasis, or Repeater). When in this mode, it is easy to view the membership of all emotes and change them at will.

emoter-collection-display-mode

  1. Enable Collection Display Mode
  2. Add emotes to available emote pools/memberships.
  3. Toggle preview mode on/off.
  4. An additional consolidated display is available in the Collection Display Mode by clicking on the emote name. This will expand the emote and display the Animation Timings of each component in the emote, giving the designer an easy way to edit all of the Animation Timings in a convenient, compact view.
  5. When an emote is added to the Repeater pool, an additional option is available, the Repeater Delay. This delay adds additional time to the Repeater's individually timed cycle. For example, if an emote's longest component takes 1.0 + 0.5 + 0.75 (On + Hold + Off) 2.25 seconds and the Repeater Delay is set for 0.5 seconds, the cycle will be triggered every 2.75 seconds, allowing the cycle to repeat with a half second delay after the end of the animation, before it begins again.
  6. When an emote is added to the Emphasizer pool, an additional option is available, the Always Fire option. For more details, see the Emphasis Pool documentation below.

Expand Shortcuts

These buttons operate exactly as the SALSA shortcuts. Please refer to that section for details. Hover over each button for a brief tooltip on the action available.

emoter-configuration-expand-collapse

  1. Collapsed -- displays only the emote header.
  2. Expand all emotes -- expands the emotes, but leaves the components collapsed.

  3. Expand all emotes and components -- expands the emotes as well as the components. NOTE: this view can greatly impact Inspector performance if many emotes and components are configured.

  4. In normal display mode, the emotes are visible with buttons at the bottom to add additional components to an emote or additional emotes to EmoteR. The details of an emote are as follows:

    • Name is solely for organizational purposes. When an emote is added, a default name is given. It is recommended to change this name to something more meaningful to the design.
    • The Memberships section defines which pools an emote belongs to. An emote may belong to any number of pools as the design requires.

  5. Preview Display Mode sets the game object to the full On state of the configured emote's components. Only one emote can be in preview display mode at a time. Activating preview mode will deactivate it on any other emote in preview mode.

EMOTER MEMBERSHIP POOLS

EmoteR currently has four pool memberships it uses to drive various types of emotes. Each configured emote can exist in one or more of the available pools (all configured emotes are part of the manual pool).

emoter-configuration-pool-memberships

  1. Manual Pool -- every emote that is configured is added to this pool in the background. There is no option to add or remove it since it is added by default and removed when an emote is deleted. The manual pool is, as the name implies, triggered manually through the API.

  2. Random Pool -- emotes in this pool are triggered randomly according to the rule specified in the Settings section > Random Emote Settings. Random emotes can be triggered one-at-a-time or all-together, or any combination between. The timer triggering the emotes has a configurable randomness.

  3. Emphasizer Pool -- contains emotes that respond to SALSA-triggers. When SALSA is linked to an EmoteR, a trigger can be specified -- when analysis values exceed this threshold, SALSA notifies the linked EmoteR and EmoteR then calculates a chance of activating one of the configured emotes. See SALSA-linked Settings for more information. Emphasis emotes may now be selected to always fire. When enabled, the Emphasis emote will fire every SALSA tick cycle where the SALSA Emphasis Trigger is exceeded. This is a good option when you would like to see the same emphasis emote consistently animated and not randomly chosen. Mixing always-fired emotes with random-emotes creates a very compelling audio-timed animation set.

  4. Repeater Pool -- is a collection of emotes that are designed to operate cyclically on their own auto-calculated timers. Each emote will animate On + Hold + Off and then repeat. A delay can be added to each emote to insert extra time into the cycle, between Off and On. Repeaters now have added functionality, configuring the start timing of the repeater. Previously all repeaters began animating using the After Full Cycle option below. Now the default is Immediately.

    • Immediately: Animation begins immediately upon Player activation.
    • After Delay: Animation begins after the configured repeater delay time has passed.
    • After Base Cycle: Animation waits one cycle of total timings (DELAY/ON/HOLD/OFF) to begin.
    • After Full Cycle: Animation waits one full cycle of timings + repeater delay (DELAY/ON/HOLD/OFF/repeater delay) to begin.

      Please refer to the Animation Timings document for more information on the animation cycle.

Special Pool Exemptions

  • Hold Variation Exempt -- is a new option that excludes individual emotes from Hold Time Duration Variation processing (configured in Settings > Common Settings). Simply enable the Hold Variation Exempt option in an emote's memberships section.

This option is only available when the Hold Time Duration Variation option is enabled in Settings.

  • Dynamics is an emote-scoped global configuration. The value computes a percentage-based override of the max extent which applies to all expression components configured for an emote. This is an easy way for an emote to be configured and then adjusted if the expression is too "large".

  • Persistence mode prevents an emote's components from ever leaving the QueueProcessor. This is an effective means of overriding external animation drivers as long as any component animating a conflicting bone/shape operates earlier than the LateUpdate loop. The SALSA Suite operates in the LateUpdate loop in an attempt to override any external animation drivers. The better practice is to design around this behavior and remove conflicting processing. However, when this cannot be done, this option may be a suitable resolution.

Special Timing Options

The EmoteR Component interface is identical to a viseme component in SALSA except for the animation timings. "Hold" and "Delay" animation timings are configurable for emotes, while they are not configurable for SALSA visemes. See the documentation for ExpressionComponent for more information on the common component settings/values.

Hold Timing

  • Hold Timing is only applicable to RoundTrip handler types. EmoteR allows both OneWay and RoundTrip handlers to be leveraged. General configuration of an emote assumes a RoundTrip and if fired by the various pool mechanisms (see above), will be treated as RoundTrip. A OneWay handler can only be invoked on an emote manually -- using the API or a proxy to the API (such as Timeline).

Delay Timing

  • Delay Timing inserts a delay loop for an animation component to delay its start animation phase. This is a special use case where the designer may prefer to start some components at later times for an overall emote, to give time for other components to start or finish. For example: an emote to stick out a tongue may require an "open mouth" shape to start and once sufficiently open, the "tongue out" shape can begin its animation.