Runtime Configuration Data Explained

Overview

Each expression type (viseme, emote, head, eye, blink, etc.) is composed of one or more ExpressionComponent configurations. For example, SALSA maintains a list of viseme configurations as List<LipsyncExpression> visemes. Each viseme in the list has its configuration definition held in the expData field, which is of the type Expression. Within this data type, there are two lists you need to be aware of for runtime configuration. Each expression component has an entry in the following lists:

  1. List<ExpressionComponent> components: contains runtime data which is registered in the QueueProcessor each time an animation is triggered. If this data is not complete, it can result in runtime null-references.
  2. List<InspectorControllerHelperData> controllerVars: holds the serialized animation controller data as it is configured in Inspector. Since the individual controller types are extended from the base controller interface, serializing the controller data for the Custom Inspector is problematic. This collection of helper data is baked into the ExpressionComponent.controller at runtime startup.

NOTE: While it is technically not necessary to configure the expData.controllerVars data in a runtime setup, it is recommended to populate this data set to ensure the Inspector is updated correctly while testing design-time implementations and to utilize the existing codebase for controller baking.

Summarizing the above, using SALSA as an example:

  • SALSA has one or more viseme configurations stored in List<LipsyncExpression> visemes
  • Each viseme is made up of one or more expression components and each have a definition contained in Salsa.visemes[index].<Expression>expData.
  • Within expData each ExpressionComponent configuration is defined in two index-matched lists:
    1. List<ExpressionComponent> components: defining the expression component's runtime data. This includes some serialized data and some non-serialized data (which needs to be baked in at run-time from helper data).
    2. List<InspectorControllerHelperData> controllerVars: defining the serialized data for design-time use in the Custom Inspector. Controller data, which cannot be serialized is derived from this helper data.

Path to Configuration Data

Each SALSA Suite module utilizes the same expression component configuration data structure. To access the configuration lists, the paths are as follows:

SALSA would look like this:

Salsa.visemes[index].expData.components[componentIndex]
Salsa.visemes[index].expData.controllerVars[componentIndex]

EmoteR is very similar:

Emoter.emotes[index].expData.components[componentIndex]
Emoter.emotes[index].expData.controllerVars[componentIndex]

And Eyes:

Eyes.eyes[index].expData.components[componentIndex]
Eyes.eyes[index].expData.controllerVars[componentIndex]
Eyes.heads[index].expData.components[componentIndex]
Eyes.heads[index].expData.controllerVars[componentIndex]
Eyes.blinklids[index].expData.components[componentIndex]
Eyes.blinklids[index].expData.controllerVars[componentIndex]
Eyes.tracklids[index].expData.components[componentIndex]
Eyes.tracklids[index].expData.controllerVars[componentIndex]

Configuration Data Detail

This section will focus on the ExpressionComponent data in the two expData lists as they pertain to runtime configuration.

Component Data Fields 'expData.components'

i.e. Salsa.visemes[index].expData.components[componentIndex]

Component data defines all fields required for runtime processing of the ExpressionComponent. All aspects of this definition can be defined at runtime. This data is not baked into the controller data, instead, it is used by the specific modules (SALSA, EmoteR, and Eyes) to define how the animation controller should operate.

Common Component Data

For runtime configuration, the following fields are available for component configuration.

NOTE: Some fields are only used by certain modules (i.e. EmoteR) and others may only be used by certain controller types (i.e. bone controllers). For example: Timing usage varies between Suite modules. SALSA does not use the 'hold' or 'delay' timing and EmoteR can use both.

Notable fields within an <Expression>expData.List<ExpressionComponent>components list element:

  • <string> name: the name of the component.
  • <ExpressionComponent.ControlType> controlType: animation controller enum (i.e. Shape, Bone, Sprite, etc.).
  • <float> durationDelay: animation timing Delay -- only used by emotes.
  • <float> durationOn: animation timing On.
  • <float> durationHold: animation timing Hold -- usable by EmoteR and Eyes:Blink.
  • <float> durationOff: animation timing Off.
  • <bool> isPersistent: see module documentation for persistence.
  • <bool> useOffset: for bone definition only.
  • <bool> useOffsetFollow: for bone definition only.
  • <LerpEasings.EasingType> easing: Easing type, default: Cubic Out.
  • <bool> isAnimatorControlled: force animation mergeback mode. This is an advanced mode that is only used in specific scenarios.
  • <IExpressionController> controller: typically baked with serialized field data. The controller is created when baking the serialized data into the controller.

Usage example (SALSA):

NOTE: This is not a fully operational copy/paste example since some variables will need to be set from your specific project implementation. For a fully operational example that utilizes the included BoxHead model, please see this API example.

Salsa salsa;
salsa.visemes.Clear(); // ensure a clean viseme list.
salsa.visemes.Add(new LipsyncExpression("saySmall", new InspectorControllerHelperData(), 0f)); // add a viseme...
// now let's cache that new viseme's expression root...
Expression expressionData = salsa.visemes[0].expData;
// and let's cache the fist component...created when we added the viseme.
ExpressionComponent expressionComponent = expressionData.components[0];
// When adding additional visemes or emotes...i.e.
// expressionData.components.Add(new ExpressionComponent());
// expressionData.controllerVars.Add(new InspectorControllerHelperData());
// ...and then work with these data elements...i.e.
// expressionComponent = expressionData.components[1];

// configure the component's common fields:
expressionComponent.name = "component #1";
expressionComponent.controlType = ExpressionComponent.ControlType.Shape; // choose the appropriate animation controller.
expressionComponent.durationOn = 0.08f; // animation ON timing.
//expressionComponent.durationHold = 0.00f; // animation HOLD timing -- not used in SALSA.
expressionComponent.durationOff = 0.08f; // animation OFF timing.
//expressionComponent.durationDelay = 0.00f; // ONLY used in EmoteR

expressionComponent.easing = LerpEasings.EasingType.CubicOut;

// Now we need to work with the controllerVars...

Animation Controller Data 'expData.controllerVars'

i.e. Salsa.visemes[index].expData.controllerVars[componentIndex]

The controllerVars data serializes the animation controller definitions in the Inspector and can also be used for run-time configurations. Due to their modular/inherited nature the animation controller configurations are reused data structures and designed to eliminate garbage collection and cannot be serialized. Therefore, we hold their configurations in the controllerVars list and bake them into the controllers at runtime. See the documentation for controller baking for more information. It is not absolutely necessary to use the controllerVars list, but the Inspector is built and updated using this data. Therefore, it is recommended to use the List<InspectorControllerHelperData> controllerVars list to store the animation controller data and then bake the data into the controller using the built-in methods. The added benefit to this is you don't have to re-invent the wheel for baking data and it also populates the Inspectors if you are testing and monitoring the outcome.

Controller Types

The data required for the animation controller depends on the controller type. Over time, as new controllers have been added, the controllerVars dataset has grown and some fields have been re-used from other controller-specific data in an effort to reduce unused data fields. Unfortunately, this has created a bit of ambiguity in some of the class's fields. Hopefully we can sort that out here in this documentation.

Next we will discuss the data fields used by each controller type in expData.controllerVars and expData.components where applicable:

NOTE: expData.controllerVars data is defined by the InspectorControllerHelperData class -- a slightly misleadingly named class at this point, since we are using it for run-time configuration as well.

[2022-02-23] NOTE: Work in progress...more updates soon.