Current version: v2.3.0

Compatible with SALSA LipSync Suite v2.3.0+

Microphone Issues on MacOS

NOTE: If you are experiencing microphone issues with MacOS, you could be seeing permissions issues with Unity Hub and MacOS Privacy security. When Unity Hub is installed, you should be prompted for microphone permissions and that does not always appear to happen. Please see this thread for more detail. Be aware, we are NOT suggesting this as a fix; however, it does demonstrate the case and condition. We have confirmed this to be an issue ourselves and have successfully performed the resolution presented here on a test machine to validate the condition. You must determine your own course of action.

Quick-Links:

Implementation Details
Instructions
Troubleshooting & Operational Notes
Release Notes

Implementation Details:

SalsaMicInput takes a very basic approach to adding real-time Unity microphone input to your SALSA lip-sync project and while still intended as an example code-base for real-time input, it does include some robust capabilities for both design-time and run-time usage. Currently, SalsaMicInput only requires a Unity AudioSource for microphone input operation. However, a linked SALSA component is required for real-time lip-synchronization. Additionally, SalsaMicInput can be selectively connected to any Unity AudioSource.

The simplicity of SalsaMicInput remains a priority and the same automated feature set exists as in previous versions, plus some new capabilities. At run-time, SalsaMicInput will attempt to gain reference to a local AudioSource component if one has not been specified (linked in the custom inspector). The process of dynamically getting the AudioSource component will operate in a non-blocking coroutine and will continue to look for an AudioSource on the gameObject until one is present. NOTE: Without an AudioSource, this asset cannot function and will simply continue to wait for an AudioSource. The benefit of this method is a flexible start-up solution, allowing programmatic implementations of an AudioSource (such as UMA2 character startup procedures). Once the AudioSource is connected, the asset will automatically complete the wire-up of the Unity microphone (using the default microphone if one has not been selected in the Custom Inspector). It will then start microphone recording (by default - selectable by the developer) and terminate the coroutine.

In addition to all of the automated processes, SalsaMicInput now has an option to disable the auto-start functionality. During startup, automatic wire-up of the Unity microphone will occur, but recording will not start automatically. This feature is very useful for implementations where microphone input is desired in an on-demand scenario. As expected, SalsaMicInput can be started and stopped on demand via the API calls, StartMicrophone() and StopMicrophone().

Run-time lip syncing with a 2D or 3D avatar with SALSA and a microphone is easy and fun! SALSA LipSync is a Unity game engine asset, available now on the Unity Asset Store. SalsaMicInput is currently only available as an addon for SALSA LipSync customers.


Instructions (using SalsaMicInput with SALSA LipSync v2.3.0+):

  1. SalsaMicInput v2.3.0 is the current compatible micInput version for SALSA LipSync v2.3.0+. micInput versions prior to v2.3.0 will not work appropriately and should be removed from your project if they exist.

  2. Ensure you have at least one microphone attached to your system (see caveats in the troubleshooting section below.

    • Ensure it is enabled and working outside of Unity.
    • It is also a good practice to make sure it is not being used by another application while trying to use it as an input source for lip synchronization.

    NOTE: [on PC] the 'Default' microphone selection will attach to the operating system default microphone. [on Mac] the 'Default' microphone selection will attempt to attach to the input device currently selected in the operating system's 'Sound' utility applet.

  3. Import/install SALSA LipSync Suite into your Unity lip-sync project. \

    NOTE: Ensure the version requirements are met.

  4. Download and import the SalsaMicInput unitypackage into your Unity lip-sync project.

    NOTE: Ensure the version requirements are met.

  5. Configure and test SALSA LipSync on a character model in your scene (without using SalsaMicInput) to ensure SALSA is configured properly and is working.

  6. Add SalsaMicInput to a GameObject in your scene (preferably the same SALSA-configured GameObject). If you are not adding it to the same GameObject that contains the AudioSource linked to SALSA LipSync, it will be necessary to link SalsaMicInput to the AudioSource driving SALSA.

    NOTE: You can also add the SalsaMicInput component to your GameObject from the Component menu: Crazy Minnow Studio > SALSA LipSync > Add-ons.

  7. Configure SalsaMicInput as per the screenshot.
    micInput-settings

    NOTE: SalsaMicInput will now add another component (SalsaMicPointerSync) to your game object when "Link with SALSA" is enabled. In order to facilitate removal of this helper script, it is only added when the "Link with SALSA" option is enabled after having been disabled. This helper script (or similar) is required for proper SALSA Core audio processing of the microphone buffer. It lets SALSA know where the current buffer record head position is. Please see the SALSA API information for the microphoneRecordHeadPosition field description for more information if you wish to write your own procedure to update SALSA Core.

  8. Ensure a SalsaMicPointerSync component is applied to the same GameObject as SalsaMicInput. See previous step notes for information.
    micInput-syncScript

    NOTE: The helper script may also be added manually, dragging and dropping from the SalsaMicInput installation directory, or by using 'Add Component' at the bottom of the GameObject's Inspector window.

  9. Ensure SALSA and SalsaMicInput are linked to the same/appropriate AudioSource if they are not attached to the same GameObject.

    NOTE: SALSA needs to connect to the same AudioSource component micInput will use. This can be wired up any way you desire, the AudioSource used by SALSA and micInput need to match.

  10. Also ensure SALSA's "Data Analysis" settings are configured to auto-adjust and use microphone.
    micInput-salsaDataAnalysis

    NOTE: If "Link with SALSA" is enabled in SalsaMicInput, these settings should already be enabled.

  11. You might need to adjust your Trigger levels in SALSA's "Viseme Configuration" section. This depends on the sensitivity of your mic and how dynamic you want the lip sync movement to be. If your microphone produces lower input, you may need to adjust the sliders to the left to take advantage of the lower input levels (or turn up the microphone levels).
    micInput-salsaTriggers

  12. Hit [Play] and enjoy lip-syncing your model with your own voice!


Troubleshooting and Operational Notes:

  • Microphone muting now works as would be expected in conjunction with SALSA LipSync v2+. It is no longer necessary to create attenuated mixer groups.

  • (OS X): Default microphone selection may not work reliably. Select the microphone explicitly if 'Default' does not work as expected.

IMPORTANT! (OS X): Selecting multiple microphones for input may not be supported on your platform. Save your project before testing this feature. In our testing, Unity didn't quite know what to do with itself and required a force-quit.

  • If microphone statuses change, use Refresh Mic List to update the Available Microphones list for selection.

  • If there are no microphones available, the Available Microphones list will display 'ERROR - no microphones available'.

  • Lip synchronization with a microphone also requires an AudioSource component to be present on your object or, if using an AudioSource on a different object, it must be linked to SalsaMicInput.

  • On mobile devices (Android), lip sync works well, but we have seen some anomalous issues when the application is task-switched to another application and then back. If the microphone seems to misbehave after doing this, try switching to the home screen and then back into the application.

  • As would be expected, microphone input will only work with one application asset at a time. Windows operating systems may allow more than one microphone input to be active at once on the same computer - this is not guaranteed behavior. If it is desireable to use microphone input for multiple assets/models from the same computer, it would be necessary to programatically disconnect one asset and then connect another.

  • Similar to the above caveat, SalsaMicInput cannot work in conjunction with other microphone "input" scripts/components (i.e. Dissonance, Photon Networking, etc.) on the same microphone.


Release Notes:

v2.3.0 - (2020-02-03):

  • Updated for SALSA LipSync v2.3.0+: SALSA Core v2.3.0+ requires the virtual pointer for the microphone record head to be updated externally -- the Unity Microphone dependency was removed from the SALSA Core code-base. SalsaMicInput now includes a helper script to perform this action. You are free to write your own helper script if you do not wish to use this one.

NOTE: SALSA Core v2.3.0+ is required for this version of SalsaMicInput.


v2.0.0-beta - (2019-06-22):

  • Initial release for SALSA LipSync v2.0.0.