SalsaMicInput is a real-time, Unity microphone input asset
(currently free for use with SALSA LipSync Suite)
Using a microphone in Unity as a real-time input source for SALSA lip synchronization is a popular feature. While it can be used in a stand-alone fashion, SalsaMicInput is designed for use with SALSA and may require some modification to use by itself. This asset is free for SALSA customers and is available via the download links below.
[UPDATE 2019-06-22] v2.0.0-beta: Initial release for use with SALSA LipSync Suite v2
SalsaMicInput takes a very basic approach to adding real-time Unity microphone input to your SALSA lip-sync project and while still intended as an example code-set for real-time input, it does include some robust capabilities for both design-time and run-time usage. Currently, SalsaMicInput only requires a Unity AudioSource for microphone input operation. However, a linked SALSA component is required for real-time lip-synchronization. Additionally, SalsaMicInput can be selectively connected to any Unity AudioSource.
The simplicity of SalsaMicInput remains a priority and the same automated feature set exists as in previous versions, plus some new capabilities. At run-time, SalsaMicInput will attempt to gain reference to a local AudioSource component if one has not been specified (linked in the custom inspector). The process of dynamically getting the AudioSource component will operate in a non-blocking coroutine and will continue to look for an AudioSource on the gameObject until one is present. NOTE: Without an AudioSource, this asset cannot function and will simply continue to wait for an AudioSource. The benefit of this method is a flexible start-up solution, allowing programmatic implementations of an AudioSource (such as UMA2 character startup procedures). Once the AudioSource is connected, the asset will automatically complete the wire-up of the Unity microphone (using the default microphone if one has not been selected in the Custom Inspector). It will then start microphone recording (by default - selectable by the developer) and terminate the coroutine.
In addition to all of the automated processes, SalsaMicInput now has an option to disable the auto-start functionality. During startup, automatic wire-up of the Unity microphone will occur, but recording will not start automatically. This feature is very useful for implementations where microphone input is desired in an on-demand scenario. As expected, SalsaMicInput can be started and stopped on demand via the API.
Run-time lip syncing with a 2D or 3D avatar with SALSA and a microphone is easy and fun! SALSA LipSync is a Unity game engine asset, available now on the Unity Asset Store. SalsaMicInput is currently only available as an addon for SALSA LipSync customers.
- SalsaMicInput v2+ is the current compatible micInput version for SALSA LipSync v2+. micInput versions prior to v2.0.0 will not work appropriately and should be removed from your project if they exist.
- Ensure you have at least one microphone attached, enabled, and working. It is also a good practice to make sure it is not being used by another application while trying to use it as an input source for lip synchronization. NOTE: [on PC] the 'Default' microphone selection will attach to the operating system default microphone. [on Mac] the 'Default' microphone selection will attempt to attach to the input device currently selected in the operating system's 'Sound' utility applet.
- Import SALSA LipSync into your Unity lip-sync project.
- Download and import the SalsaMicInput unitypackage into your Unity lip-sync project.
- Configure and test a SALSA LipSync on a character model in your scene.
- Add SalsaMicInput to a GameObject in your scene (preferrably the same GameObject SALSA is configured on. If you are not adding it to the same GameObject that contains the AudioSource linked to SALSA LipSync, it will be necessary to link SalsaMicInput to the AudioSource driving SALSA. You can also add the SalsaMicInput component to your GameObject from the Component menu: Crazy Minnow Studio > SALSA LipSync > Add-ons.
- Configure SalsaMicInput as per the screenshot.
- Ensure SALSA is linked to the appropriate AudioSource if it is not attached to the same GameObject as the AudioSource micInput is using. NOTE: SALSA needs to connect to the same AudioSource component micInput will use.
- Also ensure SALSA's Data Analysis settings are configured to auto-adjust and use microphone (as in the screenshot to the right)
- You might need to adjust your Trigger levels in SALSA's "Speech Properties". This depends on the sensitivity of your mic and how dynamic you want the lip sync movement to be. It is recommended to use a Linear trigger distribution, but you may desire a different look-and-feel.
- Hit <Play> and enjoy lip-syncing your model with your own voice!
NOTE: While every attempt has been made to ensure the safe content and operation of these files, they are provided as-is, without warranty or guarantee of any kind. By downloading and using these files you are accepting any and all risks associated and release Crazy Minnow Studio, LLC of any and all liability.Download Files
- Microphone muting now works as would be expected in conjunction with SALSA LipSync v2+. It is no longer necessary to create attenuated mixer groups.
- IMPORTANT! (OS X): Selecting multiple microphones for input may not be supported on your platform. Save your project before testing this feature. In our testing, Unity didn't quite know what to do with itself and required a force-quit.
- (OS X): Default microphone selection may not work reliably. Select the microphone explicitly if 'Default' does not work as expected.
- If microphone statuses change, use Refresh Mic List to update the Available Microphones list for selection.
- If there are no microphones available, the Available Microphones list will display 'ERROR - no microphones available'.
- Lip synchronization with a microphone also requires an AudioSource component to be present on your object or, if using an AudioSource on a different object, it must be linked to SalsaMicInput.
- On mobile devices (Android), lip sync works well, but we have seen some anomalous issues when the application is task-switched to another application and then back. If the microphone seems to misbehave after doing this, try switching to the home screen and then back into the application.
- As would be expected, microphone input will only work with one application asset at a time. Windows operating systems may allow more than one microphone input to be active at once on the same computer - this is not guaranteed behavior. If it is desireable to use microphone input for multiple assets/models from the same computer, it would be necessary to programatically disconnect one asset and then connect another.
v2.0.0-beta - (2019-06-22):
- initial release for SALSA LipSync v2.0.0.
SALSA: Simple Automated Lip Sync Approximation - available on the Unity Asset Store
Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~