micInput-lite is a real-time, Unity microphone input asset
(currently free for use with SALSA lip-sync)
micInput-lite v1.7.0-beta is now available and may reduce microphone input lag by up to 10x.
Using a microphone in Unity as a real-time input source for SALSA lip synchronization is a popular feature. However, micInput is continuing to grow into our vision of a stand-alone Unity microphone asset that has much more flexibility than simply being an add-on for SALSA. While the same uber cool capability and simplicity exists for creating lip-synced avatars with real-time microphone input, micInput-lite has reached the next step in becoming self-sufficient as a microphone input asset for any usage scenario. This asset is currently still free for SALSA customers and is available via the download links below.
[UPDATE 2018-04-14] v1.7.0-beta: Experimental change, may significantly reduce microphone input lag. If this is stable and works well for you, please send us an email and let us know!
- Implementation Details
- Download micInput
- Troubleshooting & Operational Notes
- Release Notes
micInput-lite takes a very basic approach to adding real-time Unity microphone input to your SALSA lip-sync project and while still intended as an example code-set for real-time input, it does include some robust capabilities for both design-time and run-time usage. As of version 1.5.0+, micInput functions differently from previous versions. Currently, micInput-lite only requires a Unity AudioSource for microphone input operation. Previously, a SALSA component was also required. However, as of SALSA 1.3.3+, SALSA's lip-sync functionality can operate passively from an AudioSource, rather than aggressively attaching to it. This greatly increases SALSA's flexibility and allows us to reduce this asset to a single API script vice having separate implementations for 2D and 3D SALSA components. Additionally, micInput-lite can be selectively connected to any Unity AudioSource; therefore, micInput-lite no longer automatically adds an AudioSource for Unity microphone use.
The simplicity of micInput-lite remains a priority and the same automated feature set exists as in previous versions, plus some new capabilities. At run-time, micInput-lite will attempt to gain reference to a local AudioSource component if one has not been specified (linked in the custom inspector). The process of dynamically getting the AudioSource component will operate in a non-blocking coroutine and will continue to look for an AudioSource on the gameObject until one is present. NOTE: Without an AudioSource, this asset cannot function and will simply continue to wait for an AudioSource. The benefit of this method is a flexible start-up solution, allowing programmatic implementations of an AudioSource (such as UMA2 character startup procedures). Once the AudioSource is connected, the asset will automatically complete the wire-up of the Unity microphone (using the default microphone if one has not been selected in the Custom Inspector). It will then start microphone recording (by default - selectable by the developer) and terminate the coroutine.
In addition to all of the automated processes, micInput-lite now has an option to disable the auto-start functionality. During startup, automatic wire-up of the Unity microphone will occur, but recording will not start automatically. This feature is very useful for implementations where microphone input is desired in an on-demand scenario. As expected, micInput can be started and stopped on demand via the API.
Run-time lip syncing with a 2D or 3D avatar with SALSA and a microphone is easy and fun! SALSA lip-sync is a Unity game engine asset, available now on the Unity Asset Store. micInput-lite is currently only available as an addon for SALSA.
- NOTE: a video is available demonstrating how to use micInput-lite v1.5.0+ (this video is also applicable to v1.6.0).
- The directory structure for micInput-lite has changed. It is highly recommended that micInput versions prior to v1.5.0 be deleted from your project prior to installing the latest version. to ensure the latest version is appropriately the only instance of micInput in your project.
- Ensure you have at least one microphone attached, enabled, and working. It is also a good practice to make sure it is not being used by another application while trying to use it as an input source for lip synchronization. NOTE: [on PC] the 'Default' microphone selection will attach to the operating system default microphone. [on Mac] the 'Default' microphone selection will attempt to attach to the input device currently selected in the operating system's 'Sound' utility applet.
- Import SALSA with RandomEyes into your Unity lip-sync project.
- Download and import the micInput-lite unitypackage into your Unity lip-sync project. Scripts are imported into "Crazy Minnow Studio/SALSA with RandomEyes/Addons/micInput/".
- Add micInput to a GameObject in your scene. If you are not adding it to the same GameObject that contains the AudioSource linked to SALSA lip-sync, it will be necessary to link micInput to the AudioSource driving SALSA. You can also add the micInput component to your GameObject from the Component menu: Crazy Minnow Studio > Addons.
- For Unity 5.2+, follow the steps in this blog entry to configure and link an attenuated AudioMixerGroup to the AudioSource linked to micInput. Ensure Mute Microphone is *not* enabled.
- Ensure you have added a Salsa2D or Salsa3D component to the appropriate character model in your scene. SALSA needs to connect to the same AudioSource component micInput will use.
- Configure the SALSA component per the documentation. At a minimum, link target mouth shapes to your model's blendshapes (3D) or sprites (2D) in SALSA [SaySmall/SayMedium/SayLarge indexes].
- Ensure SALSA is linked to the appropriate AudioSource if it is not attached to the same GameObject as the AudioSource micInput is using.
- You might need to adjust your Trigger levels in SALSA's "Speech Properties". This depends on the sensitivity of your mic and how dynamic you want the lip sync movement to be.
- Hit <Play> and enjoy lip-syncing your model with your own voice!
- This add-on is provided in .unitypackage format [micInput-lite_version.unitypackage]
- CM_MicInput.cs is the core helper class and it also includes a Custom Inspector (editor script) for your convenience.
NOTE: While every attempt has been made to ensure the safe content and operation of these files, they are provided as-is, without warranty or guarantee of any kind. By downloading and using these files you are accepting any and all risks associated and release Crazy Minnow Studio, LLC of any and all liability.Download Files
- micInput-lite is no longer bound to a SALSA component, its only dependency is an AudioSource component that SALSA then uses for lip-sync. This AudioSource does not necessarily need to exist on the same GameObject, nor does it need to exist on micInput startup.
- IMPORTANT! all-versions/platforms: (Unity v5.2+) for proper lip-sync functionality, the microphone/AudioSource cannot be muted. Please see the following blog post for information on setting up real-time microphone lip-sync with SALSA.
- IMPORTANT! (OS X): Selecting multiple microphones for input may not be supported on your platform. Save your project before testing this feature. In our testing, Unity didn't quite know what to do with itself and required a force-quit.
- (OS X): Default microphone selection may not work reliably. Select the microphone explicitly if 'Default' does not work as expected.
- If microphone statuses change, use Refresh Mic List to update the Available Microphones list for selection.
- If there are no microphones available, the Available Microphones list will display 'ERROR - no microphones available'.
- Lip synchronization with a microphone also requires an AudioSource component to be present on your object or, if using an AudioSource on a different object, it must be linked to micInput.
- On mobile devices (Android), lip sync works well, but we have seen some anomalous issues when the application is task-switched to another application and then back. If the microphone seems to misbehave after doing this, try switching to the home screen and then back into the application.
- Problems with onFocus/Pause changes on desktop clients has been repaired with the latest script version (v1.0.2+).
- As would be expected, microphone input will only work with one application asset at a time. Windows operating systems may allow more than one microphone input to be active at once on the same computer - this is not guaranteed behavior. If it is desireable to use microphone input for multiple assets/models from the same computer, it would be necessary to programatically disconnect one asset and then connect another.
v1.7.0-beta - (2018-04-14)[experimental]:
- + added 1/10 sec timing delay to startup -- gives Unity Microphone and AudioClip classes time to spin up and respond to data requests. In testing, this is imperceptable to startup time and has shown a 10x improvement in microphone input lag/delay.
- ~ moved micInput component menu location to Crazy Minnow Studio > SALSA > Addons > micInput
v1.6.0 - (2015-12-22) [recommended]:
- + new isRecording read-only property to see if mic is started and recording.
- + new isAutoStart mode.
- + added additional checks and logging for missing AudioSource or mic references. Breaking actions now throw LogWarning() in isDebug.
- + added logging during coroutines if isDebug.
- + added sampleRate setting prior to starting the microphone. Ensures valid freq with Editor script processing during runtime.
- ~ StartMicrophone(): removed the blocking while loop.
- ~ cleaned up and optimized logic and logging.
- ~ default sampleRate changed to 22050.
- ~ CheckFreqCapability() now processes all sampleRate-related functionality.
v1.5.0 BETA - (2015-12-07):
- - Removed dependency on Salsa2D or Salsa3D type (requires SALSA 1.3.3+). SALSA components are no longer auto-added to the gameObject when micInput is attached.
- - Removed dependency on local attached AudioSource: now public and if not found, will look for local AudioSource component. An AudioSource component is no longer automatically added to the gameObject when micInput is attached.
- ~ micInput now looks for an AudioSource in a coroutine to ensure runtime-created AudioSources have sufficient time to spin up. This is beneficial for workflows such as: SALSA with UMA2.
- ~ It is no longer necessary to attach the micInput addon to the same GameObject SALSA is attached to.
v1.0.8 BETA - (2015-11-13):
- + Available Microphones selection option in a new Custom Inspector.
- + Refresh Mic List button added to update the list if available microphones changes prior to running the app/game.
- ~ bool isMuted (Mute Microphone) now defaults to false.
v1.0.7 - (2015-06-10) [legacy]:
- ~ Additional error trapping and restructuring to prevent errors if no microphone is attached/found.
v1.0.6 - (2015-05-20):
- ~ StartMicrophone() and StopMicrophone() are now public functions.
- + Demo scene added.
- ~ Unity 5 ready!
SALSA: Simple Automated Lip Sync Approximation - available on the Unity Asset Store
Watch our lip-sync tutorial video for Real-Time Microphone Input:
version 1.0 legacy
Simple Automated Lip Sync Approximation
~ We look forward to seeing what you create! ~