WebAudioUrls - Amplitude Add-on

Overview

Freely Available to Crazy Minnow Studio Customers

As a courtesy to our customers using Amplitude with web-based audio files, we have put together this information in hopes it may assist anyone trying to implement such a solution. Amplitude itself does not care where audio files come from; however, accessing audio-files on the web, by nature, can be problematic due to the very asynchronous nature of the Internet. In response to common support questions and to help out other customers, we created a small framework for dealing with web-based audio files -- WebAudioUrls. Please note that these files and this information are outside the scope of Amplitude or SALSA Suite support and are only provided as a courtesy and example of how it might be possible to implement an asynchronous fetch of audio files within the framework of the core Unity API. We do NOT provide support for these files, the related examples, nor the documentation/video.

You can also watch our video tutorial for WebAudioUrls. This is an older video that was made for use with the previous (legacy) version of SALSA, but the information is still pertinent.

It will most likely be necessary to understand Unity's implementation of web-based resources, especially if you wish to modify or troubleshoot issues you may be having. WebAudioUrls is available as a free package for Amplitude for WebGL customers -- as a starting point for their async audio projects. Please feel free to make modifications to the API files if necessary or desired. Since Unity's implementation of WebGL is so basic and constantly changing, we do not provide support for web-based-audio, Unity's implementation of web-request resources, etc. Consider this package a starting point for asynchronous operations in your project. You can get the files in our download section. We have also published an Examples package, which includes single-action and batch-mode implementation examples, as well as a demo scene.

Cross Origin Resource Scripting (CORS)

PLEASE NOTE: Most browsers do not like cross-site access and block it with exuberance. If your project attempts to access audio from another site, you will likely experience a CORS issue. If you are only testing, you can probably bypass the issue with a browser plug-in. Google is your friend -- the Mozilla site has some good information about CORS. If the examples are not working in a browser, use your browser's development mode (usually F12) to see if you are receiving errors.

Files and Examples

Handling web audio files and dealing with the asynchronous nature of such files is beyond the scope of support for Amplitude and SALSA Suite. The documentation, video tutorial, and files are contained here as a courtesy to our customers and are provided with an expectation of no additional support.

NOTE: While every attempt has been made to ensure the safe content and operation of these files, they are provided as-is, without warranty or guarantee of any kind. By downloading and using these files you are accepting any and all risks associated and release Crazy Minnow Studio, LLC of any and all liability.

Follow this link to download the add-on files and examples.

Requirements

WebAudioUrls does not have many requirements. It is a passive framework that responds only when asked and it abstracts the async nature away from the developer as much as possible. It was built on Unity 2017.1 and has been tested through Unity 2017.3. If you choose to download the examples package (recommended), it does have some dependencies to be aware of if you want to use the included demo scene:

  • Amplitude for WebGL
  • SALSA LipSync Suite v2
  • AmplitudeSALSA

If you simply want to take a look at an example implementation, there are no real requirements for having other packages in your project. Two example implementations are included:

  • Single-action implementation: WebAudioActionExample.cs
  • Batch implementation: WebAudioBatchExample.cs

Both of these example files have been removed from the CrazyMinnow namespace and are free to be copied or modified for your needs.

API

The API for WebAudioUrlProcessor.cs is very simple to use. This documentation will cover the API in two sections, one for single-action use and one for batch usage. Before we get into the specifics of the API calls, there are a couple of things that are helpful to know and understand outside of the API calls themselves.

Things to Know and Understand

A Reference to the WebAudioUrlProcessor

This goes pretty much without saying, but to access the API of the WAUP, you will need a reference to it. You are free to establish this reference at any point, but it is recommended to do so as a cached global field reference, linked in the inspector or via GetComponent<WebAudioUrlProcessor> in Awake or Start. Without a reference, it will not be possible to access the features and functions of the add-on API.

The Callback System

The WebAudioUrlProcessor (WAUP) defines two callback delegates: one for successful returns and one for failed processes. When calling the API functions, most calls require you to include delegate methods in the parameters, indicated as SuccessCallback and FailCallback.

SuccessCallback

The SuccessCallback expects to be able to pass and AudioClip back to your delegate function, so simply ensure your method defines one in its signature. This simply looks like this:

private void MySuccessMethod(AudioClip processedClip) {
    // custom code processing...
}

You may include any processing you wish in this method. The WAUP has converted a url-based audio file to an AudioClip and passed it back to you. What you do with it at this point, is entirely up to you.

FailCallback

The FailCallback works similarly to the SuccessCallback. However, this time it expects a signature parameter of type . You can find the definition of the Notification struct in the WebAudioUrlProcessor.cs file. It consists of:

  • An enum type, indicating info, warning, error modes.
  • A string clip name, indicating the name of the AudioClip (the full URL), or a brief message indicating where a problem might lie.
  • A string message providing more detail for the issue or error encountered.

As mentioned, the FailCallback expects a delegate method with a Notification signature parameter. This simply looks like this:

private void MyFailMethod(Notification noteMsg){
    // custom code processing...
}

You may process this information in any way you choose. The example files have a couple of different implementations for your viewing pleasure.

The AudioUrl Structure

When interacting with the WAUP audio urls are expected in a custom structure. This includes two fields:

  • The URL of the audio file as a string.
  • The Unity AudioType enumeration, such as AudioType.WAV.

This could simply be built like so:

string url = "someUrl...";
AudioType type = AudioType.WAV;
var myAudioUrl = new AudioUrl(url, type);

Now, let's get on to the good stuff!

Single-Action Mode

Single-Action mode is just what it sounds like. Send a single AudioUrl to the WAUP and asynchronously get back an AudioClip (if all goes well). The examples package includes a single-action example called WebAudioActionExample.cs. It is highly recommended to check out this file (and even use it as a framework for your own implementation).

The following public API calls are available for single-action mode:

GetAudioClip()

public void GetAudioClip(AudioUrl audioUrl, SuccessCallback<AudioClip> successCallback, FailCallback failCallback)

Once you have your supporting callback methods setup, it is super simple to call the WAUP to have it process and convert your AudioUrl to an AudioClip. For example:

string url = "someUrl...";
AudioType type = AudioType.WAV;
WAUP.GetAudioClip(new AudioUrl(url, type), OnSuccess, OnFailed);

Assuming your success callback delegate looks something like the following:

private void OnSuccess(AudioClip processedClip)
{
    audioSource.clip = processedClip;
    audioSource.Play();
}

The above GetAudioClip call will send the AudioUrl to the WAUP and register the SuccessCallback and FailCallback delegate functions. After the WAUP performs its async magic and converts the URL to an AudioClip, it will call the OnSuccess method, passing the new AudioClip in to be played by the referenced AudioSource. The amount of time it takes to perform the processing is not relevant to your code. You do not have to specify any waits or loops to try and accurately time the conversion process. And, if there is a problem, it will call the FailCallback delegate method, letting you know what the problem was (assuming it knows).

Batch Mode

Batch-mode is designed for those instances where you want to process one or more AudioUrls and have them ready for consumption instantly. Consider batch-mode the ability to preprocess an AudioUrl or set of AudioUrls for zero-delay playback (after they have been processed, of course). An example implementation is available in the examples package as WebAudioBatchExample.cs. If batch-mode processing is of interest to you, it is highly recommended to download and examine this example file.

On the backend, the WAUP uses two queues to create this batching magic for you. You submit your AudioUrls to the WAUP, it pushes them to its queue and spins up a processing thread to work through the queue, asynchronously converting the AudioUrls to AudioClips and pushing those results into the second queue, waiting for you to request them at your leisure. You can push as many AudioUrls to the WAUP as quickly as you want.

The following public API calls are available for batch-mode:

EnqueueAudioUrl()

public void EnqueueAudioUrl(AudioUrl audUrl, FailCallback failCallback)

This call sends an AudioUrl to the WAUP for batching. Note, this is the only processing call that does not require a SuccessCallback to be passed. The reason is the queue is processed independently and the processed AudioClips are reserved for later consumption using GetBatchedAudioClip (see below).

To send multiple AudioUrls to the WAUP for processing, call EnqueueAudioUrl for each AudioUrl you want to send. It is perfectly fine to wrap this call in a for/foreach loop (or other looping mechanism). It is non-blocking on the API side.

QueueCount

public int QueueCount

QueueCount is a readonly property, exposing the number of AudioClips available for retrieving. This may be useful in any number of instances, such as: determining whether continued queries of the batched clips is required or even to easily confirm if all AudioUrls were converted to AudioClips -- although this would highly depend on timing. If AudioUrls are still processing, the count may not be accurate until processing is finished. To confirm whether the queue is still being processed, see IsQueueProcessing below.

IsQueueProcessing

public bool IsQueueProcessing

IsQueueProcessing simply returns a bool indicating whether the queue processor co-routine is active or not. Use this in conjunction with QueueCount to determine if batch-submitted AudioUrls have been completed and were all successful. For example: if (WAUP.IsQueueProcessing && WAUP.QueueCount == myUrlsArray.Length) //...

GetBatchedAudioClip()

public void GetBatchedAudioClip(SuccessCallback<AudioClip> successCallback, FailCallback failCallback)

Once an AudioUrl has been processed, it is pushed to the WAUP's AudioClip queue. Use this call to request an AudioClip from the WAUP in first-in-first-out fashion. In other words, the first AudioUrl passed to the WAUP is the first AudioClip returned via GetBatchedAudioClip (assuming it was successfully converted) and so forth. If this call is made prior to any AudioUrls being sent to the WAUP, it will simply return a message via the FailCallback indicating no AudioUrls have been sent. If this call is made after AudioUrls have been successfully converted and subsequently consumed to the point there are no longer any items in the queue, it will return a response via the FailCallback, indicating there are no items in the queue.

ClearAudioClipQueue()

public void ClearAudioClipQueue()

ClearAudioClipQueue is provided to wipe the queue. The main reason for wanting to do this assumes one or more AudioUrls have been converted to AudioClips and the AudioClips will not be consumed (removed from the queue) for whatever reason. This call simply frees up memory. If it is later deemed necessary to retreive the AudioClips, the associated AudioUrls may be sent (again) to the WAUP for processing and consumption.

Known Issues

  • Invalid data exception thrown in Unity Editor: if a URL is incorrect and leads to valid HTTP response data, the returned data will be linked and attempted to be converted into an AudioClip. This will likely still success all checks until it gets to FMOD. If this is a WebGL build, FMOD does not exist and the exception will not be thrown. This only exists in the Unity Editor and unfortunately, there does not seem to be a way to trap it. If you receive this error Error: Cannot create FMOD::Sound instance for resource W*C, (Unsupported file or audio format. ) confirm you have the correct, working URL.

Release Notes

  • 2018-04-02: v0.5.0 (beta): Initial release.