ETA: Submission to Unity by February 28th
STATUS: development finalization, documentation
Addon/System Integration Status:
A status of 'Functional' means (at least) the functionality of SALSA v1 has been implemented for SALSA v2. In some cases, additional functionality is available in v2.
'Pending' status means conversion has not begun, but support is planned.
Add-ons with a status of 'Deprecated' indicates support for this v1 add-on is not planned for SALSA v2. Discontinued support can occur for various reasons from being unnecessary in v2 to being unsupported by the original publisher. If they don't support it, neither do we.
|RT-Voice Native Mode (& iOS native mode)||Functional|
|Cinematic Sequencer Slate||Pending|
|Unity Timeline Core||Functional|
|Unity Timeline TextSync||Functional|
As you may have already seen on the Unity forum for SALSA, we still aren't quite there. We have made a lot of progress, but the Eyes module is still undergoing development and integration. The core functionality is complete and said functionality is now being brought into the SALSA family of core modules. This requires some changes to the queue and controller systems as we discover new requirements since Eyes does not work the same as SALSA or EmoteR. Eyes has more randomness requirements, whereas SALSA responds to audio analysis and EmoteR fires off emote expressions according to rules. Eyes continually drives in random or tracking mode, creating new destinations for the same animated objects. We are still able to leverage the queue conflict resolution system which was a major goal and this will benefit designers tremendously. The queue incorporates over-ride rules that allow certain expression types to take over from each other if they share the same core expression components. This allows animated components to be re-used across SALSA LipSync modules (SALSA, EmoteR, and Eyes) and also within each module. SALSA v1 had a strict single-use requirement for each blendshape, otherwise much herky-jerkiness would ensue. SALSA v2 does not have this limitation. None-the-less, we have been working dilligently to ensure the Eyes module utilizes the queue for animation processing to gurantee conflict resolution across the modules. Needless to say, Eyes processing looks great! Tracking now incorporates adjustable saccades to eliminate that robotic, locked-on stare that SALSA v1 had. Sure it was great for its time, but v2 ushers in a new era of cool for spicing up your application characters.
While Mike has been knocking out the Eyes todo's, I've been working with SALSA and EmoteR while creating the 1-click settings for our supported model types. It should come as no surprise that we are dropping 1-click support for MCS characters. You will still be able to use SALSA with the MCS models if you wish; we will simply not be supporting or creating a one-click addon for them. We currently have some (subjectively) pretty good settings for DAZ, iClone, and Fuse. We will probably tweak all of these over time as we discover better looking setups, but out-of-the-gate, they work great. With the loss of MCS, we've seen a lot of chatter in the forums about what character system people are jumping to. UMA DCS and Fuse seem likely candidates. Fuse is easy and we've already set it up. UMA is a bit more difficult in the use of its runtime-generated expression system -- so, we're still working on the best solution to handle it.
We are holding off on converting the codeless systems to SALSA v2 until the Eyes integration is done to prevent having to redo work in the event some of our core API changes. We will be supporting them and will likely have them completed by the time SALSA is generally available or shortly afterward. Once the core SALSA systems (LipSync, EmoteR, and Eyes) are complete, our current plan is to submit and finish up the add-on conversions while SALSA is in Unity's approval queue. This allows us to get it into your hands faster at the expense of potentially not having all of the add-ons completed. We will appreciate your understanding and patience and we will continue to be transparent in our endeavors. This blog diary will list our progress with the add-on conversions until they are complete.
We didn't quite make the end of January, but we are really close. We are now targeting the middle of Februrary for submission. That's only a couple of weeks away, and may be optimistic, but we are hopeful. Eyes is getting final features treatment and polish and is looking great! As you can see in the above chart, we're also working on getting as many of the add-ons functional as possible. Some of the existing add-ons will be deprecated since their functionality is no longer needed or is included in the core product. Updating the add-ons has also afforded a new look at the SALSA and EmoteR code and has prompted a few changes and fixes. SALSA and EmoteR rc1 was built today and will facilitate updating the remaining add-ons (which I'm targeting to be completed in the next week).
Our new server is running quite well, we are super happy with the performance and site responsiveness!
Not entirely SALSAv2-related, but we wrapped up migrating our server hosting last night. We've been somewhat disappointed with the performance of our previous shared hosting and renewal was coming up so we decided to make the jump. We're now on a VPS server and the performance is quite good. Let us know if you have any issues.
The Eyes module continues the integration process and is nearly done. We are still hopeful for submission by the end of the month, but it may slip a week or two depending on whether we hit any hurdles or not.
We have ended the closed-beta testing and received some good feedback. We are now integrating the eyes module in with the main package and expect to have that completed soon. Our current plan is to be submission-ready by the end of this month. In the meantime, we are updating existing add-ons to make them compatible with the new system and getting documentation and tutorials ready for release. Please understand, this is not a promise that we will release the product at the end of January. This is our estimated plan and intent. We are a small team and this is not our only job, so we work with what we have. :)
As we mentioned near the beginning of this development blog/log, we will be increasing the price of SALSA Lip-Sync v2 upon release. The expected release price will be $39. Shortly after release, the price will bump up. Also note, this will be a paid upgrade. As a thank you to those that have supported version 1, we will be offering a price-difference upgrade at launch, so you can continue to purchase version 1 now and upgrade upon release and not pay any extra over waiting for the new version. While the new-purchase price is $39, the upgrade price will be $4. Once the full price changes from $39, the upgrade price will increase and will no longer adhere to the price-difference model. So, get your upgrade quickly!
Here's a quick update to show some progress on eye control on a 2D character in SALSA 2.0. The video demonstrates three available methods:
- Eye transform animation.
- Nine sprites (forward, up, upper right, right, lower right, down, lower left, left, and upper left.
- Five sprites (forward, up, right, down, and left)
SALSA and EmoteR are pretty much finalized. Documentation is nearly complete. RandomEyes development continues. We are about 3 weeks into our closed-beta test of SALSA and EmoteR and have had some really good, positive feedback thus far. Very few bugs have turned up and our testers are enjoying the new version and its capabilities.
We have added a few new features and modifications from tester requests. One new key feature is blending back to Animator control in the EmoteR module using Bone components. Previoiusly, if a bone was used in an emote, EmoteR would take over animation and then release the animation at a set position, usually resulting in a jarring snap as Unity's Animator takes control back. Now, EmoteR will track the Animator control of the bone and smoothly deactivates its emote towards the updating Animator pos/rot/scale. The result is a seamless transition back to Animator control. There is also an offset mode available which EmoteR uses to apply relative animations to an Animator's animations. It's all good stuff and opens up a lot of great possibilities.
Here is a view of an expanded SALSA inspector (non-configured) and an expanded configured-for-boxHead) implementation.
UPDATE (2018-10-05): Inspector Design
The advanced elements of the SALSA inspector are more or less complete. We've implemented shape, bone/transform, and sprite switcher interface functionality. Additionally, we've been squashing some bugs and optimizing some core code, along with copious amounts of housekeeping and general cleanup. Document rough-drafts are nearly complete for SALSA. Emoter will be next in line for finalized touches, followed by Eyes. Why the huge delay? There are a few reasons, the biggest being trying to implement the inspector interface for the advanced flexibility the suite uses. We've tried to make SALSA 2.0 compatible with as many Unity versions as possible while keeping the inspector up-to-date and fresh enough. That said, our current version cutoff for Unity is planned at v5.6+. Stay tuned for more info, I'll try to post up a new view of the inspector soon!
UPDATE (2018-07-25): Inspector Design, Optimizing, & Unity Timeline
Since the feature set is pretty much complete for SALSA, we have been working on a fresh inspector implementation. The goal is to make the inspector at-a-glance informative, without being cumbersome -- providing feedback for proper implementation and key settings while the sections are completely collapsed. The animation demonstrates some of the interface ideas we have implemented.
We have also been trying out some additional ideas and working through some scenarios to try and ensure the product is as forward looking as possible. This includes optimization and eliminating unnecessary code and libraries. There is still work to be done here, but currently, the profiler is showing zero GC impact and the queue item registration processes in our sample scene are taking just a little over 1ms to process 36 simultaneous SALSA objects! Additionally, the per frame queue process (animation interpolations) for these 36 registered test items are under 1ms!
Finally, we have been ensuring our current code base is compatible with our Timeline add-ons and in fact, we have some added capabilities in Emoter for custom expression firing that are not available in SALSA 1.0, such as shape jitter variation over time when using one-way shape handling.
UPDATE (2018-06-03): technology preview video 2D (progress-to-date)
Here is another technology preview video. This time we used our new combined 2D/3D workflow, our unlimited sprite/texture/material/trigger capability (sprites for this test), and multi-image sprite sequences for each of the eight visemes used to achieve smoother 2D traditional animation. This workflow could also be used to add full lip-sync and emotion expression to low poly 3D characters using texture or material swapping. We think the results look great.
UPDATE (2018-05-23): Refactoring and Fixes
The past few days have been spent refactoring and tweaking, while testing the feature sets currently in-place. Mike has been working with an advanced 2D model from Digital Puppets and it has brought quite a few things to light. SALSA and EmoteR have been more than up for the task and it has simply been a matter of adding to the feature set rather than needing broad sweeping changes. The whole system is very modular in design so it has been quite pleasing to work with. The new 2D array controllers have really opened up the possibilities of what the SALSA suite can do for 2D. We will be putting out some 2D tech video demonstrations soon.
UPDATE (2018-05-18): EmoteR updated
The EmoteR (emote randomizer working name) received some lovin' today. Implemented manual emote triggering as well as random triggering (in addition to the already-implemented emphasizer triggering). There are now three pools of emotes: emphasizer, random, manual. An emote can belong to any/all of these groups (all emotes belong to the manual group). The randomizer functionality can be strictly a random emote, triggered by a percentage chance at random (min/max) intervals. Or it can take on some dynamics of its own with random fractional extents and/or randomized hold (min/max) durations. So, it is pretty flexible, but easy to implement, providing quite a bit of variation.
We foresee some cool value add for developer/designers who wish to create demeanors for their characters. By adding multiple EmoteR components to a character, each could be configured with emotes associated with a demeanor (i.e. "happy" or "angry"). Then, programmatically or via a mechanism like Timeline, the demeanor could be switched so that SALSA triggers "happy" emphasis emotes and EmoteR triggers "happy" random emotes. Or, likewise the demeanor could be switched to "angry" emphasis and random emotes. The possibilities are endless!
UPDATE (2018-05-17): technology preview video (progress-to-date)
SALSA 2.0 is shaping up to be an amazing upgrade to the product line. With the new design elements, we have discovered additional possibilities to make the system even more flexible and allows us to give you, the designer, a solution which breathes even more life into your characters without sacrificing SALSA's original goal's: to be simple and automagic! Of course, keep in mind, any sweet, new features we discuss here are not considered set-in-stone until the product ships. In other words, something might come up that prevents us from including a feature in the final product.
Our new sound processing algorithm has allowed us to experiment with some forward looking processing, allowing us to adjust the processing bias forward/backward. The only really useful direction seems to go forward and the benefit is eliminating the apparent processing lag visible in the current version of SALSA. This has a dramatic effect on the perceived lipsync approximation, making it feel even more accurate without having to bake in phoneme maps. We believe this will potentially work for micInput as well, further reducing the visual separation between recording speech and performing lipsync.
In SALSA 1.x, we have a custom shapes interface that allows the designer to include and randomly play emotes on the character in an attempt to give them a little more life and not be so static. Unfortunately, this is somewhat of a double-edged sword and can easily cause the character to look twitchy. SALSA 2.0 goes a few steps further. You will still be able to define shapes and have them randomly play if that is your thing. However, we just introduced a new capability that links SALSA's audio processing algorithm with our new Emoter system, allowing audio timing queues to influence a set of emphasis emotes while the character talks. The result is quite awesome - timing is everything!
Previously, we spoke about the potential of 2D animation arrays (vice single-image switching) and at that point, had not been able to test it. We can now report, the results look pretty amazing. While the system has no limit on the number of mouth shapes, we tested a simple set of 3, 10-frame animations to mimic SALSA 1.x's limit (for small, medium, large mouth shapes - 30 frames total). We will also be testing an advanced 2D character with a larger set of visemes to see how that looks -- SALSA 2.0 imposes no limits on the number of 2D/3D shapes or triggers. We do not have any best-practice recommendations at this point on how many frames are effective for an animation, but the inclusion of this capability should make some 2D enthusiasts super happy! Of course, it all depends on the 'look' you are trying to acheive and 2D is much more artistically motivated (for the most part) than 3D. No matter how you look at it, options are awesome!
The RandomEyes overhaul is also getting lots of love. Movement algorithms are applying some scientific research to the model and, I have to say, eye-movement is looking amazing! In addition to the eyes, head-movement is now a thing. Tying head and eyes together creates an even more compelling character. And similar to SALSA's emphasis emote timing, head movement timing will coordinate with other elements as well, like blinking. Super cool!!!
We realize talk is cheap and to that point, we will be releasing a new teaser shortly, demonstrating just how much life the SALSA package will infuse into your characters with so little effort.
We have made some massive progress on SALSA 2.0 over the past couple of weeks. A vast majority of the underlying logic and structure has been completed. We are now confident to say the whole package, in general, is moving to a time-based animation queueing system. As mentioned previously, this queue provides for flexibility and efficiency and is a huge contributor to the new core features of SALSA. In SALSA 1.x, animations conflicted with each other and required limited or zero reuse of shapes (reusing shapes from other triggers and emotes would have caused jittery, push-pull moments and simply wasted processor cycles). Now, the magic of the queue gracefully eliminates this problem by seemlessly tracking and taking over animations using intelligent overrides. Additionally, the queue now operates on time verses speed. Other than allowing for different easing types, this will give cinematic creators a better workflow for timing emotes, being able to separately (and accurately) control animation on, hold, and off times.
We have removed the separate 2D/3D lip-sync elements. Now it is simply SALSA lip synchronization for your 2D or 3D needs. It all uses the same queue and the 2D options have expanded. SALSA 1.x was limited internally to sprite swapping. That process was fine for many customers, but others had different needs, which we were able to accomodate with some bolt-on processes. SALSA 2.0 abstracts the animation control process to a point where we should be able to accomodate any sort of animation or switching needs our customers may have. If it does not come in the box, extending with new functionality is much easier. At present, we have 2D switching processes for sprite, material, and texture. We are even experimenting with an animated array that could offer artists a pathway to smoother 2D animation progression (multiple animation frames per emote or lip-sync trigger). This is still experimental, but the initial results are pretty cool!
One issue some of our customers had with SALSA 1.x was with 3D sound. Basically it broke the lip-sync when the character moved away from the listener. Of course, in true Minnow fashion, we were able to come up with some usable workarounds, but they were not the preferred implementation. Last night, significant progress was made on removing 3D sound shortcomings and implementing a spacially agnostic processing algorithm. Wooohooo!
RandomEyes is getting lots of love as well. We will update on its progress shortly! Stay tuned!!! ;)
SALSA v2.0 development continues. The ability to group shapes/expressions within SALSA (from multiple shapes) brings much more flexibility, but certainly can increase complexity within the inspector.The new inspector is shaping up nicely and providing for a much more streamlined user experience. The queue system is also being refined, unifying the ability for SALSA to use blendshapes and bones to drive lipsync or expressions. This added functionality will enhance SALSA's ability to work with even more model systems. Additionally, it may be possible to implement this same functionality for the 2D system or even implement 2D into the same system. Time will tell if that is going to be possible.
Lately, we have received quite a few inquiries on the release date of SALSA v2.0. It is still too early to establish a release date, but there have been a lot of advances in the development.
In addition to SALSA, two more assets should be available on the AssetStore soon that will enhance the functionality of SALSA (versions 1+). Amplitude for WebGL will provide the ability to link up audio analysis in WebGL projects (not specific to SALSA -- but yes, SALSA will work in WebGL with the product)! The second new asset, MorphMixer, will allow developers/designers to create unique blendshape instances where multiple shapes were previously required. The biggest benefit here is the creation of better defined shapes and the elimination of shape conflicts. This will benefit SALSA v1+.
Neither of these two products require SALSA to operate. They were however, inspired by SALSA in their creation. Amplitude will provide audio amplitude analysis functionality to any project/need. And MorphMixer could be used in any project to blend existing blendshapes together to form new shapes - inside the Unity Editor. Both were submitted over 30 days ago, so we are hoping Unity approves them very soon. SALSA owners will be able to purchase both products at a special price for a time.
Most of the 3D features are finalized and in place. Now we're focusing on 2D elements. Still no release date -- sorry.
SALSA v2.0 is coming!
Likely you have already seen some of the teaser videos we have published, showing off the quality gains over SALSA 1.x. If not, head over to our YouTube channel to take a peek. The response and excitement for SALSA 2.0 is growing and with that, there are a lot of questions. This post will discuss the most common questions we get and (for now) will serve as a devBlog for the goings on.
When is SALSA v2.0 going to be released?
This is our most frequently asked question. We currently do not have a date we're comfortable sharing. This version is, for the mostpart, a complete re-write of the SALSA core. Development is advancing quickly and nearly all of the advanced feature-set is already prototyped and maturing. Stumbling blocks have pretty much been kicked to the side of the road -- full steam ahead. In other words, we don't know, but check back here for updates.
Is SALSA v2.0 going to be a free upgrade to existing SALSA 1.x owners?
Ahh, the age-old question -- is it free? Nope, it's not. And there are a few reasons behind this decision. First and foremost, SALSA is being re-written. As mentioned earlier, this is a big code change and is not going to be a drop-in replacement for v1.x. Due to Unity's distribution platform, we felt it necessary to ensure projects were not accidentally broken due to an inadvertent upgrade. However, if you already own SALSA, it will be nearly free to upgrade it -- at least initially. After nearly 3 years of support and updates, we've decided the old SALSA needs a face-lift and face-lifts cost money. With the new release will come a price increase and the upgrade cost will be the difference of the increase and the current price of SALSA. SALSA v2.0 is expected to release at $39, meaning the upgrade will cost you $4 ($39 - $35). This upgrade window will likely be pretty narrow and the price will increase again (along with the upgrade cost) shortly after release. We're still working out the details, so some of this may change. Check back here for updates.
What sorts of new features will my hard-earned $4 get me with v2.0?
There are a couple of big changes and you've already seen the biggest one -- better real-time, lip-synciness in the form of vastly superior quality. The Minnow doesn't lie! It looks really good. And the quality doesn't sacrifice the core SALSA values and goals. It's still real-time and the pipeline is still quick and easy. In fact, you can still achieve compelling lip-sync approximation just like the SALSA 1.x with the same 3 shapes you used before. You can even use a single shape if you want -- although you probably wouldn't want to 'cause jaw flapping is just plain ol' ugly. With SALSA 2.0, you can use as many shapes as you want for lip-synchronization (yup! that's new!).
Behind the scenes, there's also a new trigger processing algorythm. This new tech brings a boatload of efficiency and flexibility. With the existing version of SALSA, re-using shapes in different mouth-shapes or even in the custom-shapes can cause unexpected, unattractive results like stuttering of blendshapes. It also meant animations would be pulling over-time on the processor cycles fighting each other. With the new queue-based system, duplicate shapes can be used and much more gracefully transition control and processing. Fewer shapes animating means more processor cycles in your back pocket. Priority-based transitions mean your blendshapes will spend less time duking it out and more time looking deliciously cool.
What about my 1-clicks?
SALSA 2.0 will still have them. In fact, with the new core, 1-clicks will be even more streamlined and efficient. The core engine will handle multiple skinned-mesh-renderers and won't need an additional intermediary to translate and sync things up.
Will upgrading to SALSA 2.0 be difficult?
That's a hard question to answer. It really depends on your existing integration of SALSA. For example, if you're using one of the character systems we support with a respective 1-click setup, you would likely only need to install SALSA 2.0 along with the new 1-click setup scripts. If you implement several group expressions in 1.x, these may need to be setup again in SALSA 2.0. The new 1-click setups will likely add expressions automatically, but it depends on what your needs are and how you've implemented and customized your project.
That's it for now folks! As mentioned, check back for updates!