If you want perfect lipsyncing, then you may want to try Reallusion's iClone AccuLIPS, which detects and extracts text and viseme (the position of the face and mouth when saying a word) from audio, or imports a prepared script for precise lipsyncing. You can create natural speak with the co-articulation design, and further fine-tune every word’s visemes and strength level.
AccuLIPS contains an English dictionary with 200,000 default words, which are easily customised by adding new words, or changing others. In the video above, and outlined below, I show you how to use it. I believe that my mini-guide will prove useful, as existing resources focus on AccuLIPS and Face Puppet separately without much integration. With this short piece, I aim to emphasise the effective combination of these tools to achieve a more dynamic and dramatic character performance, moving beyond simple lip-syncing techniques.
AccuLIPS is part of Reallusion's iClone, which is just really simple. The pipeline between Character Creator 4, iClone 8, and Unreal Engine is pretty seamless once you’ve done it a few times.
How to make a lip-sync animation
01. Enhance the AccuLIPS clip
Before I do anything else, I like to enhance the AccuLIPS clip by going into the lip options and setting the smoothness to 1 and the strength to 1.2. I find this tends to make the movements look a bit more natural.
As well as setting the smoothness and strength, you can also set the talking style.
02. Set the base emotion
I then like to set what I call my 'base emotion' for the clip, which can be done with an animation from the Digital Soul pack of pre-animated facial expressions, or by capturing my own expression from my iPhone via Motion LIVE for facial mocap.
03. Make another pass
After that, I make another pass, often with face keys, to enhance and finetune particular moments in the dialogue.
04. Make more passes in Face Puppet
Then I'll make another number of passes with iClone's Face Puppet, adding more emotion at key points and generally moving the mouth in a more expressive way.
05. Add some head motion
Finally, I add some head motion with Face Puppet, and then whatever body motion the character requires for the scene.
Learn more
To learn more:
