msLipSync

This script allows you to easily do lip sync for both, characters using switch layers for mouth shapes, and characters using bones to control the mouth (bone names need to be a single word). The script will work with Anime Studio 11 up to and including the latest version of Moho. I haven't tested recently with versions older than 11, but it might work there as well.


Installation
=====
The following files should be downloaded from the script. Place them in the script directories as shown. Then restart moho.

scripts\tool\msLipSync.lua - the main code for the lipSync script
scripts\tool\msLipSync.png - the icon displayed for the tool
scripts\utility\msDialog.lua - dialog utility routines
scripts\utility\msHelper.lua - debugging routines
scripts\utility\msPhonemes.lua - routines for breaking text into phonemes


Apparently, text files cannot be uploaded here, so the following lines will need to be copied into a file called, "lipSync.txt" and placed in the scripts\utilty directory. It is a bone map (described below) for characters that have mouths controlled by bones. It needs to be in the utility directory, even if you use only switch layers, because the lip sync script expects it to be there. You can change the bone angles to your liking. The discussion below explains how.

Scarlett 2
Open/Close Squash/Stretch
AI 304 349
E 278 349
L 267 353
etc 258 349
FV 233 349
MBP 203 349
rest 223 368
O 306 300
U 306 325
WQ 193 266
Dragon 1
B70
AI 280
E 285
L 285
etc 290
FV 295
MBP 298
rest 298
O 285
U 290
WQ 295
Gorilla 1
Menace
AI 156
E 161
L 167
etc 169
FV 169
MBP 176
rest 187
O 160
U 160
WQ 160
Bigsby 1
Open/Close
AI 245
E 216
L 216
etc 198
FV 183
MBP 180
rest 180
O 203
U 197
WQ 188
Gruille 1
Mouth
AI 371
E 380
L 380
etc 390
FV 395
MBP 403
rest 403
O 380
U 390
WQ 395
end

Functionality
=============
The script allows the user to specify a phrase in a dialog box, as well
as a starting and ending frame. It will then create keyframes for the
character, matching the phrase entered by the user.

Usage
=====
This video describes the script
https://www.youtube.com/watch?v=bb8dVbtTRrI

The script is named Lip Sync. In order to select the script you
must select either a switch layer, that controls the standard mouth
shapes, or a bone layer that controls mouth movements. If you are
not on one of those layers the script will not be enabled.

After you have selected a mouth switch layer or bone layer, the Lip Sync
tool will be enabled. Next, move the timeline marker to the starting point
of the audio you want to lip sync. Now, move the cursor to the workspace
and drag until the timeline marker is moved to the place where you want
the lip sync to end. (Note: You need to drag in the workspace area, because
the scripting does not allow you to drag within the timeline ares.)

When you have finished dragging the mouse, let the mouse button up and
a dialog will come up. The start frame and end frame will automatically
be populated by your mouse drag. Next enter the text for the audio.
The dialog allows you to select Phonetic spelling and Debug.

When you select ok the script will add key frames to the layer you
have selected. It will not delete existing key frames.

Tips for Lip Sync
=================
- Use phonetic spelling for greater control.
- Repeat vowels to have them sound for a longer time.
- For pauses, just lip sync each section separately.
- Use phonetic spelling and captial letters for vowels that you want long pronunciation.
- Use a hyphen whenever you want to have the "rest" mouth shape.
- Increase the consonant frame count if you consonants seem to be held too short.
- Tweak the result after generating key frames, to meet your needs.

Intention of the Script
=======================
While the script does an excellent job spacing out phonemes, the
intention is not specifically to get a perfect match. Rather the
intention is to get the proper sequence of phonemes in the correct
order and close to the correct location. You can then tweak the
speech by dragging key frames as normal.

Consonants, Vowels and Held Tones
=================================
The script recognizes that, in general, many consonants cannot be held.
For example, in the word, "cat", "c" and "t" cannot really be held a
long time. By default, those kinds of letters are only given one
frame. The script does allow you to specify the length of the
consonant sounds.

Vowels, on the other hand can easily be held for a long time. As
a result the script will assign all the short consonants the specified
short time, subtract them from the total number of frames and then divide the
result by the number of vowels. This creates a phoneme spacing that
matches the speech in most cases.

If your speech has certain sounds
that are held inordinantly long you simply need to list those sounds
multiple times. For example "the faaat cat" would hold the "a" sound,
in "fat", three times longer than the other vowels in the phrase. The
script also recognizes that certain consonants, like "n" and "m" can
be held, so they act like vowels.

Non-letters
===========
Punctuation and numbers should not cause the script to crash, but the
symbols will be unrecognized and will, in most cases, result in an
"etc" phoneme.

Phonetic or not
===============
The script does a fair job at converting basic English phrases into the
appropriate phonemes, but using phonetic spelling matches the phonemes
exactly to what you want. You can even use it to match foreign language
by and non-language, just by selecting whatever phonetic symbol you
want. The exact phonetic mapping is given at the end of this file.
For vowels, capital letters produce long vowel sounds. Lower cases
vowels result in short pronunciation.

Debug
=====
You can select Debug if something goes wrong in the lip sync. With
Debug selected, statements will be printed to hopefully help fix the
issue. Note, since I do not earn my income from the script I cannot
support problems that might occur, but hopefully the Debug option can
help if you encounter a problem.

Switch Layers
=============
If you are using switch layers for lip sync, it needs to have the
standard mouth shapes. It can have other shapes, and they can be in
any order, but it must have the shapes AI, E, L, FV, etc, MBP, O, U,
WQ, rest. Note that capitalization does matter.

Bone Layers
===========
If you are using bones to control your mouth, the dialog will show a
selection box where you will choose the appropriate bone map to use
with your character. You can modify the file Utility\lipSync.txt to
lip sync any bones controlling you character's speech.

Bone Maps
=========
A bone map tells the script what angles to set for which bones in order
to produce the mouth look you want for any of the standard mouth
phonemes. You can have any number of bones controlling your mouth. Bone
maps that come with this script include Scarlett (used with Scarlett Riggs),
Bigsby, Dragon, Gorilla, and Gruille.

Creating Your Own Bone Maps
===========================
Creating your own bone maps is easy, but it follows very specific
rules. Failure to follow the rules will cause the script to break.
The easiest way to create a bone map is to copy an existing one and
modify it. You don't add a new file, you just insert another map into
the same file. The format of the map is

mapName numberOfBones
boneName1 boneName2 ... boneNameN
phoneme boneAngle1 boneAngle2 ... boneAngleN

Rules of the Bone Maps
======================
- The last line in the file must be the word "end"
- mapName and boneNames must be a single word
- the mapName can be anything, but the boneNames must match real bones
in the character
- a forward slash, /,  is allowed in names, but other characters might cause the script to
break
- capitalization matters for bone names
- you need to include bone angles for each phoneme
  - AI E L etc FV MBP rest O U WQ 
- capitalization matters for phoneme names
- bone angles need to be in degrees

A Word of Advice About Creating Your Own Maps
=============================================
Before you create your own bone map, study the ones that come with the
script. Next create a very simple map with a single bone. If a bone map
doesn't seem to work, try to simplify it and test with a very simple
phrase.

Phonetic Spelling
=================
The following list shows the phonetic letter and the phoneme that will
be created

    a = AI
    b = MBP
    c = etc
    d = etc
    e = E
    f = FV
    g = etc
    h = etc
    i = E
    j = etc
    k = etc
    l = L
    m = MBP
    n = etc
    o = O
    p = MBP
    q = WQ
    r = etc
    s = etc
    t = etc
    u = U
    v = FV
    w = WQ
    x = etc
    y = etc
    z = etc
    A = AI
    E = E
    I = AI
    U = U
    O = O
    R = O
  - = rest

Notice that in many cases the long and short vowel have the same mouth
phoneme. That just makes it easier to specify the spelling. Notice
also, that you may specify "-" so signify the "rest" mouth position.