- 23 minutes ago
RBA2D3D lip sync sound are mouth shape phoneme and visemes info
Category
🦄
CreativityTranscript
00:02Hello and welcome. Welcome to RBA, Red Band Alley. Welcome to Emma, age 12. The game itself is about a
00:10one-year-old kid. She's the same kid when she grows up.
00:13Now, what I'm talking about is, if you watch new videos, how to settle polypsynch, I'll be talking about phomees.
00:21That's spelled P-H-O-M-E. Phomees. What on earth is a phomee? And the phomee is the shape
00:34of your mouth.
00:36And of course, if you do it in 2D, they're called vizemes. What? Phomees and vizemes. Okay, I'm going to
00:45translate that for you.
00:47The idea of the phomee and vizeme, I'll show you the character, if you watch her mouth. The phomee is
00:56the shape of the mouth.
00:57Okay, so as you can see, this is the shape of her mouth, which pronounces certain letters, right? Certain words.
01:08Okay, and whatever comes out, I'm assuming, is the vizeme.
01:16Okay, I'm not going to explain it to you like that. Here's my simplified version. What you want to do
01:24is set up your cameras.
01:26I want to show you how to do that in your video. Okay, I've got a body cam. If I
01:32go to the camera, if I go to body cam, that's the body shot.
01:36I'm going to face cam, that's the face shot. Okay, and down here, I've got another camera, which zooms into
01:45the face, so it's closer.
01:47So you can see the actual lips. Just in case you want to make yourself look like a sprite sheet.
01:53Just in case you want to do that.
01:55Anyway, this is how I'm working. Right, so what I'm going to do, I'm going to come out the camera.
01:59I don't need to be in the camera. I just need to select the camera.
02:03As I said, I recommend that you set your cameras up the way I've got them, because it makes life
02:07a whole lot easier.
02:08I'm going to show you this part here. I think you're going to like it, ideally. So here we go.
02:13The idea is to move your mouth in certain positions of each letter.
02:19So when you add audio and you play it, the software can lip sync it, let's say, to your character.
02:27Now, I do the lip sync manual. That's what I do.
02:30But I want the standard animations here, just in case I want to stick her face in a certain position.
02:38Right? So to speak.
02:41So, here we go.
02:43First of all, you've got to turn on the character's skeleton.
02:46This is her skeleton. Right?
02:49Now, if you're watching the video, you know I've got all my face as separate.
02:52OK? The face, this part here, is in the face skeleton, so I can move any part of her face
02:59to match the face.
03:01OK. All I like to do is simple. Right?
03:03I like to do the mouth on its own first.
03:07Then I like to do the eyes. Then the eyebrows. Then the ears. Then the tongue.
03:13And then when I've got all my animations exactly as I want them, I mix and match my animation however
03:19I want.
03:21That's how I do it.
03:22I think you might want to do it a different way. You might want to do it all at the
03:25same time.
03:26Yeah. If you do it all at the same time, it's going to get really complicated really quick.
03:32You'll get yourself lost, ideally.
03:34So here we go.
03:35So, as I say, if you haven't watched my videos, how to set up the skeleton, how to set up
03:39your character,
03:41and watch the video, and you'll understand exactly how to do it.
03:44OK? I'll show you how to do the cameras.
03:46So, in this video, I'm only going to show you, basically, more or less, how to do the mouth.
03:51OK. So, I want to pick any shape of the face that I want.
03:57I want to pronounce the word, OK?
04:02So, if you position your lips as if you're going to say the word, OK?
04:08That's what you've got to do.
04:09So, what I'm going to do, I'm going to take my character.
04:12Now, the lips are a bit too far down.
04:14So, I'm going to bring this up just a tiny bit.
04:17Because this actual shape is almost the same shape as this one.
04:23That's why I chose this animation.
04:26OK?
04:26So, if you type the word, not type the word, if you position the lips as FV.
04:35OK? So, FV is this shape, and IH is almost this shape.
04:40What you can do, if you pronounce this particular word, this part, moves up.
04:50So, you can see a little bit of a teeth.
04:53My character does not have no teeth.
04:55Just so you want to say that.
04:57OK?
04:58So, what I'll do, once I position her the way I want her, I think my tutorials are super simple.
05:04OK?
05:04You select the bones, right-click, you go Insert, Visual Location.
05:09OK?
05:09Come down here, Asset, Create.
05:13There's my new Asset app.
05:14Come over here.
05:16If you don't have to do this, watch my video.
05:18And as I said, I'll show you how to set all this up.
05:20So, it's nice, easy, plain, and simple.
05:23OK?
05:23Go here.
05:24You want to type, I want to type the word B.
05:27I'll click I, H.
05:29Because that's exactly what you see.
05:32OK?
05:32I want to scroll down to the photo.
05:34I want a new photo.
05:35My camera's already set up.
05:36I'll click my camera.
05:38I'll go to Render, and say, yeah.
05:42And here is the new photo for my character.
05:46So, now I'll go over here, and I'm going to say, yeah.
05:49And then everything I do is proper easy, proper simple, and proper plain.
05:52OK?
05:53Of course, it's only easy and simple plain, if you've actually set up the content the way
05:59I'll show you.
06:00OK?
06:00So, it's I, H.
06:03Cuff it.
06:04Say, yeah.
06:05You want to change the photo?
06:06You go here.
06:08Look down.
06:09Find I, H.
06:10Say, yeah.
06:12OK.
06:13Now, obviously, this is all I do.
06:16And I build up my character's animations, so she could be happy, she could be sad.
06:23It doesn't matter which way can you do it.
06:27You want to think of, let's say, her face is in this position.
06:30You want to think of every single expression that her face can do in that particular shape.
06:38When she's doing this, think of all the other shapes that you can do.
06:42I'll just turn off the skeleton.
06:45I'll say, imagine how you're looking at your character, right?
06:47Just look at the character, look at your character, as if you're looking at yourself.
06:51OK, if you basically do, let's say, that expression.
06:57OK, ideally, if you're doing that, your eyes are probably going to open pretty wide.
07:03So, as I said, I do all my animations separate.
07:07I have my eyebrows separate, my nose separate, ears separate.
07:12Just in case I want to lift one eyebrow and not two eyebrows.
07:17Of course, if you're going to do that, you're going to lift probably,
07:19I'm going to remove the lightning lift up too, my eyebrows.
07:22I'm not in that position, I'm here.
07:25As I said, I'm showing you from the first beginning, right?
07:27So, as you can see, I'm just starting to build up my animation archive.
07:34I'm also building a spreadsheet.
07:36That's what we're doing.
07:37So, later on, you can see all the expressions of Emma's face.
07:40Just in case you're curious.
07:42Or just in case you want to use them in future reference,
07:45so you can set up your face almost basically to match my face.
07:51Not my face, it's Emma's face.
07:53I apologize.
07:54OK, what would be the next letter?
07:58Now, of course, click on the closest match.
08:02When I say closest match, I mean, obviously,
08:05you have to kind of pronounce the word yourself.
08:09I'm thinking, OK, the next expression is almost,
08:16I want to say, exactly the same expression.
08:21Except, I'm going to take this,
08:25and I'm going to move her lip down.
08:29And then, there's something.
08:32Because, when she pronounces this, kind of word,
08:37yeah, the lips are almost closed.
08:40Not quite closed, but open a little bit.
08:43Because you can actually see the teeth.
08:45The top teeth.
08:47You can see those.
08:48You can't see nothing else.
08:50OK, kind of word.
08:51Let's take that.
08:52Let's take this.
08:54Insert key for you.
08:55Save that.
08:56And then to save, and save that.
09:01Then, what am I going to call that?
09:03Well, I'm going to call it B, to start with,
09:05because it's a Blender Face animation.
09:09Then, it's more Blender Face animation.
09:11I'm going to call this K, G, H, N, G.
09:19That's what I'm going to call it.
09:21K, G.
09:23So, I'm going to press Ctrl C to copy the name.
09:28That's what I'm going to do.
09:30I'm going to go to the picture.
09:32I'm going to render that.
09:34Say, yeah.
09:35Go to Render.
09:37Go to Save.
09:38Save it as.
09:39Go back here, and press Ctrl V.
09:42And it just pays save.
09:43Press, yeah.
09:44Press OK.
09:45Go back to your photograph.
09:47As I said, you can type the word.
09:50You don't have to type what I'm typing.
09:52You can type whatever you want to type.
09:54I'm just going to do this, because all of the letters that comes with this kind of shape,
10:01this there, that uses, let's say, these words.
10:05I'll show you.
10:07You see each letter.
10:10Now, each letter, the reason why I'm putting on that is because when you pronounce that particular
10:15letter, that particular word, your mouth is in this shape.
10:21So, if you try and pronounce any of those, you will notice that your shape of your mouth
10:28will basically stay in that position.
10:32Oh, yeah.
10:33Oh, OK.
10:34So, basically, really, it's like a bit of a default animation, ideally.
10:39That's what I'm going to tell you.
10:41Now, as I said, you can't see it because the mouth is on the skeleton, you don't see it like
10:45this.
10:45So, this is basically, it's more or less an idle pose.
10:49That's what it is.
10:50OK, I'm going to do one more with you.
10:54And this is...
10:56This is OH.
10:58Right.
10:59So, let's turn the skeleton back on.
11:02I'm going to show you, basically, how do I go about animating my content.
11:06So, we're going to drop something down.
11:08Right.
11:10And I'm going to move this one probably up a little bit.
11:14Let me take a look at that.
11:16It's not bad.
11:17Too much.
11:19I'm going to drop it to about there.
11:21Because, obviously, it's a poo or that.
11:24But, I think that's too much as well.
11:26I'm going to grab this.
11:27I'm going to move this.
11:29That's my tongue.
11:30I don't want to move the tongue.
11:31I want to move the lip.
11:33So, what I'm going to do, I'm going to turn it to the side.
11:36I'm going to grab this.
11:37It's the top of the lip.
11:38And I'm going to pull this up a little bit.
11:41Just so I can kind of get the idea of the shape.
11:45And the tongue is almost probably in a wet place.
11:48I can do removing the tongue.
11:50So, I can select the tongue.
11:53And I can basically bring it forward a little bit.
11:57Go down a little bit.
11:59Of course, when you do this kind of word,
12:01and you can basically almost see the tongue.
12:05I think you can hear.
12:06Great.
12:09Well, you can.
12:11If I pronounce the word.
12:12So, I think I'll go over here.
12:13You've got an insert animation.
12:15Say yeah.
12:16Go here.
12:17Create.
12:19As I say, it's a very super word.
12:22I want to put B on it for Barlinda.
12:24And it's O-H.
12:27Right.
12:28That's the word she's pronouncing.
12:31Go to Barlinda.
12:33I think you can do this at your own speed.
12:37At your own pace.
12:38As I say, yeah.
12:40The idea of my tutorials are to make it nice and easy.
12:44Go to the page.
12:46Now, of course, because of my face animations,
12:49I'll tell you this here on Sunday.
12:51If you're using a specific character,
12:54which I am,
12:55I'm using a skeleton I created in Blender
13:00from a make-human character,
13:02which is Emma.
13:03This is her own skeleton.
13:06So, none of these animations,
13:07not a single one of them,
13:10will work on any other character
13:12except Emma.
13:14Okay?
13:15Just so you understand that.
13:17So, if you're my thinking,
13:19oh, well, can I download your content?
13:20You can download my content.
13:23And you can use my Blender assets.
13:27If my assets are specifically for Emma,
13:31yes, you have to use the Emma skeleton.
13:34You can put it in any character you want.
13:36But you have to use Emma's skeleton.
13:38Because all animations are made for Emma.
13:42And it don't matter what age the character is,
13:45don't matter if she's one or 101,
13:47all animations work on Emma,
13:50regardless of her age.
13:53They work a bit like the Mixamo animations,
13:55a bit like that.
13:56You know, when you get a Mixamo,
13:57and let's say you do a certain animation,
14:01like running, jumping, crawling.
14:03Yeah, my animations work the exact same way,
14:05except Mixamo's work with Mixamo's.
14:07Other websites work with those skeletons.
14:10And my one specifically works with Emma.
14:14So, yeah.
14:14So, all animations, I'm going to tell you,
14:16at the moment,
14:18they're all created by me,
14:20specifically, for Emma.
14:22So, yes.
14:22So, you won't see any animation.
14:24That's inappropriate.
14:25That's what I'm going to tell you.
14:27Okay?
14:27So, if you're watching my content,
14:29I will say,
14:29thanks very much for watching my content.
14:31Thank you for your interest.
14:32As I said,
14:32I'm trying to do something a little bit different,
14:34a little bit unique.
14:35And I'm trying to create my character.
14:37So, she looks different.
14:39I'm going to close her mouth,
14:41like this.
14:42So, you can see,
14:43now, depending on the conversation she's having,
14:45we can position her face,
14:48however, with one.
14:50So, of course, yeah.
14:52Anyway, you get the idea.
14:54It's a way of storing animations.
14:56That's what it is.
14:57They're making them running, jumping, crawling.
14:59And we're going to get to those later.
15:01As I said,
15:02keep following the content.
15:02And as I said,
15:03you're going to have yourself
15:05a really, really nice archive.
15:08By the time we get to the end of it,
15:10yeah.
15:11Every single thing that you want to do with your character,
15:14even lip-syncing,
15:16that's exactly what you're doing.
15:17You're lip-syncing your character,
15:20step by step.
15:23So, that's what you say.
15:23Keep following me.
15:25See you soon.
15:27Bye.
15:27Bye.
15:27Bye.
15:28Bye.
15:28Bye.
15:28Bye.
15:28Bye.