Visual Narrative – Lip Sync

BRIEF:

The brief for Lip Sync is to make a 10-15 second animated piece using the audio that I recorded around the theme and topic of London.

During this project, we will be focusing on the mouth movement and body language in relation to the dialogue and sound.

AUDIO:

The first thing I had to do was ask someone for permission to use their voice and interview them based on the topic and theme of London. The audio or dialogue had to relate to London.

We were meant to interview someone we didn’t know to practice going outside our comfort zone, but I didn’t want to do that because I felt very uncomfortable and panicked, so I decided to interview a close relative instead, mostly my mother.

We planned to go out for a walk to record it, but it wasn’t a good day and we were tired, so we did it in the garden. There was a lot of background noise like wind, trees moving, pigeons, planes, and sometimes trains.

I asked questions like what does London mean to you, do you feel at home in London, and if there is a place you feel attached to.

After recording, I had to cut and simplify the audio to around 10–15 seconds using Premiere Pro. I chose one audio, I don’t fully remember which question it was, but it was something about why my mother came to London. In the dialogue, she mentioned living in America before, which surprised me because I didn’t know that. That audio stood out because it gave me more emotion and confusion than the others, and it was close to 15 seconds, so I picked that one.

MY CONCEPT:

After I chose the audio, I took inspiration from the wind and pigeon sounds, which made me think of setting it high above London in the clouds instead of a forest or normal city. I preferred that because you would hear wind and birds, so my concept is just being in the clouds with London below in view.

For the dialogue, I just used my imagination like the brief said. The story comes from being surprised that my mother lived in America, so the dialogue is about why she came to London and then mentioning America.

To show the story, I thought about adding a second, smaller character on the shoulder, like the one interviewing. I would use different camera angles to show both the speaking and the reaction. When America is mentioned, the second character would react with shock or surprise. I need to make sure that the audience understands this, because they don’t know that I didn’t know, so I have to show the surprise clearly through the reaction and camera. I haven’t fully developed it yet, but that’s the idea.

MY MOODBOARD:

For my moodboard, I first listened to the audio. I mostly listened to the background first. The main things I could hear were wind and pigeons. That immediately gave me the idea of being high above the clouds, because you can mostly hear wind up there.

Because of that, I used an image of someone sitting on a cloud, a pigeon, and people holding onto a tree. The tree image was mainly to show the wind, because there were not really any other images that showed wind clearly.

After that I listened to the dialogue. At first there was a lot of talking about living in America. That gave me a feeling of surprise and confusion, so I picked two images. One was someone looking confused, and the other was more like a thinking face, almost contemplating what the comment was and how they lived in America first.

Then they talked about Tokyo. Tokyo was already known, but I still grabbed an image of it. In the animation, though, I am probably just going to use the flag instead of trying to animate an entire country.

The main colours I wanted were white, grey, light blue, and yellow.

White and grey were mostly for the clouds. I did not want the clouds to be permanently white because if it is purely white, you are not really going to see much. So I added a bit of grey to make them feel more like clouds instead of just a big white puff of cotton.

Light blue was mainly for the sky.

Yellow was going to be the reflection of the sun. I was also thinking of maybe putting some yellow on the clouds themselves.

For the main character, I took the idea of sitting on a cloud, so they are physically sitting on one. Only half of the body is in the cloud.

The character is a mix between a mannequin, like the small humanoid figures people use for art, and a biblically accurate angel.

I combined those two ideas together, and that became the concept for the whole animation.

CHARACTER DESIGN & CONSTRUCTION SHEET:

Main Character:

For the character, it’s kind of like a biblically accurate angel, but simplified because the full version would be too complex. I made it more like a marionette, with separated joints and limbs, and wings partly covering the face with eyes on them.

For the sketches, my first attempt was to draw the character more like the way I normally draw, more alien looking, and I tried to make the character look more feminine based on the voice. But the more I kept sketching, the more I realised the design would probably be confusing to animate, so I scrapped the first idea and moved onto a new design.

The second design looked more promising and was much easier to draw, so I decided to keep it as the final design. I also experimented with different eye designs to make the character more expressive. At first, I wanted to place the eyes on the wings or the head, but they looked too small, so I decided to move the eyes somewhere else, like the chest area, because there was more space.

I also tried making another version that was more bulky, while still keeping parts of the second design, but it did not work out because it looked too human and too bulky. In the end, I stuck with the second design because it looked much better than the others.

Attempt #3, #4 & Construction sheet
Final design with eye

Character Inspiration:

I originally got inspired by using small mannequin models for the body design of my angel character instead of using a more traditional angel design. As time went on, I realised I maybe was inspired by the idea of the DevilTechs, especially the way their limbs are separated instead of connected with normal joints. The limbs float slightly apart from the body, almost like magnetic robot parts.

Persona:

Instead of sketching a whole new character, I decided to use an already existing character which my persona. I sketched it using photoshop to be my side character and it will be on the main character shoulder interviewing them. As my persona will be showing both my reaction and my body language in the animation.

My Persona

Character Turnarounds:

First attempt:

I disliked doing the character turnaround because it was one of the most difficult and stressful exercises I have done. Even though I understood why turnarounds are important, I still struggled with them a lot. I understood that they are used to show the character from all angles, like the front, side, and back, almost like showing a 3D model instead of just a flat 2D drawing. They are also important for animation because characters need to be shown from different angles in scenes instead of always facing forward.

I found it difficult because it was my first time doing turnarounds, and my character has a curved shape which made it harder to draw from specific angles. I also struggle with proportions, especially when dealing with perspective and 3D shapes. Things like hands appearing bigger when closer to the camera and smaller when further away made the exercise confusing and stressful for me.

Even though I struggled with it, I still want to practice turnarounds more in my own free time without the pressure of assignments.

Second attempt:

I struggled with turnarounds at first because they were not explained properly, so I did not understand the structure. My first attempt was difficult because I tried to draw every angle, including the front, side, and back views. I also found it hard because I usually draw characters very large, so shrinking them down to fit on one canvas was challenging.

For my second attempt, my teacher advised me to draw over my original drawing and adjust it instead of starting from scratch. I decided to do two main views and then create the back view by copying and flipping the front view. After that, I erased parts of it and redrew around the joints rather than drawing the whole character again.

This method worked much better, and I was able to complete the turnaround. I am happier with my second attempt because it looks much cleaner than my first version, which looked quite messy.

Pose sheets:

I did not have enough time to make a pose sheet for my character. There also was not a lot of poses for the character to do because most of the movement was only head movement, arm movement, hand and finger movement, and blinking. There was not really any full body movement because half of the character’s body is hidden in the clouds and not fully visible.

If I had more time, I would have experimented more with different poses and movements, maybe even making the character move in a more dramatic way like a ballerina. Instead, I mainly worked on the poses directly during the animatic and animation process.

Character Colour Concept/Palette:

Brain Storming: For the main character, I’m thinking of using a basic monochrome or simple colour, like white with a bit of other colour, so it doesn’t blend into the background and actually sticks out more. I want it to almost feel illuminated, especially at the joints, since they are slightly separated and hovering. So I might add a glow in between the joints and also around the head since it’s floating. I’m not fully sure on the exact colour yet. Something like yellow would stand out a lot, but it might be too obvious, so I need to think more about a colour that still glows but isn’t too distracting.

For the small shoulder character (my persona), I’ll also use a monochrome colour that stands out, but doesn’t clash with the main character or the background.

STORYBOARD:

For this storyboard, the first frame is a rough sketch above the clouds, high above London, with London peeking through the clouds.

The second frame has the main character with my little persona sitting on their shoulder, talking about London high above it.

The third frame is a close-up of them and they continue talking.

The fourth frame is when the character mentions living in the US. The persona on the shoulder looks confused and shocked, with question marks. I did not know, so it was a huge surprise.

The fifth frame is when they continue talking about going to Tokyo, but the side character is still confused about the US. They are very deep in thought and still confused.

The sixth frame continues with talking about getting a job in London. The persona on the shoulder is still stuck on the US, still deep in thought, and just cannot get past it.

That is how it ends. It just cuts to the ending.

ANIMATIC:

My animatic was made to help me plan out the full animation. It helped me figure out the poses, actions, and where the characters were supposed to be positioned on screen. It also helped me understand where the eye should be looking and where the hand gestures should go so the animation would not become confusing.

The animatic also helped make sure my storyboard made sense, even if it was still rough. Another reason I used it was to separate the two characters more clearly by using different colours. The main character was coloured bright blue while the other character was purple, because if they were both the same colour it would have been confusing to tell who was who.

BACKGROUNDS:

Background Rough Layout:

For my rough layout and background, I started by making rough sketches of the background concept for the audio. The idea was to have the main character high above London, with half of their body hidden inside the clouds. I drew the city of London below them, almost like a pocket opening in the clouds where you can see the city underneath while waves of clouds surround it above.

I also made other rough background sketches that were mostly focused on the clouds. I wanted the character to feel far above London, looking down and talking about the concept of London, instead of standing somewhere normal like on top of a building.

Background Colour Layout:

For the background colour, I used a mix of photobashing and regular painting in Krita. For the London background, I photobashed an image of London because I did not want to fully draw or paint an entire city since it would have taken too long. I cropped the image and positioned it onto my rough layout, then painted around it and blended it into the background.

I painted the clouds using different shades of white, light grey, and dark grey to create shadows and wave-like patterns instead of making the clouds look flat. I wanted the colours to feel soft and smooth, so I painted over the rough lines and blended everything together to make the background look more calming and natural, almost like carving or smoothing the colours together.

I also added small amounts of yellow onto the clouds and blended them out to make them look like soft sunlight shining across the clouds. The overall mood I wanted was something calm and relaxing, almost like a peaceful field made out of clouds.

I am happy with the results of my background and I am hoping on making more backgrounds like this one in the future.

X-SHEET:

I found the process extremely frustrating and annoying because it was my first time using an X-sheet. It took much longer than I expected and I spent a lot of hours working on it.

One of the hardest parts was creating and editing the transcript. I used Premiere Pro to transcribe my audio and then had to separate the dialogue into individual sounds and letters so I could place them correctly on the X-sheet. For example, I had to work out where each sound started and ended within a word.

Another problem was that the transcript audio sometimes got confused with certain sounds and words. Some sounds could be treated as separate words, which made it harder to break the dialogue down into the exact letters and sounds I needed.

Even though I found the process difficult, I understand why X-sheets are important. They are used to sync the lips with the dialogue and help get the mouth movements right.

I am not completely sure how much the X-sheet helped me with the actual animation process, but I understand why it is important and why people use it.

If I create animations with dialogue in the future, I will probably have to use X-sheets again because they help keep the speech and lip movements synchronised. However, I still find the process frustrating and would not choose to use it unless it was necessary.

MOUTH SHAPES:

Mouth shapes are used to make a character pronounce different letters and sounds when speaking. Different mouth positions are needed for sounds such as A, L, E, H, V, F, and others.

I found the process difficult and confusing because most mouth shape references use teeth, tongues, and detailed mouth movements. My character does not have teeth, so I had to improvise.

I took inspiration from how frogs and birds move their mouths, especially how birds are animated when they are made to talk like humans. Instead of using teeth, I focused on changing the shape of the lips and using the tongue to show different sounds.

For example:

  • For A, the tongue rests inside the mouth.
  • For L, the tongue lifts upwards.
  • For sounds like TH, the tongue almost sticks out.
  • For V and F, I made one lip almost bite the other because my character has no teeth.
  • Simpler shapes such as MO, and U were easier to create.

The hardest mouth shapes were sounds such as T, CH, N, G, S, SH, Z, and D because the reference sheets showed lots of teeth. Since my character only has lips, these shapes were difficult to adapt, so some of them are still a work in progress.

Reflection: When I started, I found the process complicated and stressful. It was good practice, but I did not find it very enjoyable because creating mouth shapes for a character without teeth was difficult.

If I need to improve these mouth shapes, I would research animations with characters that have no teeth, such as birds in Rio, and study how their lip-syncing works.

Whether I use mouth shapes again depends on the type of character and project I am making. If I need to create a speaking character or do lip-syncing in the future, then I would use this skill again.

KEY FRAMES:

At first, I was unsure what to do with keyframes. I understood them as where the action starts and where it ends. For example, if I wanted my character to raise their hand up, I would need to know where it starts and where it ends. My teacher gave me a useful tip by starting from a point like the edge of the wrist or hand and then following it from one frame to the next, from point A to point B, until the final position. The same idea applies to other actions, like if I want a leg to kick, I would pick a point like the ankle and move it from point A to point B.

Once the teachers explained it and gave me a simple tip, it helped me understand it better. Thinking about it as ease in and ease out helped me more than just thinking of it as keyframes.

I was also told that keyframes are more like pose to pose and I need to think about where the joint actions are supposed to start and end and what angle they should be at.

I will use keyframes in the future for certain animations, especially smoother ones or smaller movements, like someone walking into a café. But for faster animations, like fight scenes or very quick actions, it can get confusing because everything moves too fast to clearly plan the start and end of each action.

I used Toon Boom for this work.

FEEDBACK:

  • Your keys are looking excellent – I think it’s time to move into inbetweens. Good job on following the process and getting to know ToonBoom.
  • You should have one drawing every 2 frames, unless your character is in a ‘hold’ or pausing in a pose.
  • You should add your mouth shapes at the end, again working on 2s, so having one new mouth shape every two frames.
  • Pay attention to ease in and ease out when animating – make sure if you want to ease an arm move, your inbetweens are drawn more similar to the first key, with the movement increasing in distance, and then decreasing again into the second key (the difference in the drawings getting closer and closer together).
  • Be confident in your artistic style! It’s great to see your unique artistic voice moving into motion.

IN-BETWEENS:

I understand in-betweening as creating the movement between one action and the next. For example, if I was animating a punch, I would start with the beginning, middle and end poses, then create the actions between those points. I worked frame by frame between the beginning and middle, and then between the middle and end, to create the full movement.

When working on the in-betweens, I found it both difficult and easy. Some movements were straightforward, but I found it harder when I needed to get specific angles right. This was especially difficult when rotating the hands because the fingers were separated and moving at the same time. To help with this, I sometimes copied and pasted the fingers and blended the different positions together to create the movement I needed.

The most difficult part was turning the head. I needed to know exactly where the face was facing and keep the mouth and wings at the correct angle. I had to make sure I did not accidentally make them look flat because I was trying to make the movement feel more 3D. Getting the exact angles and movement right was one of the main challenges during this process.

Throughout the project, I received positive feedback from Jess, my teacher and, earlier in production, from my peers. The feedback was about areas such as the wings, the body movement and the angles used in the animation.

Overall, in-betweening helped me work from the beginning pose to the end pose by creating the actions between them. Although some parts were challenging, especially the angles and head turns, it is a technique that I will continue to use in future animations.

MOUTH SHAPES WITH TEETH:

During production of my animation, I made another set of mouth shapes instead of creating them separately. I decided to do this because it took less time and was easier to manage. If I had made another full set separately, I would have spent too much time on it and taken time away from working on the animation itself before the deadline. Instead, I created the mouth shapes and added the teeth to them during production, which helped me focus on completing the animation.

I am happy with the results of the mouth shapes in my animation. If I had more time, I would have practised mouth movement more and tried to improve it further. I would also have recorded myself moving my lips and used that as a reference instead of mostly eyeballing the movement.

During production, I found it frustrating to get the mouth shapes working with the dialogue audio because the lip sync was not matching properly. The audio was moving too quickly and the frames were not keeping up, so I had to adjust the X-sheet dialogue to match the main audio and make the lips sync better.

Although I am happy with the final result, I think the lip sync could have been improved. I found that some parts of the audio did not match the timing in the X-sheet because of very small pauses and differences in timing, which made syncing difficult. Even though it was not fully synced in every place, I am happy that it works well overall and that the mouth shapes are on the character and follow the dialogue.

PERSONA:

For my Persona, I did not have enough time to animate the character the way I originally planned because of the short deadline. Instead, I drew over my original sketch, copied and pasted the drawing onto three frames, and adjusted the angles before placing the Persona on the shoulder of the main character. Since I could not add the movements and expressions I had planned, such as looking at the main character and then looking away, I used a speech bubble showing the Persona’s face with different reactions. I also added exclamation marks and question marks to help show emotions such as surprise and confusion.

I am happy with the final result, although I wish I had more time to develop the Persona further and include the additional movements and expressions I originally intended. Despite the time constraints, I think the Persona worked well and successfully communicated the reactions I wanted to show.

TOONBOOM:

Toon Boom was not my favourite software to use because I found it difficult compared to Photoshop and other software. It felt old-fashioned and sometimes made me stressed while working. However, I used Toon Boom throughout the project to create the animation, including the keyframes, rough animation and clean-up process.

Although I was not always sure about all of Toon Boom’s features, I understood that it was useful for constructing the animation and organising movement. It helped me create key poses and build the animation frame by frame rather than drawing everything from scratch. While I found the software challenging to use, it was still an important part of producing the final animation and helped me develop the movement and structure of the project.

FINAL ANIMATION:

FINAL REFLECTION:

1. Reflecting on My Research

Research helped me decide what I wanted to make for the character and environment. From the audio, I focused on the strong winds, pigeons and the feeling of being outside. Because of that, I decided on something high above London in the clouds.

For the character, I researched angels and biblical angels. I thought if the character was high above the clouds, I should think about what would be on the clouds. That led me to the idea of an angel.

I also researched virtual models with separated limbs. Originally, I wanted to separate mannequin limbs, but I did not want them to have joints or balls around the limbs because I thought it would be awkward to animate and an eyesore. Looking at these models helped me decide on separated limbs instead.

The audio was the main influence on my idea rather than the visual research. I kept listening to the audio and focused on the background sounds, which helped me think about the atmosphere and feeling I wanted to create. The research helped me visualise the character and environment more clearly and refine the concept. It was most useful for helping me create the background, atmosphere and overall feeling of the animation.

2. Reflecting on My Concept

The biggest surprise from the recording was finding out that my mother used to live in America. I already knew she had lived in Tokyo before and then lived in the UK, but America was the one thing that surprised me more than anything else.

That surprise gave me the idea for my concept. I decided to have my persona on the main character’s shoulder so they could show the surprised reaction when the recording reaches the part about living in America.

The audio also made me think about being high above London in the clouds, which led to the idea of creating an angel-like character.

I chose to focus on the surprise moment because I wanted to show my reaction in the animation. If it was only the interview by itself, it would have felt more like a normal interview and less interesting. Having my persona react made the animation more entertaining and helped show something that could not be heard in the audio itself. Although the audio was quite monotone, the animation allowed me to show the surprise visually and make the scene feel more expressive.

I decided to place my persona on the angel’s shoulder because I imagined the angel-like character as a huge figure inspired by biblical accurate angels, while my persona remained human-sized. This helped show the difference in scale between the two characters.

I also chose to interview my mother because I felt more comfortable talking to someone I knew. I was shy about interviewing strangers, friends or neighbours, so speaking to my mother felt less stressful and allowed me to focus on the recording itself.

I decided to focus on one main idea rather than developing multiple concepts because I wanted to keep the project focused and manageable within the deadline. Overall, the surprise about America stood out the most in the recording and helped shape the direction of my concept.

3. Reflecting on My Process

Some parts of the process were more challenging than others. The turnarounds were stressful because I struggled to draw the shape correctly and found it difficult to keep the character consistent from different angles. I eventually found a solution by drawing over my original sketch, which helped me understand the structure better.

The X-sheet was one of the most stressful parts of the project because I struggled with the timing, wording and sentence placement. I had to go back and forth many times to correct mistakes and make sure everything matched the audio correctly. I also found the mouth shapes difficult because my character originally did not have teeth. After receiving feedback, I decided to add metallic teeth so the character could perform the full range of mouth shapes needed for the lip sync.

Working in Toon Boom was also challenging because it is not my strongest software and is very different from programs such as Photoshop. The keyframes and in-betweens were confusing at first, and I found the animation process stressful because I was still learning how the software worked. However, as the project progressed, I became more familiar with the process and gained more experience using the animation tools.

Time management was another challenge throughout the project. As the deadline got closer, I realised I would not have enough time to animate my persona the way I had originally planned. Instead of removing the persona completely, I found another solution by tracing my drawing, copying it onto the frames and positioning it correctly on the character’s shoulder. I then used speech bubbles, facial expressions and text to show the persona’s reactions. Although this was different from my original idea, it allowed me to complete the project on time while still communicating the reactions I wanted to show.

Looking back, I think the biggest thing I need to improve is time management. If I had managed my time better, I would have had more opportunity to develop some of my ideas further, animate the persona more fully and add colour to the final animation. Despite the challenges, the project helped me improve my understanding of the animation process and taught me how to adapt when things did not go according to plan.

4. Reflecting on My Final Outcome

Overall, I am happy with the final outcome, although it does not completely match my original idea. The main character followed my original concept closely and developed in the way I had planned. The design remained consistent with my research into biblical accurate angels and the idea of being high above London in the clouds.

The biggest difference between my original idea and the final animation was the persona. Originally, I wanted the persona to be animated and interact more with the main character. However, because of time limitations, I was not able to complete that part of the animation as planned. Instead, I used speech bubbles and facial expressions to show the persona’s reactions. Although this was not my first choice, I think it still worked because the persona was showing thoughts and reactions rather than speaking directly.

I am particularly happy with the character and environment designs because they match the atmosphere and concept I wanted to create. The background supports the idea of being above London in the clouds, and the angel-like character fits the setting well. I also think the ending works effectively when the main character notices my persona’s confused reaction, as it helps communicate the idea I wanted to show.

There are still aspects I would improve if I were to do the project again. I would spend more time animating the persona, adding colour and developing some of my original ideas further. I would also like to make the animation connect more closely with elements of the audio, such as the birds and other background sounds. Looking back, time management had the biggest impact on the final outcome, and it is the main area I need to improve in future projects. Even so, I am pleased that I was able to complete the animation and find solutions to problems when they arose.

Leave a Reply

Your email address will not be published. Required fields are marked *