"Wayang" Interactive Puppet project
at The Korea National University of Arts

Director : Suzung Kim,
Researcher : So-youn Park



by So-youn Park

Digital Puppet Animation these days, seem to moving towards expressing actuality like a movie by showing up realistic character made by 3D and natural movement using motion capture techniques. However, if we think about the origin of the puppet animation, today's puppet animation which mimics the character's behavior and that originated from cartoon and flip book has different history with puppet show that has long history that came from the East and the West tradition.     
Puppet in puppet show is symbolic and is opposed to realistic try. Puppet cannot express complexity of human life, but rather expresses the reality in a form of poetic and intuitive way. If we can speak of today's puppet animation as a movie, then the puppet show can be figuratively spoken as a high standard play. Puppeteer transfers the meanings, not just copying the behavior. Because ordinary man or even children can learn to control the puppet while expressing oneself, much research is also being made in using it for the mental treatment and the education of children.  

Interactive Puppet Theater makes users control digital puppet on the screen using their own hands. The Silhouette Character has the strong self-projection and the symbolism.  
User can easily control puppet on the screen by wearing the prepared globes or color chips and putting his/her hands inside a small box with curtains just like the stage.
Using the theatrical expression from simplistic movement of the puppet that is made from computational codes, and the hands and the fingers that are one of the most intimate interfaces of human being, we can re-illuminate the artistic form, named puppet show in digital media

For the interactive theatre, adopting narrative style of story line seems to have a bit of difficulties, so as a start, the linear style of story telling like the one in a music video has been chosen.  
 Three or four different music and the well matched character for each music has been pre-defined and the each character s behavior responses in different way. For example, each different puppet like the Indonesian silhouette puppet, a character from a fairly tale, or some geometric form has its matched music fixed, and this character or puppet reacts in different manner with its own personality.   
The user can select the music on the first screen and can control the actions of the puppet as the music plays along.


Operating environment
Mac G4/350
Director 7.0
Lingo script  
TrackThemColors Xtras
Samsung Web Camera


 Further interest

1. lingo and TrackThemColors have difficulty to express a subtle movement due to memory problem.
It needs more advanced technology to show the delicacy of silhouette animation.
I wish the puppets movement be expressed as an optimized beauty in its own.  
2. I should find the way to recognize the shape and the movement of fingers besides the tracking of colors. One possible way could be recognizing x, y, z of the coordinates of a hand, then adding the sense of distance on the puppets.
3. Research about the finger expression. Can think of a method of interaction between a computer and a human by utilizing the characteristics of human hands that can be moved freely.
For example, scrolling the page up by shaking the palm upwards and scrolling down the page by shaking the palm downwards.  
Can make a program using this type of interaction. If successfully implemented, it could be used in

 help deaf since the finger language can possibly be translated to the spoken words.