G3-Final Report

DVFX2007

CoWeb

Participants

Groups

Other Updates:

1. Story

It was a tough time for us to plan the story. When we decided to do Ping Pong, we didn't know what exactly we would do. It took us two long meetings to brainstorm the ideas and techniques. We thought about Ripple effect, particle effect, ghosting, portals and etc (In the end, we achieved most of the effects, although we didn't put all of them in our final video). Eventually, we hit the "remote" idea: playing Ping Pong through a "modern" Television.

However, simply playing Ping Pong back and forth thought a TV would be tedious. Another long meeting was devoted. We decided to have something fun with the TV, such as "suddenly turn off the TV", so that Ping Pong ball would either disappear in the TV or bounce back. We also decided to have one player plays with herself/himself. Movie "The ring" triggered some of our thoughts.

2. Shooting

Initially, we had two shooting plans to achieve the remote Ping Pong effect:
1). Use the entire Ping Pong table
Cut a square hole (act as the TV) in a huge white paper board, which pretends to be the wall
Put the Ping Pong table through the square hole of the "wall"
Two players play with each other on two sides of the wall
Advantage: easy to synchronize, since players are really playing with each other
Disadvantage: Hard to set up the environment, especially we need a HUGE wall, and the player would not be able to play with himself

2). Use the Ping Pong table in half and half
Put them in different rooms and place at least one of them against the wall.
Use green screen on the wall to add TV
Shoot separately
Advantage: relatively easier to set up compared to the first plan. Can achieve "playing with self" effect
Disadvantage: hard to choreograph and synchronize, hard to align the table to make it look natural.

We had a meeting with Irfan. Thanks to Irfan's suggestion, we decided to begin with the second plan, and it turned out to be feasible

We did four times of shooting in total. Here are the details:

Shooting 1:

We had a first test shoot as a prototype to make sure that we were able to align the half-half table and achieve the remote effect. Although it was only a test, it took us an entire Sunday to disassemble the Ping Pong table, make up green screen, calibration and do all the alignments.

The alignment of two half tables turned out to be successful, but we realized the difficulty of choreography and decided to use metronome.

Shooting 2:

We did a second time-consuming shooting, but found that we were not able to use the footage due to the lack of careful plan. After several discussions among group members, we eventually finished this detailed shooting plan, in which we carefully planned each scene and camera positions. Shot_Planning.doc

Shooting 3:

Carrying the plan, we had the third shooting, which took us another Sunday¡
We decided to shoot the near-end player separately in front of a green screen, instead of shooting it with the near half table and the wall. The reason is that: we want to add the CG ball later as a middle layer in between the player and the table, so that when the player serves the ball, the ball can be above the table layer but under the near-end player, therefore half occluded, like this:

Here are the three main layers we shot:

We successfully finished all of the planned shots that day.

Shooting 4:

When we started processing our footage, we realized that in particular parts of the video, there were scenes where the television was turned off. With televisions, they are usually covered in some reflective layer like the glass so we decided to repeat this effect in our video. Therefore, we did the forth shooting. Because the television was fake in the first place, we had to go about creating the reflection from scratch. This is how it was done..

Watch the video that is to be reflected many times to get the movements memorized
Perform those movements in a room suitable for use as a reflection (because the background of the scene itself was revealed at the beginning of the video, it was important to make sure that the reflection closely matches the same background in terms of objects or colors on the wall)
Place the video, mirror, and align it on the television
Decrease the opacity and apply a Gaussian blur to simulate the light spreading across the surface of the television screen as well as hide potential imperfections in the acting
Take the table layer from a scene with the remote player and place it on top of the reflection and apply the same filters on it.

Here is the result.

3. Pipeline and post processing:

There are two streams in our pipelines:

(1) Maya CG animation.

It contains the following effects:

a. Channel selection:
This could also be done in Premiere, but we decided to do it in Maya. Each moving video channel is a billboard with video textures in Maya. We move them together from right to left.

b. Single ball animation:
After calibrating the camera, we know the position of the table. Then, all of the ball animation was done by using physics simulation (active ball, passive table, gravity field, speed and etc). Initial position, speed, gravity field and bounciness were carefully adjusted, which is a very iterative process. Positions were specially designed, so that it looks like the paddle hits the ball accurately. Multiple lights (ambient and directional lights) and high resolution depth map shadow were used in order to create natural-looking shadow. It took us a LONG time¡

c. Multi-ball effect:
This effect was done by rendering three times of the following layers:
(1). Maya software renders the balls
(2). Maya hardware renders the particle effect: the balls' trails
(3). Maya depth map shadow

Due to the hurry in the last moment, we forgot to add the rendered shadow layer into the final composition... Another defect in this scene is that: we didn't have time to add content on the TV in the last moment.

d. Balls were bounced back:

Due to the hurry in the last moment, we forgot the rendered shadow again. It's a pity.

e. Reflection layer:
As stated in the shooting part.

f. Ripple effect:
It was also achieved, but we didn't put it into our video due to time constraint:

(2). Composition:

a. What is hard in composition?

Working in Adobe Premiere slowly became a very tedious task over time mostly because the project started to become too big and there were a lot of layers with different types of effects applied to them. What would've been better is to have cut the project into smaller projects comprised of parts of the movie. What we originally did was create multiple sequences in one project to mix together in the end, which ended up slowing down the program more than anything.

b. Chroma key:
We tried premiere Chromakey and AfterEffect color key, but they didn’t work very well. In the last moment, we used AfterEffect plug-in: Primatte. But we didn’t have enough time to redo every layer. For example, the near-end player layer still has quite visible artifacts.

c. TV layer:
We thought we had to model a TV and added it in the scene based on calibration result. But we eventually used a cheap method: use TV picture and deform it according to the perspective. Photoshop shadow was added to make it look natural:

d. TV effects:
Noise and white-black color were applied to achieve some other effects:

e. In-Out TV:
When we composite the CG ball into the scene, we changed the ball’s color when each time it goes into the TV. Compare the following two images:

f. Match:
Although we don’t have perfect choreography when we shot the footage, we carefully cut the footage and speeded up some parts to match the ball’s movement, but still make it look natural human motion.

g. Motion blur:
Motion blur is achieved by overlaying multiple layers of the same video sequence and make them differ a little bit in time. Then, assign them with reasonable transparency.

Contributions List

Christopher Octa - Adobe Premiere/Photoshop compositions, acting, making-of narration

Yu-Hsien Su - Maya compositions, cinematographer, making-of composition

Qiushuang Zhang - Maya compositions, acting, making-of composition

4. Reflection

In the world of movie special effects, there are a lot of different approaches to achieving the effects seen on television or in movies. Throughout the semester, we were introduced to these different approaches. For the most part, they were primarily techniques or methods that are done by artists or programmers. For example, using Maya to keyframe every single movement of a particular character like Davy Jones from Pirates of the Carribean 2 is an artistic method. Creating lines of code that will generate waves, mist, or bubbles like those seen in the movie Poseidon is a programming method. Both sides have their advantages and disadvantages. For artistry, the advantage usually includes things like full control and disadvantage would be the amount of time spent. The advantage of programming (mostly simulations) includes adopting a sort of hands-off approach when creating the actual scene itself and lets the simulations run the scene. The disadvantage of course would include the sense of not having full control of animations for cases where there may be an imperfection in motion or looks.

It becomes important that both sides are mixed in together to create a balanced and most realistic effect possible. Depending on the effect, a simulation or key-framed animation would be best. For our project, we learned a lot of how much time, effort, and the frustrations (slow computers, imperfect animations, re-rendering for lighting, etc.) effects artists go through in trying to create effects. In the end, creating special effects becomes an iterative loop of problem solving (how do we put a ball in the scene, how do we match the lighting, how do we make the television look like a television) until the goals are reached for that particular scene. Special effects is also a very time consuming job, as we learned first-hand.