View this PageEdit this Page (locked)Attachments to this PageHistory of this PageHomeRecent ChangesSearch the SwikiHelp Guide

Group 4 Final Report

Process

Here is a descrpition of the steps taken to render the morph of a moving object, as seen in "The Getaway". View the associated "Making Of" segment prior to reading these steps. Some provided programs rely on LIBKINO and LIBDV, and static versions of these libs compiled for x86 are included. You need GTK for the "align" program.


Alignment

When doing even a simple morph between two static images, the objects of interest in each must be in approximately the same location, and have approximately the same orientation. Otherwise, as the morph progresses, the object will appear to shift or translate position instead of just its features transforming.

Uploaded Image: hondafront.jpg Uploaded Image: wrxfront.jpg

This problem is compounded when dealing with a video stream, since the objects must be lined up in each frame pair.

To address this problem, we first set some physical limitations to make things easier: The car must travel at a constant velocity, and it must be about the same velocity as the other car.

Under these conditions, footage of either car driving toward the camera can be aligned so that when both are played simultaneously the cars "line up". (Varying speed is pretty much impossible to reproduce manually [so we can't line the streams up] and adds little to the effect).

Use the align program to figure out the offset between aligned streams: align.tar.gz

Here is an example command line:
> ./align2
usage: ./align2 input_AVI start_A start_B [ output_AVI frames ]
> ./align2 ../../raw/cameraA_raw.avi 0 0"


The video should start playing like normal. It MUST be DV encoded to work. Use the yYuU keys to rewind through channel A at various frames. You should see one channel "seperate" from another. Use iIoO to fast forward through the same channel. hHjJkKlL is used for the second video channel.

Use the keys to "line up" the video.

When you quit you should see a message like this:
"QUIT. Final: a_frame = 7410 b_frame = 310 difference = -7100"

The two frame values tell you where either channel left off playing. This information can be used to extract two sequences that are "lined up".


We extracted two seperate video files from one using the information provided by the align program. The original video file is quite large, so we've provided the results instead. Experiment with the align program on your own video file.

Here are two AVI's with the Honda and Subaru driving towards the camera, with each approximately lined up at each frame: hondafront.avi, wrxfront.avi



Feature Selection

Here is our slightly modified version of the XMRM source: xmrm20-src.tar.gz. We got it to compile on a RH8.0 box.

This step is relatively straightforward. Using XMRM's vector tool highlight corresponding feature points between the FIRST and LAST frame pairs ONLY.

You will have to load a "destination" and a "source" image. The XMRM buttons pretty much explain themselves.

NOE: You will have to extract each frame pair from the two video streams into a unique directory. This is for later processing. Extract the first frame pair into 000/wrx.tif and 000/honda.tif, and so on.. You can do this manually with Premiere or write a program to do it for you.

Now, to the actual picking of the feature points...


For an object like a car, choose the obvious: the headlights, bumper, windshield, etc. Certain features, such as that for the Subaru's hood scoop, had to be created from "nowhere" on the other image. In the hood scoop case, a horizon reflection and other creases on the Honda's hood are chosen so that it appears to "grow" out of the hood.

Because of constraints of the point interpolation program, which we will get to in the next step, you MUST select the vectors in the same order in the first project as you did in the second one. This is because the interpolation program assumes the first line specified in the initial vector file corresponds to the first line specified in the final vector file.

A way to do the former is to write the points and their directions down as you go, or have two people do it simultaneously, picking points while sitting next to each other.

Uploaded Image: fm000.jpg Uploaded Image: fm006.jpg
Uploaded Image: wrxend2.jpg Uploaded Image: hondaend2.jpg Uploaded Image: demoshot.jpg

Use the "CALCULATE" tool to generate 1 frame of the middle most point of the transition to get a preview of the morph with your points.


Point Interpolation

Now we will run the "intp" program and generate the vector files for the intermediate frame pairs.

Here is the source to the interpolate program: intp.tar.gz

The command line should look something like:
> ./intp
> usage: ./intp initial_vec final_vec output_name steps
> ./intp 000/output.prj.vec 030/output.prj.vec output.prj.vec 30

Make sure your project is named "output.prj" (and consequently, the associated vector file "output.prj.vec") . The program will interpolate the intermediate points over the specified number of iterations, generating the vector files in "0XX/output.prj.vec".


Render

We can now finally run the modified "XMRM" program. A shell script will run XMRM on each project, and XMRM will automatically render the image and save it to "output.000.tif" in that frame's directory.

Here are the scripts: scripts.tar.gz

You must first generate the project file's for each "0XX/" directory as well. Look at "generate_project_files". Change TOTAL_STEPS to the appropriate number. Get your project file ("output.prj") and split it into two parts, just as "output.prj-1" and "output.prj-2" are split up.

Finally, execute "run_for_all", make sure it points to the correct location of XMRM.

After rendering each TIFF frame, you will have to combine them into some video format. You can use Premiere to do this or write a program to do it yourself.


Insert in between practical video

After producing the short effect video, you will need to insert either car immediately before, and the other immediately after the effect so the entire sequence will be complete. Simply use Premiere to do this.

Here is the final output: final.avi


Problems

At first we tried rendering the morph sequences having only specified the feature points of the object of interest (the car). This leaves the background vulnerable to undesired warping. As a fix, you can either background subtract, render the morph, then composite it back over the original background, OR specify unchanging feature points between frames. We chose the later, easier solution. It also ends up giving a sort of neat effect where warping occurs slightly in the environment, contributing to the sense that some kind of force field is causing the transformation.





Link to this Page