G4-Update4

Input

• Video of a football game with a sufficient viewpoint of the field and players
• User input indicating which players are whom and who should be tracked

Layers

• The input football video
• The CG indicators that track the player and display additional information (ie. name)

Pipeline

1. Raw video from DVDs of Georgia Tech football games.
2. Convert raw video to format suitable for OpenCV
3. Perform segmentation on video to identify different components
4. Camera calibration
5. Player tracking
6. Visualization of tracked players

Progress

• Obtained football video in raw form
• Converted some video to format for OpenCV
• Examined samples of OpenCV tracking and drawing code
• Applied tracking sample programs to football footage
• Modified tracking code to display text under tracked objects
• Modified tracking code to track multiple objects at once

Camera tracking

Done

• (old) Tracking works in general setting
• (old) Has problems with horizontal lines leaving and entering the camera view
• system for manual corrections to alignment
• created tracking for one sequence from the side
• end view works not very well (not enough features), de-interlacing is bad for line segmentation

To Do

• Improve stability in border cases
• Filter results to remove little hic ups and smooth the results
• write out tracking over time to be played back later

Segmentation

Done

• Iterate over pixels and throw out those belonging to the field and stadium/crowd
• Allow user to pick players to get better idea of color of jerseys and throw out those pixels
• Allow control of various parameters (blob size, color differences) to improve segmentation dynamically
• Ability to let tracker know whether a particular point belongs to a probable player. If it doesn't, finds the closest point of a player.
• Use of pyramid segmentation to possibly improve results over regular method (Abandoned as it's unreliable)

To Do

• Look into segmentation via HSI
• Work on visualization for top-down view tracking of players

Tracking

Done

• Researched various tracking algorithms
• Tested several blob tracking samples in openCV with our football footage
• Tested CAMSHIFT tracking algorithm on football footage, modified to display text information for tracked objects and to track multiple user-selected objects
• Abandoned CAMSHIFT and moved to the Lucas-Kanade point-based optical flow tracking. This is giving us better results and allows us to refine the tracking more by manipulating the individual points if they move off a player.
• We use the information provided by segmentation of the frame to determine where a given track point should be. If it is no longer tracking a player, then we move it to the nearest player, which should be the one it was on previously, because players can't move too far from one frame to the next. For example, in the two images below, the player moves behind the goal post and some of the track points get left behind. Using the segmentation data, we should be able to automatically detect that those points are not on a player, so they should move to the closest player available, which would be the correct result.
• We have started looking at the direction of movement of track points to determine whether they have moved to a different player or object

To Do

• Focus on groups of players instead of individuals, especially groups that tend to stick to themselves. For instance, it should be fairly simple to track a wide receiver-cornerback pair as they run down the field. This could be useful when determining what caused a receiver to run his route incorrectly, for example.
• Related to the above, single players may sometimes cross paths. We should be able to detect this and continue to track the appropriate player(s).
• Modify sample tracking algorithms to improve performance.

Timeline (completed steps/next steps)

Links to this Page

• DVFX 2007 Groups last edited on 2 May 2007 at 12:51 am by coppola-win.cc.gatech.edu
• G4-Final Report last edited on 1 May 2007 at 12:26 am by c-67-191-163-57.hsd1.ga.comcast.net