Other EM and GMM Project Ideas

• Finding moving objects when the camera is also moving is a hard problem. One approach is to segment the image into regions have different parametric flow fields. For example, I have a dataset taken with a highly zoomed-in aerial spotter lens of a car driving on a runway, and have computed sparse optic flow by tracking corners through subsets of the image sequence (CSE586emflow.zip). To a first approximation, both the car and the background move with (different) translational motions. Therefore, one could presumably use EM to estimate two Gaussian distributions in flow dx, dy space, with perhaps a uniform outlier distribution to handle bad flow vectors. A sample is shown below, with two classes shown in green and red, and outliers shown in blue. Try this for several of the images in this sequence. Discuss the feasibility of using this approach for tracking. What happens when the car is not visible in the image? Also, consider implementing an extension where you estimate Gaussians in 4D (x,y,dx,dy) space, and see how well this allows you to "segment" the position and shape of the car as an ellipse in the image.



• Jones and Rehg develop two Gaussian mixture models (GMM) representing color of skin and not-skin in "Statistical Color Models with Application to Skin Detection." The MATLAB routine jrmogskin.m implements a likelihood ratio classifier using these color models to label pixels in an RGB color image as being skin vs nonskin. However, these models are quite generic, since they were trained using many different images from the web under arbitrary illumination conditions. Implement an extension that uses EM to learn more specific skin and background color distributions for a particular set of images taken of a single person under some constant illumination. I don't have a ready source of data for this one, but image frames from a movie with a person in it would be a start. One approach to do this is to use EM to learn a new distribution, but using Rehg and Jones GMM color models as initial estimates of mixture weights, means and covariances. Compare labeling results from the original vs revised color models, hopefully showing improvement due to greater specificity of the model (but it's OK if it doesn't work, either, since I am interested in seeing you implement EM for GMM).