The Problem with Fingers

The Kinect has really been a revolutionary tool for both developers and independent researchers. However, that is not to say that the Kinect is not limited. In fact, the Kinect is very limited by its low resolution. Tracking body and arm gestures have been no sweat, but when it comes to hand, and finger tracking, the difficulty just seems to ramp up. This is because the hand is a significantly smaller object. Viewing the hand through the Kinect ends with a very noisy result. It has been the goal for researchers some time to get accurate hand tracking through inexpensive devices, such as the Kinect. Part of the problem is different lighting can cause different readings. Other issues include a noisy or busy background, and so the challenge becomes how do you differentiate between the background and the hand you are trying to read.

The researchers at Nanyang Technology University suggest using a technique called EMD (Earth Movers Distance). EMD is basically the difference between two probability distributions. The dumbed down idea of EMD is to find the pattern the reading is most like and lob it in that category. This is actually a very similar technique to Information Retrieval's Classification problem. You would use an algorithm such as K-Nearest Neighbors to determine which pattern (or in this case gesture) is closest to the given input, rather than trying to rely on an exact matching. It never occurred to me that IR might be helpful in my Kinect project, but hopefully I'll be able to keep my eyes open.

Kao Pyro of the Azure Flame

Source:
Ren, Z., Yuan, J., & Zhang, Z. (2011). Robust hand gesture recognition based on finger-earth mover's distance with a commodity depth camera. MM '11 Proceedings of the 19th ACM international conference on Multimedia , 1093-1096.