Gesture Recognition in Flash

I was recently presented with an opportunity to experiment with a motion sensitive kiosk. It immediately got me thinking about how I could implement gesture recognition in Flash. While gestures might not be the most accurate or quickest way to input into a device, they can certainly be a lot of fun.

My earliest experiences with gesture recognition were with the Palm OS’s graffiti as well as in the Peter Molyneux designed game Black and White (which earns the distinction of being the most innovative game I ever hated).

More recently Nintendo has released a few interesting games for it’s touch screen enabled DS handheld which take advantage of gesture recognition. Specifically the games Trauma Center and Pac Pix take very innovative approaches to gesture recognition. Pac Pix literally brings your gestures to life, and is a fantastic example of some of the creative things you can do with gesture recognition.

Gesture recognition in flash is territory that has already been explored by famed Flash developer Grant Skinner. However, the resource which I found most valuable in my research was the Gamedev.net article Recognition of Handwritten Gestures by Oleg Dopertchouk. Mr. Dopertchouk provides a step by step tutorial and example source code for implementing gesture recognition. It’s a fantastic resource, and I highly recommend it as a starting point to understand the techniques involved in gesture recognition.

I’ve taken the lessons learned from Mr. Dopertchouk’s article and built out a gesture recognition Actionscript library which I plan to use in future projects. I’ve released the source code packaged in a zip file below, which can be downloaded and used in your own projects. You can also see an example of the gesture recognition in action below. Unfortunately, the library of gestures it is currently trained on is recorded from my crude handwriting while using a laptop trackpad. It’s not perfect, but with a little training on neater handwriting, or by using a simplified stroke such as is used in the Palm OS’s graffiti, I’m sure it’s accuracy could be vastly improved.