Touch gestures on Windows Phone 7

Update: My gesture sample is also now available for download which shows an example of mixing awesome gestures with awesome cats. You can find that here: https://creators.xna.com/en-us/sample/gestures

In the Windows Phone Developer Tools Beta released today, there are a lot of little bug fixes, tweaks, and a few new things to play with. One of the biggest new features (in my opinion, anyway :)) is the inclusion of a high-level touch gesture system in the XNA Framework. These touch gestures allow you to program your games and applications using a built in gesture system, rather than reinventing the wheel on what it means to tap, drag, or flick. This should not only save you time as a developer, but ensure that all games on Windows Phone 7 have a consistent feel.

Here is a list of the gestures being provided along with brief descriptions of them:

GestureType	Description
Tap	A finger touches the screen and releases without much movement.
DoubleTap	Two taps in succession.
Hold	A finger touches the screen and holds it in place for a brief period of time.
FreeDrag	A finger touches the screen and moves in any direction.
VerticalDrag	A finger touches the screen and moves in an up/down direction.
HorizontalDrag	A finger touches the screen and moves in a left/right direction.
DragComplete	Marks the end of a FreeDrag, VerticalDrag, or HorizontalDrag gesture.
Flick	A finger drags across the screen and is lifted up without stopping.
Pinch	Two fingers press on the screen and move around.
PinchComplete	Marks the end of a Pinch gesture.

As you can see, it’s a good list of gestures that most games will find suitable for use. One thing to note as you are designing your game is that all gestures, except for pinch, are single touch gestures. This means if you press two fingers down and move them around, you will not see two drag gestures occurring; you will see a single pinch gesture (if you have those enabled) or you will see one drag gesture based on the average of the two finger positions.

So, now that you know about gestures, how do you use them? The first step for any game or application to use gestures is to enable the gestures you want to read. You must enabled the gestures before using any of the other gesture APIs or you’ll get an exception. Setting the gestures is as easy as setting a static property with a bitflag enumeration:

 TouchPanel.EnabledGestures =
    GestureType.Tap |
    GestureType.DoubleTap |
    GestureType.FreeDrag;

You can enable any or all of the gestures, however for the best results you should always enable only the gestures you need. Enabling extra gestures does affect the gesture engine’s logic and performance. If you need them all, by all means enable them all, but if you only need tap and vertical drag (for example, on your level select screen), you should only enable those two in order to get better accuracy and performance.

Once you have gestures enabled, you just need to read them each frame. Gestures are queued up behind the scenes which means that unlike the TouchPanel.GetState() API, you can see multiple gestures each frame. This gives a higher resolution for gestures as well as ensuring no gestures are missed. In your game’s update loop, you will want to check if there are available gestures and read them all. Failing to read them all can cause the queue to back up and may result in your app or game seeing a lot of old gestures at a later point.

To read the gestures, you can use a simple while loop:

 while (TouchPanel.IsGestureAvailable)
{
    GestureSample gesture = TouchPanel.ReadGesture();

    switch (gesture.GestureType)
    {
        // TODO: handle the gestures
    }
}

Now you have the gestures and know what they are, how do you get data? The GestureSample type exposes five properties (besides GestureType) to give you information about the gesture. These properties are Timestamp, Position, Position2, Delta, and Delta2. Timestamp is a TimeSpan that indicates when the gesture occurred and the other four are Vector2 values related to position and direction of the gestures. Note that Timestamp is not based on GameTime and is available primarily to enable games to figure out the relationship between gestures in the timeline of game events (i.e. you can see that a tap came in exactly 3.14 seconds after a flick by doing subtraction on the Timestamps of the gestures).

Each gesture only uses some of the Vector2 values as needed, leaving the rest set to the default (0, 0). For each gesture type, here is the breakdown of the data available to you:

GestureType	Valid Fields
Tap	Position
DoubleTap	Position
Hold	Position
FreeDrag	Position, Delta
VerticalDrag	Position, Delta
HorizontalDrag	Position, Delta
DragComplete	None
Flick	Delta
Pinch	Position, Position2, Delta, Delta 2
PinchComplete	None

Mostly what you would have expected. There a few notable points:

• Flick uses the Delta field to expose the flick speed. This speed is reported in pixels per second which makes it quite nice to use for flinging around your game objects or adding velocity to scrolling lists.
• VerticalDrag and HorizontalDrag give their actual position data (where the finger is on the screen), but their deltas are constrained to their respective axis. This means the Delta of a VerticalDrag will always have a 0 X component and a HorizontalDrag will always have a 0 Y component. This is very nice for things like menus where you only want to scroll on a certain axis because you don’t have to 0 out the axis yourself.
• Pinch reports back two fingers of data where Position/Delta map to one finger and Position2/Delta2 map to the other. This allows games to easily reconstruct the previous positions and handle the gesture appropriately, whether using the gesture to scale an object or rotate a camera.

So those are the new gestures in the Windows Phone Developer Tools Beta. If you have any questions, I recommend asking them over on the Creator’s Club forum, or feel free to leave them in the comments below.

Comments

• Anonymous
  July 12, 2010
  Thank you for the firsthand account! The funktions are very interesting for using it in a Game. What a pity, we can't test it on Hardware! Ganda

• Anonymous
  July 12, 2010
  How does the system respond to overlapping gestures, such as a drag and a tap?  Does the system ignore the tap, discard the drag in favour of the tap, or treat it as a new pinch gesture?

• Anonymous
  July 12, 2010
  Unfortunately the gestures only work on the phone and not in x86.  Which makes it difficult to support because an x86 version of your game is required to do performance or memory profiling.  I am currently writing a wrapper for these to simulate gesture support on x86 so we can still do performance and memory profiling.

• Anonymous
  July 12, 2010
  @Ganda While you can only test pinch on a multitouch monitor, all of these gestures work in the XDE so you can start building with them now. @Oggyb It depends on what you enable. If you have pinch enabled, you will likely see a pinch for the time when both fingers are on the screen and then a drag once the second finger is released. If you don't have pinch enabled, you will likely see the tap ignored in favor of the original drag contact.

• Anonymous
  July 12, 2010
  does the touch functionality work also on Windows platform? i know that there's a native support for touch in Windows 7, can we use it as well?

• Anonymous
  July 14, 2010
  The comment has been removed

• Anonymous
  July 14, 2010
  These are very useful indeed, and thanks for the informative article. I have one question: the Flick gesture has Delta information which is great, but it doesn't have any Position information. This means that to know where the object is being flicked from, I have to retrieve coordinate information from a previous update. Is there any reason why Flick couldn't include the start position in its (currently unused) Position property?

• Anonymous
  July 15, 2010
  Also is it possible to set the amount of time that must pass before the Hold gesture triggers? I would like to use this in my game but I want the Hold gesture to fire quite a lot more quickly than it is at the moment, can I configure this?

• Anonymous
  July 16, 2010
  Re Flick: We expose delta because the position isn't part of what flick really means. A flick really is just a velocity. If you want to scroll a list or change pages of your game, you just need to know which way and how fast. Position isn't as important there, so it's not included. The easiest thing to do is simply track drags (FreeDrag or one of the two axis-based ones) and use the position from that whenever you see a flick. Re Hold: No, the time is not configurable because it is based on the platform such that the time for a Hold gesture in an XNA Game Studio game is the same as the hold gesture for the rest of Windows Phone. If you want to do your own time, you'll have to implement your own code to use the raw touch points and handle your own hold gesture, however I'd recommend sticking with the platform provided gestures as it will make your application fit better with the rest of the platform.

• Anonymous
  August 21, 2010
  Nick, here is real world example where Position matters for Flick: In a shopping list application (such as HandyShopper, see my website), there can be a list of items.  Tapping a checkbox makes sense for checking off the item.  But Flicking the checkbox makes sense for deferring the item until a future shopping trip.  A right-pointing arrow can mean an item is deferred (I lifted this notation from time management tools).  So Flicking right, starting from the item's checkbox, makes sense as a gesture for deferring the item. The Flick gesture should really give enough positional information to optionally interpret it as a location-specific gesture, rather than an apositional full-screen gesture.

• Anonymous
  August 21, 2010
  Indeed there are times when position of a flick matters, however we expose both drags and raw touch data for you to build this functionality yourself. At the core, the Windows Phone platform defines flick simply as a direction and speed which is why our API exposes it as such.

• Anonymous
  August 28, 2010
  The comment has been removed

• Anonymous
  August 30, 2010
  Ok I finnaly found the solution and how to use raw touch thanks to that document : creators.xna.com/.../gestures

• Anonymous
  September 07, 2010
  Touch gestures are very important for a mobile software. Recently I found out an interesting thread about some kind of gesture framework : ProDragDrop framework: www.fluentcomponents.com/.../ProDragDrop They also have some interesting concept and very cool posts like this for the  iPhone panel for Wp7: blog.fluentcomponents.com/.../Building-an-animated-iPhone-like-panel-with-drag-and-drop-in-one-minute.aspx

• Anonymous
  November 06, 2010
  In a game I'm making, i will have two objects on the screen, I would like to simulatneosly freedrag each of these objects with two independent fingers, i.e. two players. Is there a basic sample somewhere showing this. Also, I'll be using silverlight primarily, but xna for touch. A sample, as simple as two boxes on the screen, independently draggable would be cool.

• Anonymous
  November 27, 2010
  How do we do if I want to drag an sprite and I want to know the initial position of the drag and its final position?

• Anonymous
  February 26, 2011
  Hi, I'm trying to create a small application for Windows Phone 7, that does the foloowing : detects whether there were 20 finger taps on the screen, within 5 seconds. Any ideas ? thanks for your help

• Anonymous
  May 31, 2011
  blogs.msdn.com/.../anonymous.gif

• Anonymous
  July 15, 2011
  Hi, I'm making a game for windows phone and i have a problem. i need to select a certain sprite from a collection of sprites on the screen then free drag it. any ideas please thx

• Anonymous
  October 20, 2011
  Hi does FreeDrag capture horizontal draging on the screen? I am trying to develop an app where i can freely drag elements on the screen.  But when i drag in horizontal direction (when Horizontal Drag disabled,  Free drag enabled) nothing happens. if i enable both.  once i drag something horizontally, and then(without removing finger from screen) in some other direction only horizontal drag happens. What is the solution for this..   pls help thnx in advance

• Anonymous
  April 30, 2012
  Hi, Is there any built-in way to detect a "circle shape gesture"?

• Anonymous
  December 22, 2012
  this doesn't build in VS2012 for the phone.