Tutorial: add move-look controls to your DirectX game
[This article is for Windows 8.x and Windows Phone 8.x developers writing Windows Runtime apps. If you’re developing for Windows 10, see the latest documentation]
Learn how to add traditional mouse and keyboard move-look controls (also known as mouselook controls) to your DirectX game.
We also discuss move-look support for touch devices, with the move controller defined as the lower-left section of the screen that behaves like a directional input, and the look controller defined for the remained of the screen, with the camera centering on the last place the player touched in that area.
If this is an unfamiliar control concept to you, think of it this way: the keyboard (or the touch-based directional input box) controls your legs in this 3D space, and behaves as if your legs were only capable of moving forward or backward, or strafing left and right. The mouse (or touch pointer) controls your head. You use your head to look in a direction -- left or right, up or down, or somewhere in that plane. If there is a target in your view, you would use the mouse to center your camera view on that target, and then press the forward key to move towards it, or back to move away from it. To circle the target, you would keep the camera view centered on the target, and move left or right at the same time. You can see how this is a very effective control method for navigating 3D environments!
These controls are commonly known as WASD controls in gaming, where the W, A, S, and D keys are used for x-z plane fixed camera movement, and the mouse is used to control camera rotation around the x and y axes.
Objectives
- Add basic move-look controls to your DirectX game for both mouse and keyboard, and touch screens.
- Implement a first-person camera used to navigate a 3D environment.
A note on touch control implementations
For touch controls, we implement two controllers: the move controller, which handles movement in the x-z plane relative to the camera's look point; and the look controller, which aims the camera's look point. Our move controller maps to the keyboard WASD buttons, and the look controller maps to the mouse. But for touch controls, we need to define a region of the screen that serves as the directional inputs, or the virtual WASD buttons, with the remainder of the screen serving as the input space for the look controls.
Our screen looks like this.
.png)
When you move the touch pointer (not the mouse!) in the lower left of the screen, any movement upwards will make the camera move forward. Any movement downwards will make the camera move backwards. The same holds for left and right movement inside the move controller's pointer space. Outside of that space, and it becomes a look controller -- you just touch or drag the camera to where you'd like it to face.
Set up the basic input event infrastructure
First, we must create our control class that we use to handle input events from the mouse and keyboard, and update the camera perspective based on that input. Because we're implementing move-look controls, we call it MoveLookController.
using namespace Windows::UI::Core;
using namespace Windows::System;
using namespace Windows::Foundation;
using namespace Windows::Devices::Input;
// Methods to get input from the UI pointers
ref class MoveLookController
{
}; // class MoveLookController
Now, let's create a header that defines the state of the move-look controller and its first-person camera, plus the basic methods and event handlers that implement the controls and the update the state of the camera.
#define ROTATION_GAIN 0.004f // sensitivity adjustment for look controller
#define MOVEMENT_GAIN 0.1f // sensitivity adjustment for move controller
ref class MoveLookController
{
private:
// properties of the controller object
float3 m_position; // the position of the controller
float m_pitch, m_yaw; // orientation euler angles in radians
// properties of the Move control
bool m_moveInUse; // specifies whether the move control is in use
uint32 m_movePointerID; // id of the pointer in this control
float2 m_moveFirstDown; // point where initial contact occurred
float2 m_movePointerPosition; // point where the move pointer is currently located
float3 m_moveCommand; // the net command from the move control
// properties of the Look control
bool m_lookInUse; // specifies whether the look control is in use
uint32 m_lookPointerID; // id of the pointer in this control
float2 m_lookLastPoint; // last point (from last frame)
float2 m_lookLastDelta; // for smoothing
bool m_forward, m_back; // states for movement
bool m_left, m_right;
bool m_up, m_down;
public:
// Methods to get input from the UI pointers
void OnPointerPressed(
_In_ Windows::UI::Core::CoreWindow^ sender,
_In_ Windows::UI::Core::PointerEventArgs^ args
);
void OnPointerMoved(
_In_ Windows::UI::Core::CoreWindow^ sender,
_In_ Windows::UI::Core::PointerEventArgs^ args
);
void OnPointerReleased(
_In_ Windows::UI::Core::CoreWindow^ sender,
_In_ Windows::UI::Core::PointerEventArgs^ args
);
void OnKeyDown(
_In_ Windows::UI::Core::CoreWindow^ sender,
_In_ Windows::UI::Core::KeyEventArgs^ args
);
void OnKeyUp(
_In_ Windows::UI::Core::CoreWindow^ sender,
_In_ Windows::UI::Core::KeyEventArgs^ args
);
// set up the Controls that this controller supports
void Initialize( _In_ Windows::UI::Core::CoreWindow^ window );
// accessor to set position of controller
void SetPosition( _In_ float3 pos );
// accessor to set position of controller
void SetOrientation( _In_ float pitch, _In_ float yaw );
// returns the position of the controller object
float3 get_Position();
// returns the point which the controller is facing
float3 get_LookPoint();
void Update( Windows::UI::Core::CoreWindow ^window );
}; // class MoveLookController
Our code contains 4 groups of private fields. Let's review the purpose of each one.
First, we define some useful fields that hold our updated info about our camera view.
- m_position is the position of the camera (and therefore the viewplane) in the 3D scene, using scene coordinates.
- m_pitch is the pitch of the camera, or its up-down rotation around the viewplane's x-axis, in radians.
- m_yaw is the yaw of the camera, or its left-right rotation around the viewplane's y-axis, in radians.
Now, let's define the fields that we use to store info about the status and position of our controllers. First, we'll define the fields we need for our touch-based move controller. (There's nothing special needed for the keyboard implementation of the move controller. We just read keyboard events with specific handlers.)
- m_moveInUse indicates whether the move controller is in use.
- m_movePointerID is the unique ID for the current move pointer. We use it to differentiate between the look pointer and the move pointer when we check the pointer ID value.
- m_moveFirstDown is the point on the screen where the player first touched the move controller pointer area . We use this value later to set a dead zone to keep tiny movements from jittering the view.
- m_movePointerPosition is the point on the screen the player has currently moved the pointer to. We use it to determine what direction the player wanted to move by examining it relative to m_moveFirstDown.
- m_moveCommand is the final computed command for the move controller: up (forward), down (back), left, or right.
Now, we define the fields we use for our look controller,- both the mouse and touch implementations.
- m_lookInUse indicates whether the look control is in use.
- m_lookPointerID is the unique ID for the current look pointer. We use it to differentiate between the look pointer and the move pointer when we check the pointer ID value.
- m_lookLastPoint is the last point, in scene coordinates, that was captured in the previous frame.
- m_lookLastDelta is the computed difference between the current m_positionand m_lookLastPoint.
Finally, we define 6 Boolean values for the 6 degrees of movement, which we use to indicate the current state of each directional move action (on or off):
- m_forward, m_back, m_left, m_right, m_up and m_down.
We use the 6 event handlers to capture the input data we use to update the state of our controllers:
- OnPointerPressed. The player pressed the left mouse button with the pointer in our game screen, or touched the screen.
- OnPointerMoved. The player moved the mouse with the pointer in our game screen, or dragged the touch pointer on the screen.
- OnPointerReleased. The player released the left mouse button with the pointer in our game screen, or stopped touching the screen.
- OnKeyDown. The player pressed a key.
- OnKeyUp. The player released a key.
And finally, we use these methods and properties to initialize, access, and update the controllers' state info.
- Initialize. Our app calls this event handler to initialize the controls and attach them to the CoreWindow object that describes our display window.
- SetPosition. Our app calls this method to set the (x, y, and z) coordinates of our controls in the scene space.
- SetOrientationOur app calls this method to set the pitch and yaw of the camera.
- get_Position. Our app accesses this property to get the current position of the camera in the scene space. You use this property as the method of communicating the current camera position to the app.
- get_LookPoint. Our app accesses this property to get the current point toward which the controller camera is facing.
- Update. Reads the state of the move and look controllers and updates the camera position. You continually call this method from the app's main loop to refresh the camera controller data and the camera position in the scene space.
Now you have here all the components you need to implement your move-look controls.
So, let's connect these pieces together.
Create the basic input events
The Windows Runtime event dispatcher provides 5 events we want instances of the MoveLookController class to handle:
These events are implemented on the CoreWindow type. We assume that you have a CoreWindow object to work with. If you don't know how to obtain one, see How to set up your Windows Store C++ app to display a DirectX view.
As these events fire while our app is running, the handlers update the controllers' state info defined in our private fields.
First, let's populate the mouse and touch pointer event handlers. In the first event handler, OnPointerPressed(), we get the x-y coordinates of the pointer from the CoreWindow that manages our display when the user clicks the mouse or touches the screen in the look controller region.
OnPointerPressed
void MoveLookController::OnPointerPressed(
_In_ CoreWindow^ sender,
_In_ PointerEventArgs^ args)
{
// get the current pointer position
uint32 pointerID = args->CurrentPoint->PointerId;
float2 position = float2( args->CurrentPoint->Position.X, args->CurrentPoint->Position.Y );
auto device = args->CurrentPoint->PointerDevice;
auto deviceType = device->PointerDeviceType;
if ( deviceType == PointerDeviceType::Mouse )
{
// Action, Jump, or Fire
}
// check if this pointer is in the move control
// Change the values to percentages of the preferred screen resolution.
// You can set the x value to <preferred resolution> * <percentage of width>
// for example, ( position.x < (screenResolution.x * 0.15) )
if (( position.x < 300 && position.y > 380 ) && ( deviceType != PointerDeviceType::Mouse ))
{
if ( !m_moveInUse ) // if no pointer is in this control yet
{
// process a DPad touch down event
m_moveFirstDown = position; // save location of initial contact
m_movePointerPosition = position;
m_movePointerID = pointerID; // store the id of pointer using this control
m_moveInUse = TRUE;
}
}
else // this pointer must be in the look control
{
if ( !m_lookInUse ) // if no pointer is in this control yet
{
m_lookLastPoint = position; // save point for later move
m_lookPointerID = args->CurrentPoint->PointerId; // store the id of pointer using this control
m_lookLastDelta.x = m_lookLastDelta.y = 0; // these are for smoothing
m_lookInUse = TRUE;
}
}
}
This event handler checks whether the pointer is not the mouse (for the purposes of this sample, which supports both mouse and touch) and if it is in the move controller area. If both criteria are true, it checks whether the pointer was just pressed, specifically, is this click unrelated to a previous move or look input, by testing if m_moveInUse is false. If so, the handler captures the point in the move controller area where the press happened and sets m_moveInUse to true so when this handler is called again it won't overwrite the start position of the move controller input interaction. It also updates the move controller pointer ID to the current pointer's ID.
If the pointer is the mouse or if the touch pointer isn't in the move controller area, it must be in the look controller area. It sets m_lookLastPoint to the current position where the user pressed the mouse button or touched and pressed , resets the delta, and updates the look controller's pointer ID to the current pointer ID. It also lets sets the state of the look controller to active.
OnPointerMoved
void MoveLookController::OnPointerMoved(
_In_ CoreWindow ^sender,
_In_ PointerEventArgs ^args)
{
uint32 pointerID = args->CurrentPoint->PointerId;
float2 position = float2( args->CurrentPoint->Position.X, args->CurrentPoint->Position.Y );
// decide which control this pointer is operating
if ( pointerID == m_movePointerID ) // this is the move pointer
{
// Move control
m_movePointerPosition = position; // save current position
}
else if (pointerID == m_lookPointerID ) // this is the look pointer
{
// Look control
float2 pointerDelta;
pointerDelta = position - m_lookLastPoint; // how far did pointer move
float2 rotationDelta;
rotationDelta = pointerDelta * ROTATION_GAIN; // scale for control sensitivity
m_lookLastPoint = position; // save for next time through
// update our orientation based on the command
m_pitch -= rotationDelta.y; // mouse y increases down, but pitch increases up
m_yaw -= rotationDelta.x; // yaw defined as CCW around y-axis
// Limit pitch to straight up or straight down
m_pitch = (float) __max( -M_PI/2.0f, m_pitch );
m_pitch = (float) __min( +M_PI/2.0f, m_pitch );
}
}
The OnPointerMoved event handler fires whenever the pointer moves (in this case, if a touch screen pointer is being dragged, or if the mouse pointer is being moved while the left button is pressed). If the pointer ID is the same as the move controller pointer's ID, then it's the move pointer; otherwise, we check if it's the look controller that's the active pointer.
If it's the move controller, we just update the pointer position. We keep updating it as long the PointerMoved event keeps firing, because we want to compare the final position with the first one we captured with the OnPointerPressed event handler.
If it's the look controller, things are a little more complicated. We need to calculate a new look point and center the camera on it, so we calculate the delta between the last look point and the current screen position, and then we multiply versus our scale factor, which we can tweak to make the look movements smaller or larger relative to the distance of the screen movement. Using that value, we calculate the pitch and the yaw.
Finally, we need to deactivate the move or look controller behaviors when the player stops moving the mouse or touching the screen. We use OnPointerReleased, which we call when PointerReleased is fired, to set m_moveInUse or m_lookInUse to FALSE and turn off the camera pan movement, and to zero out the pointer ID.
OnPointerReleased
void MoveLookController::OnPointerReleased(
_In_ CoreWindow ^sender,
_In_ PointerEventArgs ^args)
{
uint32 pointerID = args->CurrentPoint->PointerId;
float2 position = float2( args->CurrentPoint->Position.X, args->CurrentPoint->Position.Y );
if ( pointerID == m_movePointerID ) // this was the move pointer
{
m_moveInUse = FALSE;
m_movePointerID = 0;
}
else if (pointerID == m_lookPointerID ) // this was the look pointer
{
m_lookInUse = FALSE;
m_lookPointerID = 0;
}
}
So far we handled all the touch screen events. Now, let's handle the key input events for a keyboard-based move controller.
OnKeyDown
void MoveLookController::OnKeyDown(
__in CoreWindow^ sender,
__in KeyEventArgs^ args )
{
Windows::System::VirtualKey Key;
Key = args->VirtualKey;
// figure out the command from the keyboard
if ( Key == VirtualKey::W ) // forward
m_forward = true;
if ( Key == VirtualKey::S ) // back
m_back = true;
if ( Key == VirtualKey::A ) // left
m_left = true;
if ( Key == VirtualKey::D ) // right
m_right = true;
}
As long as one of these keys is pressed, this event handler sets the corresponding directional move state to true.
OnKeyUp
void MoveLookController::OnKeyUp(
__in CoreWindow^ sender,
__in KeyEventArgs^ args)
{
Windows::System::VirtualKey Key;
Key = args->VirtualKey;
// figure out the command from the keyboard
if ( Key == VirtualKey::W ) // forward
m_forward = false;
if ( Key == VirtualKey::S ) // back
m_back = false;
if ( Key == VirtualKey::A ) // left
m_left = false;
if ( Key == VirtualKey::D ) // right
m_right = false;
}
And when the key is released, this event handler sets it back to false. When we call Update, it checks these directional move states, and move the camera accordingly. This is a bit simpler than the touch implementation!
Initialize the touch controls and the controller state
Let's hook up the events now, and initialize all the controller state fields.
Initialize
void MoveLookController::Initialize( _In_ CoreWindow^ window )
{
// opt in to recieve touch/mouse events
window->PointerPressed +=
ref new TypedEventHandler<CoreWindow^, PointerEventArgs^>(this, &MoveLookController::OnPointerPressed);
window->PointerMoved +=
ref new TypedEventHandler<CoreWindow^, PointerEventArgs^>(this, &MoveLookController::OnPointerMoved);
window->PointerReleased +=
ref new TypedEventHandler<CoreWindow^, PointerEventArgs^>(this, &MoveLookController::OnPointerReleased);
window->CharacterReceived +=
ref new TypedEventHandler<CoreWindow^, CharacterReceivedEventArgs^>(this, &MoveLookController::OnCharacterReceived);
window->KeyDown +=
ref new TypedEventHandler<CoreWindow^, KeyEventArgs^>(this, &MoveLookController::OnKeyDown);
window->KeyUp +=
ref new TypedEventHandler<CoreWindow^, KeyEventArgs^>(this, &MoveLookController::OnKeyUp);
// Initialize state of the controller
m_moveInUse = FALSE; // no pointer is in the Move control
m_movePointerID = 0;
m_lookInUse = FALSE; // no pointer is in the Look control
m_lookPointerID = 0;
// need to init this as it is reset every frame
m_moveCommand = float3( 0.0f, 0.0f, 0.0f );
SetOrientation( 0, 0 ); // look straight ahead when the app starts
}
Initialize takes a reference to the app's CoreWindow instance as a parameter and registers the event handlers we developed to the appropriate events on that CoreWindow. It initializes the move and look pointer's IDs, sets the command vector for our touch screen move controller implementation to zero, and sets the camera looking straight ahead when the app starts.
Getting and setting the position and orientation of the camera
Let's define some methods to get and set the position of the camera with respect to the viewport.
void MoveLookController::SetPosition( _In_ float3 pos )
{
m_position = pos;
}
// accessor to set position of controller
void MoveLookController::SetOrientation( _In_ float pitch, _In_ float yaw )
{
m_pitch = pitch;
m_yaw = yaw;
}
// returns the position of the controller object
float3 MoveLookController::get_Position()
{
return m_position;
}
// returns the point at which the camera controller is facing
float3 MoveLookController::get_LookPoint()
{
float y = sinf( m_pitch ); // vertical
float r = cosf( m_pitch ); // in the plane
float z = r*cosf( m_yaw ); // fwd-back
float x = r*sinf( m_yaw ); // left-right
return m_position + float3( x, y, z );
}
Updating the controller state info
Now, we perform our calculations that convert the pointer coordinate info tracked in m_movePointerPosition into new coordinate information respective of our world coordinate system. Our app calls this method every time we refresh the main app loop. So it is here that we compute the new look point position info we want to pass to the app for updating the view matrix before projection into the viewport.
void MoveLookController::Update( CoreWindow ^window )
{
// check for input from the Move control
if ( m_moveInUse )
{
float2 pointerDelta = m_movePointerPosition - m_moveFirstDown;
// figure out the command from the touch-based virtual joystick
if ( pointerDelta.x > 16.0f ) // leave 32 pixel-wide dead spot for being still
m_moveCommand.x = 1.0f;
else
if ( pointerDelta.x < -16.0f )
m_moveCommand.x = -1.0f;
if ( pointerDelta.y > 16.0f ) // joystick y is up so change sign
m_moveCommand.y = -1.0f;
else
if (pointerDelta.y < -16.0f )
m_moveCommand.y = 1.0f;
}
// poll our state bits set by the keyboard input events
if ( m_forward )
m_moveCommand.y += 1.0f;
if ( m_back )
m_moveCommand.y -= 1.0f;
if ( m_left )
m_moveCommand.x -= 1.0f;
if ( m_right )
m_moveCommand.x += 1.0f;
if ( m_up )
m_moveCommand.z += 1.0f;
if ( m_down )
m_moveCommand.z -= 1.0f;
// make sure that 45 degree cases are not faster
float3 command = m_moveCommand;
if ( fabsf(command.x) > 0.1f || fabsf(command.y) > 0.1f || fabsf(command.z) > 0.1f )
command = normalize( command );
// rotate command to align with our direction (world coordinates)
float3 wCommand;
wCommand.x = command.x*cosf( m_yaw ) - command.y*sinf( m_yaw );
wCommand.y = command.x*sinf( m_yaw ) + command.y*cosf( m_yaw );
wCommand.z = command.z;
// scale for sensitivity adjustment
wCommand = wCommand*MOVEMENT_GAIN;
// our velocity is based on the command,
// also note that y is the up-down axis
float3 Velocity;
Velocity.x = -wCommand.x;
Velocity.z = wCommand.y;
Velocity.y = wCommand.z;
// integrate
m_position = m_position + Velocity;
// clear movement input accumulator for use during next frame
m_moveCommand = float3( 0.0f, 0.0f, 0.0f );
}
Because we don't want jittery movement when the player uses our touch-based move controller, we set a virtual dead zone around the pointer with a diameter of 32 pixels. We also add velocity which is the command value plus a movement gain rate. (You can adjust this behavior to your liking, to slow down or speed up the rate of movement based on the distance the pointer moves in the move controller area.)
When we compute the velocity, we also translate the coordinates received from the move and look controllers into the movement of the actual look point we send to the method that computes our view matrix for the scene. First, we invert the x coordinate, because if we click-move or drag left or right with the look controller, the look point rotate in the opposite direction in the scene, as a camera might swing about its central axis. Then, we swap the y and z axes, because an up/down key press or touch drag motion (read as a y-axis behavior) on the move controller should translate into a camera action that moves the look point into or out of the screen (the z-axis).
The final position of the look point for the player is the last position plus the calculated velocity, and this is what is read by the renderer when it calls the get_Position method (most likely during the setup for each frame). After that, we reset the move command to zero.
Updating the view matrix with the new camera position
We can obtain a scene space coordinate that our camera is focused on, and which is updated whenever you tell your app to do so (every 60 seconds in the main app loop, for example). This pseudocode suggests the calling behavior you can implement:
myMoveLookController->Update( m_window );
// update the view matrix based on the camera position
myFirstPersonCamera->SetViewParameters(
myMoveLookController->get_Position(), // point we are at
myMoveLookController->get_LookPoint(), // point to look towards
float3( 0, 1, 0 ) // up-vector
);
Congratulations! You've implemented basic move-look controls for both touch screens and keyboard/mouse input touch controls in Windows Store C++!