Handle device orientation with MediaCapture
When your app captures a photo or video that is intended to be viewed outside of your app, such as saving to a file on the user's device or sharing online, it's important that you encode the image with the proper orientation metadata so that when another app or device displays the image, it is oriented correctly. Determining the correct orientation data to include in a media file can be a complex task because there are several variables to consider, including the orientation of the device chassis, the orientation of the display, and the placement of the camera on the chassis (whether it is a front or back-facing camera).
To simplify the process of handling orientation, we recommend using a helper class, CameraRotationHelper, for which the full definition is provided at the end of this article. You can add this class to your project and then follow the steps in this article to add orientation support to your camera app. The helper class also makes it easier for you to rotate the controls in your camera UI so that they are rendered correctly from the user's point of view.
Note
This article builds on the code and concepts discussed in the article Basic photo, video, and audio capture with MediaCapture. We recommend that you familiarize yourself with the basic concepts of using the MediaCapture class before adding orientation support to your app.
Namespaces used in this article
The example code in this article uses APIs from the following namespaces that you should include in your code.
using Windows.Devices.Enumeration;
using Windows.UI.Core;
The first step in adding orientation support to your app is to lock the display so that it doesn't automatically rotate when the device is rotated. Automatic UI rotation works well for most types of apps, but it is unintuitive for users when the camera preview rotates. Lock the display orientation by setting the DisplayInformation.AutoRotationPreferences property to DisplayOrientations.Landscape.
DisplayInformation.AutoRotationPreferences = DisplayOrientations.Landscape;
Tracking the camera device location
To calculate the correct orientation for captured media, you app must determine the location of the camera device on the chassis. Add a boolean member variable to track whether the camera is external to the device, such as a USB web cam. Add another boolean variable to track whether the preview should be mirrored, which is the case if a front-facing camera is used. Also, add a variable for storing a DeviceInformation object that represents the selected camera.
private bool _externalCamera;
private bool _mirroringPreview;
DeviceInformation _cameraDevice;
Select a camera device and initialize the MediaCapture object
The article Basic photo, video, and audio capture with MediaCapture shows you how to initialize the MediaCapture object with just a couple of lines of code. To support camera orientation, we will add a few more steps to the initialization process.
First, call DeviceInformation.FindAllAsync passing in the device selector DeviceClass.VideoCapture to get a list of all available video capture devices. Next, select the first device in the list where the panel location of the camera is known and where it matches the supplied value, which in this example is a front-facing camera. If no camera is found on the desired panel, the first or default available camera is used.
If a camera device is found, a new MediaCaptureInitializationSettings object is created and the VideoDeviceId property is set to the selected device. Next, create the MediaCapture object and call InitializeAsync, passing in the settings object to tell the system to use the selected camera.
Finally, check to see if the selected device panel is null or unknown. If so, the camera is external, which means that its rotation is unrelated to the rotation of the device. If the panel is known and is on the front of the device chassis, we know the preview should be mirrored, so the variable tracking this is set.
var allVideoDevices = await DeviceInformation.FindAllAsync(DeviceClass.VideoCapture);
DeviceInformation desiredDevice = allVideoDevices.FirstOrDefault(x => x.EnclosureLocation != null
&& x.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Front);
_cameraDevice = desiredDevice ?? allVideoDevices.FirstOrDefault();
if (_cameraDevice == null)
{
System.Diagnostics.Debug.WriteLine("No camera device found!");
return;
}
var settings = new MediaCaptureInitializationSettings { VideoDeviceId = _cameraDevice.Id };
mediaCapture = new MediaCapture();
mediaCapture.RecordLimitationExceeded += MediaCapture_RecordLimitationExceeded;
mediaCapture.Failed += MediaCapture_Failed;
try
{
await mediaCapture.InitializeAsync(settings);
}
catch (UnauthorizedAccessException)
{
System.Diagnostics.Debug.WriteLine("The app was denied access to the camera");
return;
}
// Handle camera device location
if (_cameraDevice.EnclosureLocation == null ||
_cameraDevice.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Unknown)
{
_externalCamera = true;
}
else
{
_externalCamera = false;
_mirroringPreview = (_cameraDevice.EnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Front);
}
Initialize the CameraRotationHelper class
Now we begin using the CameraRotationHelper class. Declare a class member variable to store the object. Call the constructor, passing in the enclosure location of the selected camera. The helper class uses this information to calculate the correct orientation for captured media, the preview stream, and the UI. Register a handler for the helper class's OrientationChanged event, which will be raised when we need to update the orientation of the UI or the preview stream.
private CameraRotationHelper _rotationHelper;
_rotationHelper = new CameraRotationHelper(_cameraDevice.EnclosureLocation);
_rotationHelper.OrientationChanged += RotationHelper_OrientationChanged;
Add orientation data to the camera preview stream
Adding the correct orientation to the metadata of the preview stream does not affect how the preview appears to the user, but it helps the system encode any frames captured from the preview stream correctly.
You start the camera preview by calling MediaCapture.StartPreviewAsync. Before you do this, check the member variable to see if the preview should be mirrored (for a front-facing camera). If so, set the FlowDirection property of the CaptureElement, named PreviewControl in this example, to FlowDirection.RightToLeft. After starting the preview, call the helper method SetPreviewRotationAsync to set the preview rotation. Following is the implementation of this method.
PreviewControl.Source = mediaCapture;
PreviewControl.FlowDirection = _mirroringPreview ? FlowDirection.RightToLeft : FlowDirection.LeftToRight;
await mediaCapture.StartPreviewAsync();
await SetPreviewRotationAsync();
We set the preview rotation in a separate method so that it can be updated when the phone orientation changes without reinitializing the preview stream. If the camera is external to the device, no action is taken. Otherwise, the CameraRotationHelper method GetCameraPreviewOrientation is called and returns the proper orientation for the preview stream.
To set the metadata, the preview stream properties are retrieved by calling VideoDeviceController.GetMediaStreamProperties. Next, create the GUID representing the Media Foundation Transform (MFT) attribute for video stream rotation. In C++ you can use the constant MF_MT_VIDEO_ROTATION, but in C# you must manually specify the GUID value.
Add a property value to the stream properties object, specifying the GUID as the key and the preview rotation as the value. This property expects values to be in units of counterclockwise degrees, so the CameraRotationHelper method ConvertSimpleOrientationToClockwiseDegrees is used to convert the simple orientation value. Finally, call SetEncodingPropertiesAsync to apply the new rotation property to the stream.
private async Task SetPreviewRotationAsync()
{
if (!_externalCamera)
{
// Add rotation metadata to the preview stream to make sure the aspect ratio / dimensions match when rendering and getting preview frames
var rotation = _rotationHelper.GetCameraPreviewOrientation();
var props = mediaCapture.VideoDeviceController.GetMediaStreamProperties(MediaStreamType.VideoPreview);
Guid RotationKey = new Guid("C380465D-2271-428C-9B83-ECEA3B4A85C1");
props.Properties.Add(RotationKey, CameraRotationHelper.ConvertSimpleOrientationToClockwiseDegrees(rotation));
await mediaCapture.SetEncodingPropertiesAsync(MediaStreamType.VideoPreview, props, null);
}
}
Next, add the handler for the CameraRotationHelper.OrientationChanged event. This event passes in an argument that lets you know whether the preview stream needs to be rotated. If the orientation of the device was changed to face up or face down, this value will be false. If the preview does need to be rotated, call SetPreviewRotationAsync which was defined previously.
Next, in the OrientationChanged event handler, update your UI if needed. Get the current recommended UI orientation from the helper class by calling GetUIOrientation and convert the value to clockwise degrees, which is used for XAML transforms. Create a RotateTransform from the orientation value and set the RenderTransform property of your XAML controls. Depending on your UI layout, you may need to make additional adjustments here in addition to simply rotating the controls. Also, remember that all updates to your UI must be made on the UI thread, so you should place this code inside a call to RunAsync.
private async void RotationHelper_OrientationChanged(object sender, bool updatePreview)
{
if (updatePreview)
{
await SetPreviewRotationAsync();
}
await Dispatcher.RunAsync(CoreDispatcherPriority.Normal, () => {
// Rotate the buttons in the UI to match the rotation of the device
var angle = CameraRotationHelper.ConvertSimpleOrientationToClockwiseDegrees(_rotationHelper.GetUIOrientation());
var transform = new RotateTransform { Angle = angle };
// The RenderTransform is safe to use (i.e. it won't cause layout issues) in this case, because these buttons have a 1:1 aspect ratio
CapturePhotoButton.RenderTransform = transform;
CapturePhotoButton.RenderTransform = transform;
});
}
Capture a photo with orientation data
The article Basic photo, video, and audio capture with MediaCapture shows you how to capture a photo to a file by capturing first to an in-memory stream and then using a decoder to read the image data from the stream and an encoder to transcode the image data to a file. Orientation data, obtained from the CameraRotationHelper class, can be added to the image file during the transcoding operation.
In the following example, a photo is captured to an InMemoryRandomAccessStream with a call to CapturePhotoToStreamAsync and a BitmapDecoder is created from the stream. Next a StorageFile is created and opened to retrieve an IRandomAccessStream for writing to the file.
Before transcoding the file, the photo's orientation is retrieved from the helper class method GetCameraCaptureOrientation. This method returns a SimpleOrientation object which is converted to a PhotoOrientation object with the helper method ConvertSimpleOrientationToPhotoOrientation. Next, a new BitmapPropertySet object is created and a property is added where the key is "System.Photo.Orientation" and the value is the photo orientation, expressed as a BitmapTypedValue. "System.Photo.Orientation" is one of many Windows properties that can be added as metadata to an image file. For a list of all of the photo-related properties, see Windows Properties - Photo. For more information about working with metadata in images, see Image metadata.
Finally, the property set which includes the orientation data is set for the encoder by with a call to SetPropertiesAsync and the image is transcoded with a call to FlushAsync.
private async Task CapturePhotoWithOrientationAsync()
{
var captureStream = new InMemoryRandomAccessStream();
try
{
await mediaCapture.CapturePhotoToStreamAsync(ImageEncodingProperties.CreateJpeg(), captureStream);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("Exception when taking a photo: {0}", ex.ToString());
return;
}
var decoder = await BitmapDecoder.CreateAsync(captureStream);
var file = await KnownFolders.PicturesLibrary.CreateFileAsync("SimplePhoto.jpeg", CreationCollisionOption.GenerateUniqueName);
using (var outputStream = await file.OpenAsync(FileAccessMode.ReadWrite))
{
var encoder = await BitmapEncoder.CreateForTranscodingAsync(outputStream, decoder);
var photoOrientation = CameraRotationHelper.ConvertSimpleOrientationToPhotoOrientation(
_rotationHelper.GetCameraCaptureOrientation());
var properties = new BitmapPropertySet {
{ "System.Photo.Orientation", new BitmapTypedValue(photoOrientation, PropertyType.UInt16) } };
await encoder.BitmapProperties.SetPropertiesAsync(properties);
await encoder.FlushAsync();
}
}
Capture a video with orientation data
Basic video capture is described in the article Basic photo, video, and audio capture with MediaCapture. Adding orientation data to the encoding of the captured video is done using the same technique as described earlier in the section about adding orientation data to the preview stream.
In the following example, a file is created to which the captured video will be written. An MP4 encoding profile is create using the static method CreateMp4. The proper orientation for the video is obtained from the CameraRotationHelper class with a call to GetCameraCaptureOrientation Because the rotation property requires the orientation to be expressed in counterclockwise degrees, the ConvertSimpleOrientationToClockwiseDegrees helper method is called to convert the orientation value. Next, create the GUID representing the Media Foundation Transform (MFT) attribute for video stream rotation. In C++ you can use the constant MF_MT_VIDEO_ROTATION, but in C# you must manually specify the GUID value. Add a property value to the stream properties object, specifying the GUID as the key and the rotation as the value. Finally call StartRecordToStorageFileAsync to begin recording video encoded with orientation data.
private async Task StartRecordingWithOrientationAsync()
{
try
{
var videoFile = await KnownFolders.VideosLibrary.CreateFileAsync("SimpleVideo.mp4", CreationCollisionOption.GenerateUniqueName);
var encodingProfile = MediaEncodingProfile.CreateMp4(VideoEncodingQuality.Auto);
var rotationAngle = CameraRotationHelper.ConvertSimpleOrientationToClockwiseDegrees(
_rotationHelper.GetCameraCaptureOrientation());
Guid RotationKey = new Guid("C380465D-2271-428C-9B83-ECEA3B4A85C1");
encodingProfile.Video.Properties.Add(RotationKey, PropertyValue.CreateInt32(rotationAngle));
await mediaCapture.StartRecordToStorageFileAsync(encodingProfile, videoFile);
}
catch (Exception ex)
{
System.Diagnostics.Debug.WriteLine("Exception when starting video recording: {0}", ex.ToString());
}
}
CameraRotationHelper full code listing
The following code snippet lists the full code for the CameraRotationHelper class that manages the hardware orientation sensors, calculates the proper orientation values for photos and videos, and provides helper methods to convert between the different representations of orientation that are used by different Windows features. If you follow the guidance shown in the article above, you can add this class to your project as-is without having to make any changes. Of course, you can feel free to customize the following code to meet the needs of your particular scenario.
This helper class uses the device's SimpleOrientationSensor to determine the current orientation of the device chassis and the DisplayInformation class to determine the current orientation of the display. Each of these classes provide events that are raised when the current orientation changes. The panel on which the capture device is mounted - front-facing, back-facing, or external - is used to determine whether the preview stream should be mirrored. Also, the EnclosureLocation.RotationAngleInDegreesClockwise property, supported by some devices, is used to determine the orientation in which the camera is mounted on the chassis.
The following methods can be used to get recommended orientation values for the specified camera app tasks:
- GetUIOrientation - Returns the suggested orientation for camera UI elements.
- GetCameraCaptureOrientation - Returns the suggested orientation for encoding into image metadata.
- GetCameraPreviewOrientation - Returns the suggested orientation for the preview stream to provide a natural user experience.
class CameraRotationHelper
{
private EnclosureLocation _cameraEnclosureLocation;
private DisplayInformation _displayInformation = DisplayInformation.GetForCurrentView();
private SimpleOrientationSensor _orientationSensor = SimpleOrientationSensor.GetDefault();
public event EventHandler<bool> OrientationChanged;
public CameraRotationHelper(EnclosureLocation cameraEnclosureLocation)
{
_cameraEnclosureLocation = cameraEnclosureLocation;
if (!IsEnclosureLocationExternal(_cameraEnclosureLocation))
{
_orientationSensor.OrientationChanged += SimpleOrientationSensor_OrientationChanged;
}
_displayInformation.OrientationChanged += DisplayInformation_OrientationChanged;
}
private void SimpleOrientationSensor_OrientationChanged(SimpleOrientationSensor sender, SimpleOrientationSensorOrientationChangedEventArgs args)
{
if (args.Orientation != SimpleOrientation.Faceup && args.Orientation != SimpleOrientation.Facedown)
{
HandleOrientationChanged(false);
}
}
private void DisplayInformation_OrientationChanged(DisplayInformation sender, object args)
{
HandleOrientationChanged(true);
}
private void HandleOrientationChanged(bool updatePreviewStreamRequired)
{
var handler = OrientationChanged;
if (handler != null)
{
handler(this, updatePreviewStreamRequired);
}
}
public static bool IsEnclosureLocationExternal(EnclosureLocation enclosureLocation)
{
return (enclosureLocation == null || enclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Unknown);
}
private bool IsCameraMirrored()
{
// Front panel cameras are mirrored by default
return (_cameraEnclosureLocation.Panel == Windows.Devices.Enumeration.Panel.Front);
}
private SimpleOrientation GetCameraOrientationRelativeToNativeOrientation()
{
// Get the rotation angle of the camera enclosure
return ConvertClockwiseDegreesToSimpleOrientation((int)_cameraEnclosureLocation.RotationAngleInDegreesClockwise);
}
// Gets the rotation to rotate ui elements
public SimpleOrientation GetUIOrientation()
{
if (IsEnclosureLocationExternal(_cameraEnclosureLocation))
{
// Cameras that are not attached to the device do not rotate along with it, so apply no rotation
return SimpleOrientation.NotRotated;
}
// Return the difference between the orientation of the device and the orientation of the app display
var deviceOrientation = _orientationSensor.GetCurrentOrientation();
var displayOrientation = ConvertDisplayOrientationToSimpleOrientation(_displayInformation.CurrentOrientation);
return SubOrientations(displayOrientation, deviceOrientation);
}
// Gets the rotation of the camera to rotate pictures/videos when saving to file
public SimpleOrientation GetCameraCaptureOrientation()
{
if (IsEnclosureLocationExternal(_cameraEnclosureLocation))
{
// Cameras that are not attached to the device do not rotate along with it, so apply no rotation
return SimpleOrientation.NotRotated;
}
// Get the device orienation offset by the camera hardware offset
var deviceOrientation = _orientationSensor.GetCurrentOrientation();
var result = SubOrientations(deviceOrientation, GetCameraOrientationRelativeToNativeOrientation());
// If the preview is being mirrored for a front-facing camera, then the rotation should be inverted
if (IsCameraMirrored())
{
result = MirrorOrientation(result);
}
return result;
}
// Gets the rotation of the camera to display the camera preview
public SimpleOrientation GetCameraPreviewOrientation()
{
if (IsEnclosureLocationExternal(_cameraEnclosureLocation))
{
// Cameras that are not attached to the device do not rotate along with it, so apply no rotation
return SimpleOrientation.NotRotated;
}
// Get the app display rotation offset by the camera hardware offset
var result = ConvertDisplayOrientationToSimpleOrientation(_displayInformation.CurrentOrientation);
result = SubOrientations(result, GetCameraOrientationRelativeToNativeOrientation());
// If the preview is being mirrored for a front-facing camera, then the rotation should be inverted
if (IsCameraMirrored())
{
result = MirrorOrientation(result);
}
return result;
}
public static PhotoOrientation ConvertSimpleOrientationToPhotoOrientation(SimpleOrientation orientation)
{
switch (orientation)
{
case SimpleOrientation.Rotated90DegreesCounterclockwise:
return PhotoOrientation.Rotate90;
case SimpleOrientation.Rotated180DegreesCounterclockwise:
return PhotoOrientation.Rotate180;
case SimpleOrientation.Rotated270DegreesCounterclockwise:
return PhotoOrientation.Rotate270;
case SimpleOrientation.NotRotated:
default:
return PhotoOrientation.Normal;
}
}
public static int ConvertSimpleOrientationToClockwiseDegrees(SimpleOrientation orientation)
{
switch (orientation)
{
case SimpleOrientation.Rotated90DegreesCounterclockwise:
return 270;
case SimpleOrientation.Rotated180DegreesCounterclockwise:
return 180;
case SimpleOrientation.Rotated270DegreesCounterclockwise:
return 90;
case SimpleOrientation.NotRotated:
default:
return 0;
}
}
private SimpleOrientation ConvertDisplayOrientationToSimpleOrientation(DisplayOrientations orientation)
{
SimpleOrientation result;
switch (orientation)
{
case DisplayOrientations.Landscape:
result = SimpleOrientation.NotRotated;
break;
case DisplayOrientations.PortraitFlipped:
result = SimpleOrientation.Rotated90DegreesCounterclockwise;
break;
case DisplayOrientations.LandscapeFlipped:
result = SimpleOrientation.Rotated180DegreesCounterclockwise;
break;
case DisplayOrientations.Portrait:
default:
result = SimpleOrientation.Rotated270DegreesCounterclockwise;
break;
}
// Above assumes landscape; offset is needed if native orientation is portrait
if (_displayInformation.NativeOrientation == DisplayOrientations.Portrait)
{
result = AddOrientations(result, SimpleOrientation.Rotated90DegreesCounterclockwise);
}
return result;
}
private static SimpleOrientation MirrorOrientation(SimpleOrientation orientation)
{
// This only affects the 90 and 270 degree cases, because rotating 0 and 180 degrees is the same clockwise and counter-clockwise
switch (orientation)
{
case SimpleOrientation.Rotated90DegreesCounterclockwise:
return SimpleOrientation.Rotated270DegreesCounterclockwise;
case SimpleOrientation.Rotated270DegreesCounterclockwise:
return SimpleOrientation.Rotated90DegreesCounterclockwise;
}
return orientation;
}
private static SimpleOrientation AddOrientations(SimpleOrientation a, SimpleOrientation b)
{
var aRot = ConvertSimpleOrientationToClockwiseDegrees(a);
var bRot = ConvertSimpleOrientationToClockwiseDegrees(b);
var result = (aRot + bRot) % 360;
return ConvertClockwiseDegreesToSimpleOrientation(result);
}
private static SimpleOrientation SubOrientations(SimpleOrientation a, SimpleOrientation b)
{
var aRot = ConvertSimpleOrientationToClockwiseDegrees(a);
var bRot = ConvertSimpleOrientationToClockwiseDegrees(b);
//add 360 to ensure the modulus operator does not operate on a negative
var result = (360 + (aRot - bRot)) % 360;
return ConvertClockwiseDegreesToSimpleOrientation(result);
}
private static VideoRotation ConvertSimpleOrientationToVideoRotation(SimpleOrientation orientation)
{
switch (orientation)
{
case SimpleOrientation.Rotated90DegreesCounterclockwise:
return VideoRotation.Clockwise270Degrees;
case SimpleOrientation.Rotated180DegreesCounterclockwise:
return VideoRotation.Clockwise180Degrees;
case SimpleOrientation.Rotated270DegreesCounterclockwise:
return VideoRotation.Clockwise90Degrees;
case SimpleOrientation.NotRotated:
default:
return VideoRotation.None;
}
}
private static SimpleOrientation ConvertClockwiseDegreesToSimpleOrientation(int orientation)
{
switch (orientation)
{
case 270:
return SimpleOrientation.Rotated90DegreesCounterclockwise;
case 180:
return SimpleOrientation.Rotated180DegreesCounterclockwise;
case 90:
return SimpleOrientation.Rotated270DegreesCounterclockwise;
case 0:
default:
return SimpleOrientation.NotRotated;
}
}
}
Related topics