Mixed reality cloud services and applications

  • 5 minutes

When you're developing mixed reality content, you might encounter scenarios where you need to store object placement information or render complex 3D models. Or you need to take advantage of Azure AI services like Speech and Vision. With Microsoft Mesh, people can collaborate on 3D models from remote locations, as if they're in the same room working with each other.

Mixed reality cloud services

You can use Azure Remote Rendering to render highly complex 3D models in real time and then stream them directly to a device by moving the rendering workload to high-end GPUs in the cloud. This service is especially important for applications that you might be developing for untethered devices, because they have less computational rendering power.

Photograph of two 3D car engine models. The left model has a high triangle count, and the right model has a low triangle count.

You can use Azure Spatial Anchors to build spatially aware applications. You can map, persist, and share holographic content across multiple devices at real-world scale. This service is cross-platform and is uniquely suited to way finding, multiuser experiences, and saving your object information.

Animated illustration of a spatial anchor persisting on a table between application sessions.

Microsoft has several cognitive services that can be integrated into mixed reality applications. For example, the Speech service enables the integration of speech-processing capabilities into any app or service. You can convert spoken language into text or produce natural-sounding speech from text by using standard or customizable voice fonts.

Another example is the Vision service, which lets your app recognize, identify, caption, index, and moderate your pictures, videos, and digital ink content. It's even possible for your apps to accurately identify and analyze content within an image or video.

Dynamics 365 mixed reality applications

The Dynamics 365 Remote Assist application allows your users to get help or collaborate efficiently from different physical locations by using HoloLens, HoloLens 2, Android, or iOS devices. This feature is highly valuable in scenarios like maintenance and repair, remote inspections, or training sessions.

Photograph of a HoloLens user performing mechanical repair while using Remote Assist to communicate with a coworker.

Microsoft Dynamics 365 Guides is a mixed reality application for Microsoft HoloLens. It lets operators learn during the flow of work by providing holographic instructions when and where they're needed. These instruction cards are visually tethered to the place where the work must be done. They can include images, videos, and 3D holographic models.

Microsoft Mesh

Mesh enables people to connect with presence, share across space, and collaborate with one another, from anywhere in the world. By bringing Mesh-enabled mixed reality experiences to your organization, you can enhance virtual meetings, conduct virtual design sessions, help others remotely, and host immersive virtual meet-ups to boost productivity.

Photograph of people using HoloLens to collaborate.

Mesh provides a natural collaboration experience in mixed reality. People are represented as 3D avatars in a shared space. Proximity and spatial audio let people know where they are relative to one another. Users can visualize and annotate content together in a shared 3D space. A user can look at the person they're talking to and point to features on a shared 3D object.

Scenarios

Here are some common use cases for Microsoft Mesh:

  • Virtual collaboration: Colleagues working time zones apart can collaborate as if they're physically in the same room. Mesh integrates with Microsoft 365, so connections, calendars, content, and workflows naturally transition to mixed reality. These shared experiences help deepen understanding, increase employee engagement and improve productivity.

    Photo of a virtual collaboration experience of Microsoft Mesh: people collaborating on the hologram of a car.

  • Spatially aware design reviews: Mesh enhances 3D design reviews by enabling users to join from anywhere using any device. Whether physically present or holoported, colleagues can see and annotate 3D models in real-time. All content persists between design sessions so teams can quickly start where they left off. This common understanding ignites ideas, sparks creativity, and forms powerful bonds.

    Photo of a spatially aware design experience of Microsoft Mesh: two people collaborating on th 3D hologram of product assembly.

  • Help others remotely: When employees need help, there's nothing like having an expert right next to them to provide guidance and offer new perspectives. Mesh enables remote experts to be present wherever they're needed. They can overlay contextual data and help share insights quickly and effectively, resolving issues faster.

    Photo of remote assistance experience of Microsoft Mesh: two people across the globe visualizing 3D holographic image of data as shown on global scale.

  • Train and learn together: Training is hard to do without being in the same room with the instructor and looking at the same set of objects from multiple perspectives. This fact is especially true for complex topics such as surgery, equipment maintenance, and traffic control. With Mesh, employees can learn together from anywhere thanks to holoportation, holographic sharing, and visualization. Mesh helps improve the efficacy of virtual training while reducing travel and logistics costs.

    Photo of remote learning experience of Microsoft Mesh: remote training of calligraphy techniques using paint brushes.

  • Host virtual meet-ups: Foster a deeper sense of connection and community by hosting virtual meet-ups in Mesh-powered mixed reality experiences. Form powerful bonds with others by letting your personality shine through as technology fades away.

    Photo of virtual meet-up experience of Microsoft Mesh: doctor and patient meeting remotely to check on rehabilitation progress.