Overview of orchestrate multimodal AI insights (preview) in healthcare data solutions
Important
- This is a preview feature.
- Preview features aren't meant for production use and might have restricted functionality. These features are available before an official release so that customers can get early access and provide feedback.
- To review the terms of service, see Healthcare data solutions in Microsoft Fabric.
The orchestrate multimodal AI insights (preview) capability in healthcare data solutions provides a structured framework that lets you create, evolve, and manage fully traceable healthcare AI data enrichments. It helps you connect AI services and models with healthcare data and integrate AI-generated insights back into your healthcare data estate. The capability supports advanced analytics while ensuring consistency, accuracy, and reliability.
Since the data transformation capabilities in healthcare data solutions in Microsoft Fabric let you ingest, store, and analyze multimodal data, the healthcare foundation models can use this standardized data to generate more insights. When you combine these AI insights with the existing healthcare data in your data estate, it unlocks new analytics scenarios.
The orchestrate multimodal AI insights (preview) framework enables you to generate enrichments in a scalable and efficient manner. It uses a hierarchy of data assets such as data stores, datasets, and views, that help avoid redundant API calls. The capability uses a transformation pipeline to ingest AI generated insights through the healthcare data solutions medallion lakehouse layers and persists the insights in an enrichment store within the silver layer. It supports the following enrichment types:
- Text
- Object
- Embeddings
- Image 2D segmentation
- Image 3D segmentation
The capability also offers the following five ready-to-use first-party model use cases for automated AI enrichment:
Text Analytics for health: Use Text Analytics for health to extract medical entities such as diagnoses & medications from unstructured data.
Sentiment analysis with OpenAI: Use GPT-4o to analyze sentiments from text transcripts for categories such as doctors' services, staff services, facilities, and cost.
Social determinants of health (SDOH) extraction with OpenAI: Use GPT-4o to extract SDOH factors from conversational data based on the categories defined by CMS (Centers for Medicare & Medicaid Services).
MedImageInsight: Use the MedImageInsight healthcare AI model to generate medical image embeddings.
MedImageParse: Use the MedImageParse healthcare AI model to enable segmentation, detection, and recognition across different object types and imaging modalities.
Orchestrate multimodal AI insights (preview) is an optional capability under healthcare data solutions in Microsoft Fabric. You have the flexibility to decide whether to use it, depending on your specific needs or scenarios.
Note
This article provides prerelease information and may change before the feature is available.