Data marketplace
Data marketplaces have a strong relationship with metadata. A data marketplace offers data consumers an intuitive, secure, centralized, and standardized data shopping experience. It brings data closer to data analysts and scientists by utilizing the underlying metadata. It also tracks all your data products, which are often stored across a range of data domains.
To democratize data via your data mesh architecture, focus on several important areas:
Data Product Experience Plane: Allows data providers and data consumers to work together on what data can be made available. Interfaces should provide extensive search capabilities that allow users to search using keywords, business terms, and natural language. Collaboration in data democratization is often linked to data catalogs or fully managed metadata management services that enable metadata search and discovery. Microsoft Purview is a proven approach to having a self-service collaboration portal. It supports data discovery, including glossaries and classifications. Data discovery enables your data consumers to easily find data. Microsoft Purview also supports data owner access policies so you can provide self-serve data access.
Data Infrastructure (Utility) Plane: Helps you automate the deployment and provisioning of common and reusable consumption patterns. Consumption patterns can include storage accounts, databases, compute, identity management, and so on. Best practices for allowing your users to set up and launch their own data services can be found in Organize data operations team members for cloud-scale analytics in Azure, Deployment and management services, and Development services.
Data Mesh Experience Plane: Helps you keep sight of the health status of all interfaces, data pipelines, data contracts, provisioned components, central tools, and so on. Azure Monitor helps you maximize the availability and performance of your applications and services and achieve monitoring and insight. For data observability, create an umbrella on top of your self-service collaboration portal and other metadata services. Consider designing your own metadata lake using services like Azure Cosmos DB and Azure Event Hubs.
A data marketplace is typically a thin orchestration layer with an appealing look and feel, which offers unique user experiences. Data marketplaces utilize underlying metadata repositories, which can be a mixture of homegrown metadata stores and services like Microsoft Purview. You can extend your data marketplace with extra analytical capabilities like Cognitive Services and Machine Learning. For more information on how to adopt AI/ML in data mesh, see Operationalize data mesh for AI/ML.
Building a data marketplace involves structure, culture, and people. It requires you to trust users, train people, and work on awareness. Don't underestimate these activities. Your users are valuable resources; they own or use specific parts of the data landscape. Making better use of your users increases the efficiency of your data knowledge and usage.
In some cases, you might need an external data marketplace. External data marketplaces enable sharing of your data products with external partners. You can use Azure Data Share as a component.