is there a spark connector for Azure Cosmos DB MongoDB API RU?

Question

I'm using Azure Databricks to develop a proof of concept. Is there support for a Spark Connector for Azure Cosmos DB MongoDB API? If yes, what versions are supported. Is both batch and stream operations supported?

Answer 1

Hi @Lum, Joey

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

Is there support for a Spark Connector for Azure Cosmos DB MongoDB API? Is both batch and stream operations supported?

Yes, there is support for a Spark Connector for Azure Cosmos DB’s MongoDB API. You can use the MongoDB Connector for Spark to connect to Azure Cosmos DB for MongoDB. The Spark Connector for Azure Cosmos DB MongoDB API supports both batch and stream operations. The connector provides a high-performance interface for Apache Spark to Azure Cosmos DB using the MongoDB API.

Why MongoDB Spark Connector?

Direct Integration with Spark: It allows you to leverage the powerful capabilities of Spark for large-scale data processing and analysis directly on your Cosmos DB data.

Flexible Data Processing: You can perform a wide range of operations, including data ingestion, transformation, aggregation, and machine learning, using Spark's rich API.

Scalability and Performance: Spark's distributed processing capabilities enable you to handle large datasets efficiently and scale your workloads as needed.

For additional information, please refer: https://learn.microsoft.com/en-us/azure/cosmos-db/mongodb/vcore/connect-from-databricks#configure-dependencies-for-connectivity

what versions are supported.

Version 10.x of the MongoDB Spark Connector is an all-new connector based on the latest Spark API. Install and migrate to version 10.x to take advantage of new capabilities, such as tighter integration with Spark Structured Streaming.

For more details, please refer the below links:

• MongoDB Connector for Spark
• Azure Cosmos DB Spark connector

I hope this information helps. Please do let us know if you have any further queries.

---

If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.

Thank you.