Edit

Share via

Add MySQL Database CDC as source in Real-Time hub (preview)

  • Article

This article describes how to add MySQL Database Change Data Capture (CDC) as an event source in Fabric Real-Time hub.

The Azure MySQL Database Change Data Capture (CDC) connector allows you to capture a snapshot of the current data in an Azure MySQL database. You specify the tables to be monitored and get alerted when any subsequent row-level changes to the tables. Once the changes are captured in a stream, you can process this CDC data in real-time and send it to different destinations within Fabric for further processing or analysis.

Note

Real-Time hub is currently in preview.

Prerequisites

  • Get access to the Fabric premium workspace with Contributor or above permissions.
  • Created an instance of Azure Database for MySQL – Flexible Server.

Set up MySQL database

The Azure MySQL database connector uses the Debezium MySQL connector to capture changes in your MySQL Database. You must define a MySQL user with permissions on all databases that connector monitors. For step-by-step instructions, see Set up MySQL DB.

Enable the binlog

You must enable binary logging for MySQL replication. The binary logs record transaction updates for replication tools to propagate changes. For example, Azure Database for MySQL.

  1. In the Azure portal, navigate to your Azure MySQL database.

  2. On the left navigation menu, select Server parameters.

  3. Configure your MySQL server with the following properties.

    • binlog_row_image: Set the value to full.
    • binlog_expire_logs_seconds: The number of seconds for automatic binlog file removal. Set the value to match the needs of your environment. For example, 86400.

    Screenshot that shows the Server parameters page for the Azure MySQL database.

Get events from Azure MySQL Database (CDC)

You can get events from an Azure MySQL Database CDC into Real-Time hub in one of the ways:

Launch Add source experience

  1. Sign in to Microsoft Fabric.

  2. Select Real-Time hub on the left navigation bar.

  3. On the Real-Time hub page, select + Add source in the top-right corner of the page.

    Screenshot that shows how to launch Real-Time hub In Microsoft Fabric.

Use instructions from the Add Azure MySQL Database CDC as a source section.

Microsoft sources page

  1. In Real-Time hub, select Microsoft sources.

  2. In the Source drop-down list, select Azure MySQL DB (CDC).

  3. For Subscription, select an Azure subscription that has the resource group with your Azure MySQL Database account.

  4. For Resource group, select a resource group that has the database.

  5. For Region, select a location where your database is located.

  6. Now, move the mouse over the name of the Azure MySQL DB CDC source that you want to connect to Real-Time hub in the list of databases, and select the Connect button, or select ... (ellipsis), and then select the Connect button.

    Screenshot that shows the Microsoft sources page with filters to show Azure MySQL CDC and the connect button.

    To configure connection information, use steps from the Add Azure MySQL DB CDC as a source section. Skip the first step of selecting Azure MySQL DB CDC as a source type in the Add source wizard.

Add Azure MySQL Database CDC as a source

  1. On the Select a data source screen, select MySQL DB (CDC).

    Screenshot that shows the Select a data source page with Azure MySQL DB (CDC) selected.

  2. On the Connect page, select Go to resource to navigate to the Azure PostgreSQL database. Take a note of the server name on the Overview page. It's in the following format: mysqlserver.mysql.database.azure.com.

    Screenshot that shows the Connect page with Go to resource link highlighted.

  3. On the Connect page, select New connection.

    Screenshot that shows the Connect page of the Add source wizard with the **New connection** link highlighted.

  4. In the Connection settings section, do these steps:

    1. For Server, enter the URI for your Azure MySQL server.

    2. For Database, enter the name of your database.

      Screenshot that shows the Connection settings section.

  5. In the Connection credentials section, do these steps:

    1. For Connection, select if there's an existing connection to the MySQL database. If not, keep the default value: Create new connection.

    2. For Authentication kind, select Basic. Currently, only Basic authentication is supported.

    3. Enter values for User name and Password.

    4. Specify whether you want to use an encrypted connection.

    5. Select Connect.

      Screenshot that shows the Connection credentials section.

  6. Back on the Connect page, do these steps:

    1. Enter the table name.
    2. Enter the server ID.
    3. Enter the port number or keep the default value.

  7. In the Stream details section to the right, do these steps:

    1. Select Fabric workspace where you want to save this connection and the eventstream that the wizard creates.

    2. Enter a name for the eventstream.

    3. The name of the stream in Real-Time hub is automatically created for you.

      Screenshot that shows the Connect page with all the required fields specified.

  8. Now, select Next at the bottom of the page.

  9. On the Review and create, review settings, and select Create source.

    Screenshot that shows the Review and create page with all the required fields specified.

View data stream details

  1. On the Review and create page, if you select Open eventstream, the wizard opens the eventstream that it created for you with the selected Azure MySQL DB CDC as a source. To close the wizard, select Close or X* in the top-right corner of the page.

    Screenshot that shows the Review and create page after successful creation of the source.

  2. In Real-Time hub, select All data streams. To see the new data stream, refresh the All data streams page. For detailed steps, see View details of data streams in Fabric Real-Time hub.

Related content

To learn about consuming data streams, see the following articles: