Tutorial: Deploy a Spring Boot app connected to Apache Kafka on Confluent Cloud with Service Connector in Azure Spring Apps

Learn how to access Apache Kafka on Confluent Cloud for a Spring Boot application running on Azure Spring Apps. In this tutorial, you complete the following tasks:

• Create Apache Kafka on Confluent Cloud
• Create a Spring Cloud application
• Build and deploy the Spring Boot app
• Connect Apache Kafka on Confluent Cloud to Azure Spring Apps using Service Connector

Warning

Microsoft recommends that you use the most secure authentication flow available. The authentication flow described in this procedure requires a very high degree of trust in the application, and carries risks that are not present in other flows. You should only use this flow when other more secure flows, such as managed identities, aren't viable.

Prerequisites

• An Azure account with an active subscription. Create an account for free.

• Java 8 or a more recent version with long-term support (LTS) 1.

• Use the Bash environment in Azure Cloud Shell. For more information, see Quickstart for Bash in Azure Cloud Shell.

  

• If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.

  • If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Sign in with the Azure CLI.

  • When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use extensions with the Azure CLI.

  • Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

Clone or download the sample app

1. Clone the sample repository:

   git clone https://github.com/Azure-Samples/serviceconnector-springcloud-confluent-springboot/
   

2. Navigate into the following folder:

   cd serviceconnector-springcloud-confluent-springboot
   

Prepare cloud services

Create an instance of Apache Kafka for Confluent Cloud

Create an instance of Apache Kafka for Confluent Cloud by following this guidance.

Create Kafka cluster and schema registry on Confluent Cloud

1. Sign in to Confluent Cloud using the SSO provided by Azure

   

2. Use the default environment or create a new one

   

3. Create a Kafka cluster with the following information:

   • Cluster type: Standard
   • Region/zones: eastus(Virginia), Single Zone
   • Cluster name: cluster_1 or any other name.

4. In Cluster overview -> Cluster settings, note the Kafka Bootstrap server URL.

   

5. Create API keys for the cluster in Data integration -> API Keys -> + Add Key with Global access. Note down the key and secret.

6. Create a topic named test with partitions 6 in Topics -> + Add topic

7. Under default environment, select the Schema Registry tab. Enable the Schema Registry and note down the API endpoint.

8. Create API keys for schema registry. Save the key and secret.

Create an Azure Spring Apps instance

Create an instance of Azure Spring Apps by following the Azure Spring Apps quickstart in Java. Make sure your Azure Spring Apps instance is created in a region that has Service Connector support.

Build and deploy the app

Build the sample app and create a new spring app

1. Sign in to Azure and choose your subscription.

   az login
   
   az account set --subscription <Name or ID of your subscription>
   

2. Build the project using gradle.

   ./gradlew build
   

3. Create the app with a public endpoint assigned. If you selected Java version 11 when generating the Spring Cloud project, include the --runtime-version=Java_11 switch.

   az spring app create -n hellospring -s <service-instance-name> -g <your-resource-group-name> --assign-endpoint true
   

Create a service connection using Service Connector

CLI

Run the following command to connect your Apache Kafka on Confluent Cloud to your spring cloud app.

az spring connection create confluent-cloud -g <your-spring-cloud-resource-group> --service <your-spring-cloud-service> --app <your-spring-cloud-app> --deployment <your-spring-cloud-deployment> --bootstrap-server <kafka-bootstrap-server-url> --kafka-key <cluster-api-key> --kafka-secret <cluster-api-secret> --schema-registry <kafka-schema-registry-endpoint> --schema-key <registry-api-key> --schema-secret <registry-api-secret>

Replace the following placeholder texts with your own data:

• Replace <your-resource-group-name> with the resource group name that you created for your Apps Spring Apps instance.
• Replace <kafka-bootstrap-server-url> with your Kafka bootstrap server URL. For example: pkc-xxxx.eastus.azure.confluent.cloud:9092.
• Replace <cluster-api-key> and <cluster-api-secret> with your cluster API key and secret.
• Replace <kafka-schema-registry-endpoint> with your Kafka Schema Registry endpoint. For example: https://psrc-xxxx.westus2.azure.confluent.cloud.
• Replace <registry-api-key> and <registry-api-secret> with your kafka Schema Registry API key and secret.

Note

If you see the error message "The subscription is not registered to use Microsoft.ServiceLinker", please run az provider register -n Microsoft.ServiceLinker to register the Service Connector resource provider and run the connection command again.

Portal

Select Service Connector and enter the following settings.

Setting	Suggested value	Description
Service Type	Apache Kafka on Confluent cloud	Target service type. If you don't have an Apache Kafka on Confluent Cloud target service, complete the previous steps in this tutorial.
Name	Generated unique name	The connection name that identifies the connection between your Spring Cloud and target service.
Kafka bootstrap server url	Your Kafka bootstrap server url.	Enter the value from earlier step: "Create Kafka cluster and schema registry on Confluent Cloud".
Cluster API Key	Your cluster API key.	Your cluster API key.
Cluster API Secret	Your cluster API secret.	Your cluster API secret.
Create connection for schema registry	Checked	Also create a connection to the schema registry.
Schema Registry endpoint	Your Kafka Schema Registry endpoint.
Schema Registry API Key	Your Kafka Schema Registry API Key.	Your Kafka Schema Registry API Key.
Schema Registry API Secret	Your Kafka Schema Registry API Secret.	Your Kafka Schema Registry API Secret.

Select Review + Create to review the connection settings. Then select Create to create start creating the service connection.

Deploy the JAR file

Run the following command to upload the JAR file (build/libs/java-springboot-0.0.1-SNAPSHOT.jar) to your Spring Cloud app.

az spring app deploy -n hellospring -s <service-instance-name> -g <your-resource-group-name>  --artifact-path build/libs/java-springboot-0.0.1-SNAPSHOT.jar

Validate the Kafka data ingestion

Navigate to your Spring Cloud app's endpoint from the Azure portal and select the application URL. You'll see "10 messages were produced to topic test".

Then go to the Confluent portal and the topic's page will show production throughput.

Next steps

Follow the tutorials listed below to learn more about Service Connector.

Learn about Service Connector concepts