Tutorial: Deploy a Spring Boot app connected to Apache Kafka on Confluent Cloud with Service Connector in Azure Spring Apps

Learn how to access Apache Kafka on Confluent Cloud for a Spring Boot application running on Azure Spring Apps. In this tutorial, you complete the following tasks:

  • Create Apache Kafka on Confluent Cloud
  • Create a Spring Cloud application
  • Build and deploy the Spring Boot app
  • Connect Apache Kafka on Confluent Cloud to Azure Spring Apps using Service Connector

Warning

Microsoft recommends that you use the most secure authentication flow available. The authentication flow described in this procedure requires a very high degree of trust in the application, and carries risks that are not present in other flows. You should only use this flow when other more secure flows, such as managed identities, aren't viable.

Prerequisites

  • An Azure account with an active subscription. Create an account for free.

  • Java 8 or a more recent version with long-term support (LTS) 1.

Clone or download the sample app

  1. Clone the sample repository:

    git clone https://github.com/Azure-Samples/serviceconnector-springcloud-confluent-springboot/
    
  2. Navigate into the following folder:

    cd serviceconnector-springcloud-confluent-springboot
    

Prepare cloud services

Create an instance of Apache Kafka for Confluent Cloud

Create an instance of Apache Kafka for Confluent Cloud by following this guidance.

Create Kafka cluster and schema registry on Confluent Cloud

  1. Sign in to Confluent Cloud using the SSO provided by Azure

    The link of Confluent cloud SSO login using Azure portal

  2. Use the default environment or create a new one

    Cloud environment of Apache Kafka on Confluent Cloud

  3. Create a Kafka cluster with the following information:

    • Cluster type: Standard
    • Region/zones: eastus(Virginia), Single Zone
    • Cluster name: cluster_1 or any other name.
  4. In Cluster overview -> Cluster settings, note the Kafka Bootstrap server URL.

    Cluster settings of Apache Kafka on Confluent Cloud

  5. Create API keys for the cluster in Data integration -> API Keys -> + Add Key with Global access. Note down the key and secret.

  6. Create a topic named test with partitions 6 in Topics -> + Add topic

  7. Under default environment, select the Schema Registry tab. Enable the Schema Registry and note down the API endpoint.

  8. Create API keys for schema registry. Save the key and secret.

Create an Azure Spring Apps instance

Create an instance of Azure Spring Apps by following the Azure Spring Apps quickstart in Java. Make sure your Azure Spring Apps instance is created in a region that has Service Connector support.

Build and deploy the app

Build the sample app and create a new spring app

  1. Sign in to Azure and choose your subscription.

    az login
    
    az account set --subscription <Name or ID of your subscription>
    
  2. Build the project using gradle.

    ./gradlew build
    
  3. Create the app with a public endpoint assigned. If you selected Java version 11 when generating the Spring Cloud project, include the --runtime-version=Java_11 switch.

    az spring app create -n hellospring -s <service-instance-name> -g <your-resource-group-name> --assign-endpoint true
    

Create a service connection using Service Connector

Run the following command to connect your Apache Kafka on Confluent Cloud to your spring cloud app.

az spring connection create confluent-cloud -g <your-spring-cloud-resource-group> --service <your-spring-cloud-service> --app <your-spring-cloud-app> --deployment <your-spring-cloud-deployment> --bootstrap-server <kafka-bootstrap-server-url> --kafka-key <cluster-api-key> --kafka-secret <cluster-api-secret> --schema-registry <kafka-schema-registry-endpoint> --schema-key <registry-api-key> --schema-secret <registry-api-secret>

Replace the following placeholder texts with your own data:

  • Replace <your-resource-group-name> with the resource group name that you created for your Apps Spring Apps instance.
  • Replace <kafka-bootstrap-server-url> with your Kafka bootstrap server URL. For example: pkc-xxxx.eastus.azure.confluent.cloud:9092.
  • Replace <cluster-api-key> and <cluster-api-secret> with your cluster API key and secret.
  • Replace <kafka-schema-registry-endpoint> with your Kafka Schema Registry endpoint. For example: https://psrc-xxxx.westus2.azure.confluent.cloud.
  • Replace <registry-api-key> and <registry-api-secret> with your kafka Schema Registry API key and secret.

Note

If you see the error message "The subscription is not registered to use Microsoft.ServiceLinker", please run az provider register -n Microsoft.ServiceLinker to register the Service Connector resource provider and run the connection command again.

Deploy the JAR file

Run the following command to upload the JAR file (build/libs/java-springboot-0.0.1-SNAPSHOT.jar) to your Spring Cloud app.

az spring app deploy -n hellospring -s <service-instance-name> -g <your-resource-group-name>  --artifact-path build/libs/java-springboot-0.0.1-SNAPSHOT.jar

Validate the Kafka data ingestion

Navigate to your Spring Cloud app's endpoint from the Azure portal and select the application URL. You'll see "10 messages were produced to topic test".

Then go to the Confluent portal and the topic's page will show production throughput.

Sample metrics

Next steps

Follow the tutorials listed below to learn more about Service Connector.