Kubernetes Load Balancer 403 AuthorizationFailed and InvalidApiVersionParameter

Question

When deploying a Kubernetes Service (type LoadBalancer), the service shows the following event:

Warning  SyncLoadBalancerFailed  1s (x2 over 7s)  service-controller  Error syncing load balancer: failed to ensure load balancer: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 403, RawError: {"error":{"code":"AuthorizationFailed","message":"The client '<redacted>' with object id '<redacted>' does not have authorization to perform action 'Microsoft.Resources/subscriptions/resourceGroups/<redacted>/rg/Microsoft.Network/read' over scope '/subscriptions/<redacted>/resourceGroups/subscriptions/<redacted>/resourceGroups/rg/providers/Microsoft.Network/publicIPAddresses' or the scope is invalid. If access was recently granted, please refresh your credentials."}}

But I already granted the mentioned permission by assigning "Network Contributor" role on the resource group to the managed identity.

I can "fix" this by assigning the "Owner" role on the subscription to the managed identity. But then the following is shown:

Warning  SyncLoadBalancerFailed  1s    service-controller  Error syncing load balancer: failed to ensure load balancer: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: {"error":{"code":"InvalidApiVersionParameter","message":"The api-version '2022-07-01' is invalid. The supported versions are '2024-08-01,2024-07-01,2024-06-01-preview,2024-03-01,2023-07-01,2023-07-01-preview,2023-03-01-preview,2022-12-01,2022-11-01-preview,2022-09-01,2022-06-01,2022-05-01,2022-03-01-preview,2022-01-01,2021-04-01,2021-01-01,2020-10-01,2020-09-01,2020-08-01,2020-07-01,2020-06-01,2020-05-01,2020-01-01,2019-11-01,2019-10-01,2019-09-01,2019-08-01,2019-07-01,2019-06-01,2019-05-10,2019-05-01,2019-03-01,2018-11-01,2018-09-01,2018-08-01,2018-07-01,2018-06-01,2018-05-01,2018-02-01,2018-01-01,2017-12-01,2017-08-01,2017-06-01,2017-05-10,2017-05-01,2017-03-01,2016-09-01,2016-07-01,2016-06-01,2016-02-01,2015-11-01,2015-01-01,2014-04-01-preview,2014-04-01,2014-01-01,2013-03-01,2014-02-26,2014-04'."}}

This results in the configured external ip not being used by the load balancer.

UPDATE

Here's the full output of kubectl describe svc:

Name:                     traefik
Namespace:                traefik
Labels:                   app.kubernetes.io/instance=traefik-traefik
                          app.kubernetes.io/managed-by=Helm
                          app.kubernetes.io/name=traefik
                          helm.sh/chart=traefik-33.0.0
Annotations:              meta.helm.sh/release-name: traefik
                          meta.helm.sh/release-namespace: traefik
                          service.beta.kubernetes.io/azure-load-balancer-resource-group:
                            /subscriptions/<redacted>/resourceGroups/<redacted>
                          service.beta.kubernetes.io/azure-pip-name: pip-lb
Selector:                 app.kubernetes.io/instance=traefik-traefik,app.kubernetes.io/name=traefik
Type:                     LoadBalancer
IP Family Policy:         SingleStack
IP Families:              IPv4
IP:                       <redacted>
IPs:                      <redacted>
Port:                     web  80/TCP
TargetPort:               web/TCP
NodePort:                 web  31856/TCP
Endpoints:                
Port:                     websecure  443/TCP
TargetPort:               websecure/TCP
NodePort:                 websecure  32096/TCP
Endpoints:                
Session Affinity:         None
External Traffic Policy:  Cluster
Events:
  Type     Reason                  Age   From                Message
  ----     ------                  ----  ----                -------
  Normal   EnsuringLoadBalancer    2s    service-controller  Ensuring load balancer
  Warning  SyncLoadBalancerFailed  1s    service-controller  Error syncing load balancer: failed to ensure load balancer: Retriable: false, RetryAfter: 0s, HTTPStatusCode: 400, RawError: {"error":{"code":"InvalidApiVersionParameter","message":"The api-version '2022-07-01' is invalid. The supported versions are '2024-08-01,2024-07-01,2024-06-01-preview,2024-03-01,2023-07-01,2023-07-01-preview,2023-03-01-preview,2022-12-01,2022-11-01-preview,2022-09-01,2022-06-01,2022-05-01,2022-03-01-preview,2022-01-01,2021-04-01,2021-01-01,2020-10-01,2020-09-01,2020-08-01,2020-07-01,2020-06-01,2020-05-01,2020-01-01,2019-11-01,2019-10-01,2019-09-01,2019-08-01,2019-07-01,2019-06-01,2019-05-10,2019-05-01,2019-03-01,2018-11-01,2018-09-01,2018-08-01,2018-07-01,2018-06-01,2018-05-01,2018-02-01,2018-01-01,2017-12-01,2017-08-01,2017-06-01,2017-05-10,2017-05-01,2017-03-01,2016-09-01,2016-07-01,2016-06-01,2016-02-01,2015-11-01,2015-01-01,2014-04-01-preview,2014-04-01,2014-01-01,2013-03-01,2014-02-26,2014-04'."}}

Answer 1

Hi @Timo Notheisen,

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

Based on your inquiry, we understand you're facing an issue. The error message you received indicates that the API version 2022-07-01 in your request is invalid. It also provides a list of supported API versions for the operation you're attempting.

Please review your configuration or code to locate where the API version is set to 2022-07-01 and update it to one of the supported versions listed in the error message.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.ContainerService/managedClusters/{resourceName}?api-version=2024-08-01

If you have any further queries, do let us know.

If the answer has been helpful, we appreciate hearing from you and would love to help others who may have the same question. Accepting answers helps increase visibility of this question for other members of the Microsoft Q&A community.

Answer 2

Hi @Timo Notheisen ,

Thank you for reaching out to us again and confirming that the issue has been resolved.

As an original poster cannot accept their own answer, I am reposting it so that you can accept it an answer. Accepted answer will help other community members navigate to the appropriate solutions.

Issue: Kubernetes Load Balancer 403 AuthorizationFailed and InvalidApiVersionParameter

Solution: Turned out we had an error in the value of the service.beta.kubernetes.io/azure-load-balancer-resource-group annotation. We've been passing the resource group id the whole time. But it seems you have to use the resource group name instead.

This fixes both problems (the missing permission and the wrong API version).