Connecting to Azure Storage Queue using Celery

Question

Hello, I am trying to use Celery with Azure Storage Queues as a message broker. I have written the following code to connect to the queue:

import os
from celery import Celery
from kombu.utils.url import safequote

## CELERY CONFIGURATION
STORAGE_ACCOUNT_NAME = storage_name
STORAGE_ACCOUNT_ACCESS_KEY = (
    safequote(storage_account_key),
)
STORAGE_QUEUE_NAME = storage_queue_name


# create a Celery instance and configure it with your Django settings.
app = Celery(
    "app",
    broker_url=f"azurestoragequeues://{STORAGE_ACCOUNT_NAME}:{STORAGE_ACCOUNT_ACCESS_KEY}@{STORAGE_QUEUE_NAME}",
)

However, I am receiving the following error:

ValueError: Unable to determine account name for shared key credential.  

I would like to confirm that I have configured the settings correctly. If not, could you please advise me on how to correct my configuration settings? Thanks.

Answer 1

@Henry Page Adding more information to the above repsponse

It seems like you are trying to use Celery with Azure Storage Queues as a message broker, but you are receiving a ValueError with the message "Unable to determine account name for shared key credential."

This error typically occurs when the STORAGE_ACCOUNT_NAME variable is not set correctly. To resolve this issue, you can try setting the STORAGE_ACCOUNT_NAME variable to the name of your Azure Storage account.

Here is an example of how to set the STORAGE_ACCOUNT_NAME variable:

import os
from celery import Celery
from kombu.utils.url import safequote
## CELERY CONFIGURATION
STORAGE_ACCOUNT_NAME = "your_storage_account_name"
STORAGE_ACCOUNT_ACCESS_KEY = safequote("your_storage_account_access_key")
STORAGE_QUEUE_NAME = "your_storage_queue_name"
# create a Celery instance and configure it with your Django settings.
app = Celery(
    "app",
    broker_url=f"azurestoragequeues://{STORAGE_ACCOUNT_NAME}:{STORAGE_ACCOUNT_ACCESS_KEY}@{STORAGE_QUEUE_NAME}",
)

Make sure to replace the your_storage_account_name, your_storage_account_access_key, and your_storage_queue_name placeholders with the appropriate values for your environment.

Also The broker_url should not include the access key directly. Instead, you need to provide the storage account name and access key separately.

CELERY CONFIGURATION
STORAGE_ACCOUNT_NAME = storage_name STORAGE_ACCOUNT_ACCESS_KEY = ( safequote(storage_account_key), ) STORAGE_QUEUE_NAME = storage_queue_name
create a Celery instance and configure it with your Django settings.
app = Celery( “app”, broker_url=f"azurestoragequeues://{STORAGE_ACCOUNT_NAME}:{STORAGE_ACCOUNT_ACCESS_KEY}@{STORAGE_QUEUE_NAME}", )

If the issue still persists, I would like wo work closer on this issue, could you send an email toAzCommunity[at]Microsoft[dot]com referencing this thread. Please mention "ATTN subm" in the subject field and subscription ID. Thank you for your cooperation on this matter and look forward to your reply.

Additional information:

Azure Queue Storage can be used to build flexible applications and separate functions for better durability across large workloads.

• A queue message must be in a format compatible with an XML request using UTF-8 encoding.
• A message may be up to 64 KB in size.
• If a message contains binary data, Base64-encode the message.
• A queue may contain millions of messages, up to the total capacity limit of a storage account.
• Access to queue data via the Azure portal, PowerShell, or Azure CLI can be authorized either by using the user's Microsoft Entra account or by using the account access keys (Shared Key authorization).
• Authorization with Shared Key is not recommended as it may be less secure
• The password is the part of the Transport connection that needs to be safely quoted.
• Dedicated worker processes constantly monitor task queues for new work to perform.
• Celery communicates via messages, usually using a broker to mediate between.

Please let us know if you have any further queries. I’m happy to assist you further.     

---Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.