Unable to create a directory named space character using ADLS Gen2 REST API

Question

I am unable to create a directory in "Azure Data Lake Storage Gen2" that is named as just a single space character, despite reviewing the documentation and there being no indication this is a disallowed name.

My primary access to "Azure Data Lake Storage Gen2" is using the Azure Data Lake Storage Gen2 REST API - most notably using the Hadoop ABFS File System provider.

In the REST API / ABFS, a dedicated operation to create a directory occurs. When it attempts the creation of the directory simply named " " (space) we get an exception.

2024-12-09 14:30:12.713  WARN - [action-pool-2       ] - a.b.c.RedactedClass           : Exception migrating /test-data/dir1/  - will retry:
org.apache.hadoop.fs.azurebfs.contracts.exceptions.AbfsRestOperationException: Operation failed: "The request URI is invalid.", 400, PUT, https://REDACTED.dfs.core.windows.net/REDACTED/test-data/dir1/%20?resource=directory&timeout=90, InvalidUri, "The request URI is invalid. RequestId:00000000-0000-0000-0000-000000000000 Time:2024-12-09T14:30:12.7006415Z"

As noted, using the BLOB API, such as using azcopy, this works perfectly fine, as the space just becomes encoded as part of the BLOBs name

2024/12/05 16:19:24 ==> REQUEST/RESPONSE (Try=1/104.067886ms, OpTime=228.146948ms) -- RESPONSE SUCCESSFULLY RECEIVED
   PUT https://REDACTED.blob.core.windows.net/REDACTED/dummy%2Fwp%2F%20%2Ffile?sdd=1&se=2024-12-06T00%3A13%3A02Z&sig=-REDACTED-&sp=racwdlmeop&sr=d&st=2024-12-05T16%3A13%3A02Z&sv=2022-11-02
   X-Ms-Request-Id: [00000000-0000-0000-0000-000000000000]

According to the documentation, it identifies and confirms that BLOB storage does support this provided it is correctly escaped.The ADLS Gen2 documentation lacks any direct statement of non/support for this, but does state that it is built on BLOB with a direct statement.> Most Blob Storage features are fully supported, but some features might be supported only at the preview level and there are a handful of them that aren't yet supported

Reviewing the Features page https://learn.microsoft.com/en-us/azure/storage/blobs/storage-feature-support-in-storage-accounts does not rule them in or out.

Reviewing the Known Issues page https://learn.microsoft.com/en-us/azure/storage/blobs/data-lake-storage-known-issues does not list it either as a known issue, or unsupported concept.

For all intent and purposes, there is an indication that this is a supportable concept.

Can anybody clarify this? Most ideally, have it documented?

Answer 1

Hi Phil R ,

Greetings & Welcome to Microsoft Q&A forum! Thanks for posting your query!

Based on the information you are provided, it appears that creating a directory with a single space character in ADLS Gen2 using the REST API or Hadoop ABFS File System provider is not supported, even though it works with Blob Storage.

The error message "The request URI is invalid" suggests that the space character in the directory name is causing issues with the URI encoding. While Blob Storage can handle this by encoding the space, ADLS Gen2 seems to have stricter URI validation rules.

Please consider below steps:

URI Encoding: Ensure that the space character is properly encoded as %20 in the URI. However, it seems you are already tried this and encountered the same issue.

Naming Conventions: ADLS Gen2 might have different naming conventions or restrictions compared to Blob Storage. The documentation does not explicitly state this, but the error message indicates a potential limitation.

Alternative Naming: Consider using an alternative naming convention that avoids special characters like spaces. For example, you could use underscores (_) or hyphens (-) instead.

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.           

If you have any other questions or are still running into more issues, let me know in the "comments" and I would be happy to help you