Azure container instance (ACI) was very slow when creating tremendous small files on an azure file share disk mount point

Question

Hi,

I created a linux azure container instance (aci), and mounted an azure file share disk to it in ARM when creating the ACI, the mount point was at "/mytest". The file share disk wasn't SSD, it's HDD, and it's "transaction-optimized" type.And then I have the following code to create tremendous small files in ACI.##### start

try:

os.makedirs(staging_path, exist_ok=True)

start_time1 = time.time()

for i in range(10000):

file_path = os.path.join(staging_path, f"file_{i}.txt")

with open(file_path, "w") as f:

f.write("abcdefghij") # 10 bytes of content

end_time1 = time.time()

total_time1 = end_time1 - start_time1

logger.log(ST_LOG, f"Total time to create 10,000 files: {total_time1:.2f} seconds")

except Exception as e:

logger.log(ST_ERROR, f"test: {e}")

##### end

It's very slow as the following for the file share disk mount point:

10/29 03:11:38 MSG "Total time to create 10,000 files: 602.12 seconds" - task_handler.py->process_unzip_request():117

But the interesting thing is, if I removed the mount point, and ran the above code in the disk allocated to ACI by default, which is 46 gb. The performance is very well as the following.

26 140221229517696 10/29 02:13:26 MSG "Total time to create 10,000 files: 0.75 seconds" - task_handler.py->process_unzip_request():117

Then my question: Why did the mounted file share disk so slow for above code? How to increase the default 46 gb allocated disk for ACI? Thanks.

Answer 1

@Jun Ge Apologies for the delay response.

• Have you considered using Azure Premium Files instead of standard Azure Files?
• Can you please share the screenshot of the metrics?

How to increase the default 46 GB allocated disk for ACI?

To increase the default allocated disk space for your Azure Container Instance, you can follow these steps:

1. Request a Quota Increase: Azure services include certain default limits and quotas for resources and features. You can request an increase in the quota for the allocated disk space by submitting an Azure support request
2. Use Premium Storage: Consider using premium storage options for your file shares, which offer higher IOPS and throughput compared to standard storage (OPS and throughput requirements: Premium file shares support larger IOPS and throughput limits than standard file shares. See file share scale targets for more information.)
3. Optimize File Operations: If possible, optimize your code to reduce the number of small file operations or batch them to minimize the overhead.
   Understand and optimize Azure file share performance

The performance issue you are experiencing when creating small files on the mounted Azure file share disk in your Linux Azure Container Instance (ACI) is likely due to the transaction-optimized type of the disk. This type of disk is optimized for handling large files and sequential I/O operations, rather than small files and random I/O operations. When you create a large number of small files, it can result in a large number of random I/O operations, which can cause performance issues on a transaction-optimized disk.

To improve the performance of creating small files on the mounted Azure file share disk, you can try using a different disk type that is optimized for random I/O operations, such as a general-purpose SSD or premium SSD disk. You can also try increasing the size of the disk to provide more space for the files and improve performance.

To increase the default 46 GB allocated disk for your ACI, you can specify a larger disk size when creating the ACI using the --storage-account-name and --storage-account-key parameters. For example, you can use the following command to create an ACI with a 100 GB disk:

az container create --resource-group <resource-group-name> --name <aci-name> --image <image-name> --cpu <cpu-count> --memory <memory-size> --azure-file-volume-account-name <storage-account-name> --azure-file-volume-account-key <storage-account-key> --azure-file-volume-share-name <share-name> --azure-file-volume-mount-path /mytest --os-type Linux --ip-address Public --ports <port-number> --storage-size 100

In this command, the --storage-size parameter specifies the size of the disk in GB. You can adjust this value as needed to meet your requirements.

Why is the Azure file share disk slow? The performance issues you're encountering with the Azure file share disk can be attributed to several factors:

1. IOPS and Throughput Limits: Azure file shares have specific limits on Input/Output Operations Per Second (IOPS) and throughput. When these limits are reached, the performance can degrade significantly. For example, transaction-optimized file shares have lower IOPS and throughput compared to premium file shares
   Troubleshoot Azure Files performance issues
2. Throttling: If the IOPS, ingress, or egress limits for the file share are exceeded, the Azure Files service may throttle the requests, resulting in poor performance.
3. File System Overhead: Creating a large number of small files can introduce significant overhead due to metadata operations, which can further slow down the performance

Additional information:
Mount an Azure file share in Azure Container Instances
Limitations

• Azure Storage doesn't support SMB mounting of file share using managed identity
• You can only mount Azure Files shares to Linux containers. Review more about the differences in feature support for Linux and Windows container groups in the overview.
• Azure file share volume mount requires the Linux container run as root.
• Azure File share volume mounts are limited to CIFS support.

If the issue still perists, I would like to work closer on this issue.

---

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.