Load H5 file saved in Blob Storage

Question

Hi all,

I am trying to load my model from file I saved in Azure blob storage.

This is the function that downloads the file to my local machine:

def download_weights_to_temp_file():
    """
    Downloads the model weights from Azure Blob Storage to a temporary file.
    Returns the local path to the temporary file.
    """
    try:
        # Authenticate with Azure
        default_credential = DefaultAzureCredential()
        blob_service_client = BlobServiceClient(account_url, credential=default_credential)
        blob_client = blob_service_client.get_blob_client(container=CONTAINER_NAME_MODEL, blob=MODEL_BLOB_NAME)
        
        print(f"Downloading model weights from: {blob_client.url}")

        # Create a temporary file
        with tempfile.NamedTemporaryFile(delete=False, suffix=".h5") as temp_file:
            temp_file.write(blob_client.download_blob().readall())
            temp_model_path = temp_file.name

        print(f"Model weights downloaded to: {temp_model_path}")

        if os.path.exists(temp_model_path):
            print(f"Downloaded file exists at: {temp_model_path}")
        else:
            print("Error: Downloaded file does not exist.")

        return temp_model_path
    except Exception as e:
        print(f"Error downloading model weights: {e}")
        raise

And this is the function that loads my model:

def load_model():
    temp_model_path = download_weights_to_temp_file()
    try:
        # Create the model architecture
        model= create_enet_model(input_shape=(256, 256, 1), num_classes=1)
        model.compile(optimizer="adam", loss=weighted_binary_crossentropy, metrics=["accuracy"])
        model.load_weights(temp_model_path)
        print(f"Successfully loaded model weights from: enet_light_fold_4.weights.h5")

    except Exception as e:
        print(f"Error loading model weights: {e}")
        raise
    finally:
        # Clean up the temporary file
        if os.path.exists(temp_model_path):
            os.remove(temp_model_path)

    return model

However, I get following error:

File "C:\Users\Amanda\anaconda3\envs\tf17\lib\site-packages\keras\src\utils\traceback_utils.py", line 122, in error_handler raise e.with_traceback(filtered_tb) from None File "C:\Users\Amanda\anaconda3\envs\tf17\lib\site-packages\keras\src\legacy\saving\legacy_h5_format.py", line 357, in load_weights_from_hdf5_group raise ValueError( ValueError: Layer count mismatch when loading weights from file. Model expected 13 layers, found 0 saved layers.

But if I run my code with the original file I have, it works fine. I imagine it is something when reading the file from Azure.

Answer 1

Hi ,

Thanks for reaching out to Microsoft Q&A.

A common cause of this exact error (“Layer count mismatch … found 0 saved layers”) is that the H5 file being loaded is effectively empty or corrupted from the perspective of the HDF5 loader. Even though you are writing the blob data out, on Windows especially, if the temporary file is opened in text mode instead of binary mode, you can end up with a corrupted file.

Try adjusting your code that creates the temp file. On Windows, you should explicitly use mode="wb" with NamedTemporaryFile to avoid writing bytes in text mode. By specifying mode="wb", we ensure that Python does not transform line endings or otherwise treat the file as text. On Windows, this can be crucial.

import tempfile
import os
from azure.identity import DefaultAzureCredential
from azure.storage.blob import BlobServiceClient
def download_weights_to_temp_file():
    """
    Downloads the model weights from Azure Blob Storage to a temporary file.
    Returns the local path to the temporary file.
    """
    try:
        # Authenticate with Azure
        default_credential = DefaultAzureCredential()
        blob_service_client = BlobServiceClient(account_url, credential=default_credential)
        blob_client = blob_service_client.get_blob_client(
            container=CONTAINER_NAME_MODEL, 
            blob=MODEL_BLOB_NAME
        )
        
        print(f"Downloading model weights from: {blob_client.url}")
        # Create a temporary file in binary mode
        with tempfile.NamedTemporaryFile(delete=False, suffix=".h5", mode="wb") as temp_file:
            temp_file.write(blob_client.download_blob().readall())
            temp_file.flush()  # Make sure everything is written to disk
            temp_model_path = temp_file.name
        print(f"Model weights downloaded to: {temp_model_path}")
        if os.path.exists(temp_model_path):
            print(f"Downloaded file exists at: {temp_model_path}")
        else:
            print("Error: Downloaded file does not exist.")
        return temp_model_path
    except Exception as e:
        print(f"Error downloading model weights: {e}")
        raise

Sometimes your blob download might end up being an empty file. As a quick debug, check the file size before attempting to load:

file_size = os.path.getsize(temp_model_path)
print(f"Size of downloaded H5 file: {file_size} bytes")
if file_size == 0:
    print("The downloaded file is empty! Something went wrong with the download.")

If you used something like model.save('my_model.h5') originally (which saves both architecture and weights by default), you must load it with keras.models.load_model('my_model.h5').

If, on the other hand, you used model.save_weights('my_weights.h5'), then you load it with the below, Make sure you are consistent about which approach you use. Mismatching “full model” vs. “weights only” can also cause the “found 0 saved layers” error.

model = create_enet_model(input_shape=(256, 256, 1), num_classes=1)
model.compile(...)
model.load_weights('my_weights.h5')

Please feel free to click the 'Upvote' (Thumbs-up) button and 'Accept as Answer'. This helps the community by allowing others with similar queries to easily find the solution.