Is there a way to upload/use pre-labeled images to Custom Vision object detection?

Question

I have images that are already tagged with me. How can use it instead of loading images to the portal and labeling them manually again?

Answer 1

I have built a Python package called PyLabel that can be used to import pre-labelled images to Azure Custom Vision. You can see a proof of concept in this notebook https://github.com/pylabel-project/samples/blob/main/pylabel2azure_custom_vision.ipynb.

PyLabel can read annotations from COCO, YOLO, or VOC format into a dataframe. Once they are in the data frame you can loop through the dataframe of annotations and use the Custom Vision APIs to upload the images and annotations.

Here is a snippet of the code from the notebook mentioned above:

#Iterate the rows for each image in the dataframe
for img_filename, img_df in dataset.df.groupby('img_filename'):
    img_path = str(PurePath(dataset.path_to_annotations, str(img_df.iloc[0].img_folder), img_filename))
    assert exists(img_path), f"File does not exist: {img_path}"

    #Create a region object for each bounding box in the dataset 
    regions = []
    for index, row in img_df.iterrows():

        #Normalize the boundings box coordinates between 0 and 1
        x = Decimal(row.ann_bbox_xmin / row.img_width).min(1)
        y = Decimal(row.ann_bbox_ymin / row.img_height).min(1)
        w = Decimal(row.ann_bbox_width / row.img_width).min(1-x)
        h = Decimal(row.ann_bbox_height / row.img_height).min(1-y)

        regions.append(Region(
                tag_id=tags[row.cat_name].id, 
                left=x,
                top=y,
                width=w,
                height=h
            )
        )

    #Create an object with the image and all of the annotations for that image
    with open(img_path, mode="rb") as image_contents:
        image_and_annotations = [ImageFileCreateEntry(name=img_filename, contents=image_contents.read(), regions=regions)]

    #Upload the image and all annnotations for that image
    upload_result = trainer.create_images_from_files(
            project.id, 
            ImageFileCreateBatch(images=image_and_annotations)
        )

    #If upload is not successful, print details about that image for debugging 
    if not upload_result.is_batch_successful:
        print("Image upload failed.")
        for image in upload_result.images:
            print(img_path)
            print("Image status: ", image.status)
            print(regions)

#This will take a few minutes 
print("Upload complete")

Note, I am a Microsoft employee but I am not a member of the Azure Custom Vision team. But I think PyLabel can help with this task so I wanted to share this with you.

Answer 2

@Namita Menon Thanks for the question. If you would like to use the code, please follow the below doc for the same. https://github.com/microsoft/computervision-recipes/tree/master/scenarios/detection

Answer 3

@Namita Menon To answer your question you can upload existing images using the SDK but the co-ordinates of these images will vary based because custom vision uses normalized co-ordinates which needs a conversion tool to convert the co-ordinates and then upload them using the SDK. Here is some info of a similar discussion.

Normalized co-ordinates are calculated using the below methods for an area and the bounding boxes:

Normalized Left = Left / Width (in Pixels)
Normalized Top = Top / Height (in Pixels)
Normalized Bounding Box Width = (Right - Left) / Width (in Pixels)
Normalized Bounding Box Height = (Bottom - Top) / Height (in Pixels)

Once your co-ordinates are calculated you can use the SDK to upload them. The section upload and tag images in this document provides you an example to upload images with regions/boxes as calculated from above. Once these are uploaded you can confirm if they are tagged correctly from the portal.

From the portal though you can upload new images or use smart labeler instead.