Unable to use Custom Vision Model with latest library "Azure.AI.Vision.ImageAnalysis"

Deepankar 60

Hi,

In the latest update of the library, I am unable to use the Analyze method or pass my custom model name through the endpoint to get the confidence score or other data for the image URI I am analyzing. Can someone provide the latest syntax for using a custom AI Vision Model deployed in C#?

I am unable to find the syntax on the Microsoft portal. It mentions passing the model name in the endpoint as a Model-name parameter for a custom vision model, but I need guidance on how to use the analyze function to get the confidence score of the image I am passing. The function requires both the feature parameter and the options parameter.

Ketsha 250 Reputation points Microsoft Employee

2024-12-19T16:27:40.5433333+00:00
Hi Deepankar -

To use the Analyze method with a custom AI Vision Model in C#, you’ll need to ensure you’re passing the correct parameters, including the model name, feature, and options. Here’s a basic example of how you can achieve this:

Set up your client:

var client = new ComputerVisionClient(new ApiKeyServiceClientCredentials("<Your API Key>"))

{ Endpoint = "<Your Endpoint>"};

Define the parameters:

string imageUri = "<Your Image URI>";

string modelName = "<Your Custom Model Name>";

var features = new List<VisualFeatureTypes?> { VisualFeatureTypes.Objects };

var options = new ImageAnalysisOptions

{

ModelVersion = modelName

};

Call the Analyze method:

var analysisResult = await client.AnalyzeImageAsync(imageUri, features, options);

Extract the confidence score:

foreach (var obj in analysisResult.Objects)

{

Console.WriteLine($"Object: {obj.ObjectProperty}, Confidence: {obj.Confidence}");

}

In this example, replace "<Your API Key>", "<Your Endpoint>", "<Your Image URI>", and "<Your Custom Model Name>" with your actual values. The AnalyzeImageAsync method takes the image URI, a list of features to analyze, and options that include your custom model name.
Deepankar 60 Reputation points

2024-12-19T17:00:57.87+00:00

I am getting error "The new Azure.AI.Vision.ImageAnalysis does not exist in current version for VisualFeatureType and similarly for the Image Analysis options.

I have asked the same question to copilot and it is telling that these classes have been removed from the recent package Azure.AI.Vision.ImageAnalysis. Can you please provide the correct method and syntax to be used in this latest package?
Ketsha 250 Reputation points Microsoft Employee

2024-12-19T17:22:09.2233333+00:00
I see what you are saying. The classes and methods you mentioned, such as VisualFeatureType and ImageAnalysisOptions, have been removed or updated in the recent package. The other option is as follows:

To use the latest syntax for analyzing images with a custom AI Vision Model in C#, you can follow these updated steps. I am positive that this should work, please let me know the outcome.

Set up your client:

var client = new ImageAnalysisClient(new Uri("<Your Endpoint>"), new AzureKeyCredential("<Your API Key>"));

Define the parameters:

string imageUri = "<Your Image URI>";

string modelName = "<Your Custom Model Name>";

Call the Analyze method:

var analysisOptions = new ImageAnalysisOptions

{

ModelVersion = modelName, Features = new List<ImageAnalysisFeature> { ImageAnalysisFeature.Objects }

};

var analysisResult = await client.AnalyzeImageAsync(new Uri(imageUri), analysisOptions);

Extract the confidence score:

foreach (var obj in analysisResult.Objects)

{

Console.WriteLine($"Object: {obj.Name}, Confidence: {obj.Confidence}");

}

In this example, replace <Your Endpoint>, <Your API Key>, <Your Image URI>, and <Your Custom Model Name> with your actual values. The AnalyzeImageAsync method now takes a URI and an ImageAnalysisOptions object that includes your custom model name and the features to analyze.

I hope this helps! Let me know if you have any further questions or need additional assistance.
Deepankar 60 Reputation points

2024-12-19T17:53:58.0466667+00:00

Now I am getting ImageAnalysisFeature not found error.

I have tried using Visual Feature but again I am getting the error that ImageAnalysisOptions does not contain a definition of Features.

But I have the latest version installed. I am using .Net 9 and Version 1.0.0 of Azure.AI.Vision.ImageAnalysis
Deepankar 60 Reputation points

2024-12-20T06:35:21.61+00:00

I have checked the decompiled version of ImageAnalysisOptions structure, it does not contain any definition related to Features in the latest version. See the ss below:

navba-MSFT 26,890 Microsoft Employee

@Deepankar Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

Could you please try with the below nuget packages and check if that helps ?

Computer Vision	Microsoft.Azure.CognitiveServices.Vision.ComputerVision	GitHub
Computer Vision	Microsoft.Azure.CognitiveServices.Vision.ComputerVision	GitHub
Custom Vision Prediction	Microsoft.Azure.CognitiveServices.Vision.CustomVision.Prediction	GitHub
Custom Vision Training	Microsoft.Azure.CognitiveServices.Vision.CustomVision.Training	GitHub

More info here.

Hope this helps.

navba-MSFT 26,890 Reputation points Microsoft Employee

2024-12-20T07:43:27.7666667+00:00

@Deepankar Adding on to the above, the sample code for Custom Vision, both ImageClassification and ObjectDetection is available here.
Deepankar 60 Reputation points

2024-12-20T07:51:38.13+00:00

No, I am sorry, but this does not fulfill my objective. I have my own custom-trained and deployed model, and I simply want to send the image to the model using the key and endpoints to get the confidence score and others value.

However, the available package "Azure.AI.Vision.ImageAnalysis" is not very helpful, and many properties have been removed. Could you please provide sample code that works with the latest available version .NET 9 and ImageAnalysis package version 1.0.0? I am using C# as my programming language.

I believe there might be some bugs or missing properties in the latest Azure.AI.Vision.ImageAnalysis package.
Deepankar 60 Reputation points

2024-12-21T08:12:48.59+00:00
I have tried using Rest API for custom model. Can you confirm what is going wrong which is throwing error that "Resource Not Found 404"

@Ketsha @navba-MSFT

I have checked the endpoint link and key - All are correct.

I have even tried api-version=2024-02-01

navba-MSFT 26,890 Microsoft Employee

@Deepankar Thanks for getting back. I tried with the below sample using SDK and it worked fine.

using Azure;
using Azure.AI.Vision.ImageAnalysis;

private static readonly string key = "AJOJLXXXXXXXXCOGSb9F";
private static readonly string endpoint = "https://XXX.cognitiveservices.azure.com/";

ImageAnalysisClient client = new ImageAnalysisClient(
new Uri(endpoint),
new AzureKeyCredential(key));

ImageAnalysisResult result = client.Analyze(
    new Uri("https://learn.microsoft.com/azure/ai-services/computer-vision/media/quickstarts/presentation.png"),
    VisualFeatures.Caption | VisualFeatures.Read,
    new ImageAnalysisOptions { GenderNeutralCaption = true });

Console.WriteLine("Image analysis results:");
Console.WriteLine(" Caption:");
Console.WriteLine($"   '{result.Caption.Text}', Confidence {result.Caption.Confidence:F4}");

Console.WriteLine(" Read:");
foreach (DetectedTextBlock block in result.Read.Blocks)
    foreach (DetectedTextLine line in block.Lines)
    {
        Console.WriteLine($"   Line: '{line.Text}', Bounding Polygon: [{string.Join(" ", line.BoundingPolygon)}]");
        foreach (DetectedTextWord word in line.Words)
        {
            Console.WriteLine($"     Word: '{word.Text}', Confidence {word.Confidence.ToString("#.####")}, Bounding Polygon: [{string.Join(" ", word.BoundingPolygon)}]");
        }
    }

User's image

If you wish to make your application work only with the REST API, then please try with the api-version and 2023-10-01

Hope this helps.

Deepankar 60 Reputation points

2024-12-21T19:35:13.2+00:00

Thanks @navba-MSFT , but again this snippet does not fulfil my requirement where I wanted to use my custom own model prepared. Can you please help where shall I pass my model-name in this snippet, so that I can analyze the image with my own prepared custom model? I cannot find the method with this latest version, where I shall be passing my own prepared custom model name.

Currently, this is using prebuilt model for image analysis.

If I have to pass in the endpoint, but what will be its syntax? I tried with the rest API syntax as well where it require model name to be passed in the endpoint itself, but here it does not work.
Deepankar 60 Reputation points

2024-12-22T08:23:47.25+00:00

I am able to achieve my requirement through Rest API using the below endpoint syntax:

string requestUrl = $"{endpoint}/computervision/imageanalysis:analyze?model-name={modelName}&api-version=2023-04-01-preview";

This is the syntax which is working for my requirement. As of now, I think .Net SDK is not allowing to use custom model. The only way to achieve is to through REST API.

Accepted answer

navba-MSFT 26,890 Reputation points Microsoft Employee

2024-12-24T06:47:38.37+00:00

@Deepankar Thanks for getting back. I had a discussion internally with our Product owners.

.

Please note that the SDK does not support custom models as of now. There is no ETA on when this would be supported. We will update this thread and our documentation once the support is added.

.

Workaround:

The only workaround available as of now is to leverage and invoke the REST API for custom model.

.

Hope this answers.

**

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.

0 comments No comments
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Unable to use Custom Vision Model with latest library "Azure.AI.Vision.ImageAnalysis"

0 additional answers

Your answer