SimilarProduct Insight not working in visual search API

Martin Meyer 0

i have this trying to find similiar products for an image of an product. Following the documentation
i just need put "SimilarProducts" in the invodedSkills of the api call
but i not get any answer with this code

endpoint = "https://api.bing.microsoft.com/v7.0/images/visualsearch"

image_url = "https://m.media-amazon.com/images/I/71CBk3JtF6L.__AC_SY300_SX300_QL70_ML2_.jpg"

# Specify the image insights token
knowledge_request = {
    "imageInfo": {
        "url": image_url  # Provide the URL of the image you want insights for
        #"imageInsightsToken": "ccid_cixTCTqz*cp_D7F0A068A89FA15BECFF781E1353DC4F"

    },
    "knowledgeRequest" : {


    "invokedSkills": ["SimilarProducts"],
    
}

    }


# Set the unique boundary
boundary = "boundary_1234abcd"

# Prepare the headers with multipart form-data
headers = {
    "Ocp-Apim-Subscription-Key": subscription_key,
    "Content-Type": f"multipart/form-data; boundary={boundary}"
}

# Format the multipart form-data body
body = (
    f"--{boundary}\r\n"
    f"Content-Disposition: form-data; name=\"knowledgeRequest\"\r\n\r\n"
    f"{json.dumps(knowledge_request)}\r\n"
    f"--{boundary}--"
)

# Send the POST request
response = requests.post(endpoint, headers=headers, data=body)

i get this response for an amazon product image

{"_type": "ImageKnowledge", "instrumentation": {"_type": "ResponseInstrumentation"}, "tags": [{"displayName": "", "actions": []}], "image": {"imageInsightsToken": "ccid_cixTCTqz*cp_D7F0A068A89FA15BECFF781E1353DC4F"}, "debugInfo": {}}

navba-MSFT 25,765 Microsoft Employee

@Martin Meyer Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

.
I tried to paste your image url in below and it worked fine:

https://www.bing.com/visualsearch

User's image

Could you please use the "invokedSkills": ["SimilarImages"] and test ?

I am sharing my working sample code here if that helps ?

import json
import requests
from pprint import pprint

'''
This sample uses the Bing Visual Search API to find similar images based on a provided image URL.
'''

# Add your Bing Search V7 subscriptionKey and endpoint.
endpoint = "https://api.bing.microsoft.com/v7.0/images/visualsearch"
subscription_key = "94d4XXXXXXXX7b142"

# Specify the image URL
image_url = "https://m.media-amazon.com/images/I/71CBk3JtF6L.__AC_SY300_SX300_QL70_ML2_.jpg"

# Prepare the headers
headers = {
    'Ocp-Apim-Subscription-Key': subscription_key,
    'Content-Type': 'multipart/form-data'
}

# Prepare the knowledge request
knowledge_request = {
    "knowledgeRequest": {
        "invokedSkills": ["SimilarImages"]
    }
}

# Prepare the file upload
files = {
    'image': ('image.jpg', requests.get(image_url).content)  # Download the image content
}

# Call the API
try:
    response = requests.post(endpoint, headers=headers, files=files, data={'knowledgeRequest': json.dumps(knowledge_request)})
    response.raise_for_status()

    print("\nHeaders:\n")
    print(response.headers)

    print("\nJSON Response:\n")
    pprint(response.json())
except Exception as ex:
    print(f"An error occurred: {ex}")

Hope this helps.

Martin Meyer 0 Reputation points

2024-11-07T07:48:05.35+00:00
@navba-MSFT Thanks for the answer!

the SimilarImages is working for me, but I am especially looking to also get the response object "aggregateOffer" with price and rating for each similar images.

Is there any other way to get the response object "aggregateOffer" https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/reference/response-objects ?

The invokedSkill "SimilarProducts" is mentioned here in the docu. https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/how-to/get-insights

The ImageInfo object includes the following fields:

url — The URL to the Internet-accessible image that you want insights of. The maximum supported image size is 1 MB. Read more.

imageInsightsToken — The insights token that Image Search API returned for an image. Read more.

cropArea — The bounding box that identifies the area of interest in the image. Read more.

The url and imageInsightsToken fields are mutually exclusive. If the request uploads the image binary in the body of the request, don’t includes the url and imageInsightsToken fields.

The KnowledgeRequest object includes the following fields:

filters — A Filter object that you use to filter the results.

site — The site filter returns similar images and similar products results from the specified site only. Read more.

invokedSkillsRequestData — A SkillsRequestData object that you use to request additional insights.

enableEntityData — A Boolean value that if true, returns information about the well-known person, place, or thing seen in the image. Read more.

invokedSkills — A list of insights to invoke. The string array may include only SimilarImages or SimilarProducts but not both. Include this field if you want to page visually similar insights or images of visually similar products. Read more. Specifying the image's information
navba-MSFT 25,765 Reputation points Microsoft Employee

2024-11-08T07:24:09.19+00:00

@Martin Meyer I am checking this internally. I will get back to you once I have an update on this.
Martin Meyer 0 Reputation points

2024-11-19T11:30:28.39+00:00

@navba-MSFT Did you manage to find any more information regarding this issue? If not, I may need to start evaluating other options.
navba-MSFT 25,765 Reputation points Microsoft Employee

2024-11-19T11:54:37.8433333+00:00

@Martin Meyer I heard back from the Product Owners. They are unable to reproduce this issue.
.

This is their response:

We’re seeing similar products alongside similar images, which is expected behavior. It's expected to see similar imager and products rather than only getting exact images and products.

.

Also, For example, If the user is shopping for a black cocktail dress and wants to see if the store they shop at carries it, use the site filter. The site filter restricts the similar images and similar products results to a specific domain.

Please refer for more details: - https://learn.microsoft.com/en-us/bing/search-apis/bing-visual-search/how-to/get-insights#filtering-similar-images-and-products
Martin Meyer 0 Reputation points

2024-11-19T13:56:00.34+00:00

@navba-MSFT Thanks for researching this !

Could you provide an example request body where it is working like described with similar images and products?
navba-MSFT 25,765 Reputation points Microsoft Employee

2024-11-20T04:33:00.63+00:00

@Martin Meyer Sure. I will share it once I get it from the Product owners.
navba-MSFT 25,765 Reputation points Microsoft Employee

2024-11-24T12:08:34.74+00:00

@Martin Meyer Attaching the Request headers used and the response headers received in txt file.

Request.txt
Response.txt

Share via

SimilarProduct Insight not working in visual search API

Your answer