Hi, I am trying to convert text data in "txt "column in azure sql db to vector data and saving the data in "txt_vector" attribute in index below, but I see nothing in index when I run the indexer. Does anyone give me a clue for where I got wrong on the code?? Here is the skillset and indexer. (Skillset) (indexer)

Hi, @REX_mentin Your indexer outputFieldMappings is not correct. In embedding skillset, you given context as /document and in the same context the target field will be populated. So, add mappings for embedding output like below. "outputFieldMappings":[ { "sourceFieldName":"/document/txt_vector", "targetFieldName":"txt_vector" }, ... ]

Index doesn't appear in the process of indexing and skillset

REX_mentin 0

Hi,

I am trying to convert text data in "txt "column in azure sql db to vector data and saving the data in "txt_vector" attribute in index below, 20-01-25 午後4.48の画像

but I see nothing in index when I run the indexer. 36C8A381-DB07-420B-AB5F-138C54DADAC6

Does anyone give me a clue for where I got wrong on the code?? Here is the skillset and indexer.

(Skillset) 25E1C2C3-7649-474A-B3DF-D5BF8EB06CC7

F9677FF8-9C03-4DEC-90A9-984CA481BB97

(indexer) 8F30D455-4746-4129-9A58-42F350192C4D

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-01-29T03:54:37.4233333+00:00
@REX_mentin Hi, the output field mappings should be like below

{ "sourceFieldName":"/document/txt_vector", "targetFieldName":"txt_vector" }

Please try above and let me know if it is working or not.

1 answer

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-01-30T06:13:18.6166667+00:00
Hi, @REX_mentin

Your indexer outputFieldMappings is not correct.

In embedding skillset, you given context as /document and in the same context the target field will be populated.

So, add mappings for embedding output like below.

"outputFieldMappings":[ { "sourceFieldName":"/document/txt_vector", "targetFieldName":"txt_vector" }, ... ]
Please sign in to rate this answer.
JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-01-31T07:12:40.95+00:00

Hi, @REX_mentin

Did you get a chance to try the above-mentioned resolution steps and see if it helped or not. Let us know if there is anything else we can help on. Thanks

REX_mentin 0 Reputation points

2025-02-03T03:01:38.3966667+00:00

@JAYA SHANKAR G S
Hi, I have edited the output field mapping like you have mentioned, but I still see nothing in the index like I previously posted.

REX_mentin 0 Reputation points

2025-02-03T03:12:24.69+00:00

Hi, thank you for your support. I have edited the code like you mentioned, but I still see nothing in the index.

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-03T03:25:03.4133333+00:00

@REX_mentin You reset and re-run indexer after making changes. This should re-index the data. check this documentation for more information.

After checking let me know if it's working or not.

REX_mentin 0 Reputation points

2025-02-03T03:44:16.15+00:00

@JAYA SHANKAR G S I have reset and re-run indexer and I got the failed status and encountered these errors and warnings.

REX_mentin 0 Reputation points

2025-02-03T04:01:50.9233333+00:00

@JAYA SHANKAR G S

I have reset and re-run indexer after making changes and I got the failed status.

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-03T06:31:37.41+00:00

@REX_mentin

The error says input to skill is empty, that is /document/txt is empty.

Make sure you have the data in it from source.

Also, the way you given /document/txt as context in other skillsets may causing the issue.

Change context like below in all of your skillset

"context":"/document"

and do output field mappings like below

"outputFieldMappings":[ { "sourceFieldName":"/document/translated_text", "targetFieldName":"translated_text" }, { "sourceFieldName":"/document/keyphrases", "targetFieldName":"keyphrases" } { "sourceFieldName":"/document/txt_vector", "targetFieldName":"txt_vector" } ...remaining all same ]

After making these changes reset and run the indexer.

REX_mentin 0 Reputation points

2025-02-03T06:58:46.82+00:00

I have changed the "context" in the skillset and updated output field mappings like above but the status hit the error still.

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-03T07:12:39.0366667+00:00

Is it the same error input to skill is empty, check the data source whether the data present for field /document/txt, please provide indexer and skillset definition in text format not as image so that i can check the configuration from my end.

REX_mentin 0 Reputation points

2025-02-03T10:36:30.76+00:00

I really appreciate your help. Here is the code for skillset data source, and indexer

<data source json>

{ "@odata.context": "https://ais-aisearch-blob.search.windows.net/$metadata#datasources/$entity", "@odata.etag": "\"0x8DD3F205A78505C\"", "name": "sql-datasrc-v", "description": null, "type": "azuresql", "subtype": null, "credentials": { "connectionString": "Data Source=tcp:server-demo-001.database.windows.net,1433;Initial Catalog=mySampleDatabase;User ID=...;Password=...;Connect Timeout=30;Encrypt=True;Trust Server Certificate=False" }, "container": { "name": "[sites]", "query": "" }, "dataChangeDetectionPolicy": { "@odata.type": "#Microsoft.Azure.Search.HighWaterMarkChangeDetectionPolicy", "highWaterMarkColumnName": "txt" }, "dataDeletionDetectionPolicy": null, "encryptionKey": null, "identity": null } ```<skillset> ```json { "@odata.etag": "\"0x8DD441F22EED874\"", "name": "azuresql-skillsetc-v", "description": "Skillset created from the portal. skillsetName: azuresql-skillsetc-v; contentField: txt; enrichmentGranularity: document; knowledgeStoreStorageAccount: ;", "skills": [ { "@odata.type": "#Microsoft.Skills.Text.TranslationSkill", "name": "#1", "context": "/document", "defaultToLanguageCode": "en", "suggestedFrom": "en", "inputs": [ { "name": "text", "source": "/document/txt", "inputs": [] } ], "outputs": [ { "name": "translatedText", "targetName": "translated_text" } ] }, { "@odata.type": "#Microsoft.Skills.Text.LanguageDetectionSkill", "name": "#2", "context": "/document", "inputs": [ { "name": "text", "source": "/document/txt", "inputs": [] } ], "outputs": [ { "name": "languageCode", "targetName": "language" } ] }, { "@odata.type": "#Microsoft.Skills.Text.KeyPhraseExtractionSkill", "name": "#3", "context": "/document", "defaultLanguageCode": "en", "inputs": [ { "name": "text", "source": "/document/txt", "inputs": [] }, { "name": "languageCode", "source": "/document/language", "inputs": [] } ], "outputs": [ { "name": "keyPhrases", "targetName": "keyphrases" } ] }, { "@odata.type": "#Microsoft.Skills.Text.AzureOpenAIEmbeddingSkill", "name": "#4", "description": "Connects a deployed embedding model.", "context": "/document", "resourceUri": "https://oai-ai-search-blob.openai.azure.com", "apiKey": "<redacted>", "deploymentId": "text-embedding-ada-002", "dimensions": 1536, "modelName": "text-embedding-ada-002", "inputs": [ { "name": "text", "source": "/document/txt", "inputs": [] } ], "outputs": [ { "name": "embedding", "targetName": "txt_vector" } ] } ] }

<indexer>

{ "@odata.context": "https://ais-aisearch-blob.search.windows.net/$metadata#indexers/$entity", "@odata.etag": "\"0x8DD443DC6598BB8\"", "name": "azuresql-indexer-v", "description": "", "dataSourceName": "sql-datasrc-v", "skillsetName": "azuresql-skillsetc-v", "targetIndexName": "azuresql-index-v", "disabled": null, "schedule": { "interval": "PT5M", "startTime": "2025-01-14T06:30:25Z" }, "parameters": { "batchSize": null, "maxFailedItems": 0, "maxFailedItemsPerBatch": 0, "base64EncodeKeys": false, "configuration": {} }, "fieldMappings": [], "outputFieldMappings": [ { "sourceFieldName": "/document/translated_text", "targetFieldName": "translated_text", "mappingFunction": null }, { "sourceFieldName": "/document/language", "targetFieldName": "language", "mappingFunction": null }, { "sourceFieldName": "/document/keyphrases", "targetFieldName": "keyphrases", "mappingFunction": null }, { "sourceFieldName": "/document/txt_vector", "targetFieldName": "txt_vector", "mappingFunction": null } ], "cache": null, "encryptionKey": null }

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-03T13:40:19.46+00:00

Hi @REX_mentin ,

You configuration is perfect and working fine.

you query index in Json view like below.

The error because of empty/null data in index txt field.

So only the records having data in txt fields indexed and gives warning for empty/null values, you remove those records and try re-indexing the data to avoid warnings.

REX_mentin 0 Reputation points

2025-02-04T03:20:29.6433333+00:00

@JAYA SHANKAR G S

Hi, thank you for the consistent support.

I have deleted the record which has null in the txt column by query editor in the sql db and reset and rerun the indexer but index still says there are 8 documents (not 5 ) and it still have no vectorised data in json view. When json query editor is

{ "search": "*", "count": true, "vectorQueries": [ { "kind": "text", "text": "*", "fields": "txt_vector" } ] }

When json query editor is

{ "search": "*", "count": true }

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-04T05:58:05.55+00:00

When you do vector query on empty vector field you will get empty results. Please add the warning or error details from indexer for the latest run.

REX_mentin 0 Reputation points

2025-02-04T09:56:32.1366667+00:00

All warning disappeared and now I got only one error.

Also, what does it mean, "When you do vector query on empty vector field" ?

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-04T10:25:57.4233333+00:00

Web api response is unauthorised. Make sure you giving the embedding skill apikey is correct. Regarding vector query, you have empty vector and the vectorQueries you given in first json view makes search on that empty vector field which gives empty results,later when you made only text search it gave proper results. Please update the apikey everytime you make changes in skillset because it will be saved as redacted literal value.

JAYA SHANKAR G S 245 Reputation points Microsoft Vendor

2025-02-05T07:55:16.8466667+00:00

Hi @REX_mentin , is the issue resolved? Let me know if you have any query.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.