Vmware inventory not up-to-date in Azure Arc due to SyncFailed

Brian Holm - Admin 0

vCenter inventory is not up-to-date in Azure Arc. I cannot see newly create machines, resources, networks, etc from Vmware vCenter in Azure Arc portal. However, the integration through the resource-bridge works fine on already connected machines.

Looking into "Resource Health" it states "vCenter health is degraded right now due to SyncFailed" which probably causes the problem.

Anybody who knows how to solve this?

Brian Holm - Admin 0

Where are some content from Arc appliance logfile:

{"LogLevel":"ERROR","Timestamp":"2024-10-07T07:17:39.595Z","Name":"controllers.VCenter","Message":"recovered in refreshInventory","resource":{"name":"8c18b543-f5be-4c56-a1c9-07764b444da9","namespace":"DC"},"OperationID":"a3a18940-fa45-5f88-802d-b2215ff630a2","CorrelationID":"8c4cdg13-1a1a-42fb-ba73-efc59d29a161","state":"Ok","phase":"Ok","error":"error is runtime error: invalid memory address or nil pointer dereference, stack trace is goroutine 469 [running]:\nruntime/debug.Stack()\n\t/usr/local/go/src/runtime/debug/stack.go:24 +0x5e\nmsazure.visualstudio.com/DefaultCollection/One/_git/AzureArc-VMwareOperator.git/src/VMwareOperator/controllers/vcenter.refreshInventory.func1()\n\t/__w/1/s/src/VMwareOperator/controllers/vcenter/vcenter_controller_inventory_helpers.go:225 +0x69\npanic({0x1d14520?, 0x35b7e00?})\n\t/usr/local/go/src/runtime/panic.go:770 +0x132\nmsazure.visualstudio.com/DefaultCollection/One/_git/AzureArc-VMwareOperator.git/src/pkg/vsphere.(*virtualMachineOperations).populateDiskBacking(0x1cfcc40?, 0xc002441230?, {0x2441f20?, 0xc001b149a0?}, {{0x2462640?, 0xc000860690?}, 0xc0052f7e10?})\n\t/__w/1/s/src/pkg/vsphere/virtualmachine_operations.go:945 +0x7fe\nmsazure.visualstudio.com/DefaultCollection/One/_git/AzureArc-VMwareOperator.git/src/pkg/vsphere.(*virtualMachineOperations).getVirtualDisks(0xc00073b038, {0xc00061c000, 0x12, 0xa?}, 0xc002441230, {{0x2462640?, 0xc000860690?}, 0x0?})\n\t/__w/1/s/src/pkg/vsphere/virtualmachine_operations.go:1019 +0x2b2\nmsazure.visualstudio.com/DefaultCollection/One/_git/AzureArc-VMwareOperator.git/src/pkg/vsphere.(*virtualMachineOperations).getVMProperties(_, {_, _}, {{{{{...}, {...}}, {0x0, 0x0, 0x0}, {0x0, 0x0, ...}}, ...}, ...}, ...)\n\t/__w/1/s/src/pkg/vsphere/virtualmachine_operations.go:338 +0x799\nmsazure.visualstudio.com/DefaultCollection/One/_git/AzureArc-VMwareOperator.git/src/pkg/vsphere.(*virtualMachineOperations).newVirtualMachineView(...)\n\t/__w/1/s/src/pkg/vsphere/virtualmachine_operations.go:70

Pavan Minukuri 915 Reputation points Microsoft Vendor

2024-10-09T17:30:06.85+00:00

Hi @Brian Holm - Admin

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

Below are the steps for resolve Sync Failed issues in Azure Arc for VMware vCenter.

1.Check vCenter and network: Please check vCenter is running from the azure arc resources bridge.

2.Review the resources logs: Use Azure Portal logs to identify sync-related errors.

3.Manual Sync: Trigger a Manual sync in Azure portal and monitor logs.

4.Update Resource Bridge: Please check the resource bridge is up to date.

5.CheckvCenter Health: Ensure vCenter inventory is healthy with no stale resource or certificate issue.

If you have any further queries, do let us know.

If the answer is helpful, please click "Accept comment" and "Upvote it".
Pavan Minukuri 915 Reputation points Microsoft Vendor

2024-10-10T20:58:58.61+00:00

Hi @Brian Holm - Admin
If you had a chance to see my comment to your question.
If it was helpful, please click "Upvote" on my post let us know.
Thank you...!
Brian Holm - Admin 0 Reputation points

2024-10-29T12:17:34.9966667+00:00

Thanks for all the suggestions. Unfortunately none of them solved the problem. Problem still persists.
Pavan Minukuri 915 Reputation points Microsoft Vendor

2024-10-30T18:31:41.88+00:00
Hi @Brian Holm - Admin
Sorry for inconvenience,
It seems like you are experiencing an issue with the vCenter inventory not being up-to-date in Azure Arc and you are seeing a "SyncFailed" error in the Resource Health. This error is likely causing the problem with not being able to see newly created machines, resources, networks, etc. from VMware vCenter in the Azure Arc portal.

To resolve this issue, you can try the following steps:

Check the connectivity between the vCenter and the Azure Arc enabled servers. Ensure that the servers are able to communicate with the vCenter and that the necessary ports are open.

Check the credentials used to connect to the vCenter. Ensure that the credentials are correct and have the necessary permissions to access the vCenter inventory.

Check the logs for the Azure Arc enabled servers. Look for any errors related to the vCenter integration and try to troubleshoot those errors.

Restart the Azure Arc enabled servers. This may help to resolve any issues with the vCenter integration.

If none of these steps resolve the issue, you can try contacting Microsoft support for further assistance. They may be able to provide additional troubleshooting steps or escalate the issue to the appropriate team for resolution.

I hope this helps! Let me know if you have any further questions.
Pavan Minukuri 915 Reputation points Microsoft Vendor

2024-11-01T04:41:13.1633333+00:00

Hi @Brian Holm - Admin
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment.
Thank you.!
Pavan Minukuri 915 Reputation points Microsoft Vendor

2024-11-04T23:05:45.1033333+00:00

Hi @Brian Holm - Admin
Just want to check is that issue resolved or else if you need any help, please let us know we will always help as you needed.! if you had chance please review once my previous comment. Thank you.!
Brian Holm - Admin 0 Reputation points

2024-12-10T14:52:57.3133333+00:00

Sorry for the late response. The issue was solved by MS. They did find an issue in the west-europe Azure region that triggered the sync issues in our tenant. They did some changes and after that, our sync issues disappeared and sync of VMware inventory now works again.
Pavan Minukuri 915 Reputation points Microsoft Vendor

2024-12-11T19:26:33.3833333+00:00

Hi @Brian Holm - Admin
Nice to hear, Thanks for replying back.
Could you please share the solution here? If I post your solution, you can accept that answer because you own answer you cannot accept it yourself, I would appreciate it if you could share your solution so that I can post it and you can accept it for other community member reference.

Share via

Vmware inventory not up-to-date in Azure Arc due to SyncFailed

Your answer