@adityaTangirala-4084 Unfortunately. Like any preview feature, once you turn it on you cannot remove it.
That is even more true with the AKS version since Microsoft doesn’t allow you to downgrade version of AKS this would be too risky.
What you can do it that you can create a second cluster on the side and and redeploy all of your manifests to have an exact copy.
You can use this tool. It should help with this task.
Hoping this helps you out.
Please Accept as answer if it helped, so that it can help others in the community looking for help on similar topics.