Is it possible to dynamically change the path in the backend section of API management service?

Question

Hello, I am trying to rewrite the uri of an API request to our Open AI instance.

Basically, we want to be able to change the backend path from

https://{our-backend}/openai/deployments/{deployment1}/chat/completions

to

https://{our-backend}/openai/deployments/{deployment2}/chat/completions

in the <backend /> section of our API if the first call fails.

Is there a way to do this? We know <rewrite-uri> doesn’t work in the backend section.

Answer 1

Hi @alex c,

Welcome to the Microsoft Q&A Platform! Thank you for asking your question here.

To change the backend URL based on whether an API call succeeds or fails, you need to use some conditional logic and a retry mechanism in your API gateway or custom code. Since you can't use certain features in the backend section, here’s a straightforward way to do this using simple steps:

• Start by setting up your main and backup backend URLs as variables. You can do this in your API gateway settings or directly in your code.

primary_url = "https://{our-backend}/openai/deployments/{deployment1}/chat/completions"

secondary_url = "https://{our-backend}/openai/deployments/{deployment2}/chat/completions"

• Write a function that makes the API call. This function should take the URL as a parameter. Initially, set it to call the primary URL. After the first call, check the response. If the call to the primary URL fails (based on the response status), try calling the secondary URL.
• Process the response according to your application’s needs. If both calls fail, make sure to log the error or send a notification.
• Deploy these changes to your API gateway or server. Test the setup by simulating failures to ensure that it correctly switches to the secondary URL when needed.

This approach provides a simple and effective way to switch between different backend URLs based on the success or failure of API calls, using standard programming and API management techniques.

If you find the answer helpful, kindly click "Accept Answer" and upvote it. If you have any further questions or concerns, please feel free to reach out to us. We are happy to assist you.

Answer 2

Hi Shireesha,

Thank you for the advice. This is exactly what we’re trying to accomplish, but can you provide a code snippet or the policy that allows us to call the secondary url (i.e. instead of <rewrite-uri /> we can use…?)

Thanks again.