JsonReaderException when trying to connect/use Azure OpenAI assistants

Question

Hey!

I'm working on a Azure OpenAI project having to do with multiple AI assistants.

I've created them in Azure and I'm calling them from my Blazor Web Application project.

Seemingly out of nowhere, without any change to configuration all my assistants stopped responding.

Even when trying to communicate with an assistant in Azure Assistant Playground I'm receiving no answer. My resource location is eastus2.

This is what's being thrown:

Error: System.Text.Json.JsonReaderException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. LineNumber: 0 | BytePositionInLine: 0.

Answer 1

Hi Mateusz Rzepnicki (MTZP)!

Sorry for the inconvenience.

Below error message indicates that either the input from server is not received or got a malformed response not containing Json data.

Error: System.Text.Json.JsonReaderException: The input does not contain any JSON tokens. Expected the input to start with a valid JSON token, when isFinalBlock is true. LineNumber: 0 | BytePositionInLine: 0.

Could you check for any exceptions (429, 403,500 etc.) in respective OpenAI deployment through SDK or portal. If you are catching some exceptions which won't be be valid Json file, it is expected to show above message.

You can catch those exception if it has been customized through code.

Reference thread

Remedial for 429 (Rate limit failures)- increase max_token in code/ TPM in model deployment, adopt retry logic in code.

Reference thread

Remedial for 500 (internal server error)-

Create a backup model deployment regions and route traffic to available regions in case one deployment is not available. You can check for ongoing outages here in Azure status and Service health (subscription and customer specific)

Change API version of model

Leverage APIM or load balancer to route traffic to different API based on service health and traffic.

Reference for APIM/Loadbalancer

Remedial for 403 (Authorization and Authentication issues)

1. Check for change in RBAC roles (Cognitive Services OpenAI contributor to use API keys and create deployments)
2. . Network authentication- Please check whether you are in same virtual network if it is protected with virtual network. Resource Firewall should have allowed Azure OpenAI resource as trusted resources. You have to use managed identity for OpenAI and other resources. Reference on using Virtual networks
3. Take a network trace and check in activity logs for causes.

Hope it helps.

Thank you.

Answer 2

Thank you for this detailed description.

When I logged in today the problem was gone but I'll save this info for the future. Thanks :)