Server Sent Events not working in App Service (Linux, containers)

Question

We have an app running with custom containers using docker compose in Azure App Service (Linux plan). One of the containers expose an HTTP API endpoint that make call to Azure OpenAI Service and stream the response (text/event-stream mime-type). So the workflow is:

• Browser make a request to our API.
• API craft a request to Azure OpenAI.
• API streams Azure OpenAI back to browser using Server Sent Events.

It's working fine using the same docker compose and containers on local machines. But when we deploy to App Service it doesn't work as expected, it doesn't start downloading the stream until the response is fully completed. Streaming from Azure OpenAI to our container works fine, but not from our container to user's browser when deployed on App Service, we believe that there's an internal proxy buffering the streamed data on App Service side.

Expected behaviour, reasonable delay until getting the first bytes from the streamed response:

Same request when deployed in Azure App Service:

As you can see the response is not streamed until completed.

Is this a known issue? Is there a way to fix this behaviour using App Service configuration?

Kind regards.

Answer 1

¡Hola Alejandro Martín!

Esperamos que este mensaje te encuentre bien.

Lamentamos sinceramente la demora en abordar tu tema en este foro. Quiero ponerme a tu disposición si aún tienes el problema por el cual abriste este tema. Estaré atento a tu respuesta.

Si no recibimos una respuesta tuya en los próximos días, procederemos a cerrar el tema. Sin embargo, queremos asegurarte que siempre estamos disponibles para cualquier asistencia que puedas necesitar.

Agradecemos tu comprensión y estamos aquí para ayudar en lo que sea necesario.

Saludos cordiales,

 Jonathan