Azure Web Application and ChatPlayground giving different responses?

Question

Hi,

I am working on Azure AI Search. I want to use Web Application/Chat Playground for Q/A from bunch of documents at a time. I have created azure ai search service with Basic tier package for West US Region. After this, I created skillset using Text Split Skill. Then, I have created index called azureblob-index where fields are:

1.) content (Retrievable, Searchable)

2.) metadata_storage_name (Retrievable)

3.) metadata_storage_path (Retrievable, Filterable)

4.) metadata_author (Retrievable, Filterable, Facetable, Searchable)

5.) metadata_title (Retrievable, Filterable, Facetable, Searchable)

6.) metadata_creation_date (Retrievable, Filterable, Sortable, Facetable)

7.) language (Retrievable, Filterable, Facetable, Searchable)

8.) split_text (Retrievable, Searchable)

9.) keywords (Retrievable, Searchable)

10.) summary (Retrievable, Searchable)

11.) section_titles (Retrievable, Searchable)

12.) metadata_file_type (Retrievable, Filterable, Facetable, Searchable)

13.) merged_content (Filterable, Searchable)

14.) text (Retrievable, Searchable)

15.) layoutText (Retrievable, Searchable)

After this, I have created semantic configurations where fields are:

Title_field: metadata_title

Content fields: content, merged_content

Keyword fields: keywords

After this, I have created chat playground using bring your own data. When I am asking questions, then chat playground is giving response which is not very accurate or sometimes not even matching to the question that is asked. Also, sometimes it does not give any answer by stating that this question is out of my domain/scope. But again if I ask same question after like 10-15 seconds, then I am getting answer for that question for which earlier I got out of domain/scope message. Sometimes, it gives correct answer. Same happens in the web application. Also, sometimes references in the chat playground and web application doesn't even match while question being same.

Can anyone suggest why this is happening.