Azure AI Document Intelligence
An Azure service that turns documents into usable data. Previously known as Azure Form Recognizer.
1,721 questions
Concerns Regarding the tables in markdown output changes in 2024-07-31-preview
![](data:image/svg+xml, %3Csvg xmlns='http://www.w3.org/2000/svg' height='64' class='font-weight-bold' style='font: 600 30.11764705882353px "SegoeUI", Arial' width='64'%3E%3Ccircle fill='hsl(48, 0%, 14%)' cx='32' cy='32' r='32' /%3E%3Ctext x='50%25' y='55%25' dominant-baseline='middle' text-anchor='middle' fill='%23FFF' %3EXJ%3C/text%3E%3C/svg%3E)
Xi, Jonathan
20
Reputation points
Hello Azure AI Document Intelligence Team,
We have some concerns regarding the tables in markdown output changes in 2024-07-31-preview release:
Starting from 2024-07-31-preview, the representation of tables is changed to HTML tables to enable rendering of merged cells, multi-row headers, etc.
A significant concern is that HTML tables require nearly twice the number of tokens compared to markdown tables based on our testing. Since we're developing GenAI apps using Azure AI Document Intelligence, this increase in token usage will reduce chunk context and might negatively impact the accuracy of our applications.
Could we explore alternative solutions for handling merged cells and multi-row headers? E.g., could we consider duplicating cells for row spans instead of converting everything to HTML? Additionally, it would be beneficial to provide options for end-users to choose tables in Markdown or HTML, so that end-users can do customization for markdown merged cells issue (if Azure can't address the issue).
Thank you for considering these suggestions.
Best regards, Jonathan
{count} votes
-
romungi-MSFT 46,911 Reputation points Microsoft Employee2024-10-28T08:56:58.0133333+00:00 @Xi, Jonathan I understand this change might be impacting your usage w.r.t to integration with GPT scenarios where token usage determines the cost. We shall relay the feedback to the product team for consideration.
Considering that this is a preview version of the API, the GA version might or might not have the same behavior since the metrics collected from previews are taken into consideration while releasing GA versions of the API. I would recommend posting a feedback item on feedback.azure.com too for visibility for extended community and program owners who can vote or accept or decline the request. Thanks!!
-
Xi, Jonathan 20 Reputation points
2024-10-31T08:20:10.66+00:00 Thanks romungi. Do you know whether there is a timeline for the Azure Document Intelligence release schedule?
-
Xi, Jonathan 20 Reputation points
2024-10-31T08:22:31.68+00:00 Thanks @romungi-MSFT Do you know whether there's a timeline for Azure Document Intelligence GA release schedule?
-
romungi-MSFT 46,911 Reputation points Microsoft Employee
2024-11-04T11:42:40.7466667+00:00 @Xi, Jonathan I do not think a schedule is released for updates but if you are using a preview API you should get an email when GA is set to be released. I recommend following what's new page too for any updates on future releases.
Sign in to comment
Sign in to answer