Share via

Responsible AI FAQs for Financial document analysis skill for agents (preview)

  • Article

Important

This is a preview feature. This information relates to a prerelease feature that may be substantially modified before it's released. Microsoft makes no warranties, expressed or implied, with respect to the information provided here.

Frequently asked questions

This article answers frequently asked questions (FAQs) about Financial document analysis skill for agents (preview). It also explains how Microsoft uses your data responsibly.

What is Financial document analysis skill for agents?

Financial document analysis skill for agents is a skill that enables you to query your financial documents such as analyst reports and receive high quality responses, focusing on understanding financial tables and charts. Financial document analysis skill for agents is a back-end solution that you can deploy in your tenant and manage.

What can Financial document analysis skill for agents do?

Using the skill, you can ask questions over a set of your own financial documents and receive high-quality responses grounded in the set of defined documents. The solution has the following components: the ingestion process which processes the documents to query upon, and the Inference API which can be used to query the documents.

What is the intended use of Financial document analysis skill for agents?

The skill enables you to query your financial reports such as analyst reports and receive high-quality responses. The solution is designed to interpret charts and tables in addition to the text within the document. The following examples explain the intended use of the skill:

  • Investment decision support: The skill helps you ask questions related to specific investments and receive answers based on the insights provided in those reports. Investors can gather relevant information and make informed decisions about where to allocate their funds.

  • Strategic planning insights: The skill can assist strategic planning by offering a wide range of perspectives from analyst reports. You can ask questions about market trends, competitor analysis, industry forecasts, and more.

  • Asset assessment: The skill can aid in asset assessment by providing access to financial reports that offer insights into various risks associated with different investments or business decisions.

How was Financial document analysis skill for agents evaluated? What metrics are used to measure performance?

The skill was evaluated using a diverse set of financial documents, including analyst reports and 10-Q forms, along with a curated question and answer collection containing established ground-truth answers. The evaluation process assessed both the retrieval mechanism, ensuring that relevant data was accurately sourced, and the generation process, confirming that the output was precise, complete, and aligned with the ground truth. The output demonstrated improved accuracy and relevance.

Furthermore, the skill has built-in content filtering through the Azure OpenAI Service, which uses an ensemble of classification models to detect and prevent the output of harmful content, such as hate speech, sexual content, and violence.

What are the limitations of Financial document analysis skill for agents? How can users minimize the effect of Financial document analysis skill for agents’s limitations when using the system?

  • The skill doesn't provide financial advice or financial forecasts. The answers provided are based on the documentation provided for ingestion and don't reflect the opinions of Microsoft.
  • The skill is designed to answer one question at a time, so we recommend asking multiple questions separately.
  • It isn't intended for use in chat mode and only provides answers based on a predefined corpus of documents, rather than updated information from the web.
  • Although the skill is designed to prevent the output of harmful content, it can sometimes make mistakes.
  • The skill makes use of Large Language Model (LLM) technology which can make mistakes. We recommend adding a disclaimer to the output reminding users to check for accuracy before making financial decisions.
  • It relies on the use of AI services and its output might be incorrect. Customers are responsible for conducting skillropriate due diligence of AI-generated content before making any financial decision.

What operational factors and settings allowed for effective and responsible use of Financial document analysis skill for agents? 

During the deployment and configuration phase, it's important to follow the documentation. Furthermore, we recommend using the Azure OpenAI models as suggested in the documentation for best practices and high-quality questions and answers.

Overview of Financial document analysis skill for agents (preview)