Responsible AI FAQs for Factory Operations Agent in Azure AI (preview)
Important
Some or all of this functionality is available as part of a preview release. The content and the functionality are subject to change.
An AI system includes not only the technology, but also the people who use it, the people affected by it, and the environment in which it's deployed. Microsoft's Responsible AI FAQs are intended to help you understand how AI technology works, the choices system owners and users can make that influence system performance and behavior, and the importance of thinking about the whole system, including the technology, the people, and the environment. You can use Responsible AI FAQs to better understand specific AI systems and features that Microsoft develops.
Responsible AI FAQs are part of a broader effort to put Microsoft's AI principles into practice. To find out more, see Microsoft AI principles.
AI-driven features in this solution
This solution contains a growing list of AI-driven features. To learn about the capabilities and impact of specific features, select a feature name from the list.
Factory Operations Agent in Azure AI (preview) is a starter solution in Microsoft Cloud for Manufacturing. It uses the capabilities of natural language interfaces to tackle manufacturing scenarios like resolving quality issues, conducting root cause analysis, and more. In this article, you find answers to some frequently asked questions (FAQs) about the Factory Operations Agent and how Microsoft uses your data responsibly.
What is Factory Operations Agent in Azure AI?
Factory Operations Agent for Manufacturing data solutions in Microsoft Fabric is being built to provide a natural language interface for generating KQL queries, tabular results, and summaries of the results related to the Manufacturing Operations Management domain.
What can Manufacturing data solutions in Microsoft Fabric and Factory Operations Agent in Azure AI do?
The default schema is standardized using an industry standardized data model based on key ISA95 entities including Person, Equipment, and Process. Visit the ISA95 website and ISA95 store to purchase the ISA95 standards.
Default metadata is preloaded. Sample data can optionally be loaded for demo purposes. API is also provided to allow customers to create their own custom entities and ingest their own data. Kusto queries (KQL) are the primary way that customers retrieve data out of Manufacturing data solutions. The Factory Operations Agent in Azure AI takes a natural language question/command (for example, "how many employees are there?") and converts it into the equivalent KQL. The KQL is then run to retrieve the data and then the data results summarized.
What is/are Manufacturing data solutions in Microsoft Fabric and Factory Operations Agent in Azure AI intended use?
The intended use for the Manufacturing data solutions and Factory Operations Agent is to do root cause analysis (RCA) and overall equipment efficiency (OEE) gain. RCA is to be done by the factory personnel asking the natural language questions to get the data results and based on that figure out factory floor problem and take corrective action. For Example: mis-welding – welding machine too old (rust, humidity). Fix: replace old equipment and inform supplier to ship newer equipment.
What languages does Factory Operations Agent support?
Factory Operations Agent is currently supported in English language only. Support for other languages becomes available based on market demand and the availability of Azure OpenAI in those regions. We plan to localize the solution to other languages where Azure OpenAI is present and based on customer asks.
How can customers use Factory Operations Agent?
The primary purpose is to:
Simplify data queries: Factory Operations Agent enables easy, SQL-free data queries through conversational UX, broadening data access across roles
Enhance responsiveness: Factory Operations Agent acts as a crucial aid for managers in handling shop floor issues like quality problems or downtime, supporting efficient root cause analysis.
Streamlining communication: Factory Operations Agent facilitates the summarization of analyses for cross-team sharing, improving collaboration and problem-solving speed.
This tool is essential for modern, agile manufacturing operations, promoting informed decision-making and team collaboration. It enables factory staff to perform root cause analysis (RCA) and improve overall equipment efficiency (OEE). They can ask questions in natural language to obtain data results, which can then be used to identify and rectify issues on the factory floor. Every output should undergo human review before use.
How was Factory Operations Agent in Azure AI evaluated? What metrics are used to measure performance?
The system was evaluated by running queries for which the KQL query was already available. The data results for the actual KQL query were compared to the one generated using the Factory Operations Agent. With these results, a consolidated F1 score was computed for the specific NL question based on column and row precision and recall.
What service does Factory Operations Agent use?
The Factory Operations Agent uses the Azure OpenAI service. We have a prompt engineering pipeline for the language models to intelligently construct a DSL based on the Natural Language Query ask.
Where is the data processed as part of the Factory Operations Agent?
The Factory Operations Agent has all the data processing within the customer Azure tenant based on the managed-on behalf of model.
What are the limitations of Factory Operations Agent in Azure AI? How can users minimize the impact of Factory Operations Agent in Azure AI limitations when using the system?
A known limitation of Factory Operations Agent is lack of generalization. Users can provide context through instructions. If these don't provide adequate context to the query, Factory Operations Agent doesn't generate an appropriate response.
Users can mitigate this limitation by formulating their natural language queries more descriptively and supplementing them with relevant instructions for better context.
What operational factors and settings allow customers to use Factory Operations Agent effectively and responsibly?
For an effective use, users should add relevant instructions and phrase questions descriptively.
Additionally, our workflow allows you to use Factory Operations Agent effectively and responsibly. We check the intent of the question and abort if the question is unrelated to manufacturing. The query also aborts if it doesn't find any relevant entities.
What operational factors and settings allow for effective and responsible use of Factory Operations Agent in Azure AI?
Our workflow allows the effective and responsible use of Factory Operations Agent in Azure AI.
The query generation workflow starts by understanding user's intent. The query aborts if intent is invalid - this handles most of the "bad" questions. The Azure OpenAI content filters are also applied all LLM calls to remove harmful content from the response. For more information, see Content filtering.
If you onboard your own Azure OpenAI resource, ensure that these filters are turned on. You can configure the filters with a Low threshold, which will flag even slightly malicious content for extra safety. This is useful when dealing with potentially harmful data. By default, we deploy with a Medium threshold.