Customize the instructions and examples for API pipeline V3

บทความ
11/18/2024

Important

Some or all of this functionality is available as part of a preview release. The content and the functionality are subject to change.

This article covers the template JSON format for an instruction and how it can be managed via API to support its lifecycle. The core objective of the instruction approach is to provide a structured representation that aids in its management and is RAG efficient for the Copilot.

As part of the deployment, out of the box (OOTB) instructions are already in an active state. Users can register and activate custom instructions as well.

Instruction template structure overview

The instructions follow a structured JSON approach where the instruction is partitioned into several sections such as for retrieval, prompt injection, and for lifecycle management and administration.

Each instruction should be a JSON object containing the following key components:

Metadata
Retrieval
Content
Relationships
Comment

For more details of the payload, see OpenAPI specifications.

1. Metadata

Metadata provides basic information about the instruction:

instructionId: Unique identifier (for example, PO, SP). This is also used to reference towards when providing examples. An alphanumeric value is preferred.
instructionVersion: The version field is a string parameter for representing a variant of the instruction. For example, instruction 1 can have two versions, A and B.
status: Status of the instruction.

Value Description

0 active

1 inactive
domain: Main domain.

Value Description

0 MOM

1 SCM

2 DE
subDomain: Specific subdomain.

Value Description

0 Production

1 Quality

2 Maintanance

3 Inventory

4 Downtime

5 Scrap

6 Mixed
category: Category classification.

Value Description

0 Transactional

1 Analytical

2 Trend

3 KPI
language: The language of the instruction.

Value Description

0 English

Value	Description
0	active
1	inactive

Value	Description
0	MOM
1	SCM
2	DE

Value	Description
0	Production
1	Quality
2	Maintanance
3	Inventory
4	Downtime
5	Scrap
6	Mixed

Value	Description
0	Transactional
1	Analytical
2	Trend
3	KPI

Value	Description
0	English

2. Retrieval

The retrieval section is crucial for helping the LLM retrieve the most relevant instructions. It should include:

subTopics: List of specific topics or tasks covered by the instruction. Phrase these as clear, concise questions, or statements (for example, Find production order information, Find schedule of past production order). This text is vectorized and used to match with the natural language questions from the client.
sampleQuestions: Example questions that demonstrate the types of queries the instruction is designed to handle. These are used for ranking the instructions.
keyWords: Keywords related to the instruction. These keywords help in indexing and searching for relevant instructions (for example, orders, production order, schedule). Avoid using generic words (for example: production order is ok, production is a generic word) as they then to get picked when not necessarily relevant. Don't use the same keyword for different instructions. These are used for ranking the instructions.

Important

Ensure that subTopics and keyWords are comprehensive and relevant to the tasks the instruction is meant to address. This aids in accurate and efficient retrieval by the LLM.

3. Content

The content section is used by the LLM to construct a working GraphKQL query. This section should include:

semanticContent
- context: Detailed description of the context, including relevant entities and their relationships. Explain how the entities are connected and the type of information they provide. It should describe the semantic layout of the graph schema where the response is contained, how data is organized in terms of entities, columns and relationships.
- steps:: Step-by-step guidelines on how to use the instruction. Clearly outline the process of constructing queries, including specific fields and filters to be used. Think of adding details like which entity to start the traversal from, how to navigate and what properties to filter on at each step.

Important

The context should be detailed and specific, explaining how to access and use the relevant entities. The steps should be clear and actionable, guiding the user through the query construction process.

4. Relationships

The relationships section lists dependencies on other instructions or entities. This helps in understanding the interconnectedness of different instructions.

dependencies: List any related instructions or entities that this instruction depends on. These instructions are used to create the prompt tree to construct the final query.
exclusions: List any instructions or entities that should be excluded for constructing the final query.

5. Comment

The comment section is for any additional notes or remarks.

comment: Additional comments or notes (optional).

6. Administrative

These aren't in the user payload and are for overall governance purposes.

source: 0: Out of the Box, 1: Custom.

Example Instruction

Here's an example of a well-structured instruction, including a linked example query:

Instruction JSON

[
  {
    "instructionId": "PO",
    "instructionVersion": "OperationOrderInfo",
    "status": "active",
    "domain": "MOM",
    "subDomain": "Production",
    "category": "Transactional",
    "language": "EN",
    "retrieval": {
        "subTopics": [
            "Find production order information.",
            "Find schedule of past production order.",
            "Find schedule of future production order.",
            "Find production order number.",
            "Find date of execution of order.",
            "Find type of order."
        ],
        "sampleQuestions": [
            "Orders planned for Machine XYZ on Feb 2025",
            "Planned quantity of SKU 12345GFA on Line G345 next month",
            "List of order per line that are running on the current Shift."
        ],
        "keyWords": [
            "orders",
            "production order",
            "schedule",
            "planned",
            "scheduled"
        ]
    },
    "content": {
        "semanticContent": {
            "context": "The manufacturing activity in a plant starts with planning for an order to take place. Based on the type of order, user may be interested in a production order or maintenance order (planned downtime). Use 'OperationsRequest' Entity to get information on operations order ID and planned date of execution of order.\n\nThe final report for any manufacturing operation is captured for a completed production order or maintenance order (planned downtime) using 'OperationsRequest' Entity. Use this entity to get information on operations order ID and actual date when the order was executed.",
            "steps": "Below are the steps to use above instruction:\n\n1. Use OperationsRequest.{id} to identify unique production orders.\n2. For date-related filtering, use OperationsRequest.{startTime} >= starting timestamp and OperationsRequest.{endTime} <= completing timestamp.\n3. Use OperationsRequest.{operationsType=Production/Maintenance} based on type of order."
        }
    },
    "relationships": {
        "dependencies": []
    },
    "comment": "This instruction is used to describe how orders can be retrieved. We provide steps how to traverse the data and what columns to use for time filters."
  }
]

Tips for Creating Effective Instructions

Be Specific: Provide detailed context and steps. Use unique Keywords and subtopics for retrieval.
Use Clear Language: Avoid jargon and complex language. Clear, concise instructions are easier to follow.
Test Example Queries: If possible, include tested example queries to demonstrate how the instruction can be used.
Review Dependencies: Ensure that all dependencies are listed and accurate to avoid any gaps in the instruction chain.
Avoid circular dependencies: While there are safeguards in place, it's good to avoid circular dependencies in the instructions. It leads to unexpected results.

Instructions endpoints

These API help manage the instructions. The instructions provide guidance to Manufacturing data solutions and the language model to create accurate and performant queries on the dataset.

Request headers

All endpoints require the following headers.

Name	Required	Description
Authorization	True	The bearer token used to authenticate the request
User-Agent	True	Short string to identify the client. See here for details

Role required

The GET-endpoints can be invoked with the Manufacturing User role assigned to the client. All other endpoints need the Manufacturing Admin role assigned to the client.

Request parameters

Name	Required	Type	Default	Description
api-version	True	string	-	The API version to use (`2024-06-30-preview`)

GET

Retrieves a single instruction.

GET https://{serviceUrl}/mds/copilot/v3/instructions/{instructionId}?api-version=2024-06-30-preview

Retrieves all instructions.

Name	Required	Description
x-ms-continuationtoken	False	Provide this value to navigate to the next page

GET https://{serviceUrl}/mds/copilot/v3/instructions?api-version=2024-06-30-preview

PUT

Creates or updates an instruction.

PUT https://{serviceUrl}/mds/copilot/v3/instructions?api-version=2024-06-30-preview

Important

Make sure you don't have multiple versions of the same instructionId in the payload. Doing so will cause issues in processing and eventually retrieving the latest state of the instructions.

DELETE

Deletion is only allowed for custom instructions and versions. For deleting a specific version of an instruction

DELETE https://{serviceUrl}/mds/copilot/v3/instructions/{instructionId}/versions/{instructionVersion}?api-version=2024-06-30-preview

If an instruction has only one version and if the instruction is being referenced as a dependency, you need to use the following method of deleting all versions of an instruction

For deleting all the versions of an instruction,

DELETE https://{serviceUrl}/mds/copilot/v3/instructions/{instructionId}?deleteAll=true&forceDelete=true?api-version=2024-06-30-preview

The deleteAll flag ensures that the entire instruction isn't accidentally removed. If deleteAll is set to false, no action is taken. If the instruction is part of a dependency graph, the forceDelete must be set to true for the deletion to proceed.

PATCH

Used to change the status of an instruction active or inactive.

PATCH https://{serviceUrl}/mds/copilot/v3/instructions/{instructionId}/versions/{instructionVersion}?api-version=2024-06-30-preview

Examples endpoints

These API help manage the examples linked with the instructions. These examples are natural language and query language pairs to aid the agent in the prompt constructed for an incoming question.

Refer to ADX functions used by Agent for usage in KQL queries in the examples.

Request headers

All endpoints require the following headers.

Name	Required	Description
Authorization	True	The bearer token used to authenticate the request
User-Agent	True	Short string to identify the client. See here for details

Role required

All these endpoints require the Manufacturing Admin role assigned to the client.

Request parameters

Name	Required	Type	Default	Description
api-version	True	string	-	The API version to use (`2024-06-30-preview`)

GET

Endpoint to get all the example queries in Manufacturing data solutions.

GET https://{serviceUrl}/mds/copilot/v3/exampleQuery?api-version=2024-06-30-preview

Endpoint to get a single example queries in Manufacturing data solutions.

GET https://{serviceUrl}/mds/copilot/v3/exampleQuery/{exampleId}?api-version=2024-06-30-preview

PUT

Endpoint to create the example query. This can take a list of examples in payload:

Note

api-version=2024-09-30-preview supports both multiple examples and single example. 2024-06-30-preview is still supported but works only with single example.

PUT  https://{serviceUrl}/mds/copilot/v3/exampleQuery?api-version=2024-09-30-preview

Example payload

[ {
    "exampleId": "Example_3",
    "userQuestion": "List of orders planned in AssemblyLine3 in March 2023?",
    "sampleQuery": {
        "language": "KQL",
        "query": "let operationsRequestIds = Graph()  | graph-match cycles = none (equipment)--(equipmentActual)--(segmentResponse)--(segmentRequirement)--(operationsRequest) where equipment.Properties.id contains 'AssemblyLine3' and todatetime(segmentResponse.Properties.postingDate) between(datetime(2023-03-01T00:00:00)..datetime(2023-03-31T23:59:59)) and equipment.ModelId has 'Equipment' and equipmentActual.ModelId has 'EquipmentActual' and segmentResponse.ModelId has 'SegmentResponse' and segmentRequirement.ModelId has 'SegmentRequirement' and operationsRequest.ModelId has 'OperationsRequest' project id = tostring(operationsRequest.Properties.id); operationsRequestIds | distinct id"
    },
    "linkedInstructions": [
        "PO"
    ]
}

]

PATCH

Endpoint to update the linked instructions to an example.

PATCH https://{serviceUrl}/mds/copilot/v3/exampleQuery/{exampleId}?api-version=2024-06-30-preview

DELETE

Endpoint to delete the example query.

DELETE https://{serviceUrl}/mds/copilot/v3/exampleQuery/{exampleId}?api-version=2024-06-30-preview

แชร์ผ่าน

Customize the instructions and examples for API pipeline V3

Instruction template structure overview

1. Metadata

2. Retrieval

3. Content

4. Relationships

5. Comment

6. Administrative

Example Instruction

Instruction JSON

Tips for Creating Effective Instructions

Instructions endpoints

Request headers

Role required

Request parameters

GET

PUT

DELETE

PATCH

Examples endpoints

Request headers

Role required

Request parameters

GET

PUT

Example payload

PATCH

DELETE

แหล่งทรัพยากรเพิ่มเติม