Consumption API

Artykuł
11/18/2024

Important

Some or all of this functionality is available as part of a preview release. The content and the functionality are subject to change.

Important

This document includes several references to the ISA95 semantic model. Visit the ISA95 website and store to purchase the ISA95 standards.

The Consumption API allows you to retrieve data from various sources within Manufacturing data solutions, such as Azure Data Explorer (ADX) and Cosmos DB, using a single query. The API fetches data directly from both ADX and Cosmos DB. However, for APIs with a time range, you can only query data in ADX, as Cosmos DB doesn't store historical data. If a filter or projection is specified for a column that exists in Cosmos DB, that column is ignored.

The DateTime format for historization queries is yyyy-MM-ddTHH:mm:ss[.sss] (ISO 8601). Time range operators are listed in the following table:

Operator	Parameters expected
lt: less than	One parameter representing the DateTime in string format. for example: 2023-01-27T02:30:00
gt: greater than	One parameter representing the DateTime in string format. for example: 2023-01-27T02:30:00
between	Two parameters, start time and end time

For more information about API payload and corresponding responses, see OpenAPI specification.

Query API

Query API is used to query entities data from different sources like Azure Data Explorer (ADX) and Cosmos DB, and it has two versions.

Version 2023-03-31 (Default Version)**

Used to fetch data from ADX and Cosmos DB only.

POST https://{serviceUrl}/mds/service/query

API Payload

{
  "entityName": "string",
  "filters": [
 {
   "fieldName": "string",
   "operator": "EQ",
   "value": "string"
 }
  ],
  "projectedFields": [
 "string"
  ],
  "timeRange": {
    "operator": "LT",
    "parameters": [
      "2023-06-07T04:55:37.026Z"
    ]
  },
  "maxRecords": "integer",
  "pageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Sample Request

{
    "EntityName": "Material Lot",
    "Filters": [
        {
                "FieldName": "id",
                "Operator": "contains",
                "Value": "HYDK3010200"
            }
    ],
  "pageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Sample Response

[
{
  "id": "HYDK3010200",
  "description": "",
  "hierarchyScope": "",
  "assemblyType": "",
  "assemblyRelationship": "",
  "storageLocation": "",
  "storageLocationType": "",
  "status": "Approved",
  "disposition": "",
  "quantity": 712,
  "quantityUnitOfMeasure": "g",
  "inventorymangementsystemrefid": "",
  "messystemrefid": "",
  "TimeStamp": "2024-03-14T11:00:59.2319555Z"
}
]

Related nodes API is used to get details of all the related entities of a particular given entity. It only fetches single hop entities.

POST https://{serviceUrl}/mds/service/query/relatedNodes

API Payload

{
  "entityName": "string",
  "filters": [
 {
   "fieldName": "string",
   "operator": "EQ",
   "value": "string"
 }
  ],
  "timeRange": {
 "operator": "LT",
 "parameters": [
   "2023-06-07T04:55:37.026Z"
 ]
  },
  "maxRecords": "integer",
  "pageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Version 2024-02-29-preview

This version aims to achieve the same objective as the previous one. However, instead of performing multiple manual joins, it utilizes the make-graph capability of Azure Data Explorer.

POST https://{serviceUrl}/mds/service/query/relatedNodes?api-version=2024-02-29-preview

Payload remains same as the previous version.

Sample Request

{
    "EntityName": "Equipment",
    "Filters": [
        {
            "FieldName": "dtId",
            "Operator": "in",
            "Value": "0C031D807658014654814055A38919132C7E0B87"
        }
    ],
  "pageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Sample Response

[
    {
        "Source": "0C031D807658014654814055A38919132C7E0B87",
        "Target": "AA16791A526CC3F26E34177ADF27653E18EAC0BC",
        "RelationshipName": "correspondsToAnElementIn",
        "RelationshipId": "AA16791A526CC3F26E34177ADF27653E18EAC0BC_0C031D807658014654814055A38919132C7E0B87",
        "TimeStamp": "2024-01-18T12:46:57.0068616Z"
    },
    {
        "Source": "0C031D807658014654814055A38919132C7E0B87",
        "Target": "A132815D213D90D77106D8CB4964D0CC56E44F3F",
        "RelationshipName": "correspondsToAnElementIn",
        "RelationshipId": "A132815D213D90D77106D8CB4964D0CC56E44F3F_0C031D807658014654814055A38919132C7E0B87",
        "TimeStamp": "2024-01-18T12:46:57.0068766Z"
    },
    // ...
]

Note

Filters are mandatory for both the versions of relatedNodes API. Atleast one filter should be present in the payload

The Related Entities Data API retrieves details of entities connected to a specified source entity. It supports maximum hop queries and fetches only outbound relationships. If the maximum hops parameter isn't provided, it defaults to 5.

POST https://{serviceUrl}/mds/service/query/relatedEntitiesData

API Payload


{
  "sourceEntity": {       // any given entity
    "entityName": "string",
    "filters": [
      {
        "fieldName": "string",
        "operator": "EQ",
        "value": "string",
      }
    ]
  },
  "relatedEntities": [   // target entity for which details are required that are connected to source entity
    {
      "entityName": "string",
      "filters": [
        {
          "fieldName": "string",
          "operator": "EQ",
          "value": "string",
        }
      ],
      "projectedFields": [
        "string"
      ],
    }
  ],
  "timeRange": {
    "operator": "LT",
    "parameters": [
      "2023-06-06T08:53:18.337Z"
    ]
  },
  "maxRecords": "integer",
  "maxHops": "integer",
  "PathEntityList":[
     "string"
  ],
  "pageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Version 2024-02-29-preview

Similar to the previous version, the objective of this version of the Related Entities Data API remains the same. However, instead of performing multiple manual joins for multi-hop queries, it uses the make-graph capability of Azure Data Explorer (ADX). This version reduced the Kusto Query Language (KQL) query more than 50% enabling code maintainability and readability.

POST https://{serviceUrl}/mds/service/query/relatedNodes?api-version=2024-02-29-preview

Payload remains same as the previous version.

Sample Request

{
    "SourceEntity": {
        "EntityName": "Operations Request",
        "Filters": [
            {
                "FieldName": "description",
                "Operator": "contains",
                "Value": "BAKING"
            },
            {
                "FieldName": "dtId",
                "Operator": "in",
                "Value": "B170F38F78C47D9CF8EF8649BC0A777D6DE7DCFE"
            }
        ]
    },
    "RelatedEntities": [
        {
            "EntityName": "Equipment",
            "Filters": [
                {
                    "FieldName": "dtId",
                    "Operator": "in",
                    "Value": "0C031D807658014654814055A38919132C7E0B87"
                }
            ]
        }
    ],
    "PathEntityList": [
        "Equipment requirement"
    ],
  "PageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Sample Response (2024-02-29-preview)

[
    {
        "Result": [
            {
                "ModelId": "dtmi:digitaltwins:isa95:Equipment;1",
                "Source": "B170F38F78C47D9CF8EF8649BC0A777D6DE7DCFE",
                "Target": "0C031D807658014654814055A38919132C7E0B87",
                "Path": [
                    "B170F38F78C47D9CF8EF8649BC0A777D6DE7DCFE_2868E331E85E13DEDB234C5E12771D8CB11DAAB9 isMadeUpOfRequirement",
                    "2868E331E85E13DEDB234C5E12771D8CB11DAAB9_C53B5EEE7BCFE84CA7A0F4566C62149707928E61 containsEquipment",
                    "C53B5EEE7BCFE84CA7A0F4566C62149707928E61_B526925FE8A50CE3204D2C7AC54612A5EA9F512A correspondsToAnElementIn",
                    "B526925FE8A50CE3204D2C7AC54612A5EA9F512A_0C031D807658014654814055A38919132C7E0B87 isMadeUpOf"
                ],
                "TimeStamp": [
                    "2024-01-18T12:46:55.9936183Z",
                    "2024-01-18T12:46:57.3352265Z",
                    "2024-01-18T12:46:58.2894347Z",
                    "2024-01-18T12:46:56.6865436Z"
                ],
                "description": "BREADBLAST05",
                "equipmentLevel": "workCell",
                "hierarchyScope": "BREADBLAST05",
                "id": "BREADBLAST05",
                "assetsystemrefid": "testequipment"
            }
        ]
    }
]

Object	Description
maxRecords	Signifies the total number of records that should be present in the result.
maxHops	Signifies the maximum number of hops between source and target. In other words, the maximum number of edges away the target node can be.
PathEntityList	It tells the entities type that should be present in the path from source to the target. For example, `"PathEntityList" :["Person Property"]`. This means that result filters out the path which doesn't have Person Property as one of the nodes in between source and target. If its `null` then all valid paths are taken.

Ingestion status API

The Ingestion Status API provides information on the number of nodes and relationships successfully created in Azure Data Explorer (ADX) for time series data and in Cosmos DB for reference data. You can filter the results over a specific time range; if no time range is specified, the API returns the total count of nodes and relationships created to date. Additionally, if the instance is Copilot-enabled, the API also provides the ingestion status count for ProperNoun.

GET https://{serviceUrl}/mds/service/query/ingestionStatus?operator=BETWEEN&endDate=2024-02-15&startDate=2024-02-21

Sample Response

{
    "TimeSeries": [
        {
            "TotalTwins": [
                {
                    "Count": 412112
                }
            ],
            "TwinsPerModelId": [
                {
                    "ModelId": "dtmi:digitaltwins:isa95:MaterialLot;1",
                    "Count": 16402
                },
                {
                    "ModelId": "dtmi:digitaltwins:isa95:Equipment;1",
                    "Count": 7111
                },
                 // ...
            ]
        },
        {
            "TotalRelationships": [
                {
                    "Count": 564241
                }
            ],
            "RelationshipsPerSourceTargetModelId": [
                {
                    "SourceModelId": "dtmi:digitaltwins:isa95:OperationsEventProperty;1",
                    "TargetModelId": "dtmi:digitaltwins:isa95:OperationsEventDefinitionProperty;1",
                    "Count": 4278
                },
                {
                    "SourceModelId": "dtmi:digitaltwins:isa95:OperationsEvent;1",
                    "TargetModelId": "dtmi:digitaltwins:isa95:OperationsEventProperty;1",
                    "Count": 10866
                },
                {
                    "SourceModelId": "dtmi:digitaltwins:isa95:OperationsEvent;1",
                    "TargetModelId": "dtmi:digitaltwins:isa95:OperationsEventDefinition;1",
                    "Count": 4281
                },
                // ...

            ]
        }
    ],
    "ReferenceData": [
        {
            "TotalTwins": [
                {
                    "Count": 412112
                }
            ],
            "TwinsPerEntity": [
                {
                    "EntityName": "Equipment",
                    "Count": 7111
                },
                {
                    "EntityName": "Person",
                    "Count": 7095
                },
                // ...
              ]
        }
     ],
    "VectorizationDataCount": {
        "TotalProperNounsCreated": [
          {
            "Count": 311
          }
        ],
        "ProperNounsCreatedPerEntity": [
          {
            "EntityName": "PersonnelClass",
            "Count": 1
          },
          {
            "EntityName": "Equipment",
            "Count": 13
          },
          // ...
        ],
        "TotalEntitySchemaVectorization": [
          {
            "SumOfCount": 19
          }
        ],
        "EntitySchemaVectorizationCountPerEntity": [
          {
            "Key": "MaterialDefinition",
            "Count": 1
          },
          {
            "Key": "OperationsEventProperty",
            "Count": 2
          },
          // ...
        ],
        "TotalEntityRelationsVectorizationCount": [
          {
            "SumOfCount": 20
          }
        ],
        "EntityRelationsVectorizationCountPerEntity": [
          {
            "Key": "SegmentResponse",
            "Count": 4
          },
          {
            "Key": "Equipment",
            "Count": 3
          },
          // ...
        ]
  }
}

Consumption history API

The Consumption History API retrieves historical data on the relationships between source and related entities. It supports various filters, similar to the Related Entities Data API, to customize the output and uses the make-graph capability to query Azure Data Explorer (ADX).

POST https://{serviceUrl}/mds/service/query/relatedEntitiesDataHistory

API Payload

{
  "sourceEntity": {              // any given entity
    "entityName": "string",
    "filters": [
      {
        "fieldName": "string",
        "operator": "EQ",
        "value": "string",
      }
    ]
  },
  "relatedEntities": [           // target entity for which details are required that are connected to source entity
    {
      "entityName": "string",
      "filters": [
        {
          "fieldName": "string",
          "operator": "EQ",
          "value": "string",
        }
      ],
      "projectedFields": [
        "string"
      ],
      }
  ],
  "timeRange": {
    "operator": "LT",
    "parameters": [
      "2023-06-06T08:53:18.337Z"
    ]
  },
  "maxRecords": "integer",
  "maxHops": "integer",
  "PathEntityList":[
     "string"
  ]
}

API payload supported query operators

The following table provides a comparison of API operators and their corresponding implementations in Cosmos DB and Azure Data Explorer (ADX):

API operator	Cosmos DB operator	Cosmos DB example	ADX operator	ADX example
EQ	=	Value ='dev_2_0_id'	==	Value =='dev_2_0_id'
GT	>	Quantity > 10	>	Quantity > 10
LT	<	Quantity < 10	<	Quantity < 10
NEQ	!=	Name!= 'Arturo'	!=	Name!= 'Arturo'
IN	in	quantity in ('9000','80000')	IN	quantity in ('9000','80000')
NIN	not in	quantity not in ('9000','80000')	!IN	quantity!in ('9000','80000')
GEQ	>=	Quantity >= 10	>=	Quantity >= 10
LEQ	<=	Quantity <= 10	<=	Quantity <= 10
BETWEEN	between	(c.Data.name between "aman" and "amaan")	between	Value between (80 .. 80000)
NBETWEEN	not between	(c.Data.name not between "aman" and "amaan")	!between	Value!between (80 .. 80000)
CONTAINS	contains	contains(c.Data.id, 'QJKmhP_Log')	contains	ID contains 'QJKmhP_Log'
NOTCONTAINS	not contains	not contains(c.Data.id, 'QJKmhP_Log')	!contains	ID!contains 'QJKmhP_Log'
STARTSWITH	startsWith	STARTSWITH(c.EntityName,'Material')	startsWith	Entityname startswith 'Material'
ENDSWITH	endsWith	ENDSWITH(c.EntityName,'Lot')	endsWith	Entityname endswith 'Lot'
NOTSTARTSWITH	not startsWith	NOT STARTSWITH(c.EntityName,'Material')	!startsWith	Entityname!startswith 'Material'
NOTENDSWITH	not endsWith	NOT ENDSWITH(c.EntityName,'Lot')	!endsWith	EntityName!endswith 'Lot'
REGEXMATCH	regexmatch	regexmatch(c.EntityName, ".*Lot")	matches regex	entityName matches regex '.*Lot'
HAS	has	contains(c.Data.id,'QJKmhP_Log')	has	ID has 'QJKmhP_Log'
NHAS	not contains	not contains(c.Data.id, 'QJKmhP_Log')	!has	ID!has 'QJKmhP_Log'

Pagination support

The API now supports pagination, allowing you to retrieve results in smaller, more manageable chunks. To enable pagination, include the paginate query parameter in your request URL and set it to true. Additionally, specify the pageIndex property in the request body. Each page contains a fixed number of 1,000 results.

Parameter	Type	Required	Description
`paginate`	Boolean	No	Enable pagination in the response. Default is `false`.

An example of a request:

POST https://{serviceUrl}/mds/service/query?paginate=true

Sample Request

{
    "EntityName": "Material Lot",
    "Filters": [
        {
                "FieldName": "id",
                "Operator": "contains",
                "Value": "HYDK3010200"
            }
    ],
  "pageIndex": 1  //Optional parameter, default value is 1 if pagination is enabled
}

Filtering data

The Consumption REST API contains several endpoints capable of filtering data. It's important to note filtering is only supported on properties with the semanticRelevantFlag set to the value true as shown in the following excerpt from the Equipment entity.

Excerpt Of Equipment Entity

{
    "name": "Equipment",
    "tags": {
        "ingestionFormat": "Batch",
        "ingestionRate": "Hourly"
    },
    "columns": [
        {
            "mappingTableForNormalization": "",
            "isProperNoun": false,
            "displayName": "",
            "description": "...",
            "type": "String",
            "mandatory": true,
            "semanticRelevantFlag": true,
            "primaryKey": true,
            "groupBy": true,
            "name": "id"
        },
        // ...
        {
            "mappingTableForNormalization": "",
            "isProperNoun": false,
            "displayName": "",
            "description": "...",
            "type": "Enum",
            "mandatory": false,
            "semanticRelevantFlag": true,
            "primaryKey": false,
            "groupBy": false,
            "enumValues": [
                "Description",
                "Operational Location"
            ],
            "name": "operationalLocationType"
        },
        {
            "mappingTableForNormalization": "",
            "isProperNoun": false,
            "displayName": "",
            "description": "...",
            "type": "Alphanumeric",
            "mandatory": false,
            "semanticRelevantFlag": false,
            "primaryKey": false,
            "groupBy": false,
            "name": "assetsystemrefid"
        },
        {
            "mappingTableForNormalization": "",
            "isProperNoun": false,
            "displayName": "",
            "description": "...",
            "type": "Alphanumeric",
            "mandatory": false,
            "semanticRelevantFlag": false,
            "primaryKey": false,
            "groupBy": false,
            "name": "messystemrefid"
        }
    ]

For this entity adding filters for id, and operationalLocationType works. When you add a filter for assetsystemrefid or messystemrefid, a response is returned stating the request isn't supported.

Known issues

The maxRecords parameter only works for time range queries and the results are sorted in descending order by default. maxRecords specified without a time range is ignored.
Data type conversion is supported in ADX queries, as all data in Cosmos DB is stored in string format.
Only the 'AND' operator is supported between different filters.

Udostępnij za pośrednictwem

Consumption API

Query API

Version 2023-03-31 (Default Version)**

Version 2024-02-29-preview

Version 2024-02-29-preview

Ingestion status API

Consumption history API

API payload supported query operators

Filtering data

Known issues

Dodatkowe zasoby

Udostępnij za pośrednictwem

Consumption API

Query API

Version 2023-03-31 (Default Version)**

Related nodes API

Version 2024-02-29-preview

Related entities Data API

Version 2024-02-29-preview

Ingestion status API

Consumption history API

API payload supported query operators

Pagination support

Filtering data

Known issues

Dodatkowe zasoby