Eventstream definition

This article provides a breakdown of the definition structure for Eventstream items.

Definition parts

The definition of Eventstream item is constructed from two parts: the item part and the platform part. Each part contains the following properties:

• Path: The file name, for example: eventstream.json
• Payload: See Example of payload content decoded from Base64
• Payload Type: InlineBase64

Example of eventstream.json part decoded from Base64

The JSON file describing the Eventstream item has the following properties:

Property	Type	Required	Description
sources	Object	true	Describes the data sources that can be ingested into Eventstream for processing.
destinations	Object	true	Describes the endpoints within Fabric where processed data can be routed to, including Lakehouse, Eventhouse, Reflex, and others.
operators	Object	true	Define the event processors that handle real-time data streams, such as Filter, Aggregate, Group By, and Join.
streams	Object	true	Describes the data streams available for subscription and analysis in the Real-time Hub. There are two types of streams: default streams and derived streams.

{
    "sources": [
      {
        "name": "SqlServerOnVmDbCdc",
        "type": "SQLServerOnVMDBCDC",
        "properties": {
          "dataConnectionId": "1186ea93-fce4-4fd7-892d-0c3742a08486",
          "tableName": ""
        }
      }
    ],
    "destinations": [{
      "name": "Lakehouse",
      "type": "Lakehouse",
      "properties": {
          "workspaceId": "fdf52f3a-b687-41b8-8ff8-aeeca4d1edd8",
          "itemId": "737d6a97-e88c-45e1-9c39-adf1c9c4e817",
          "schema": "",
          "deltaTable": "newTable",
          "minimumRows": 100000,
          "maximumDurationInSeconds": 120,
          "inputSerialization": {
              "type": "Json",
              "properties": {
                  "encoding": "UTF8"
              }
          }
      },
      "inputNodes": [
          {
              "name": "derivedStream"
          }
      ]
  }
],
    "streams": [
      {
        "name": "myEventstream-stream",
        "type": "DefaultStream",
        "properties": {},
        "inputNodes": [
          {
            "name": "SqlServerOnVmDbCdc"
          }
        ]
      },
      {
        "name": "DerivedStream",
        "type": "DerivedStream",
        "properties": {
          "inputSerialization": {
            "type": "Json",
            "properties": {
              "encoding": "UTF8"
            }
          }
        },
        "inputNodes": [
          {
            "name": "GroupBy"
          }
        ]
      }
    ],
    "operators": [
      {
        "name": "GroupBy",
        "type": "GroupBy",
        "inputNodes": [
          {
            "name": "myEventstream-stream"
          }
        ],
        "properties": {
          "aggregations": [
            {
              "aggregateFunction": "Average",
              "column": {
                "expressionType": "ColumnReference",
                "node": null,
                "columnName": "payload",
                "columnPathSegments": [
                  {
                    "field": "ts_ms"
                  }
                ]
              },
              "alias": "AVG_ts_ms"
            }
          ],
          "groupBy": [
          ],
          "window": {
            "type": "Tumbling",
            "properties": {
              "duration": {
                "value": 5,
                "unit": "Minute"
              },
              "offset": {
                "value": 1,
                "unit": "Minute"
              }
            }
          }
        }
      }

    ],
    "compatibilityLevel": "1.0"
  }

To construct an Eventstream item for the API payload, you can use the GitHub template to define an Eventstream item and refer to Eventstream REST API for more guidelines.

Platform part

The platform payload is optional. The platform part is a file that contains the Eventstream metadata information.

• Create Item with definition respects the platform file if provided.
• Get Item definition always returns the platform file.
• Update Item definition accepts the platform file if provided, if you set a new URL parameter updateMetadata=true.

Definition example

Here's an example of a Base64-encoded eventstream definition, where the content from Example of payload content decoded from Base64 is encoded in Base64 and placed in the payload field with the path set to eventstream.json :

{
  "displayName": "myEventstream",
  "type": "Eventstream",
  "description": "Create Eventstream item with definition",
  "definition": {
    "parts": [
      {
        "path": "eventstream.json",
        "payload": "ewogICAgInNvdXJjZXMiOiBbCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJTcWxTZXJ2ZXJPblZtRGJDZGMiLAogICAgICAgICJ0eXBlIjogIlNRTFNlcnZlck9uVk1EQkNEQyIsCiAgICAgICAgInByb3BlcnRpZXMiOiB7CiAgICAgICAgICAiZGF0YUNvbm5lY3Rpb25JZCI6ICIxMTg2ZWE5My1mY2U0LTRmZDctODkyZC0wYzM3NDJhMDg0ODYiLAogICAgICAgICAgInRhYmxlTmFtZSI6ICIiCiAgICAgICAgfQogICAgICB9CiAgICBdLAogICAgImRlc3RpbmF0aW9ucyI6IFt7CiAgICAgICJuYW1lIjogIkxha2Vob3VzZSIsCiAgICAgICJ0eXBlIjogIkxha2Vob3VzZSIsCiAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgIndvcmtzcGFjZUlkIjogImZkZjUyZjNhLWI2ODctNDFiOC04ZmY4LWFlZWNhNGQxZWRkOCIsCiAgICAgICAgICAiaXRlbUlkIjogIjczN2Q2YTk3LWU4OGMtNDVlMS05YzM5LWFkZjFjOWM0ZTgxNyIsCiAgICAgICAgICAic2NoZW1hIjogIiIsCiAgICAgICAgICAiZGVsdGFUYWJsZSI6ICJuZXdUYWJsZSIsCiAgICAgICAgICAibWluaW11bVJvd3MiOiAxMDAwMDAsCiAgICAgICAgICAibWF4aW11bUR1cmF0aW9uSW5TZWNvbmRzIjogMTIwLAogICAgICAgICAgImlucHV0U2VyaWFsaXphdGlvbiI6IHsKICAgICAgICAgICAgICAidHlwZSI6ICJKc29uIiwKICAgICAgICAgICAgICAicHJvcGVydGllcyI6IHsKICAgICAgICAgICAgICAgICAgImVuY29kaW5nIjogIlVURjgiCiAgICAgICAgICAgICAgfQogICAgICAgICAgfQogICAgICB9LAogICAgICAiaW5wdXROb2RlcyI6IFsKICAgICAgICAgIHsKICAgICAgICAgICAgICAibmFtZSI6ICJkZXJpdmVkU3RyZWFtIgogICAgICAgICAgfQogICAgICBdCiAgfQpdLAogICAgInN0cmVhbXMiOiBbCiAgICAgIHsKICAgICAgICAibmFtZSI6ICJteUV2ZW50c3RyZWFtLXN0cmVhbSIsCiAgICAgICAgInR5cGUiOiAiRGVmYXVsdFN0cmVhbSIsCiAgICAgICAgInByb3BlcnRpZXMiOiB7fSwKICAgICAgICAiaW5wdXROb2RlcyI6IFsKICAgICAgICAgIHsKICAgICAgICAgICAgIm5hbWUiOiAiU3FsU2VydmVyT25WbURiQ2RjIgogICAgICAgICAgfQogICAgICAgIF0KICAgICAgfSwKICAgICAgewogICAgICAgICJuYW1lIjogIkRlcml2ZWRTdHJlYW0iLAogICAgICAgICJ0eXBlIjogIkRlcml2ZWRTdHJlYW0iLAogICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgImlucHV0U2VyaWFsaXphdGlvbiI6IHsKICAgICAgICAgICAgInR5cGUiOiAiSnNvbiIsCiAgICAgICAgICAgICJwcm9wZXJ0aWVzIjogewogICAgICAgICAgICAgICJlbmNvZGluZyI6ICJVVEY4IgogICAgICAgICAgICB9CiAgICAgICAgICB9CiAgICAgICAgfSwKICAgICAgICAiaW5wdXROb2RlcyI6IFsKICAgICAgICAgIHsKICAgICAgICAgICAgIm5hbWUiOiAiR3JvdXBCeSIKICAgICAgICAgIH0KICAgICAgICBdCiAgICAgIH0KICAgIF0sCiAgICAib3BlcmF0b3JzIjogWwogICAgICB7CiAgICAgICAgIm5hbWUiOiAiR3JvdXBCeSIsCiAgICAgICAgInR5cGUiOiAiR3JvdXBCeSIsCiAgICAgICAgImlucHV0Tm9kZXMiOiBbCiAgICAgICAgICB7CiAgICAgICAgICAgICJuYW1lIjogIm15RXZlbnRzdHJlYW0tc3RyZWFtIgogICAgICAgICAgfQogICAgICAgIF0sCiAgICAgICAgInByb3BlcnRpZXMiOiB7CiAgICAgICAgICAiYWdncmVnYXRpb25zIjogWwogICAgICAgICAgICB7CiAgICAgICAgICAgICAgImFnZ3JlZ2F0ZUZ1bmN0aW9uIjogIkF2ZXJhZ2UiLAogICAgICAgICAgICAgICJjb2x1bW4iOiB7CiAgICAgICAgICAgICAgICAiZXhwcmVzc2lvblR5cGUiOiAiQ29sdW1uUmVmZXJlbmNlIiwKICAgICAgICAgICAgICAgICJub2RlIjogbnVsbCwKICAgICAgICAgICAgICAgICJjb2x1bW5OYW1lIjogInBheWxvYWQiLAogICAgICAgICAgICAgICAgImNvbHVtblBhdGhTZWdtZW50cyI6IFsKICAgICAgICAgICAgICAgICAgewogICAgICAgICAgICAgICAgICAgICJmaWVsZCI6ICJ0c19tcyIKICAgICAgICAgICAgICAgICAgfQogICAgICAgICAgICAgICAgXQogICAgICAgICAgICAgIH0sCiAgICAgICAgICAgICAgImFsaWFzIjogIkFWR190c19tcyIKICAgICAgICAgICAgfQogICAgICAgICAgXSwKICAgICAgICAgICJncm91cEJ5IjogWwogICAgICAgICAgXSwKICAgICAgICAgICJ3aW5kb3ciOiB7CiAgICAgICAgICAgICJ0eXBlIjogIlR1bWJsaW5nIiwKICAgICAgICAgICAgInByb3BlcnRpZXMiOiB7CiAgICAgICAgICAgICAgImR1cmF0aW9uIjogewogICAgICAgICAgICAgICAgInZhbHVlIjogNSwKICAgICAgICAgICAgICAgICJ1bml0IjogIk1pbnV0ZSIKICAgICAgICAgICAgICB9LAogICAgICAgICAgICAgICJvZmZzZXQiOiB7CiAgICAgICAgICAgICAgICAidmFsdWUiOiAxLAogICAgICAgICAgICAgICAgInVuaXQiOiAiTWludXRlIgogICAgICAgICAgICAgIH0KICAgICAgICAgICAgfQogICAgICAgICAgfQogICAgICAgIH0KICAgICAgfQoKICAgIF0sCiAgICAiY29tcGF0aWJpbGl0eUxldmVsIjogIjEuMCIKICB9",
        "payloadType": "InlineBase64"
      },
      {
        "path": ".platform",
        "payload": "ewogICIkc2NoZW1hIjogImh0dHBzOi8vZGV2ZWxvcGVyLm1pY3Jvc29mdC5jb20vanNvbi1zY2hlbWFzL2ZhYnJpYy9naXRJbnRlZ3JhdGlvbi9wbGF0Zm9ybVByb3BlcnRpZXMvMi4wLjAvc2NoZW1hLmpzb24iLAogICJtZXRhZGF0YSI6IHsKICAgICJ0eXBlIjogIkV2ZW50c3RyZWFtIiwKICAgICJkaXNwbGF5TmFtZSI6ICJhbGV4LWVzMSIKICB9LAogICJjb25maWciOiB7CiAgICAidmVyc2lvbiI6ICIyLjAiLAogICAgImxvZ2ljYWxJZCI6ICIwMDAwMDAwMC0wMDAwLTAwMDAtMDAwMC0wMDAwMDAwMDAwMDAiCiAgfQp9",
        "payloadType": "InlineBase64"
      }
    ]
  }
}