Edit

Share via

Eventhouse and KQL database deployment pipelines and git integration (Preview)

  • Article

Eventhouses and KQL databases integrate with the lifecycle management capabilities in Microsoft Fabric, providing a standardized collaboration between all development team members throughout the product's life. This functionality is delivered via git integration and deployment pipelines.

In this article, you learn about the configuration options available through Microsoft Fabric's lifecycle management for eventhouses and KQL databases.

Eventhouse and KQL database git integration

The eventhouse and KQL database are items that contain both metadata and data that are referenced in multiple objects in the workspace. Eventhouse and KQL database contain tables, functions, and materialized views. From a development workflow perspective, the following dependent objects might reference an eventhouse or KQL database:

The git integration applies at the platform and data level for eventhouses and KQL databases.

Platform-level integration

The following eventhouse and KQL database information is serialized and tracked in a git-connected workspace:

  • Eventhouse

    • Name
    • Description
    • Logical guid

  • KQL database

    • Name
    • Description
    • Caching Policy
    • Retention Policy
    • Logical guid

Data-level integration

Data-level integration is achieved through the use of a KQL script to create or modify database objects schemas, properties, and policies. However, it's important to note that not all commands supported in a KQL script are compatible with Microsoft Fabric ALM.

  • KQL database

    The following database objects are supported in the KQL script:

    • Table
    • Function
    • Table policy update
    • Column encoding policy
    • Materialized view
    • Table ingestion mapping

    For information about supported commands, see the DatabaseSchema.kql file description under KQL database files.

Git integration representation

Each eventhouse and KQL Database items synced with git appear in its own folder named using the following format: <ItemName>.<ItemType> where <ItemName> is the name of the item and <ItemType> is the type of the item. For example, for an eventhouse named Example that has a single KQL database named ExampleDB, the following folders appear in the git repository:

  • Example.Eventhouse
  • ExampleDB.KQLDatabase

Eventhouse files

The following files are contained in an eventhouse folder:

  • .platform

    The file uses the following schema to define an eventhouse:

    {
  "$schema": "https://developer.microsoft.com/json-schemas/fabric/gitIntegration/platformProperties/2.0.0/schema.json",
  "metadata": {
    "type": "Eventhouse",
    "displayName": "",
    "description": ""
  },
  "config": {
    "version": "2.0",
    "logicalId": ""
  }
}

  • EventhouseProperties.json

    The file allows you to configure platform-level settings for the eventhouse item.

KQL database files

The following files are contained in an KQL database folder:

  • .platform

    The file uses the following schema to define a KQL database:

    {
  "$schema": "https://developer.microsoft.com/json-schemas/fabric/gitIntegration/platformProperties/2.0.0/schema.json",
  "metadata": {
    "type": "KQLDatabase",
    "displayName": "",
    "description": ""
  },
  "config": {
    "version": "2.0",
    "logicalId": ""
  }
}

  • DatabaseProperties.json

    The file uses the following schema to configure platform-level settings for the KQL database item:

    {
  "databaseType": "ReadWrite",
  "parentEventhouseItemId": "",
  "oneLakeCachingPeriod": "P36500D",
  "oneLakeStandardStoragePeriod": "P36500D"
}

    The following table describes the properties in the DatabaseProperties.json file:

    Property Description
    databaseType Valid values: ReadWrite
    parentEventhouseItemId The logical ID of the parent eventhouse. This shouldn't be modified.
    oneLakeCachingPeriod Database level setting for the caching policy.
    oneLakeStandardStoragePeriod Database level setting for the retention policy.

  • DatabaseSchema.kql

    The file is a KQL script that configures the data-level settings for the KQL database. It's automatically generated when the KQL database is synced to git. The file is executed when syncing to your Fabric Workspace.

    You can make changes to this script by adding or modifying the following supported commands:

    Database object Supported commands
    Table Create or merge
    Function Create or alter
    Table policy update Alter
    Column encoding policy Alter
    Materialized view Create or alter
    Table ingestion mapping Create or alter

    The following is an example of a kql script to create a table and its ingestion mapping.

    // KQL script
// Use management commands in this script to configure your database items, such as tables, functions, materialized views, and more.

.create-merge table SampleTable (UsageDate:datetime, PublisherType:string, ChargeType:string, ServiceName:string, ServiceTier:string, Meter:string, PartNumber:string, CostUSD:real, Cost:real, Currency:string)
.create-or-alter table SampleTable ingestion csv mapping 'SampleTable_mapping' "[{'Properties':{'Ordinal':'0'},'column':'UsageDate','datatype':''},{'Properties':{'Ordinal':'1'},'column':'PublisherType','datatype':''}]"

Related content