Copilot for Data Factory overview

Important

This feature is in preview.

Copilot in Fabric enhances productivity, unlocks profound insights, and facilitates the creation of custom AI experiences tailored to your data. As a component of the Copilot in Fabric experience, Copilot in Data Factory empowers customers to use natural language to articulate their requirements for creating data integration solutions using Dataflow Gen2. Essentially, Copilot in Data Factory operates like a subject-matter expert (SME) collaborating with you to design your dataflows.

Copilot for Data Factory is an AI-enhanced toolset that supports both citizen and professional data wranglers in streamlining their workflow. It provides intelligent Mashup code generation to transform data using natural language input and generates code explanations to help you better understand earlier generated complex queries and tasks.

Before your business can start using Copilot capabilities in Fabric, your administrator needs to enable Copilot in Microsoft Fabric.

Note

Supported capabilities

With Dataflow Gen2, you can:

  • Generate new transformation steps for an existing query.
  • Provide a summary of the query and the applied steps.
  • Generate a new query that may include sample data or a reference to an existing query.

With Data pipelines, you can:

  • Troubleshoot data pipeline issues with the error explaination capability with troubleshooting guidance.

Get started

  1. Create a new Dataflows Gen2.

  2. On the Home tab in Dataflows Gen2, select the Copilot button.

    Screenshot showing Copilot icon on the Home tab.

  3. In the bottom left of the Copilot pane, select the starter prompt icon, then the Get data from option.

    Screenshot showing Get data from the starter prompt.

  4. In the Get data window, search for OData and select the OData connector.

    Screenshot showing Select the OData connector.

  5. In the Connect to data source for the OData connector, input the following text into the URL field:

    https://services.odata.org/V4/Northwind/Northwind.svc/
    

    Screenshot showing Connect to the data source.

  6. From the navigator, select the Orders table and then Select related tables. Then select Create to bring multiple tables into the Power Query editor.

    Screenshot showing Choose the data orders table and related tables.

  7. Select the Customers query, and in the Copilot pane type this text: Only keep European customers, then press Enter or select the Send message icon.

    Your input is now visible in the Copilot pane along with a returned response card. You can validate the step with the corresponding step title in the Applied steps list and review the formula bar or the data preview window for accuracy of your results.

    Screenshot showing Filter rows.

  8. Select the Employees query, and in the Copilot pane type this text: Count the total number of employees by City, then press Enter or select the Send message icon. Your input is now visible in the Copilot pane along with a returned response card and an Undo button.

  9. Select the column header for the Total Employees column and choose the option Sort descending. The Undo button disappears because you modified the query.

    Screenshot showing the Copilot pane and Power Query Online user interface.

  10. Select the Order_Details query, and in the Copilot pane type this text: Only keep orders whose quantities are above the median value, then press Enter or select the Send message icon. Your input is now visible in the Copilot pane along with a returned response card.

  11. Either select the Undo button or type the text Undo (any text case) and press Enter in the Copilot pane to remove the step.

    Screenshot showing the undo button.

  12. To leverage the power of Azure OpenAI when creating or transforming your data, ask Copilot to create sample data by typing this text:

    Create a new query with sample data that lists all the Microsoft OS versions and the year they were released

    Copilot adds a new query to the Queries pane list, containing the results of your input. At this point, you can either transform data in the user interface, continue to edit with Copilot text input, or delete the query with an input such as Delete my current query.

    Screenshot showing a new query being created.

Limitations of Copilot for Data Factory

Here are the current limitations of Copilot for Data Factory:

  • Copilot can't perform transformations or explanations across multiple queries in a single input. For instance, you can't ask Copilot to "Capitalize all the column headers for each query in my dataflow."
  • Copilot doesn't understand previous inputs and can't undo changes after a user commits a change when authoring, either via user interface or the chat pane. For example, you can't ask Copilot to "Undo my last 5 inputs." However, users can still use the existing user interface options to delete unwanted steps or queries.
  • Copilot can't make layout changes to queries in your session. For example, if you tell Copilot to create a new group for queries in the editor, it doesn't work.
  • Copilot may produce inaccurate results when the intent is to evaluate data that isn't present within the sampled results imported into the sessions data preview.
  • Copilot doesn't produce a message for the skills that it doesn't support. For example, if you ask Copilot to "Perform statistical analysis and write a summary over the contents of this query", it doesn't complete the instruction successfully as mentioned previously. Unfortunately, it doesn't give an error message either.