How can i create a linked service in ADF for Sharepoint online?

Question

I want to extract files from sharepoint to ADL by only using ADF. I followed few steps

Step1: Azure Active Directory

-> Registered new app

-> created new secret key

I have the Tenant ID, ClientID(App ID), Secret Key

Step2: Sharepoint online

-> I have gone to my sharepoint (https://sharepoint.com/sites/mysites/_layouts/15/appinv.aspx) and tried to add new app

So, Here I am seeing this message

Using Azure ACS (Access Control Services) for SharePoint Online has been retired as of November 27th 2023, checkout the full retirement announcement to learn more. Using Azure ACS outside of the context of SharePoint was already retired on November 7th, 2018 and is end-of-life now.

So if this service goes away then how can i add app ID and access sharepoint through ADF??

I only want to use ADF, not any other service like logic apps, powershell script etc..

Answer 1

@Siddartha Reddy Jammula As for ACS retirement, it is recommended to switch applications to use Microsoft Entra ID for authorization and authentication needs.

To extract files from SharePoint to Azure Data Lake (ADL) using Azure Data Factory (ADF) without Azure Access Control Services (ACS), you can follow these steps:

1.Register an application in Azure Active Directory: You have already completed this step.

2.Grant SharePoint Online site permission to your registered application:

• Since Azure ACS is retired, you’ll need to use the service principal authentication method.
• Go to your SharePoint Online site (replace [your_site_url] with your actual site URL): https://[your_site_url]/_layouts/15/appinv.aspx.
• Search for the application ID you registered, fill in the empty fields, and click “Create”.
• Permission Request XML:

<AppPermissionRequests AllowAppOnlyPolicy="true">

<AppPermissionRequest Scope="http://sharepoint/content/sitecollection/web" Right="Read"/>

</AppPermissionRequests>

Note: The “App Domain” and “Redirect URL” refer to the SharePoint app that you have registered in Microsoft Entra ID to allow access to your SharePoint data.

3.Create a linked service in ADF: Use the SharePoint Online List connector in ADF, which supports service principal authentication. Set up the linked service with the Application ID, Application key, and Tenant ID you obtained earlier.

4.Copy data using ADF: Use the Copy Activity in ADF with the SharePoint Online List connector as the source. If you need to copy files (not just list items), you can use a Web activity to authenticate and grab an access token from SharePoint Online, then pass it to a subsequent Copy activity with an HTTP connector as the source.

Refer: https://learn.microsoft.com/en-us/azure/data-factory/connector-sharepoint-online-list?tabs=data-factory

Let me know if you have any questions in the comments.