Make sure the projects are of .NET framework 4.5.2

To write a test client which completely mimics the ADF, write a console application with the code as shared.

1. Following nugets need to be installed first:

• Azure.Management.DataFactories – provides ADF management capabilities
• Moq – a mocking framework for .NET

 

2. Make linked services for azure table storage, batch and azure documentDB:

• Azure Storage Linked Service connection string (Input):
  • Account Name - schoolrecordsstorage
  • Account Key – can be found on the portal
  • Default Endpoint Protocol – https

 

• Azure Batch Linked Service (Batch):
  • Account Name – schoolrecordsbatch
  • Batch URI - https://northeurope.batch.azure.com
  • Access Key - can be found on the portal
  • Pool Name – streamspool
  • Linked Service Name – Azure Storage Linked Service’s name

 

• Azure documentDB linked service connection string (Output):
  • Account Endpoint - https://documents.azure.com:443/
  • Account Key – can be found on the portal

 

3. Create list of all the linked services being used by ADF

 

4. Make datasets for azure table storage and documentDB

 

• Azure Storage Dataset definition (Input):
• Structure - List<DataElement>() where new DataElement are EnglishScore, MathScore, ChemistryScore, PhysicsScore, ComputerScore, AccountsScore, BusinessStdScore, EconomicsScore
• Linked Service Name – associated storage linked service
• Type Properties – SubjectsScoreStorage is the table name
• External – true (when data in storage is not coming from some other pipeline but coming from external source)
• Availability – 1 Hourly
• Policy – Specify the retry interval and timeout and maximum no. of retries

 

• Azure Blob Dataset definition (Input):
  • Linked Service Name – associated storage linked service
  • Type Properties - FolderPath = "binariescontainer"
  • External – true
  • Availability – 1 Hourly

 

• Azure DocumentDB Dataset definition (Output):
  • Linked Service Name – associated documentDB linked service
  • Type Properties – TestAssetsR StudentCollection is the collection name
  • Availability – 1 Hourly

 

5. Create list of all the datasets being used by ADF

 

6. Create an activity within the pipeline:

 

• Assign Storage and Azure Batch as inputs to it
• Assign DocumentDB as output to it
• Type Properties – Give the following details in here:
  • Assembly Name: dll StudentGradingAdfActivity.dll
  • Entry Point:TransformActivity (i.e., namespace.className of entry class)
  • Package Linked Service: Azure Storage Linked Service Name
  • Package File: binariescontainer/ StudentGradingAdfActivity.zip (i.e., a zip containing only .dll and .pdb files which has the custom activity dll as well)
  • Extended Properties: Containing all the configurable properties, namely, sliceStart, sliceEnd, dataStorageAccountName, docDbEndpointUrl, sinkDbName, dataStorageContainer, azureStorageDatasetName, docDbDatasetName, docDbLinkedServiceName, azureStorageLinkedServiceName, blobDllStorageDatasetName, queueName, storedProcedureName, queueWrapperWriteSize, docsWriteBatchSize, (dataStorageAccountKey, docDbAuthKey, appInsightsInstrumentationKey)
  • Policy: Define the policy to follow for pipeline scheduling, namely, order of execution, retry count and timeout.
  • Scheduler: 1 Hourly

 

7. Create a mock for the IActivityLogger. This will be needed in the next step

 

8. Call Execute(IEnumerable<LinkedService>, IEnumerable<Dataset>, Activity, IActivityLogger). Pass mock logger to it as logging is not needed while running the adf test client