Asynchronous on-demand site collection provisioning to Office 365 with Azure WebJobs

Artigo
03/04/2015

This is follow up post for my previous post related on building on-demand asynchronous operations using Azure storage queues and Azure WebJobs. Like mentioned in the initial post, one of the typical use cases for this kind of pattern would be to use it for site collection provisioning in the Office 365.

In this blog post we will walkthrough our reference sample for asynchronous on-demand site collection provision to Office 365 with Azure WebJobs. This is pretty typical solution needed for customers in the Office 365 if they are looking for self service site collection capabilities for their end users. Our sample scenario is relatively simple and concentrates on provisioning new site with out of the box site definition and only customization is a custom branding, which is applied to the site. You could though easily extend this solution for example to have following typical capabilities.

Request approval – Provision requests entry to SP site and start actual site creation when item is approved
Complex templates – provision always oob site, but apply difference customizations based on end user selection
Site metadata – Collect metadata from the site and store that to reusable location
Site directory – Show information on provisioned sites with for example filtering support on collected metadata

Logical design for our solution

Here’s a logical design and usage of the different techniques for achieving this solution.

Provider hosted app is installed and hosted in Office 365
Provider hosted app shows the different template options and collects other relevant metadata for creation
- Notice that these could be abstract templates like just templates called as X, Y and Z. Each one will be based on oob team site, but difference is the changes which are applied to the site after the oob site provision is completed. You do not want to use web templates or site templates.
Actual request is added to storage queue for async processing and end user will be notified on this saved request
Storage queue will automatically fire a connected WebJob, which will then have responsibility to provision requested site with given configuration and branding
Actual site is provisioned to the Office 365 using app only token by the WebJob

Reference solution structure

Here’s the Visual Studio solution structure.

Provision.Cloud.Async.WebJob.Console.Create

This is helper project to ensure that you have properly registered app id and app secret with tenant permissions to your site collection. You can do this by using appregnew.aspx and appinv.aspx pages. See Kirk Evan’s blog post for more information on this.

You should update tenant name and admin account email to the app.config. You should also update AppId and AppSecret keys accordingly in the app.config.

 <appSettings>
   <add key="TenantName" value="contoso" />
   <add key="SiteColTestOwnerEmail" value="admin@contoso.onmicrosoft.com" />
   <!-- Register app id and secret with tenant permissions for this -->
   <add key="ClientId" value="[Your App ID]" />
   <add key="ClientSecret" value="[Your App Secret]" />
 </appSettings>

Provision.Cloud.Async.WebJob.Console.Message

This is helper project which can be used to ensure that you have storage queues connection string properly available and you can use this project to easily send site collection creation messages to the storage queues without running any UI elements. You should be updating the Azure storage connection string from the app.config project to ensure that this works properly and if you truly want to test the whole provisioning story, update also the tenant name and owner email information.

 <appSettings>
   <add key="TenantName" value="contoso" />
   <add key="SiteColTestOwnerEmail" value="admin@contoso.onmicrosoft.com" />
   <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=[AccountName];AccountKey=[AccountKey]" />
 </appSettings>

Provision.Cloud.Async.WebJob

This is the actual SP app project for the provider hosted app UI. We do ask permission to the Web level so that we can resolve details on the user who is accessing the app in the code.

Provision.Cloud.Async.WebJob.Common

This is the business logic component used by the helper projects and the actual projects for storage queue handling and also for the site collection creation.

Provision.Cloud.Async.WebJob.Job

Actual web job project, which is deployed to the Azure web site. This is continuous running WebJob which will be automatically called when new items are being added to storage queue. You will need to configure app id and app secret in the app configuration and also the connection strings for the WebJob as follows.

 <appSettings>
   <!-- Use AppRegNew.aspx and AppInv.aspx to register client id with proper secret -->
   <add key="ClientId" value="[Your Client ID]" />
   <add key="ClientSecret" value="[Your Client Secret]" />
 </appSettings>
 <connectionStrings>
   <!-- The format of the connection string is "DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" -->
   <!-- For local execution, the value can be set either in this config file or through environment variables -->
   <add name="AzureWebJobsDashboard" connectionString="DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" />
   <add name="AzureWebJobsStorage" connectionString="DefaultEndpointsProtocol=https;AccountName=NAME;AccountKey=KEY" />
 </connectionStrings>

Notice that above settings can be configured either in app.config before you deploy WebJob to Azure or from web site configuration tab in the Azure side.

Provision.Cloud.Async.WebJobWeb

This is the self service user interface to request site collections. You can use this to fill the form and submit request to the storage queue. Actual message submission logic is actually called from the Common project.

 <appSettings>
   <add key="ClientId" value="bd8b31ca-ea88-448a-875f-33eb9e99ae73" />
   <add key="ClientSecret" value="g63EIwWqa/RwrBUxLw8eVTcOftQ+WSbjMNXUJ74htJo=" />
   <add key="StorageConnectionString" value="DefaultEndpointsProtocol=https;AccountName=[AccountName];AccountKey=[AccountKey]" />
 </appSettings>

Here’s the current user interface for requesting new site collections from end user perspective.

When a request has been submitted, end user will be shown following confirmation on the received message. Request is actually added to Azure storage queue, which will then automatically start up the provision process from WebJob.

Video on the solution usage in practice

Here’s a short video on using this solution in practice, which explains the solution structure and purpose of each project. Notice that this one does not explain details on the pre-requisites related on Azure storage queues and web job deployment, which are the same as for the sample in the “Using Azure storage queues and WebJobs for async actions in Office 365”.

Office 365 Developer Patterns and Practices

Techniques showed in this blog post are part of the Provisioning.Cloud.Async.WebJob solution in the Office 365 Developer Patterns and Practices guidance, which contains more than 100 samples and solutions demonstrating different patterns and practices related on the app model development together with additional documentation related on the app model techniques.

Check the details directly from the GitHub project at https://aka.ms/OfficeDevPnP. Please join us on sharing patterns and practices for the community for the benefit of the community. If you have any questions, comments or feedback related on this sample, blog post or anything on PnP, please join us and more than 1700 others who are using Office 365 Developer Patterns and Practices Yammer group at https://aka.ms/OfficeDevPnPYammer.

“From the community for the community” – “Sharing is caring”

Comments

Anonymous
March 15, 2015
I'm having a "Access denied" error on the console program - I made sure to set the AccessLevel Permissions - but still haven't been able to get it working. I've included the correct Client Id and Secret - and made sure that I'm using the v16 DLL's - is that correct ? (not v15)
Anonymous
March 15, 2015
ACTUALLY - I had the wrong app permissions - and the console app NOW creates a site collection. But - the WebJob is complaining about needing v4.0.1.0 of the AzureStorage DLL. And - there's some issues with the SharePoint app - it's a complex solution !
Anonymous
March 15, 2015
Sorry for the SPAM - but I have the solution working now. Was some problems with the binding redirect, and the versioning of the DLL for Azure.Storage - and - I'd incorrectly registered my app with the wrong domain. It's a great pattern/function - thanks !
Anonymous
March 15, 2015
Hi Chris, no worries on the "spam", there are some challenges with the default WebJob template, which oob references different nuget version than the latest one. As long as you got it to work properly. I would also suggest to use the PnP Yammer group for any questions / comments, since that's definitely faster way to get support than these blog comments. PnP Yammer group - aka.ms/OfficeDevPnPYammer
Anonymous
March 25, 2015
Hi Vesa, Thank you for these articles on WebJobs and Storage, it's the exact information I need. I'm using webjobs to create subsites. It is a very long process which is why I'm using WebJobs. Like yourself I'm using Continuous webjobs. However, I found when I was remote debugging, before I finished stepping through the code, the same item in the queue was fired again. Is this expected? Or is there a way to ensure that the current queue item isn't already being processed? Paul.
Anonymous
March 26, 2015
Hi MrCann0nF0dder, there is indeed time out, but my understanding is that this time out is 10 minutes for WebJobs. What happens is that we take a message from queue and try to process that. If processing is not successful due exception or a timeout, we drop the message back to queue and re-try 5 times before moving it to poison queue. I could not find any way to adjust this time out, but will be digging around... 10 minutes should be sufficient in general, unless you actually do debugging like in your case.

Partilhar via