Myths and truth about the SharePoint Workflow History list
This post is about demystifying the Workflow History list and some of the confusion around it.
Exposition
To be honest, this is a second try around this topic. The whole story started with a question from my colleague Christopher Scott. His customer is in SharePoint Online now and using Workflows to automate their business logic. For various reasons they need to keep the Workflow History entries for a long time. While they've been using an on-prem installation, they simply disabled the Workflow History Cleanup timer job, but obviously in SPO this is not something you could do. So the question was, how to keep the history entries with a single click to the users?
Here comes why this is the second try. Even I got confused a bit. Let's see how.
Rising Action
Act 1
Before I dig deep into this, a little clarification for those who are not familiar with what the Workflow History Cleanup timer job does. Despite of what the name suggests, it does not clean up the Workflow History list. (This is where the log entries you add in your workflow are stored.) It does, however, two other things:
- removes the SP2010 Workflow History link from the list item
- removes the Workflow Instance reference from the workflow service's register.
This also means that the items will ever be in the Workflow History list, unless you start manually deleting them. (Which you should, by the way. Stand by for a post on that...)
As for the SP2013 style workflows, the reference is not stored in the SharePoint Workflow infrastructure, but in the Workflow Manager. So there is a different logic for that, but the behavior is kind of the same, the result is that your Workflow instance information is gone after 30 days.
Act 2
Given the fact that the entries in the Workflow History list are there, you might think, if you save the link for the Workflow History page, you can access the history information forever. Well... That's where you are wrong, kiddo!
If you are familiar with the Workflow History page (of course you are, that's why you are reading this entry) , you know there is a field called Internal Status. It is there for both SP2010 and SP2013 style workflows.
The value of this field is also responsible for displaying some context information, like stopping the workflow, list of the related Tasks, etc.
This is where the Workflow History Cleanup timer job comes into play again. As I mentioned it earlier, it cleans the reference in the SP2010 style Workflow Infrastructure, leaving you with a nice "Invalid WorkflowInstanceID parameter in URL." error message.
The error with the SP2013 style workflows is a bit different, but the result is the same, you cannot see the WF History entries.
That's pretty much it. The built-in Workflow History page is not usable. It would be a lot easier if - instead of the above error messages - the page would simply put "Already cleaned up" into the Internal Status field, and still display the rest of the page, but it does not.
Climax
Act 3
Does this mean that the information we've stored in the Workflow History list is useless? Well... Not entirely. While there is no built-in way to display the content, it does not mean that we cannot dig out this information. Let's take a quick look at the actual WF History List that you can reach on a similar URL https://www.contoso.com/sites/TestSite/Lists/Workflow%20History/AllItems.aspx. You would see something like this:
[caption id="attachment_436" align="alignnone" width="300"]
This is how the actual Workflow History List looks like[/caption]
Let's observe the columns a bit:
| Workflow engine | Workflow History Parent Instance | Workflow Association ID | Workflow Template ID | List ID | Primary Item ID |
|---|---|---|---|---|---|
| 2010 Style WF | GUID of the WF instance | GUID of the WF association | GUID of the WF association version | GUID of the List | Number of the item in the List |
| 2013 Style WF | GUID of the WF instance | Name of the WF association | N/A | N/A | N/A |
How to get back the information then?
Act 4
For the SP2010 style workflows it is relatively easy. You either know the GUID of your Workflow's association, or you will have to look it up. Hopefully you are in the lucky situation that you have not deleted the Workflow from the list yet, so you can get it with a simple REST call to the Workflow service. (Should you need help with playing with the REST calls, read this entry.)
https://www.contoso.com/sites/testsite/_api/Web/Lists/GetByTitle('MyList')/WorkflowAssociations?$filter=Name eq 'My2010WFName'&$Select=BaseID,ListID
- The BaseID will be equal to the WorkflowTemplateID.
- While the ListID is the GUID of the original list where the item is.
Thus, if you know the ID (not the GUID) of the item, you can use the below REST call to get all the related entries from the Workflow History list.
https://www.contoso.com/sites/testsite/_api/Web/Lists/GetByTitle('Workflow%20History')/Items?$filter=((substringof(' <ListGUID> ',List)) and (substringof(' <BaseID> ',WorkflowTemplate)) and (Item eq <ItemID> ))&$orderby=ID desc
To understand the above query, keep in mind that:
- Filter for the WorkflowTemplateID, where it equals the BaseID,
- Order the results by Modified descending.
- Should you have run the workflow more than once, it will give you all the entries for that Workflow definition.
- (Note that the { and } characters are forbidden in the Workflow definition, so we have to use the SubstringOf method to filter for the List and WorkflowTemplate columns.)
Act 5
The SP2013 style workflow is a bit different. Remember, the link is not cleaned up by the Workflow History timer job, so at least we have the WorkflowInstanceName in the URL.
| WF Style | Link schema |
|---|---|
| 2013 Style WF | /sites/testsite/_layouts/15/wrkstat.aspx?List=8d56ca92-0687-4032-972c-10df5ad0d82e&WorkflowInstanceName=3bf2ef56-d100-4a61-9354-2b08f4a0aacc |
This information we can use in a similar REST query:
https://www.contoso.com/sites/testsite/_api/web/lists/GetByTitle('Workflow%20History')/Items?$filter=WorkflowInstance eq ' <WFInstanceGUIDFromURL> '$orderby=ID desc
Luckily as here we are using an actual Workflow instance GUID, we won't have information about the previous runs. I assume you can guess, this means the information about previous runs is lost forever. ... ... Unless... ... By any chance you put the item's ID into one of the comment fields, in which case you can still dig the info out. But then we really are getting into the archeology of Workflows.
Falling Action
So getting the information from the Workflow History page is possible even after the related workflow instance has been cleaned up. This is not too convenient and if you really want to use it, you should create your own Workflow History page.
Dénouement
It would be a lot more comfortable if the built-in Workflow History page would not fail if the Workflow instance ID has already been cleaned up. But given the fact that both the SP2010 and the Sp2013 style workflows are already in deprecated status and that Microsoft is focusing on MSFlow which does not have a default history list anyway, it would be a good idea to get used to creating your own workflow history in case you need it for legal reasons.
(Stand by for the next entry around that topic.)
Comments
- Anonymous
June 15, 2017
Excellect post, thanks! :)To your next post: I can provide you some real screenshots why is it so important to keep the number items in WF History list on a manageable level. We had some experience in the past with 18 million items in that list:)- Anonymous
June 15, 2017
That would indeed come handy. I'll ping you for that. Thanks.- Anonymous
February 22, 2018
I'd like to read that about the unmanageable WF history lists.
- Anonymous
- Anonymous
- Anonymous
November 27, 2017
The comment has been removed- Anonymous
February 27, 2018
Whenever you get an error, I suggest you executing the query from Fiddler. There you can see the response and you'll be able to see what the error message behind the HTTP 500. Most probably you'll get the 500 error due to hitting the list view threshold. If you send the full error message, I might be able to tell more.
- Anonymous