Sending emails from SharePoint Workflows with SendGrid

This post is mainly for those who want to achieve email sending from SharePoint Designer Workflows. (If you don't want to read through all the drama, jump directly to the Falling Action section.)

Exposition

With the introduction of SharePoint 2013's new Workflow infrastructure, we have to learn new ways to get stuff done. Take for instance the simple task of sending emails outside of your organization. Say, you have a business logic that requires informing certain individuals.

When building a SharePoint 2010 Workflow, you have a nice little step for the purpose of sending out emails:

Do we not have the same in an SP2013 Workflow? Of course, we do. What happens however, when you want to send an email to someone who does not have a valid SharePoint account? You get a nice, sweet error message, stating that most probably the person won't receive these emails.

That's not nice.

So what options do we have? If it is an on premise installation, you can write your own custom Workflow Step that goes around this. Or just use one of the existing solutions out there. But what if you are in O365? Or just planning to go there in the near future? What if you cannot just install custom code? What then?
You could probably use some external web service. You could once again write your own solution, but what if you are not capable of that? Or maybe use something that already exist?

Rising action

There are a few companies already that build their business on sending massive number of emails and providing web service or other APIs for automation. As the title suggests, I will detailing my challenges with SendGrid. If you are still interested, please continue reading.

In the SharePoint 2013 Worfklow infrastructure you have an option to call a Web Service. The URL for this is: https://api.sendgrid.com/api/mail.send.json. Now if you go and read the API documetnation more thoroughly than I did, you will avoid all the mistakes that stumbled across. Unfortunately I learn by trial and error, but let's not run ahead.

So SendGird has this fine WebAPI to send emails. (They even give you a nice form to try this out.) SharePoint has a a step to call a Web Service. We just have to put them together. Cheesy easy. ( Everything is Awesome! )

Act 1

Let's see it in practice then! (Many blog posts detail how calling web service from SharePoint 2013 Workflow works and how you should create the necessary steps, so I won't go into much detail, just show you the main steps.)

You need a Dictionary for the request header first:

• Accept
  • String
  • application/json; odata=verbose
• Content-Type
  • String
  • application/x-www-form-urlencoded; charset=UTF-8
• Authorization
  • String

Note that the Authorization key is empty. We need this, because we want to make sure the Workflow does not try to authenticate with the current user's or the Workflow App's credentials.
Also note that the Content-Type is not JSON. Please note it. Did you note it? 'Coz I didn't. Once again, let's just not run ahead. (Yeah, yeah. If you're inpatient, please just jump to the Falling Action section.)

Then one Dictionary for the request content:

And of course you need the Call Web Service action as well:

• URL: https://api.sendgrid.com/api/mail.send.json
• HTTP Method: POST

Also, you need to configure the additional parameters of the Web Service call:

To make sure I know what the result was, I simply logged the SG_RespContent and SG_respCode variables into the Workflow Log. Here are the results:

• Response code was: BadRequest
• Response content was: {"errors":["Permission denied, wrong credentials"],"message":"error"}

Anything but what I wanted to see. ( You've been thunderstruck )

 

Act 2

Of course I did not understand why I get "Permission denied, wrong credentials" error message. I was sure the username and password was correct. What the hell is this then?

Fortunately all the communication is going through HTTPS, so with a little bit of tweaking and an on premise test environment I was able to tract down what is happening.

This is what you see when you use the test from SendGrid:

And this is what Workflow Manager does:

What??? ( I'm on a highway to hell... )

 

Act 3

Of course the SharePoint 2013 Workflow engine can handle only JSON. As you can see in the screenshot above, this is exactly what it does. The Dictionary I put together with so much love and devotion appears as a nicely formatted JSON body in the request, while the one that SendGrid gives you as an example is nothing else, but a set of URI arguments. (At this point I was assuming the sample is just some legacy stuff and not have been updated to JSON yet.)

But the URL says /api/mail.send.json, so it must be JSON. But why it is not working then? Especially, why I get HTTP 400 error? Maybe I am malforming the request?

After a few hundred trial and error sessions ( My mind was aching... You shook me all night long ) I gave up and opened a query with the SendGrid community, hoping someone will tell me what I am doing wrong. ( Hope of deliverance )

 

Climax

Ask and it shall be given. Jacob was so kind to highlight what I should have known - if I am not lazy enough in the beginning to read the API documentation thoroughly - that the web API is not upgraded yet to be able to handle JSON, so all the parameters should be passed as URI arguments.

Maybe a more talkative error message could have come handy? We could argue on that, as after all I have not sent the correct authentication keys, so indeed it is bad credentials from their perspective, from mine it is just a malformed request in general.

So how do I explain my SharePoint 2013 Workflow to send URI arguments instead of a well formed JSON request?
( Mission Impossible )

 

Falling Action

Fortunately enough the guys at SendGrid were smart enogh to make the current web service stupid-proof, so in case someone just sends a hell a lot of junk in the URI arguments, they just ignore it, and parse whatever they find sensible.

We know that SharePoint can send only JSON. We also know that for this we have to use a Dictionary. Here came my drift of thoughts:

• Does it say anywhere that we cannot put a big long string into the Dictionary? No, it doesn't.
• Does it say anywhere that a dictionary key must have a Name? No, it doesn't.

( Breaking all the rules )

 

So let's put it together then. ( Final Countdown )

Create a simple String variable and put all the important information into it:

&api_user=<YourSendGridUser>&api_key=<ThePasswordForIt>&to=anyone@example.com&toname=Zsolt Illes&subject=Test Subject&html=test&from=anyone@example.com&

NOTE: you have to start and end your string with the '&' sign to make sure everything before and after can be safely trashed by the web service.

 

Then create a Dictionary and put this variable into it with no Name:

 

The rest of the Web Service call just remains the same. Now issue your request and voilá it works like a charm now. Fiddler will show you something really weird, but if it looks stupid, but it works, then it ain't stupid.

{"''":"&api_user=<YourSendGridUser>&api_key=<ThePasswordForIt>&to=anyone@example.com&toname=Zsolt Illes&subject=Test Subject&html=test&from=anyone@example.com&"}

 

 

Dénouement

Moral of the story: with the change to the SharePoint 2013  Workflow infrastructure you will have to shift your way of thinking to achieve your goals. Could this be done easier? Sure. With smaller footprint on your environment? Not likely. This is what counts after all; that your SharePoint on premise installation stays as Vanilla as it can. If something is not incorporated into the product, then don't amend it, but use external web services. This will make sure that you can later migrate to the Cloud smoothly and easily. Future is the Cloud after all.

( Blue skies )

Comments

• Anonymous
  September 20, 2016
  The comment has been removed
  • Anonymous
    September 21, 2016
    Hi Vadim,Yes, I had the same issue. I could not find any workaround, plus I needed some formatting, so I ended up using HTML in my message. Not sure if elasticemail supports that or not.Cheers,Zsolt
    • Anonymous
      September 21, 2016
      Thank you Zsolt!May be it will be usefull - elasticemail supports GET requests to send mail and it works good enough with short messages. But after 2048 symbols of the whole request workflows is paused with error.
• Anonymous
  March 11, 2017
  Hi Zsolt. I need to send a rest call to azure AD from SP 2013 WF and it seems, that your post will help me a lot to achieve this. Very helpful thank you!!! (may I ask why didn't you run an SP2010 WF from SP2013 WF and delegate to it to send email outside your organization?
  • Anonymous
    March 12, 2017
    Hi Zoltan,Yes, in theory sending email with the SP2010 workflow would work, and in an on prem environment it most probably would. In O365 however the infrastructure proved to be less reliable, and troubleshooting the issues is anything, but easy.Furthermore, the solution where we implemented this method requires to know whether the email has actually been sent and viewed. With the SharePoint built-in mechanism you do not have this option, but using a 3rd party provider like SendGrid would give you all kind of statistics. Cheers,Zsolt
    • Anonymous
      March 13, 2017
      Well I understand now, thank you for clarifying!By the way, POSTing x-www-form-urlencoded request body to Azure API worked like a charm with your method. Thank you again!
• Anonymous
  May 09, 2017
  Hmmm ... SPD2013 isn't letting me create the dictionary variable without supplying a name ...
  • Anonymous
    May 09, 2017
    Never mind! My bad! I didn't see that you had put in 2 single quotes into the name field ... once I did that, it worked like a charm!!! I can't thank you enough for sharing this! I've been trying to find a workaround for DAYS!! Thanks again!
    • Anonymous
      May 09, 2017
      I'm happy it worked. :)Even though SendGrid now has a new API, unfortunately we still cannot use that from a Workflow as the WF infrastructure does not support providing the Bearer header. :(