Event Hub Publisher Policy in Action

The 2.6 SDK released last week included some important enhancements to the client experience of working with Event Hubs. This blog will describe using SAS based publisher policy with Event Hubs.

This High Level Overview of SAS with Service Bus explains the background of SAS – the rest of this blog will show SAS in action.

If you don’t already have one you’ll need to create an Event Hub in the Azure portal.

 

Once that’s done click on your Event Hub

From here you can click on the Configure tab to get to the Share Access Policies at the bottom of the page.

The first thing to do here will be to create a new Policy. Do this by typing in a name and selection the permissions – in our case only check the Send box.

From here whenever you click the Connection Information within the context of your Event Hub you will see a SAS policy as shown below.

 

Clicking this brings up the dialog below.

Here’s where a lot of people get lost with SAS – what we have here is the Policy, not a SAS token. We can use this policy to send to Event Hub, but since you can only have a limited number of policies (and it’s a small number) you would need to share this key with all your senders. This method of sharing keys with multiple senders exposes the keys directly to the senders and greatly increases the change of the keys being lost or misused. A SAS Key is like a private key and the Key Name (the policy name we typed in) is like the public key. It’s safe to share public keys, but not private ones. Worse still once you have shared the SAS Key (the private key) if you ever needed to regenerate this key (done by clicking the Regenerate button back in that Configure screen where you create this policy) it would stop everyone who has this key from sending to your Event Hub.

Instead we can use this Policy to create a SAS Token – which is a time bound permission for a specific publisher using this policy (i.e. it allows a specific identity to send to this Event Hub, inheriting the Send permission from the Policy that creates the token).

If this all sounds complex it’s really not that bad. Here’s how you do it:

Create a .NET application, I used a Console application, and pull in the latest NuGet for Service Bus – ALWAYS use NuGet to source your packages. Copy out that Connection String from the portal shown in the image above. To make my life easier I may put this in Notepad or somewhere temporary while working.

Create variables for a publisher identity and an Event Hub name, these will be useful shortly.

 

 string publisherId = "1";
string eventHubName = "ingresshub";

Now I need to create a SAS Token from the Send SAS Policy that I created in the portal. Fortunately there is a method that does this for you shown below.

 string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(
 new Uri("sb://techreadypt.servicebus.windows.net/"),
 eventHubName, publisherId, "Send",
 "<YOUR PRIMARY KEY>",
 new TimeSpan(0, 5, 0));

The first parameter is the URI of my Service Bus namespace. The second is the name of my Event Hub. The third is the publisher identity I wish to create – any unique identifier for the publisher will do, it’s your responsibility to pick and manage these. Fourth is the name of the policy that is used to create this token – in our case Send (which you typed into the portal earlier). The fifth parameter is the Primary Key for this policy – you can find this after the SharedAccessKey= part of the connection string you copied out of the portal. The sixth parameter is the timespan you wish for this token to be valid – again the choice here is yours and depends a largely on your specific scenario.

Once you have this token it can be used by an Event Hub publisher. You must provide this to your publishers either at registration time or in some other method that you determine is appropriate for your needs. Once a publisher has this token sending events to the Event Hub is three simple steps.

1) Create a connection string to the Event Hub using this token. The four parameters here are fairly self-explanatory. Be sure the URI does not have the Event Hub name in it, that’s the second parameter.

 var connStr = ServiceBusConnectionStringBuilder.CreateUsingSharedAccessSignature(
 new Uri("sb://techreadypt.servicebus.windows.net/"),
 eventHubName,
 publisherId,
 token);

2) Create an EventHubSender – it’s not really worth creating an EventHubClient here as all you can do is send with this token.

 var sender = EventHubSender.CreateFromConnectionString(connStr);

3) Call send on the EventHubSender and use an EventData object to send your data.

 sender.Send(new EventData(Encoding.UTF8.GetBytes("Hello Event Hub")));

That was easy! Now you have a unique publisher identity that is secure and has a lifetime that is in your control – at least at creation time. Suppose, however, that you want to block this publisher from sending to the Event Hub. Until the SAS Token you provided them expires there isn’t a way in SAS to do this – so Event Hubs enables publisher revocation. To do this you’ll need a connection string to your Service Bus namespace that has Manage permissions. For your entire namespace by default this will be called RootManageSharedAccessKey. If you don’t have a UNIX background you should be aware that Root means something very powerful and scary – I always create a separate Manage policy, which you now know how to do, and I create it at the Event Hub level so it is scoped only to the Event Hub and not the entire Namespace. After doing this you can copy out the Manage connection string from the portal and instantiate a NamespaceManager object using that connection string.

 var nsm = Microsoft.ServiceBus.NamespaceManager.CreateFromConnectionString(manageString);

This is a powerful object that allows you to do even more than the Azure portal with regards to Service Bus – that’s why using a scoped Manage policy is a good idea. It is with this object that we can revoke individual publishers as shown:

 nsm.RevokePublisher(eventHubName,publisherId);

After performing the revocation calling the same send code from above will result in a Microsoft.ServiceBus.Messaging.PublisherRevokedException with the message: "Publisher: 1, was revoked access to Send messages to EventHub.” You can list the revoked publishers from an Event Hub using the method shown below.

 var revokedPublishers = nsm.GetRevokedPublishers(eventHubName);

It is worth noting here that if not carefully managed this list can grow quite large and the results will be paged in pages of 100. In addition to counting as management calls (ingress) the results are counted as egress traffic and large lists can result in throttling if you don’t have many Throughput Units.

Finally you can restore a publisher with this method.

 nsm.RestorePublisher(eventHubName, publisherId);

Publisher policy is a simple and powerful capability of Event Hubs that can be used to protect your processing pipeline. It is, however, important to remember that real security lies in short token lifetimes and being careful with your keys - much like how you would with your house keys, but not giving them to people you don't trust and changing your locks if you lose them.

Comments

• Anonymous
  February 04, 2015
  Event hub API seems to buggy to me. Here are few bugs (in case my understanding is correct)

1. "A device that possesses a token can only send to one publisher, but no other publisher"!!! Right, You can't change publisherId in url and send to another publisher. It result into exception. But you don't have to do that.  You can simply create a partition sender from EventHubClient. And using PartitionSender.Send(), we can send to any random partition.
2. "After performing the revocation calling the same send code from above will result in a exception Publisher was revoked action" Right, But I can take a different route. I can create PartitionSender from EventHubClient and use PartitionSender.Send() to any partition even if 'PERMISSIONS ARE REVOKED'!!! I found these issues when i was trying to test API. Please correct me in case I am missing something.

• Anonymous
  February 06, 2015
  Dan, Thanks for very helpful post! How about this scenario? I'm creating a SAS token and want to use it for many-many publishers. Say, I don't want to manage token-per-publisher, because publisher is some device (IOT device), and EventHub publisher installed as firmware. BTW Most of this firmware would be never changed in the lifetime of this device.

• Anonymous
  February 09, 2015
  @Balesh - here's where you can read a li'l bit more about publisher policy - readtocode.blogspot.com/.../event-hubs-publisher-policy-explained.html. Publisher policy is useful only for enabling per-sender access control. If you trust all your senders and share your SAS Key directly with senders - that means, you don't need Publisher policy here! So, the 2 API's that you were referring to are for 2 different purposes. I know that many people use SAS key name and SAS key as username/password pair! This is not the right way to use SAS key. Imagine SAS key as a token generator - and use this to generate tokens. Give these tokens to the users of EventHub.

• Anonymous
  February 10, 2015
  @SreeramG- I want to enable "** per-sender access control**". To do that, I created different SAS token for different devices/senders. You seem confused. The key name and key value you are referring to - are actually 'Policy'. I will share SAS "Token" and not the Policy. I  I do not trust my sender. Now, my sender should only be able to send data using this SAS token only. And each Token has a publisherId associated to it, which is resolved into one partition. You cannot change the publisher key and start sending data to it. It should not be possible and it is not. But there is a twist. You can use this 'Token' (which is supposed to work with 'EventHubClient API) and create connectionString. And now using this connection string, create PartitionSender. Using this API, you can send data to any partition. Which is wrong, because you are send data to partition which is resolved by your partition Id. This API even works when you revoke permissions from the Token.

• Anonymous
  February 10, 2015
  @Balesh: Let me understand what you are trying to do.. var device1Token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(                ServiceBusEnvironment.CreateServiceUri("sb", "firstehub-ns", string.Empty), "firstehub", "device1", "MyDeviceKeyName", "-----SASKEY-----", TimeSpan.FromDays(10)); var connStr = ServiceBusConnectionStringBuilder.CreateUsingSharedAccessSignature(                    ServiceBusEnvironment.CreateServiceUri("sb", "firstehub-ns", string.Empty),                    "firstehub", "device1", device1Token); var eventHubClient = EventHubClient.CreateFromConnectionString(connStr); eventHubClient.CreatePartitionedSender("0"); eventHubClient.Send(new EventData(Encoding.UTF8.GetBytes("Hello Event Hub"))); You created a device token (device1Token) and using that token - created a service bus connection string - using which, you created an EventHubClient and sent a message using PartitionedSender. And did this succeed ?   I don't even need to try this and am sure that this is 1000% guaranteed to throw @ eventHubClient.Send(...) step with error like - InvalidAudience/WrongAudience. Please correct me - if this is not what you are trying... NOTE: CreatePartitionedSender(...) is a client only client only call. SDK optimizes calls to the Service and will not talk to SB service untill you actually Send a message.

• Anonymous
  February 10, 2015
  @SreeramG Thanks for showing interest! Yes, This is what I want to convey. It does not throw exception. IT WORKS. Not only that, It works even when you revoke permissions from publisher. I tried with two samples. Sample 1:            var serviceNamespace = "--- NAMESPACE NAME HERE ---";            var hubName = "--- HUB NAME HERE ---";            var deviceName = "--- DEVICE NAME HERE ---";            var sasToken = "--- SAS KEY HERE ---";            // step 1: Create factory            var factory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", serviceNamespace, ""), new MessagingFactorySettings            {                TokenProvider = TokenProvider.CreateSharedAccessSignatureTokenProvider(sasToken),                TransportType = TransportType.Amqp            });            // step 2: Create Data            var data = new EventData(Encoding.UTF8.GetBytes(JsonConvert.SerializeObject(someObject)));            //step 3: Send message            var client = factory.CreateEventHubClient(String.Format("{0}/publishers/{1}", hubName, deviceName));            var partitionSender = client.CreatePartitionedSender("0");            partitionSender.Send(data); Sample 2:      string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(             new Uri("sb://----.servicebus.windows.net/"),             eventHubName, publisherId, "send",             sasKey,             new TimeSpan(0, 5, 0));        var connStr = ServiceBusConnectionStringBuilder.CreateUsingSharedAccessSignature(             new Uri("sb://----.servicebus.windows.net/"),             eventHubName,             publisherId,             token);        var message = "Some text data " + publisherId;        var eventData = new EventData(Encoding.UTF8.GetBytes(message));        var evetHubClient = EventHubClient.CreateFromConnectionString(connectionString, eventHubName);        Console.WriteLine("Message sending to 2 different partitions: {0}", message);        await evetHubClient.CreatePartitionedSender("0").SendAsync(eventData);        await evetHubClient.CreatePartitionedSender("1").SendAsync(eventData);

• Anonymous
  February 10, 2015
  @Balesh: Thanks a lot for your response.

1. In your sample 1 - you created a Factory with a TokenProvider - which has the SaSKey (not the device specific token):       TokenProvider = TokenProvider.CreateSharedAccessSignatureTokenProvider(sasToken)      - so your token - which is generated, will, in-purpose, have permissions to send to entire EventHub.   I agree with you that the API experience with the below code snippet might not be the best:            var client = factory.CreateEventHubClient(String.Format("{0}/publishers/{1}", hubName, deviceName));            client.CreatePartitionedSender()  // should have failed - but, we took a design design to not through      - and this was the primary reason for us to come up with the EventHubSender.CreateFromConnectionString() method to create a Sender directly to publisher. Please let me know if any of our MSDN samples lead you to this path. We will clean them up.
2.  In your sample 2: I believe you have a bug in your client code where you have 2 variables 'connStr' and 'connectionString' and are using connectionString to create the EventHubClient - which I believe would be made out of the SasKey. This time - I tried this one - as you actually forwarded the code snippet that failed for you. Thanks for taking time trying our API! Sree

• Anonymous
  February 10, 2015
  @SreeramG, In both the samples, I used device specific token (using following method). Let me know if it is not device specific token or not?            var serviceUri = ServiceBusEnvironment.CreateServiceUri("sb", "NAMESPACE", String.Format("{0}/publishers/{1}", "EVENT HUB NAME", "DEVICE NAME")).ToString().Trim('/');            sas = SharedAccessSignatureTokenProvider.GetSharedAccessSignature("SAS KEY NAME", "SAS KEY", serviceUri, TimeSpan.FromDays(1)); In sample2, this was a typo (connectionString in place of connStr).  But lets focus on sample 1. And when I revoked permission using RevokePublisher, I was still able to send message using PartitionSender. Your little more effort can save my day. Please assist. Thanks Balesh Kumar

• Anonymous
  February 10, 2015
  @SreeramG, The token I used look like something similar to this: SharedAccessSignature sr=sb%3a%2f%2fNAMESPACE.servicebus.windows.net%2fPARTITION_NAME%2fpublishers%2DEVICE_NAME&sig=SOME_KEY%3d&se=1423684427&skn=send Thanks Balesh Kumar

• Anonymous
  February 10, 2015
  @SreeramG, In my previous comment, token was not printed correctly, Lemme try again. SharedAccessSignature sr=sb%3a%2f%2fnagpartition-ns.servicebus.windows.net%2fnagpartition%2fpublishers%2DeviceName&sig=3cU7GuYMok4k5oy3aBcZi40TCSqlcaonGXJfBs6LsnU%3d&se=1423684427&skn=send Thanks Balesh Kumar

• Anonymous
  February 10, 2015
  @balesh: Please donot use this overload - as you specified above: sas = SharedAccessSignatureTokenProvider.GetSharedAccessSignature("SAS KEY NAME", "SAS KEY", serviceUri, TimeSpan.FromDays(1)); instead you need to use :  GetPublisherSharedAccessSignature string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature( new Uri("sb://techreadypt.servicebus.windows.net/"), eventHubName, publisherId, "Send", "<YOUR PRIMARY KEY>", new TimeSpan(0, 5, 0)); Just follow this blog: readtocode.blogspot.com/.../event-hubs-publisher-policy-explained.html and be rest assured that you are secure! HTH! Sree

• Anonymous
  February 10, 2015
  @SreeramG, I am also facing similar issue, I used the SAS TOKEN for per publisher scenario but when i use client.CreatePartitionedSender("4") then I am able to publish event on some other partition Id (although not "4", but some random partition ID). This should not be case as any of my rouge devices can publish unwanted events in the partitions that are not mean to be used by those devices. I am rather expecting an exception from the API. It seems like a security loop in the per publisher scenario. I used the following code snippet: var publisherId = "1d8480fd-d1e7-48f9-9aa3-6e627bd38bae"; -- use any GUID string token = SharedAccessSignatureTokenProvider.GetPublisherSharedAccessSignature(                    new Uri("sb://anyhub-ns.servicebus.windows.net/"),                    eventHubName, publisherId, "send",                    sasKey,                    new TimeSpan(0, 5, 0));                var factory = MessagingFactory.Create(ServiceBusEnvironment.CreateServiceUri("sb", "anyhub-ns", ""), new MessagingFactorySettings                {                    TokenProvider = TokenProvider.CreateSharedAccessSignatureTokenProvider(token),                    TransportType = TransportType.Amqp                });                var client = factory.CreateEventHubClient(String.Format("{0}/publishers/{1}", eventHubName, publisherId));                var message = "Event  message for publisher: " + publisherId;                Console.WriteLine(message);                var eventData = new EventData(Encoding.UTF8.GetBytes(message));                await client.SendAsync(eventData);                await client.CreatePartitionedSender("5").SendAsync(eventData);                await client.CreatePartitionedSender("6").SendAsync(eventData); Here, the last three lines publish the same event on three different partitions, which should not be the case as even though I am using the SAS Token which is supposed to publish only on one partition but  I can publish on any partition using the method CreatePartitionedSender.

• Anonymous
  February 10, 2015
  @SreeramG, We tried GetPublisherSharedAccessSignature() but still we are able send message to any partition using PartitionSender(). Please check previous comment by Abhishek. He is also facing the same issue. Ideally, CreatePartitionSender() should not work with a token which is supposed to work with a particular partition key (and partition internally). Thanks Balesh Kumar

• Anonymous
  March 02, 2015
  Please try this again. Fairly certain this is no longer possible.

• Anonymous
  March 29, 2015
  Hi Dan,  is there a way to enable token/tokenfactory when listen to event hub with EventProcessorHost? For below code, is there an way to introduce token/token factory, so that dont need to pass in connectionString. _host = new EventProcessorHost(                _startHostParams.HostName,                _startHostParams.EventHubName,                consumerGroup.GroupName,                connectionString,                storageConnectionString,                _startHostParams.EventHubName.ToLowerInvariant());            var factory = new EventProcessorFactory(_startHostParams.HostName, _startHostParams.EventDataProcessorFactory);            var XmlXapResolver = new EventProcessorFactory()            try            {                Trace.TraceInformation("EventHubListenerHost.StartHost() - Registering host: {0}", _startHostParams.HostName);                var options = new EventProcessorOptions();                options.ExceptionReceived += EventProcessorOnExceptionReceived;                await _host.RegisterEventProcessorFactoryAsync(factory, options);            }

• Anonymous
  August 13, 2015
  When using a non .NET sender (e,g, postman - chrome extension) to send messages via a revoked publisher, a 403 response is returned but the message gets into the event hub.

• Anonymous
  August 16, 2015
  Do you know if the EventHubClient is thread safe ? Can I use the same EventHubClient when sending messages from different threads / tasks ?

• Anonymous
  March 03, 2016
  Great post Dan! Thank you! @EventHubClient is thread safe, yes it is