How to send a large message to a Service Bus Relay Service using Streamed transfer mode
Introduction
This solution shows how to use the TransferMode = Streamed to send a large message to a Service Bus Relay Service using a streaming approach.
Requirements
The following list contains the software required for building and running the sample:
- Windows Server 2008 R2 SP1, Windows Server 2012, Windows 7 SP1, Windows 8 or higher.
- Windows Azure Service Bus 2.1 or higher.
- NET Framework 4.5
- Visual Studio 2012
- Any Microsoft SDK which provides a gacutil version for .NET 4.0/4.5.
- A valid subscription to Windows Azure.
Building the Sample
Inside the zip file you can find a solution that contains 3 projects:
- Client: this is a Windows Forms application that you can use to select multiple files (typically very large files, images, videos, etc.) from the local file system and send them in parallel to the WCF service, running on-premises or in the cloud, which exposes an endpoint in Windows Azure via a Service Bus Relay Service.
- Service: this is a Console Application hosting the WCF service. The UploadService receives messages from the client application and stores them in a local folder whose path is defined in the appSettings section of the configuration file. Note: in a production environment the service should run in a scalable, reliable and robust environment such as Internet Information Services (IIS).
- Contracts: this project contains the data and service contracts used by both the client an service.
Open the solution in Visual Studio 2012 and make sure to replace the following placeholders in the App.config file under the Client and Service projects:
- YOUR-NAMESPACE: specify the name of your Windows Azure Service Bus namespace.
- YOUR-NAMESPACE-ISSUER-SECRET: specify the issuer secret of your Service Bus namespace. You can retrieve the issuer secret of your namespace from the Windows Azure Management Portal as illustrated in the picture below.
Solution
The following picture shows the architecture of the solution:
Message Flow:
- The user selects multiple files from the local file system, then clicks the Send button. The client application sends the selected files to the Service Bus Relay Service using creates a separate Task and WCF channel for each message. The Windows Forms application can use two different client endpoints to send messages to the underlying WCF service . These endpoints are respectively configured to use the BasicHttpRelayBinding and NetTcpRelayBinding.
- The Service Bus Relay Service re-transmits the message to the WCF service.
- The WCF service, running on-premises or in the cloud, receives files from the client via the Service Bus Relay Service. The WCF service exposes two endpoints in Windows Azure. These endpoints are respectively configured to use the BasicHttpRelayBinding and NetTcpRelayBinding.
- The WCF service stores the incoming files to a local folder configured in the configuration file.
WCF Message Encoders and Transfer Modes
WCF includes three types of encoding for SOAP messages:
- Text Message Encoding: The text encoding represented by the TextMessageEncodingBindingElement is the most interoperable, but the least efficient encoder for XML messages. Web service or Web service client can generally understand textual XML. However, transmitting large blocks of binary data as text is not efficient.
- Binary Message Encoding: The BinaryMessageEncodingBindingElement is the binding element that specified the .NET Binary Format for XML should be used for encoding messages, and has options to specify the character encoding and the SOAP and WS-Addressing version to be used. Binary encoding is most efficient but least interoperable of the encoding options.
- MTOM Message Encoding: The MTOMMessageEncodingBindingElement represents the binding element that specifies the character encoding and message versioning and other settings used for messages using a Message Transmission Optimization Mechanism (MTOM) encoding. (MTOM) is an efficient technology for transmitting binary data in WCF messages. The MTOM encoder attempts to create a balance between efficiency and interoperability. The MTOM encoding transmits most XML in textual form, but optimizes large blocks of binary data by transmitting them as-is, without conversion to their base64 encoded format.
WCF transport channels support two modes for transferring messages in each direction:
- Buffered: transmissions hold the entire message in a memory buffer until the transfer is complete. On the service side message processing cannot start until the entire message has been received. This is the default mode for all the WCF bindings.
- Streamed: transmissions only buffer the message headers and expose the message body as a stream, from which smaller portions can be read at a time.
The TransferMode property exposed by transport protocol channel (e.g.HttpTransportBindingElement, TcpTransportBindingElement, etc.) and bindings (BasicHttpBinding, NetTcpBinding, BasicHttpRelayBinding, NetTcpRelayBinding, etc.) allows to indicate whether messages are sent buffered or streamed. Streamed transfers can improve the scalability of a service by eliminating the need for large memory buffers. Whether changing the transfer mode actually improves scalability in practice depends on the size of the messages being transferred. Improvements in scalability should be more evident when sending large messages using the streamed instead of buffered transfer mode. For more information on the WCF Transfer Modes, you can read the following topic:
- “TransferMode Enumeration” on MSDN.
Client
The Windows Forms is able to upload multiple documents at the same time. The UI of the client application allows to select a list of files to be sent in parallel to the underlyng WCF service and which client endpoint to use to this purpose. The configuration file of the client application contains two client endpoints configured, respectively, to use the BasicHttpRelayBinding and NetTcpRelayBinding. Each endpoint (see below) is configured to use the Streamed transfer mode. Besides, the value of the maxBufferPoolSize, maxReceivedMessageSize, maxBufferSize properties of the two bindings have been increased to support the transmission of large files. The same approach has been used for the WCF service. When the user clicks the Send button on the main form, the event handler method sends the selected files in an asynchronous way using a separate Task and WCF channel for each message. The following table contains the code of the MainForm class.
MainForm.cs
#region Using Directives
using System;
using System.Diagnostics;
using System.Drawing;
using System.Globalization;
using System.IO;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using System.Windows.Forms;
using System.ServiceModel;
using Microsoft.WindowsAzure.CAT.Streamed.Client;
using Microsoft.WindowsAzure.CAT.Streamed.Contracts;
#endregion
namespace Microsoft.WindowsAzure.CAT.ServiceBus.Client
{
public partial class MainForm : Form
{
#region Private Constants
//***************************
// Formats
//***************************
private const string DateFormat = "<{0,2:00}:{1,2:00}:{2,2:00}> {3}";
private const string ExceptionFormat = "Exception: {0}";
private const string InnerExceptionFormat = "InnerException: {0}";
private const string NoFilesSelected = "No file has been selected.";
private const string FileDoesNotExist = "The filename {0} does not exist.";
private const string MessageSuccessfullyUploaded =
"{0} file was successfully uploaded in {1} milliseconds using the {2} endpoint.";
private const string Unknown = "Unknown";
private const int DefaultBufferSize = 65536;
//***************************
// ListView Column Indexes
//***************************
private const int NameListViewColumnIndex = 0;
private const int SizeListViewColumnIndex = 1;
#endregion
#region Private Fields
private string currentFactoryEndpoint;
private int count;
private int bufferSize = DefaultBufferSize;
private int completed;
private ProgressForm progressForm;
private ChannelFactory<IUploadMessage> channelFactory;
private readonly List<Tuple<string, string>> fileNames = new List<Tuple<string, string>>();
#endregion
#region Private Delegates
private delegate void WriteToLogDelegate(string message);
private delegate void RestoreStateDelegate();
#endregion
#region Public Constructors
public MainForm()
{
InitializeComponent();
}
#endregion
#region Public Methods
public void ThreadTerminated()
{
lock (this)
{
completed++;
if (completed == count)
{
Invoke(new RestoreStateDelegate(RestoreState), null);
}
}
}
#endregion
#region Private Methods
private void RestoreState()
{
progressForm.Close();
}
private void btnClose_Click(object sender, EventArgs e)
{
Close();
}
private void btnSubmit_Click(object sender, EventArgs e)
{
try
{
if (fileNames != null &&
fileNames.Count > 0)
{
bufferSize = txtBufferSize.IntegerValue;
var callList = new List<CallParameters>();
for (var i = 0; i < fileNames.Count; i++)
{
if (File.Exists(fileNames[i].Item1))
{
var fileInfo = new FileInfo(fileNames[i].Item1);
callList.Add(new CallParameters(i,
(int)fileInfo.Length,
fileNames[i].Item1,
cboEndpoint.Text));
}
else
{
WriteToLog(string.Format(FileDoesNotExist, fileNames[i].Item1));
}
}
count = callList.Count;
completed = 0;
progressForm = new ProgressForm(callList);
progressForm.Show();
for (var i = 0; i < fileNames.Count; i++)
{
Task.Factory.StartNew(SubmitFile, new StartState(callList[i], this));
}
}
else
{
WriteToLog(NoFilesSelected);
}
}
catch (Exception ex)
{
WriteToLog(ex.Message);
}
}
private void SubmitFile(Object stateInfo)
{
var state = stateInfo as StartState;
InternalSubmitFile(state);
GC.Collect();
}
private void InternalSubmitFile(StartState startState)
{
try
{
if (startState != null &&
startState.Parameters != null &&
!string.IsNullOrEmpty(startState.Parameters.Filename) &&
File.Exists(startState.Parameters.Filename))
{
var fileInfo = new FileInfo(startState.Parameters.Filename);
using (var fileStream = new FileStream(startState.Parameters.Filename, FileMode.Open,
FileAccess.Read, FileShare.Read, bufferSize))
{
var progressStream = new ProgressStream(startState.Parameters.Index, fileStream);
progressStream.ProgressChanged += progressForm.ProgressStream_ProgressChanged;
var uploadMessage = new UploadMessage(Guid.NewGuid().ToString(),
txtSender.Text,
fileInfo.Name,
DateTime.Now,
fileInfo.Length,
progressStream);
if (channelFactory == null || currentFactoryEndpoint != startState.Parameters.Endpoint)
{
channelFactory = new ChannelFactory<IUploadMessage>(startState.Parameters.Endpoint);
channelFactory.Endpoint.Contract.SessionMode = SessionMode.Allowed;
currentFactoryEndpoint = startState.Parameters.Endpoint;
}
var channel = channelFactory.CreateChannel();
var stopwatch = new Stopwatch();
stopwatch.Start();
channel.UploadMessage(uploadMessage);
stopwatch.Stop();
var seconds = stopwatch.ElapsedMilliseconds;
WriteToLog(string.Format(MessageSuccessfullyUploaded, fileInfo.Name, seconds,
startState.Parameters.Endpoint));
}
}
else
{
if (startState != null &&
startState.Parameters != null)
{
WriteToLog(string.Format(FileDoesNotExist,
!string.IsNullOrEmpty(startState.Parameters.Filename) ?
startState.Parameters.Filename :
Unknown));
}
}
}
catch (FaultException ex)
{
HandleException(ex);
if (channelFactory == null)
{
return;
}
channelFactory.Abort();
channelFactory = null;
}
catch (CommunicationException ex)
{
HandleException(ex);
if (channelFactory == null)
{
return;
}
channelFactory.Abort();
channelFactory = null;
}
catch (TimeoutException ex)
{
HandleException(ex);
if (channelFactory == null)
{
return;
}
channelFactory.Abort();
channelFactory = null;
}
catch (Exception ex)
{
HandleException(ex);
if (channelFactory == null)
{
return;
}
channelFactory.Abort();
channelFactory = null;
}
finally
{
if (startState != null && startState.Form != null)
{
startState.Form.ThreadTerminated();
}
}
}
private void btnClear_Click(object sender, EventArgs e)
{
lstLog.Items.Clear();
messageFileListView.Clear();
}
public void WriteToLog(string message)
{
if (InvokeRequired)
{
Invoke(new WriteToLogDelegate(InternalWriteToLog), new object[] {message});
}
else
{
InternalWriteToLog(message);
}
}
private void InternalWriteToLog(string message)
{
if (string.IsNullOrEmpty(message))
{
return;
}
var lines = message.Split('\n');
var objNow = DateTime.Now;
var space = new string(' ', 19);
for (var i = 0; i < lines.Length; i++)
{
if (i == 0)
{
string line = string.Format(DateFormat,
objNow.Hour,
objNow.Minute,
objNow.Second,
lines[i]);
lstLog.Items.Add(line);
}
else
{
lstLog.Items.Add(space + lines[i]);
}
}
lstLog.SelectedIndex = lstLog.Items.Count - 1;
}
private void MainForm_Load(object sender, EventArgs e)
{
txtBufferSize.Text = DefaultBufferSize.ToString(CultureInfo.InvariantCulture);
txtSender.Text = Environment.MachineName;
cboEndpoint.SelectedIndex = 0;
currentFactoryEndpoint = cboEndpoint.Text;
messageFileListView_Resize(null, null);
}
public void HandleException(Exception ex)
{
if (ex == null || string.IsNullOrEmpty(ex.Message))
{
return;
}
WriteToLog(string.Format(CultureInfo.CurrentCulture, ExceptionFormat, ex.Message));
if (ex.InnerException != null && !string.IsNullOrEmpty(ex.InnerException.Message))
{
WriteToLog(string.Format(CultureInfo.CurrentCulture,
InnerExceptionFormat,
ex.InnerException.Message));
}
}
private void button_MouseEnter(object sender, EventArgs e)
{
var control = sender as Control;
if (control != null)
{
control.ForeColor = Color.White;
}
}
private void button_MouseLeave(object sender, EventArgs e)
{
var control = sender as Control;
if (control != null)
{
control.ForeColor = SystemColors.ControlText;
}
}
private void aboutToolStripMenuItem_Click(object sender, EventArgs e)
{
var aboutForm = new AboutForm();
aboutForm.ShowDialog();
}
private void messageFileListView_DrawColumnHeader(object sender, DrawListViewColumnHeaderEventArgs e)
{
var startX = e.ColumnIndex == 0 ? -1 : e.Bounds.X;
var endX = e.Bounds.X + e.Bounds.Width - 1;
// Background
e.Graphics.FillRectangle(new SolidBrush(Color.FromArgb(215, 228, 242)),
startX, -1, e.Bounds.Width + 1, e.Bounds.Height + 1);
// Left vertical line
e.Graphics.DrawLine(new Pen(SystemColors.ControlLightLight),
startX, -1, startX, e.Bounds.Y + e.Bounds.Height + 1);
// Top horizontal line
e.Graphics.DrawLine(new Pen(SystemColors.ControlLightLight), startX, -1, endX, -1);
// Bottom horizontal line
e.Graphics.DrawLine(new Pen(SystemColors.ControlDark),
startX, e.Bounds.Height - 1, endX, e.Bounds.Height - 1);
// Right vertical line
e.Graphics.DrawLine(new Pen(SystemColors.ControlDark),
endX, -1, endX, e.Bounds.Height + 1);
var roundedFontSize = (float)Math.Round(e.Font.SizeInPoints);
var bounds = new RectangleF(e.Bounds.X + 4,
(e.Bounds.Height - 8 - roundedFontSize) / 2, e.Bounds.Width, roundedFontSize + 6);
e.Graphics.DrawString(e.Header.Text, e.Font, new SolidBrush(SystemColors.ControlText), bounds);
}
private void messageFileListView_DrawItem(object sender, DrawListViewItemEventArgs e)
{
//e.DrawDefault = true;
//e.DrawBackground();
//e.DrawText();
}
private void messageFileListView_DrawSubItem(object sender, DrawListViewSubItemEventArgs e)
{
e.DrawDefault = true;
}
private void messageFileListView_Resize(object sender, EventArgs e)
{
try
{
messageFileListView.SuspendLayout();
var width = messageFileListView.Width - messageFileListView.Columns[SizeListViewColumnIndex].Width;
messageFileListView.Columns[NameListViewColumnIndex].Width = width - 4;
}
finally
{
messageFileListView.ResumeLayout();
}
}
private void btnSelectFiles_Click(object sender, EventArgs e)
{
openFileDialog.Multiselect = true;
openFileDialog.FileName = string.Empty;
if (openFileDialog.ShowDialog() != DialogResult.OK ||
!openFileDialog.FileNames.Any())
{
return;
}
foreach (var fileInfo in openFileDialog.FileNames.Select(fileName => new FileInfo(fileName)))
{
var size = string.Format("{0} KB", fileInfo.Length % 1024 == 0
? fileInfo.Length / 1024
: fileInfo.Length / 1024 + 1);
var listViewItem = messageFileListView.Items.Add(new ListViewItem(new[]
{
fileInfo.FullName,
size
}));
listViewItem.Checked = true;
fileNames.Add(new Tuple<string, string>(fileInfo.FullName, size));
checkBoxFileName.Checked =
messageFileListView.Items.Cast<ListViewItem>().AsEnumerable().All(i => i.Checked);
}
}
private void checkBoxFileName_CheckedChanged(object sender, EventArgs e)
{
for (var i = 0; i < messageFileListView.Items.Count; i++)
{
messageFileListView.Items[i].Checked = checkBoxFileName.Checked;
}
}
#endregion
}
public class CallParameters
{
#region Public Constructors
public CallParameters()
{
Index = 0;
Size = 0;
Filename = null;
}
public CallParameters(int index,
int size,
string filename,
string endpoint)
{
Index = index;
Size = size;
Filename = filename;
Endpoint = endpoint;
}
#endregion
#region Public Properties
public int Index { get; set; }
public int Size { get; set; }
public string Filename { get; set; }
public string Endpoint { get; set; }
#endregion
}
public class StartState
{
#region Public Constructors
public StartState()
{
Parameters = null;
Form = null;
}
public StartState(CallParameters parameters,
MainForm form)
{
Parameters = parameters;
Form = form;
}
#endregion
#region Public Properties
public CallParameters Parameters { get; set; }
public MainForm Form { get; set; }
#endregion
}
}
The following table contains the configuration file of the client application.
App.config
<?xml version="1.0"?>
<configuration>
<system.diagnostics>
<sources>
<source name="System.ServiceModel"
switchValue="Information, ActivityTracing"
propagateActivity="true">
<listeners>
<add type="System.Diagnostics.DefaultTraceListener" name="Default">
<filter type=""/>
</add>
<add name="ServiceModelMessageLoggingListener">
<filter type=""/>
</add>
</listeners>
</source>
<source name="System.ServiceModel.MessageLogging" switchValue="Information, ActivityTracing">
<listeners>
<add type="System.Diagnostics.DefaultTraceListener" name="Default">
<filter type=""/>
</add>
<add name="ServiceModelMessageLoggingListener">
<filter type=""/>
</add>
</listeners>
</source>
</sources>
<sharedListeners>
<add initializeData="C:\servicebus_client.svclog"
type="System.Diagnostics.XmlWriterTraceListener, System, Version=2.0.0.0, Culture=neutral,
PublicKeyToken=b77a5c561934e089"
name="ServiceModelMessageLoggingListener"
traceOutputOptions="Timestamp">
<filter type=""/>
</add>
</sharedListeners>
</system.diagnostics>
<system.serviceModel>
<diagnostics>
<messageLogging logEntireMessage="false"
logMalformedMessages="false"
logMessagesAtServiceLevel="false"
logMessagesAtTransportLevel="false"
maxSizeOfMessageToLog="1000000"/>
</diagnostics>
<bindings>
<basicHttpRelayBinding>
<binding name="basicHttpRelayBinding"
closeTimeout="01:00:00"
openTimeout="01:00:00"
receiveTimeout="01:00:00"
sendTimeout="01:00:00"
maxBufferPoolSize="1048576"
maxReceivedMessageSize="1048576000"
maxBufferSize="1048576"
transferMode="Streamed">
<security mode="Transport" relayClientAuthenticationType="RelayAccessToken"/>
</binding>
</basicHttpRelayBinding>
<netTcpRelayBinding>
<binding name="netTcpRelayBinding"
closeTimeout="01:00:00"
openTimeout="01:00:00"
receiveTimeout="01:00:00"
sendTimeout="01:00:00"
maxBufferPoolSize="1048576"
maxReceivedMessageSize="1048576000"
maxBufferSize="1048576"
transferMode="Streamed"
maxConnections="200"
listenBacklog="200">
<security mode="Transport" relayClientAuthenticationType="RelayAccessToken"/>
</binding>
</netTcpRelayBinding>
</bindings>
<behaviors>
<endpointBehaviors>
<behavior name="securityBehavior">
<transportClientEndpointBehavior>
<tokenProvider>
<sharedSecret issuerName="owner" issuerSecret="YOUR-NAMESPACE-ISSUER-SECRET"/>
</tokenProvider>
</transportClientEndpointBehavior>
</behavior>
</endpointBehaviors>
</behaviors>
<client>
<endpoint address="https://YOUR-NAMESPACE.servicebus.windows.net/uploadservice/http"
binding="basicHttpRelayBinding"
bindingConfiguration="basicHttpRelayBinding"
behaviorConfiguration="securityBehavior"
contract="Microsoft.WindowsAzure.CAT.Streamed.Contracts.IUploadMessage"
name="basicHttpRelayBinding"/>
<endpoint address="sb://YOUR-NAMESPACE.servicebus.windows.net/uploadservice/nettcp"
binding="netTcpRelayBinding"
bindingConfiguration="netTcpRelayBinding"
behaviorConfiguration="securityBehavior"
contract="Microsoft.WindowsAzure.CAT.Streamed.Contracts.IUploadMessage"
name="netTcpRelayBinding"/>
</client>
<extensions>
<behaviorExtensions>
<add name="connectionStatusBehavior"
type="Microsoft.ServiceBus.Configuration.ConnectionStatusElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="transportClientEndpointBehavior"
type="Microsoft.ServiceBus.Configuration.TransportClientEndpointBehaviorElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="serviceRegistrySettings"
type="Microsoft.ServiceBus.Configuration.ServiceRegistrySettingsElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="tokenProviderEndpointBehavior"
type="Microsoft.WindowsAzure.CAT.ServiceBusForWindowsServer.Behaviors.TokenProviderBehaviorExtensionElement,
Microsoft.WindowsAzure.CAT.ServiceBusForWindowsServer.Behaviors,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=197ec3eb961f773c"/>
</behaviorExtensions>
<bindingElementExtensions>
<add name="netMessagingTransport"
type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingTransportExtensionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="tcpRelayTransport"
type="Microsoft.ServiceBus.Configuration.TcpRelayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="httpRelayTransport"
type="Microsoft.ServiceBus.Configuration.HttpRelayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="httpsRelayTransport"
type="Microsoft.ServiceBus.Configuration.HttpsRelayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="onewayRelayTransport"
type="Microsoft.ServiceBus.Configuration.RelayedOnewayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
</bindingElementExtensions>
<bindingExtensions>
<add name="basicHttpRelayBinding"
type="Microsoft.ServiceBus.Configuration.BasicHttpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="webHttpRelayBinding"
type="Microsoft.ServiceBus.Configuration.WebHttpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="ws2007HttpRelayBinding"
type="Microsoft.ServiceBus.Configuration.WS2007HttpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netTcpRelayBinding"
type="Microsoft.ServiceBus.Configuration.NetTcpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netOnewayRelayBinding"
type="Microsoft.ServiceBus.Configuration.NetOnewayRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netEventRelayBinding"
type="Microsoft.ServiceBus.Configuration.NetEventRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netMessagingBinding"
type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
</bindingExtensions>
</extensions>
</system.serviceModel>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5"/>
</startup>
</configuration>
Contracts
The Contracts project contains the definition of the IUploadService contract and the UploadMessage used to send the metadata (sender, file name, etc.) of the documents sent by the client application to the WCF service.
IUploadService.cs
#region Using Directives
using System.ServiceModel;
#endregion
namespace Microsoft.WindowsAzure.CAT.Streamed.Contracts
{
[ServiceContract(Namespace = https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice,
SessionMode = SessionMode.Allowed)]
public interface IUploadMessage
{
#region Contract Operations
[OperationContract(Action = "UploadMessage", ReplyAction = "UploadMessageResponse")]
void UploadMessage(UploadMessage message);
#endregion
}
}
UploadMessage.cs
#region Using Directives
using System;
using System.IO;
using System.ServiceModel;
using System.Net.Security;
#endregion
namespace Microsoft.WindowsAzure.CAT.Streamed.Contracts
{
[MessageContract]
public class UploadMessage
{
#region Public Constructors
public UploadMessage()
{
Id = null;
Sender = null;
Filename = null;
DateTime = DateTime.Now;
Size= 0;
Data = null;
}
public UploadMessage(string id,
string sender,
string filename,
DateTime dateTime,
long size,
Stream data)
{
Id = id;
Sender = sender;
Filename = filename;
DateTime = dateTime;
Size = size;
Data = data;
}
#endregion
#region Public Properties
[MessageHeader(Name = "id",
Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice",
ProtectionLevel = ProtectionLevel.None)]
public string Id { get; set; }
[MessageHeader(Name = "sender",
Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice",
ProtectionLevel = ProtectionLevel.None)]
public string Sender { get; set; }
[MessageHeader(Name = "filename",
Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice",
ProtectionLevel = ProtectionLevel.None)]
public string Filename { get; set; }
[MessageHeader(Name = "dateTime",
Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice",
ProtectionLevel = ProtectionLevel.None)]
public DateTime DateTime { get; set; }
[MessageHeader(Name = "size",
Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice",
ProtectionLevel = ProtectionLevel.None)]
public long Size { get; set; }
[MessageBodyMember(Name = "data",
Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice",
ProtectionLevel = ProtectionLevel.None)]
public Stream Data { get; set; }
#endregion
}
}
Service
The WCF service runs in a console application. In a production environment, the service should be hosted by a more scalable and reliable runtime environment such as IIS. The following table contains the code of the WCF service and its configuration file.
UploadService.cs
#region Copyright
//-------------------------------------------------
// Author: Paolo Salvatori
// Email: paolos@microsoft.com
// History: 2013-6-4 Created
//-------------------------------------------------
#endregion
#region Using Directives
using System;
using System.Configuration;
using System.IO;
using System.Diagnostics;
using System.ServiceModel;
using Microsoft.WindowsAzure.CAT.Streamed.Contracts;
#endregion
namespace Microsoft.WindowsAzure.CAT.Streamed.Service
{
// Service class which implements the service contract.
[ServiceBehavior(Namespace = "https://microsoft.windowsazure.cat/10/samples/servicebus/uploadservice")]
public class UploadService : IUploadMessage
{
#region Private Constants
private const string Source = "UploadService";
private const string MessageFormat = "[UploadService] {0}";
private const string MessageWritingFormat = "[UploadService] Writing message from {0} to file {1}....";
private const string MessageSuccessfullyWrittenFormat =
"[UploadService] Message from {0} successfully written to {1}.";
private const string FolderPathParameter = "targetFolder";
private const string FolderPathDefaultValue = @"C:\";
private const int BufferSize = 8192;
#endregion
#region Static Fields
private static readonly string targetFolder;
#endregion
#region Static Constructor
static UploadService()
{
try
{
targetFolder = ConfigurationManager.AppSettings[FolderPathParameter];
if (string.IsNullOrEmpty(targetFolder))
{
targetFolder = FolderPathDefaultValue;
}
}
catch (Exception ex)
{
EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
Debug.WriteLine(MessageFormat, ex.Message);
}
}
#endregion
#region Public Methods
[OperationBehavior]
public void UploadMessage(UploadMessage message)
{
try
{
var path = Path.Combine(targetFolder, message.Filename);
WriteMessageToFile(message.Data, message.Sender, path);
}
catch (Exception ex)
{
EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
Debug.WriteLine(MessageFormat, ex.Message);
}
}
#endregion
#region Private Methods
private static void WriteMessageToFile(Stream stream, string sender, string path)
{
try
{
Trace.WriteLine(string.Format(MessageWritingFormat, sender, path));
var buffer = new byte[BufferSize];
if (File.Exists(path))
{
File.Delete(path);
}
using (var fileStream = File.Open(path, FileMode.Create, FileAccess.Write, FileShare.None))
{
int bytesRead;
while ((bytesRead = stream.Read(buffer, 0, BufferSize)) > 0)
{
fileStream.Write(buffer, 0, bytesRead);
}
}
Trace.WriteLine(string.Format(MessageSuccessfullyWrittenFormat, sender, path));
}
catch (Exception ex)
{
EventLog.WriteEntry(Source, ex.Message, EventLogEntryType.Error);
Trace.WriteLine(string.Format(MessageFormat, ex.Message));
}
}
#endregion
}
}
App.config
<?xml version="1.0"?>
<configuration>
<system.diagnostics>
<trace autoflush="false" indentsize="4">
<listeners>
<add name="configConsoleListener"
type="System.Diagnostics.ConsoleTraceListener" />
</listeners>
</trace>
<sources>
<source name="System.ServiceModel"
switchValue="Information, ActivityTracing"
propagateActivity="true">
<listeners>
<add type="System.Diagnostics.DefaultTraceListener"
name="Default">
<filter type=""/>
</add>
<add name="ServiceModelMessageLoggingListener">
<filter type=""/>
</add>
</listeners>
</source>
<source name="System.ServiceModel.MessageLogging"
switchValue="Information, ActivityTracing">
<listeners>
<add type="System.Diagnostics.DefaultTraceListener"
name="Default">
<filter type=""/>
</add>
<add name="ServiceModelMessageLoggingListener">
<filter type=""/>
</add>
</listeners>
</source>
</sources>
<sharedListeners>
<add initializeData="C:\servicebus_service.svclog"
type="System.Diagnostics.XmlWriterTraceListener,
System, Version=2.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089"
name="ServiceModelMessageLoggingListener"
traceOutputOptions="Timestamp">
<filter type=""/>
</add>
</sharedListeners>
</system.diagnostics>
<system.serviceModel>
<diagnostics>
<messageLogging logEntireMessage="true"
logMalformedMessages="true"
logMessagesAtServiceLevel="true"
logMessagesAtTransportLevel="true"
maxSizeOfMessageToLog="1000000"/>
</diagnostics>
<bindings>
<basicHttpRelayBinding>
<binding name="basicHttpRelayBinding"
closeTimeout="01:00:00"
openTimeout="01:00:00"
receiveTimeout="01:00:00"
sendTimeout="01:00:00"
maxBufferPoolSize="1048576"
maxReceivedMessageSize="1048576000"
maxBufferSize="1048576"
transferMode="Streamed">
<security mode="Transport" relayClientAuthenticationType="RelayAccessToken"/>
</binding>
</basicHttpRelayBinding>
<netTcpRelayBinding>
<binding name="netTcpRelayBinding"
closeTimeout="01:00:00"
openTimeout="01:00:00"
receiveTimeout="01:00:00"
sendTimeout="01:00:00"
maxBufferPoolSize="1048576"
maxReceivedMessageSize="1048576000"
maxBufferSize="1048576"
transferMode="Streamed"
maxConnections="200"
listenBacklog="200">
<security mode="Transport" relayClientAuthenticationType="RelayAccessToken"/>
</binding>
</netTcpRelayBinding>
</bindings>
<behaviors>
<endpointBehaviors>
<behavior name="securityBehavior">
<transportClientEndpointBehavior>
<tokenProvider>
<sharedSecret issuerName="owner" issuerSecret="YOUR-NAMESPACE-ISSUER-SECRET"/>
</tokenProvider>
</transportClientEndpointBehavior>
</behavior>
</endpointBehaviors>
</behaviors>
<services>
<service name="Microsoft.WindowsAzure.CAT.Streamed.Service.UploadService">
<endpoint address="https://YOUR-NAMESPACE.servicebus.windows.net/uploadservice/http"
binding="basicHttpRelayBinding"
bindingConfiguration="basicHttpRelayBinding"
behaviorConfiguration="securityBehavior"
contract="Microsoft.WindowsAzure.CAT.Streamed.Contracts.IUploadMessage"
name="basicHttpRelayBinding"/>
<endpoint address="sb://YOUR-NAMESPACE.servicebus.windows.net/uploadservice/nettcp"
binding="netTcpRelayBinding"
bindingConfiguration="netTcpRelayBinding"
behaviorConfiguration="securityBehavior"
contract="Microsoft.WindowsAzure.CAT.Streamed.Contracts.IUploadMessage"
name="netTcpRelayBinding"/>
</service>
</services>
<extensions>
<behaviorExtensions>
<add name="connectionStatusBehavior"
type="Microsoft.ServiceBus.Configuration.ConnectionStatusElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="transportClientEndpointBehavior"
type="Microsoft.ServiceBus.Configuration.TransportClientEndpointBehaviorElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="serviceRegistrySettings"
type="Microsoft.ServiceBus.Configuration.ServiceRegistrySettingsElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="tokenProviderEndpointBehavior"
type="Microsoft.WindowsAzure.CAT.ServiceBusForWindowsServer.Behaviors.TokenProviderBehaviorExtensionElement,
Microsoft.WindowsAzure.CAT.ServiceBusForWindowsServer.Behaviors,
Version=1.0.0.0, Culture=neutral, PublicKeyToken=197ec3eb961f773c"/>
</behaviorExtensions>
<bindingElementExtensions>
<add name="netMessagingTransport"
type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingTransportExtensionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="tcpRelayTransport"
type="Microsoft.ServiceBus.Configuration.TcpRelayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="httpRelayTransport"
type="Microsoft.ServiceBus.Configuration.HttpRelayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="httpsRelayTransport"
type="Microsoft.ServiceBus.Configuration.HttpsRelayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="onewayRelayTransport"
type="Microsoft.ServiceBus.Configuration.RelayedOnewayTransportElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
</bindingElementExtensions>
<bindingExtensions>
<add name="basicHttpRelayBinding"
type="Microsoft.ServiceBus.Configuration.BasicHttpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="webHttpRelayBinding"
type="Microsoft.ServiceBus.Configuration.WebHttpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="ws2007HttpRelayBinding"
type="Microsoft.ServiceBus.Configuration.WS2007HttpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netTcpRelayBinding"
type="Microsoft.ServiceBus.Configuration.NetTcpRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netOnewayRelayBinding"
type="Microsoft.ServiceBus.Configuration.NetOnewayRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netEventRelayBinding"
type="Microsoft.ServiceBus.Configuration.NetEventRelayBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
<add name="netMessagingBinding"
type="Microsoft.ServiceBus.Messaging.Configuration.NetMessagingBindingCollectionElement,
Microsoft.ServiceBus, Version=2.1.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/>
</bindingExtensions>
</extensions>
</system.serviceModel>
<appSettings>
<add key="targetFolder" value="C:\Temp"/>
</appSettings>
<startup>
<supportedRuntime version="v4.0" sku=".NETFramework,Version=v4.5.1"/>
</startup>
</configuration>
Run the sample
To run the solution you can proceed as follows:
- Start the console application hosting the WCF service. Access your Service Bus namespace in the Windows Azure Management Portal and make sure that the both the the BasicHttpRelayBinding and NetTcpRelayBinding Relrelay services were started correctly, as shown in the following picture.
- Start the client application.
- Click the Select Files button and select 1 or multiple files to send.
- Select the basicHttpRelayBinding or netTcpRelayBinding endpoint from the Endpoint drop-down-list.
- Specify a value for the BufferSize.
- Click the Send button to send the selected files to the WCF service using the chosen client endpoint, as shown in the following picture.
- The WCF service receives the files via Service Bus Relay Service and stores them in the folder whose path is defined in the configuration file.
Conclusions
This article shows a powerful technique to transfer large files using the Service Bus Relay Service and WCF Streamed transfer mode. You can download the code on MSDN Code Gallery.
Comments
Anonymous
September 30, 2013
Hi Paolo, Thanks for this informative blog. I have tried using MtomEncoding with your sample but the Mtom encoding doesnt seems to worke with Servicebus relay and it throws error Message "There was an error encountered while processing the request." string Text encoding is very costly for huge binary data. Please advise. Thanks & Regards, Kunal DeshpandeAnonymous
September 30, 2013
Hi Kunal did you try to use the BinaryMessageEncoding? I agree with you that the TextMessageEncoding may be heavy when exchanging large messages. Ciao PaoloAnonymous
October 09, 2013
Hi Paolo, Yes, I tried using BinaryMessageEncoding with custom binding. But that also fails with nelow message. Message "The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was ''." string Seems like the Servicebus relay is not able to read the soap headers in binary encoding. FYI please find below code of custom binding I am using public class HTTPRelayBinding : CustomBinding
public HTTPRelayBinding() : base() { ReceiveTimeout = new TimeSpan(0, 15, 0); SendTimeout = new TimeSpan(0, 15, 0); m_transportSecurity = SecurityBindingElement.CreateUserNameOverTransportBindingElement(); m_encodingWeb = new BinaryMessageEncodingBindingElement(); m_encodingWeb.MessageVersion = MessageVersion.Soap12WSAddressing10; m_transport = new HttpsRelayTransportBindingElement(); m_transport.TransferMode = TransferMode.Streamed; m_transport.MaxBufferSize = Int32.MaxValue; m_transport.MaxReceivedMessageSize = Int32.MaxValue; } } Any help in this regard is highly appreciated. Thanks & Regards, Kunal Deshpande
Anonymous
October 20, 2013
Hi Paolo, Did you able to implement binary coding with Servicebus over HTTP? Can you kindly share a sample. You can reach me at below email id. kunal.deshpande@igate.com Thanks & Regards, Kunal DeshpandeAnonymous
April 14, 2014
The comment has been removedAnonymous
April 14, 2014
Hi Paul that's just the namespace of the WCF service contact, it doesn't have anything to deal with the Service Bus namespace. ;) Ciao Paolo