Troubleshooting Hybrid Connections with Logging

Hybrid Connections offer an easy way to connect your Web App to an on-premise resource. In most cases, Hybrid Connections just work, but when they don't, the only info you might have to go on is a failure in your app or a status of "Not connected" in the Azure portal or the Hybrid Connection Manager. In this post, I'll tell you how you can get some diagnostic information from a couple of different logging methods.

Note: The information in this post applies to both Azure Relay Hybrid Connections and to Classic Hybrid Connections.

Service Bus Operational Logs

Service Bus Operational logs are available on the machine running the Hybrid Connection Manager. You don't have to do anything special to enable logging. Simply open the Event Viewer in Windows and navigate to them. You'll find them under Application and Service Logs/Microsoft/ServiceBus/Client as shown in the figure below.

Service Bus Operational Logs (Click for a larger image.)

The problem in the figure above is that the Hybrid Connection called mysql doesn't actually exist in Azure. It used to, but it has been deleted. Therefore, the Hybrid Connection Manager encounters an error when attempting to connect to it.

The Service Bus Operational logs aren't going to provide you with a silver bullet to diagnose a Hybrid Connection issue, but if you do a little investigative work with the data they provide, you'll usually find the source of your problem. Keep in mind that some of the entries might be classified as informational. Most of those can be safely ignored.

System.Net Tracing

The Hybrid Connection Manager uses the .NET Framework for connectivity with Service Bus. Therefore, you can enable logging for the System.Net namespace in order to get information on what might be causing a problem with your Hybrid Connection. System.Net tracing is usually a better choice when your Hybrid Connection is showing "Connected" but your app is failing.

To enable System.Net tracing, follow these steps on the machine that is running the Hybrid Connection Manager.

1. Launch Notepad or another text editing application. You'll need to run this app as an administrator so that you can edit the Hybrid Connection Manager's configuration file.
2. Open the file called Microsoft.HybridConnectionManager.Listener.exe.config located in the folder where the Hybrid Connection Manager is installed. (By default, this folder is Program Files\Microsoft\HybridConnectionManager [version] .)
3. Add the following code before the closing </configuration> element in the file.

 <system.diagnostics>
  <trace autoflush="true" />
    <sources>
      <source name="System.Net">
        <listeners>
          <add name="System.Net"/>
        </listeners>
     </source>
     <source name="System.Net.HttpListener">
       <listeners>
         <add name="System.Net"/>
       </listeners>
     </source>
     <source name="System.Net.Sockets">
       <listeners>
         <add name="System.Net"/>
       </listeners>
     </source>
     <source name="System.Net.Cache">
       <listeners>
         <add name="System.Net"/>
       </listeners>
     </source>
   </sources>
   <sharedListeners>
     <add name="System.Net" type="System.Diagnostics.TextWriterTraceListener" initializeData="c:\temp\System.Net.trace.log" traceOutputOptions = "ProcessId, DateTime" />
   </sharedListeners>
   <switches>
     <add name="System.Net" value="Verbose" />
     <add name="System.Net.Sockets" value="Verbose" />
     <add name="System.Net.Cache" value="Verbose" />
     <add name="System.Net.HttpListener" value="Verbose" />
   </switches>
 </system.diagnostics>

 

4. Restart the Azure Hybrid Connection Manager Service in Services as shown in the figure below.

 

[caption id="attachment_2906" align="alignnone" width="630"] Restarting the Hybrid Connection Manager Service (Click for a larger image.) [/caption]

Note:  Make sure that you have a temp folder at the root of your C drive. If you don't, create one so that the trace log can be written to that folder.

After you do this, System.Net logs will be saved to c:\temp\ and the filename will be System.Net.trace.log. The code you added enables verbose logging, so there will be a LOT of information in the log. If you are having difficulty locating a problem in the log, you can search for exception to see if there are any exceptions occurring. Microsoft support staff can also assist you with interpreting the log if you open a support case.

Important note:  Once you generate your log, you should remove the code you added to the config file and restart the Azure Hybrid Connection Manager Service again. If you don't, the System.Net log will continue to grow and it will use a lot of disk space.

Hopefully these logging options will make it easier for you to troubleshoot your Hybrid Connections.