SignalR troubleshooting : “Performance Counters” and “Tracing”

Hello,

In this blogpost, I wanted to share with you some of the troubleshooting methods available on “SignalR”. Especally, SignalR tracing helps a lot to diagnose end-to-end connectivity issues. For assessment of performance, “SignalR Performance Counters” can be collected. See below, how these can be activated.

Enabling SignalR Tracing:

You can configure server (if possible also client side) side SignalR tracing to see, if you have connectivity issues, or performance problems. Tracing on the server , logs: connection, scaleout provider, and message bus events. Tracing on the client logs: connection events. In SignalR 2.1 and later, tracing on the client logs the full content of hub invocation messages. To give an example, here is a typical “transport” error written to 'transport.log' , which is indeed a failure on 'LongPolling' transport:

 

SignalR.Transports.LongPollingTransport Warning: 0 : Exception thrown by Task: System.AggregateException: One or more errors occurred. ---> System.Web.HttpException: The remote host closed the connection. The error code is 0x800704CD.
at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
at System.Web.Hosting.IIS7WorkerRequest.ExplicitFlush()
at System.Web.HttpResponse.Flush(Boolean finalFlush, Boolean async)
at System.Web.HttpWriter.WriteFromStream(Byte[] data, Int32 offset, Int32 size)
at Microsoft.Owin.Host.SystemWeb.CallStreams.OutputStream.Write(Byte[] buffer, Int32 offset, Int32 count)
at Microsoft.AspNet.SignalR.Owin.ServerResponse.Write(ArraySegment`1 data)
at Microsoft.AspNet.SignalR.Infrastructure.BinaryTextWriter.b__4(ArraySegment`1 data, Object state)
at Microsoft.AspNet.SignalR.Infrastructure.BufferTextWriter.ChunkedWriter.Flush(Byte[] byteBuffer, Boolean flushEncoder)
at Microsoft.AspNet.SignalR.Infrastructure.BufferTextWriter.Flush()
at Microsoft.AspNet.SignalR.Transports.LongPollingTransport.PerformKeepAlive(Object state)
at Microsoft.AspNet.SignalR.Infrastructure.TaskQueue.InvokeNext(Func`2 next, Object nextState)
at Microsoft.AspNet.SignalR.TaskAsyncHelper.FromMethod[T1,T2,T3](Func`4 func, T1 arg1, T2 arg2, T3 arg

 

Enabling SignalR Performance counters:

Counters can be enabled with 'SignalR.exe' utility found in the Microsoft.AspNet.SignalR.Utils package. You can get it with Nuget and install counters, in your project like described in this article. After enabling the counters, ou can define a data collector set, as usual:

After getting the package, they are activated with following command -->
c:\SRTest\packages\Microsoft.AspNet.SignalR.Utils.2.2.0\tools>signalr.exe ipc
SignalR Utility Version: 2.2.0.0
Installing performance counters...
Connections Connected
Connections Reconnected
Connections Disconnected
Connections Current ForeverFrame
Connections Current LongPolling
Connections Current ServerSentEvents
Connections Current WebSockets
Connections Current
Connection Messages Received Total
Connection Messages Sent Total
Connection Messages Received/Sec
Connection Messages Sent/Sec
Message Bus Messages Received Total
Message Bus Messages Received/Sec
Scaleout Message Bus Messages Received/Sec
Message Bus Messages Published Total
Message Bus Messages Published/Sec
Message Bus Subscribers Current
Message Bus Subscribers Total
Message Bus Subscribers/Sec
Message Bus Allocated Workers
Message Bus Busy Workers
Message Bus Topics Current
Errors: All Total
Errors: All/Sec
Errors: Hub Resolution Total
Errors: Hub Resolution/Sec
Errors: Hub Invocation Total
Errors: Hub Invocation/Sec
Errors: Tranport Total
Errors: Transport/Sec
Scaleout Streams Total
Scaleout Streams Open
Scaleout Streams Buffering
Scaleout Errors Total
Scaleout Errors/Sec
Scaleout Send Queue Length

Performance counters installed!

Thanks,

Mert