RemotingException: “All pipe instances are busy.” or “The system cannot find the file specified.”

Today, I ran into an interesting issue. One of our customers was using the .Net Remoting IPC client in the asp.net application communicating with a local service. The asp.net application was configured with impersonation. The IPC client was simply making multiple calls to the remoting service. Every now and then, the call failed RemotingException with “All pipe instances are busy.” (error=231) and “The system cannot find the file specified.” (error=2). Upon investigation, the issue seemed to be a race condition issue with creating and closing the Win32 namepiped handle. Typically, IPC .Net Remoting client will cache and reuse the namedpipe handles. Since the namedpipe creation is security context sensitive, if the calling thread is impersonated, the namedpipe will not be shared. Calling remoting service in a tight loop will cause new namedpipe handle to be created and closed. The issue is repro-able by just a native Win32 namedpipe sample posted on msdn (https://msdn.microsoft.com/en-us/library/aa365588(VS.85).aspx and https://msdn.microsoft.com/en-us/library/aa365592(VS.85).aspx). I simply put the client into a while loop. After just a few seconds on my Vista and Windows 2008 machine, the error will occur.

Good news is the workaround is simple. One should call WaitNamedPipe and retry creating the namedpipe. One catch is that WaitNamedPipe is also subject to this issue and, sometimes, return immediately with and “The system cannot find the file specified.” (error=2). You may want to retry the WaitNamedPipe a couple of time before calling recreate the namedpipe handle. I have a code snippet below that I modified from the sample mentioned above. With this, the client will run without any issue. You could adapt this in the Remoting IPC client scenario by Pinvoke WaitNamedPipe in the same manner before retrying invoking the remoting service.

// // PipeClient.cpp : Defines the entry point for the console application. // #include <windows.h> #include <stdio.h> #include <conio.h> #include <tchar.h> #include "stdafx.h" #define BUFSIZE 512   int _tmain(int argc, TCHAR *argv[]) {    HANDLE hPipe;    LPTSTR lpvMessage=TEXT("Default message from client");    TCHAR chBuf[BUFSIZE];    BOOL fSuccess;    DWORD cbRead, cbWritten, dwMode;    LPTSTR lpszPipename = TEXT("\\\\.\\pipe\\mynamedpipe");    if( argc > 1 )       lpvMessage = argv[1];      while (1)    { // Try to open a named pipe; wait for it, if necessary.       BOOL bRetried = FALSE;       while (1)       {             hPipe = CreateFile(                   lpszPipename, // pipe name                   GENERIC_READ | // read and write access                   GENERIC_WRITE,                   0, // no sharing                   NULL, // default security attributes                   OPEN_EXISTING, // opens existing pipe                   0, // default attributes                   NULL); // no template file             // Break if the pipe handle is valid.                   if (hPipe != INVALID_HANDLE_VALUE)                   break;                   // Exit if an error other than ERROR_PIPE_BUSY occurs.                   if (GetLastError() != ERROR_PIPE_BUSY)             {                   printf("Could not open pipe");                   return 0;             }             if (bRetried)             {                   printf("Could not open pipe");                   return 0;             }             // Retry for 1000ms             bRetried = TRUE;             BOOL bSuccess = FALSE;             for (INT i = 0 ; i < 10 ; ++i)             {      _tprintf( TEXT("W") );                   bSuccess = WaitNamedPipe(lpszPipename, 0);                   if (!bSuccess)                   {                         Sleep(100);                   }                   else                         break;             }             if (!bSuccess)             {                   printf("Fail WaitNamedPipe");                   return 0;             }             else      _tprintf( TEXT("R") );       }             // The pipe connected; change to message-read mode.             dwMode = PIPE_READMODE_MESSAGE;       fSuccess = SetNamedPipeHandleState(             hPipe, // pipe handle             &dwMode, // new pipe mode             NULL, // don't set maximum bytes             NULL); // don't set maximum time       if (!fSuccess)       {             printf("SetNamedPipeHandleState failed");             return 0;       }             // Send a message to the pipe server.             fSuccess = WriteFile(             hPipe, // pipe handle             lpvMessage, // message             (lstrlen(lpvMessage)+1)*sizeof(TCHAR), // message length             &cbWritten, // bytes written             NULL); // not overlapped       if (!fSuccess)       {             printf("WriteFile failed");             return 0;       }             do       {       // Read from the pipe.                   fSuccess = ReadFile(                   hPipe, // pipe handle                   chBuf, // buffer to receive reply                   BUFSIZE*sizeof(TCHAR), // size of buffer                   &cbRead, // number of bytes read                   NULL); // not overlapped                   if (! fSuccess && GetLastError() != ERROR_MORE_DATA)                   break;                   //_tprintf( TEXT("%s\n"), chBuf );             _tprintf( TEXT(".") );       } while (!fSuccess); // repeat loop if ERROR_MORE_DATA       //getch();             CloseHandle(hPipe);    }      return 0; }

Comments

• Anonymous
  October 25, 2012
  I have same issue (ASP.NET to Windows Service via IPC). I use Activator.GetObject for creating instance and frankly I am confused because I no idea how to solve this on C#.