Share via

Winsock I/O Model - Part II : Implementation

  • Article

Winsock I/O Model - Part II : Implementation

In the previous post[6], I
summarized several scalable network I/O models in theory. In this
article, I will give concrete code to show how to use each model to
build a scalable network server. Building scalable server is a
challenging task and needs a lot of considerations, we just consider
the network I/O model problem here.

When a client request comes
in, the server will do some float number calculation, get server system
time and then response the client with these information. The code can
be found in the
ServeTheClient() function in Util.cxx.

1. Multi-threading

The server consists of two parts:
- One Listening Thread: listen and accept client connection
- Many Worker Threads: one for each client

When
new client comes, the Listening Thread accepts it and creates a new
worker thread to serve it. The worker thread reads a string data from
client, do the processing and send back the output to client.

This is the simplest solution, code can be found here: Multithreading Network Server.

2. I/O Multiplexing - BSD Select

In
this model, the main thread only init winsock environment, start
listening, other worker threads do the real work. Each worker thread
handle upto N - 1 client connections(N is the 64 on most BSD compatible
systems), one slot is for listening socket.

While using BSD
Select to implement network server, you should set the server
socket(socket that is used to listen/accept client connections) into
non-blocking mode. The reasoning is that, even select() tells that some
client connection request had arrived, accept() may block since it's
possible that many threads get this notification.

There is a SELECT_CONN_CTX data structure for each client connection. It's definitely a state machine style server architecture.

One
drawback of current implementation is that the number of threads only
increases, never decrease. Worker threads should become less when
concurrent client connections drops down.

An alternative design
is use some dedicated thread(s) to Listen/Accept client connections and
other threads only serve client requests. Since it introduces many
shared data among different threads, synchronization mechanisms (such
as lock) are needed.

The code - BSD Select based Network Server

3. I/O Multiplexing - Event Select

It's very similiar to BSD Select model with some small differences:
- WSAWaitForMultipleEvents() rather than select() is used to wait for network event
-
Since the wait function only returns one array index value, we should
check all event handles that follows the returned one to ensure
fairness.
- WSAEnumNetworkEvents() is used to determine the exact network event
- each thread can only serve WSA_MAXIMUM_WAIT_EVENTS - 1 client connections

The drawback and possible new architecture is the same as BSD select.

Note: Socket should be in non-blocking mode to avoid blocking for the same reason as in BSD select model. But WSAEventSelect() will put a socket into non-blocking mode, so we don't need to call other function to do this as in BSD select model.

The code - Event Select based Network Server

4. Overlapped I/O - Event Waiting

In
this model, you associate each winsock call with an overlapped data
structure, and associate a kernel event with each overlapped data
structure. This event is used to get async/overlapped call completion
notification.

Core logic:
- The main thread only deal with Init work
- Worker threads do async accept using AcceptEx()
- When client connected, it's served by server in state machine style
- When WSAWaitForMultipleEvents() returns, we should check all following handles to ensure fairness
- When no free slot, a new worker thread will be created to serve more clients.

Since
it uses kernel event to get completioin notification, it has the same
drawback as "Kernel Event Select" model - each thread can only serve
WSA_MAXIMUM_WAIT_EVENTS - 1 client connections.

The code - Overlapped I/O with Event Waiting based Network Server

5. Overlapped I/O - Callback Routine

In this model, you pass an extra callback routine parameter when issuing overlapped I/O calls.

Since
each client connection is represented by a context data structure,
which can be accessed in the callback routine, this models seems pretty
simple:
- no multiple threading
- no synchronization
- callback routine represents state transition

Core Logic:
- Main thread is a async listen/accept loop
- AcceptEx() sends completion notification using Kernel Event
- Main thread must wait in alertable state in order to get overlapped i/o callback routine executed.

The code - "Overlapped I/O with Callbacking" based Network Server

6. Overlapped I/O - IO Completion Port

This is the most elegant solution for a scalable network server:
- The main thread is a listen/accept thread, when a client connection arrives, it post a notification to IOCP and let worker thread process it.
- Each worker thread deal with each client connection in state machine style: using network I/O completion notification to trigger state transition and next operation call.
- Let system to control thread number and scheduling.

You
can see from the source code that the logic is very clear and easy to
understand. If you are going to write a network server on windows
platform, this model is highly recommended.

The code - "Overlapped I/O with IOCP" based Network Server

Note:
The original version of CreateIOCompletionPort() is heavily overloaded on its semantic. So I created two separated function CreateNewIoCompletionPort() and AssociateDeviceWithIoCompletionPort() to make each one only do one simple function. The definition and implementation can be found in Util.h and Util.cxx

Conclusion:
Here
I listed the implementation of a scalable network server using various
network I/O models. In next article, I will do some load test on each
server and compare the performance/scalability of each model.

[Reference]
1. Write Scalable Winsock Application using I/O Completion Port
2. Writing scalable server applications using I/O Completion Port
3. Book - Network Programming for Microsoft Windows
4. Network I/O system call implementation on Linux
5. Winsock Network I/O Model - Part I : Theory
6. Scalable I/O Model

Comments

  • Anonymous
    June 09, 2015
    google is returning a 403 access denied to the code download links.