REST WCF vs. WebAPI (throughput performance)
(Updated: providing more info about provided tests)
Goal
This post compares WCF and WebAPI technologies from the performance perspective.
Service description
I implemented almost same service in two different ways. Both are accessible via HTTP/REST and both works in the way to skip the serialization in order to isolate logic which could impact the performance results.
Both services implements GET and POST requests:
- GET request to return a string with current time
- POST request to accept a data and return back the same data in the stream
I self-hosted both services and run them via Full Azure emulator.
My own test agent
The first approach was stress testing via my own implementation, very similar to the way described here: https://www.ducons.com/blog/tests-and-thoughts-on-asynchronous-io-vs-multithreading). I was able to get cca 4000 request/sec but it had the following drawback that the test agent used more CPU than server so that the results were not real, the whole system run on 100% CPU. I think, it was mainly due to overhead connected with TPL and HttpClient.
Azure Load testing
This approach wasn’t successful neither, I created and setup the azure load testing (max number of users without any delays) but I was able to run max little bit over 2000requests/second.
Appache Benchmark
I knew this tool from my past and using it was worth again! It’s built on sockets, which has the minimal overhead (compared to HttpClient). There is also a similar benchmark done by Rick Strahl but I’ve measured a little bit different numbers.
WCF service details
class TestService : ITestService
{
public Stream TestGet()
{
return new MemoryStream(Encoding.UTF8.GetBytes("Hello World. Time is: " + DateTime.Now));
}
public async Task<Stream> TestStream(Stream requestStream)
{
using (var reader = new StreamReader(requestStream))
{
var body = await reader.ReadToEndAsync();
return new MemoryStream(Encoding.UTF8.GetBytes(body));
}
}
}
WebAPI service details
public class TestController : ApiController
{
public HttpResponseMessage Get()
{
return new HttpResponseMessage() { Content = new StringContent("Hello World. Time is: " + DateTime.Now, Encoding.UTF8, "text/plain") };
}
public async Task<HttpResponseMessage> Post(HttpRequestMessage inputMessage)
{
var content = await inputMessage.Content.ReadAsByteArrayAsync();
var response = new HttpResponseMessage { Content = new ByteArrayContent(content) };
return response;
}
public async Task<HttpResponseMessage> Put(HttpRequestMessage inputMessage)
{
var content = await inputMessage.Content.ReadAsStreamAsync();
var response = new HttpResponseMessage { Content = new StreamContent(content) };
return response;
}
}
Thanks to feedback from Joakim:
As you can see, WebAPI has POST and PUT request support. The reason is, that StreamContent turns the response to the chunked transfer mode and I wanted to compare chunked and unchunked modes.
Performance tests
I run both services locally on my machine (Dell XPS12, i7, SSD). In each test I issue 60000 requests with concurrency set to 100. POST requests sent and received 500 bytes over the network. Here are the commands and results:
GET request to WebAPI
ab -n 60000 -c 100 -k https://localhost:8082/api/test
Server Software: Microsoft-HTTPAPI/2.0
Server Hostname: localhost
Server Port: 8082
Document Path: /api/test
Document Length: 41 bytes
Concurrency Level: 100
Time taken for tests: 8.042 seconds
Complete requests: 60000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 60000
Total transferred: 13860000 bytes
HTML transferred: 2460000 bytes
Requests per second: 7460.40 [#/sec] (mean)
Time per request: 13.404 [ms] (mean)
Time per request: 0.134 [ms] (mean, across all concurrent requests)
Transfer rate: 1682.96 [Kbytes/sec] received
GET request to WCF
ab -n 60000 -c 100 -k https://localhost:8080/wcf/test/testget
Server Software: Microsoft-HTTPAPI/2.0
Server Hostname: localhost
Server Port: 8080
Document Path: /wcf/test/testget
Document Length: 41 bytes
Concurrency Level: 100
Time taken for tests: 7.333 seconds
Complete requests: 60000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 60000
Total transferred: 13800000 bytes
HTML transferred: 2460000 bytes
Requests per second: 8181.73 [#/sec] (mean)
Time per request: 12.222 [ms] (mean)
Time per request: 0.122 [ms] (mean, across all concurrent requests)
Transfer rate: 1837.69 [Kbytes/sec] received
POST request to WebAPI
POST request is implemented in NON-CHUNKED transfer mode.
ab -n 60000 -c 100 -p c:\temp\data500.txt -k https://localhost:8082/api/test
Server Software: Microsoft-HTTPAPI/2.0
Server Hostname: localhost
Server Port: 8082
Document Path: /api/test
Document Length: 508 bytes
Concurrency Level: 100
Time taken for tests: 8.729 seconds
Complete requests: 60000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 60000
Total transferred: 39480000 bytes
Total POSTed: 40267000
HTML transferred: 30480000 bytes
Requests per second: 6873.25 [#/sec] (mean)
Time per request: 14.549 [ms] (mean)
Time per request: 0.145 [ms] (mean, across all concurrent requests)
Transfer rate: 4416.60 [Kbytes/sec] received
4504.64 kb/s sent
8921.24 kb/s total
PUT request to WebAPI
PUT request is implemented in CHUNKED transfer mode.
ab -n 60000 -c 100 -u c:\temp\data500.txt -k https://localhost:8082/api/test
Server Software: Microsoft-HTTPAPI/2.0
Server Hostname: localhost
Server Port: 8082
Document Path: /api/test
Document Length: 0 bytes
Concurrency Level: 100
Time taken for tests: 10.079 seconds
Complete requests: 60000
Failed requests: 0
Write errors: 0
Non-2xx responses: 60001
Keep-Alive requests: 60000
Total transferred: 10020167 bytes
Total PUT: 40207569
HTML transferred: 0 bytes
Requests per second: 5953.22 [#/sec] (mean)
Time per request: 16.798 [ms] (mean)
Time per request: 0.168 [ms] (mean, across all concurrent requests)
Transfer rate: 970.90 [Kbytes/sec] received
3895.90 kb/s sent
4866.81 kb/s total
POST request to WCF
ab -n 60000 -c 100 -p c:\temp\data500.txt -k https://localhost:8080/wcf/test/teststream
Server Software: Microsoft-HTTPAPI/2.0
Server Hostname: localhost
Server Port: 8080
Document Path: /wcf/test/teststream
Document Length: 508 bytes
Concurrency Level: 100
Time taken for tests: 9.365 seconds
Complete requests: 60000
Failed requests: 0
Write errors: 0
Keep-Alive requests: 60000
Total transferred: 41880000 bytes
Total POSTed: 40928100
HTML transferred: 30480000 bytes
Requests per second: 6407.15 [#/sec] (mean)
Time per request: 15.608 [ms] (mean)
Time per request: 0.156 [ms] (mean, across all concurrent requests)
Transfer rate: 4367.37 [Kbytes/sec] received
4268.11 kb/s sent
8635.48 kb/s total
Summary
The measured results showed that WCF is faster than WebAPI when sending GET requests but slower when executing POST request in non-chunked mode. My results are different than Rick’s results little bit. WCF proved to be faster in some scenarios and I think all depends on the data payload. My take-away from it:
- I have a base line for the further service development
- both platforms are “fast enough” for me and I’m going to stay with WebAPI as it looks to be more appropriate for building further real RESTfull service development but I wrote about it in my previous article.
Comments
Anonymous
January 06, 2014
Fun to see some simple benchmark between the two. At least just to see that it's not a 10x performance diffrence But why does the POST data state Total transferred: 10020000 bytes for WebAPI Total transferred: 41880000 bytes for WCF. Thats a 4x increment in data transfered so the test can't be fair. It would also be nice to see the test without tasks since tasks should be used for long running jobs/long running io. Something this test doesn't contain.Anonymous
January 07, 2014
Thanks Joakim, yeah, amount of data transferred is quite different, I'll look at it and provide more results for sync version, too. I had several versions but published just async.Anonymous
January 08, 2014
The comment has been removedAnonymous
January 10, 2014
Both hosted in IIS? I wonder how web-api would do on the OWIN stack.Anonymous
January 15, 2014
@Jarrod: no, selfhosted, using OWINAnonymous
June 09, 2016
Hi all! I was wondering if these tests still give the same results as in 2014 assuming the changes that may have occured in the last 2 years.