The WCF Service, the Load Balancer and the Transport Security
Looking at the title of this post it looks almost that I will be talking about a remake of some "Western Spaghetti" of the old days, but no, rest assure, that is not the case the topic is something much more dangerous.
Lets talk about WCF Services, Load Balancers and Security stuff, scary talk, right?
So, one of this day I got this ask for help in one of those scenarios, you know, the usual stuff... Yeah it was working and then stop working. And the change was that they just add a Load Balancer to the mix.
Lets then start by providing some details on the issue, we have this client application that was now trying to call a WCF Service and it was not able to reach it. The error that the client application was throwing was the below one:
There was an error downloading 'https://myamazingServer.josepen.dev/WCFBulletProof/servicenumberone.svc'. The request failed with HTTP status 403: Forbidden. Metadata contains a reference that cannot be resolved: ``'https://myamazingServer.josepen.dev/WCFBulletProof/servicenumberone.svc'``. ``The HTTP request was forbidden with client authentication scheme 'Anonymous'. `` The remote server returned an error: (403) Forbidden. ``If the service is defined in the current solution, try building the solution and adding the service reference again.
If you look close enough in the error message you not that is referring to HTTP when the communication should be on HTTPS. This fact is a clear hint why the application was working when accessing a directly the Server but not when going through the load balancer. The reason is that since the client were using a transport binding (since the requirement was to connect the client through https).
However one of the drawbacks of transport security is that it is a hop-to-hop security only.
That is, the security will be established only from the client to the load balancer. Once the information reaches the load balancer, it will terminate the SSL at the load balancer and decrypt the traffic and redirect to the appropriate server. That is the reason why the error message mentioning HTTP instead of HTTPS.
Lets then recap this, basically the issue was caused by the fact that the Server was expecting a Certificate to be presented by the client through https. However, the load balancer is sending an anonymous credential through http.
Based on the above, I would say that we could follow two approaches:
- Set first a secure transport channel between the client and the load balancer. Then, set another secure transport channel from the load balancer to the appropriate server.
- Use a message security instead of a transport level security because Message security is an end-to-end security .
See below what Microsoft quoted about message security in this article:
"Transport security, such as Secure Sockets Layer (SSL) only secures messages when the communication is point-to-point.
If the message is routed to one or more SOAP intermediaries (for example a router) before reaching the ultimate receiver, the message itself is not protected once an intermediary reads it from the wire.
Additionally, the client authentication information is available only to the first intermediary and must be re-transmitted to the ultimate receiver in out-of-band fashion, if necessary.
This applies even if the entire route uses SSL security between individual hops. Because message security works directly with the message and secures the XML in it, the security stays with the message regardless of how many intermediaries are involved before it reaches the ultimate receiver. This enables a true end-to-end security scenario."
After passing this information to the developer that was facing this issue, I was feeling really happy for being able to help another fellow dev.
He really appreciate the help, and told me that they will be following a Message based security over Transport based since there is the load balancer in the middle.
Hope that helps