Custom HTTP Authentication Schemes

Introduction

My name is Chris Ross and I work as a developer for Microsoft’s .NET Framework networking components. As part of the Network Class Library (NCL) team I get lots of networking questions from other developers. This post resulted from my research into a question about using custom HTTP authentication schemes. The custom scheme in question was Google’s GoogleLogin scheme, so I’ll use it as an example to show how System.Net.WebRequest and System.Net.WebClient can work with custom schemes. Fist though, I should explain how standard HTTP authentication works.

Standard HTTP Authentication

It has become increasingly important to secure online information from unauthorized users. But once secured, how do clients figure out how to login? Well, the HTTP protocol utilizes a back and forth negotiation pattern to let the client know that something is secured and what type of credentials they must submit to gain access.

A common HTTP authentication pattern goes as follows:

WebRequest.Credentials = new  NetworkCredential(username, password);
WebRequest sent (without Credentials)	To	Secured web page
Error 401 – List of login schemes supported (Basic, Digest…)	From	Secured web page
WebRequest +Credentials (formatted according to the selected login scheme) sent	To	Secured web page
200 – Success – Here is your secured content	From	Secured web page

 

The best part is that .NET handles this back and forth for you under the hood, so all you end up seeing is the content you asked for. The .NET Framework (WebClient & WebRequest) has built in support for Basic, Digest, NTLM, Kerberos, and Negotiate authentication schemes.

Here is an example on how you would use standard authentication schemes with WebRequest:

public static void NormalHTTPAuthExample(String username, String password)

{

    NetworkCredential creds = new NetworkCredential(username, password);

    WebRequest request = WebRequest.Create("https://www.contoso.com/");

    request.Credentials = creds;

    // Send the request and process the response

    WebResponse response = request.GetResponse();

    StreamReader responseStreamReader =

        new StreamReader(response.GetResponseStream());

    String result = responseStreamReader.ReadToEnd();

    responseStreamReader.Close();

    Console.WriteLine(result);

}

WebRequest Authentication Practices

WebRequest takes great care with the credentials you provided. Many of these precautions are to help prevent your username and password from being exposed to unauthorized servers. Two features related to our example are CredentialCache and PreAuthenticate.

WebRequest by default facilitates automatic request redirection from site to site. However if a server you have logged into decides to redirect you elsewhere, WebRequest will remove its Credentials property before automatically redirecting to prevent your username and password from being exposed. If you are aware that redirects will take place, you can use the CredentialCache class to manage what servers are allowed access to your credentials. A CredentialCache pairs credentials with the URIs and authentication schemes they’re allowed to be used with. When you have created and configured a CredentialCache object, you then assign it to the WebRequest.Credentials property. You can see this in the next code sample. A CredentialCache will not be removed from the Credentials property when redirecting because WebRequest knows where you will allow your credentials sent. You may also reuse your cache by assigning it to subsequent requests. All of this also applies to WebClient.Credentials, as it uses WebRequest for its underlying operations.

                A second WebRequest property that can be useful with HTTP authentication is PreAuthenticate. Without it, WebRequest will not include your credentials on a request until it has first received a challenge from the server. Setting this property is optional but will help boost performance on secured sites as follows. After you’ve successfully authenticated once, this flag causes your credentials to be included in the Authorization header automatically on subsequent requests and redirects that match the same URI path at the folder level. This way the server does not have to issue a challenge for authentication on each page. WebClient does not expose this property.

Here is a code sample that shows both of these properties:

public static void RedirectHTTPAuthExample(String username, String password)

{

    NetworkCredential creds = new NetworkCredential(username, password);

    CredentialCache credCache = new CredentialCache();

    // Cached credentials can only be used when requested by

    // specific URLs and authorization schemes

    credCache.Add(new Uri("https://www.contoso.com/"), "Basic", creds);

    WebRequest request = WebRequest.Create("https://www.contoso.com/");

    // Must be a cache, basic credentials are cleared on redirect

    request.Credentials = credCache;

    request.PreAuthenticate = true; // More efficient, but optional

    WebResponse response = request.GetResponse();

    StreamReader responseStreamReader =

        new StreamReader(response.GetResponseStream());

    String result = responseStreamReader.ReadToEnd();

    responseStreamReader.Close();

    Console.WriteLine(result);

}

Custom HTTP Authentication Schemes

What if the sever you’re requesting information from does not utilize one of the standard authentication schemes? Take for an example Google’s custom GoogleLogin authentication scheme. I’m no expert on Google’s APIs, but its differences from standard HTTP authentication schemes make for a good example. GoogleLogin requires first making a request to a completely different login server for an Auth token, then adding that token to every subsequent request’s Authorization header. You can still do this with .NET, but because GoogleLogin is a custom scheme you might have to handle some of the back and forth yourself, such as turning off automatic redirects so you can be sure to include the Auth token on each request.

For example, you would first make a request to the login server for your user token. Then you’d add that token to a second request to the Calendar server. The Calendar server responds with a redirect to your specific calendar at a session specific url. You then issue one final request (again including the Auth token) to that session specific url for your calendar data.

WebRequest (with user name and password) sent	To	Login server
Auth token	From	Login server
WebRequest + Auth token	To	Secured Calendar server
302 – Redirect to session specific URL	From	Secured Calendar server
WebRequest + Auth token sent to new URL	To	Your Secured Calendar
200 – Success – Here is your secured content	From	Your Secured Calendar

 

While this approach can work well enough to use once, I find it unwieldy to make three different requests where only one would normally be required, especially if you needed to do it often.

 Luckily .NET provides a better answer to this sort of problem: Custom Authentication Modules. You can add your own modules to the list of supported authentication schemes for WebClient & WebRequest, and let .NET juggle the authentication, redirection, proxies, and other complications for you. Below is an example of how to do this for Google’s custom GoogleLogin scheme.

How to use a custom module with WebRequest/WebClient

                How do we use WebRequest and a custom authentication module to log in to Google services? Here is a sample:

public static void GoogleAuthManagerExample(String username, String password)

{

    AuthenticationManager.Register(new GoogleLoginClient());

    // Now 'GoogleLogin' is a recognized authentication scheme

    GoogleCredentials creds = // user@gmail.com

        new GoogleCredentials(username, password, "HOSTED_OR_GOOGLE");

    CredentialCache credCache = new CredentialCache();

    // Cached credentials can only be used when requested by

    // specific URLs and authorization schemes

    credCache.Add(new Uri(

        "https://www.google.com/calendar/feeds/default/private/"),

        "GoogleLogin", creds);

    WebRequest request = WebRequest.Create(

"https://www.google.com/calendar/feeds/default/private/full?q=null+item");

    // Must be a cache, basic credentials are cleared on redirect

    request.Credentials = credCache;

    request.PreAuthenticate = true; // More efficient, but optional

   

    WebResponse response = request.GetResponse();

    StreamReader responseStreamReader =

        new StreamReader(response.GetResponseStream());

    String result = responseStreamReader.ReadToEnd();

    responseStreamReader.Close();

    Console.WriteLine(result);

    // Erase cached auth token unless you'll use it again soon.

    creds.PrevAuth = null;

}

                The only meaningful difference between this code block and the previous authentication example I showed is the registration of the GoogleLoginClient. The GoogleLoginClient is a custom authentication module that WebRequest can refer to for handling the special symantics of Google’s API. As I described before, the alternative is to include much of the folowing Google API code inline, as well as managing all of your own redirection. I definitely prefer the custom modules, so lets look how to implement this custom authentication module.

How to write a custom authentication module for GoogleLogin

The first minor element is a storage place for Google’s Auth token and login parameters. Caching the token is ok so long as you’re careful where you put it and you know when you need to clear it.

public class GoogleCredentials : NetworkCredential {

    private Authorization prevAuth = null;

    public Authorization PrevAuth { // Cached login token

        get { return prevAuth; }

        set { prevAuth = value; }

    }

    private String accountType;

    public String AccountType {

        get { return accountType; }

        // Validate "GOOGLE","HOSTED", or "HOSTED_OR_GOOGLE"

        set { accountType = value; }

    }

    public GoogleCredentials(String user, String pswd,

        String accountType)

        : base(user, pswd) {

        this.AccountType = accountType;

    }

}

Then we need to implement the IAuthenticationModule interface. When we are challenged for GoogleLogin authorization this will do the actual request to Google’s login server and fetch us an Auth token.

public class GoogleLoginClient : IAuthenticationModule {

    internal const string AuthType = "GoogleLogin"; // Scheme identifier

    internal static string AuthServer

        = "https://www.google.com/accounts/ClientLogin";

    public String ServiceString = "cl"; // Calendar

    public String Source = "MSTest"; // Our program name

    public Authorization Authenticate(string challenge,

        WebRequest webRequest, ICredentials credentials) {

        // Careful, if your challenge contains more than one

        // authorization scheme, this one might not be first

        // in the list. Also ignore parameter names and quoted

        // parameter values. See RFC 2617 Section 1.2

        // ie: Basic, Digest nonce=122352354,

        // realm="www.GoogleLoginDirections.com/help",

        // GoogleLogin realm="https://login.google.com/",Ntlm

        if (!challenge.Contains(AuthType)

            /* && ContainsNotInQuotes(challenege,AuthType)

    * && MoreValidation(challenege,AuthType) */) {

            return null;

        }

        return Login(webRequest, credentials);

    }

    public Authorization PreAuthenticate(WebRequest webRequest,

        ICredentials credentials) {

        return Login(webRequest, credentials);

    }

    public bool CanPreAuthenticate {

        // Some schemes do not support PreAuthentication

        get { return true; }

    }

    public string AuthenticationType {

        get { return AuthType; }

    }

    private Authorization Login(WebRequest webRequest,

        ICredentials credentials) {

        // Do we have credentials for this site?

        NetworkCredential NC = credentials.GetCredential(

            webRequest.RequestUri, AuthType);

        GoogleCredentials gcreds = NC as GoogleCredentials;

        if (gcreds == null)

            return null; // none or wrong type of credentials

        if (gcreds.PrevAuth != null)

            return gcreds.PrevAuth; // Cached from last login

       

      ICredentialPolicy policy =

            AuthenticationManager.CredentialPolicy;

        if (policy != null && !policy.ShouldSendCredential(

            webRequest.RequestUri, webRequest, NC, this))

            return null;

        WebRequest client = WebRequest.Create(AuthServer);

        client.ContentType = "application/x-www-form-urlencoded";

        client.Method = "POST";

        // Custom authentication string:

   // https://code.google.com/apis/accounts/docs/AuthForInstalledApps.html

        String requestParams = "accountType=" + gcreds.AccountType

            + "&Email=" + gcreds.UserName + "&Passwd=" + gcreds.Password

            + "&service=" + ServiceString + "&source=" + Source;

        byte[] bytes = Encoding.UTF8.GetBytes(requestParams);

        // Google's API says that the custom authentication string

        // goes in the body of this request. This is unusual.

        Stream requestStream = client.GetRequestStream();

        requestStream.Write(bytes, 0, bytes.Length);

        requestStream.Close();

        // The Auth token comes in the response body. Also unusual.

        WebResponse response = client.GetResponse();

        StreamReader responseStreamReader =

            new StreamReader(response.GetResponseStream());

        String result = responseStreamReader.ReadToEnd();

        responseStreamReader.Close();

        String authToken = ""; // Parse out the Auth token

        String[] tokens = result.Split(new String[] { "\n" },

            StringSplitOptions.None);

        foreach (String token in tokens) {

            if (token.StartsWith("Auth=",

                StringComparison.OrdinalIgnoreCase)) {

                authToken = token;

                break;

            }

        }

        if (authToken == "")

            throw new WebException("GoogleLogin authentication failed");

        // Assemble the Authorization header and cache it

        gcreds.PrevAuth = new Authorization(AuthType + " " + authToken);

        return gcreds.PrevAuth;

    }

}

                And you’re done. Now any time your application is challenged for GoogleLogin authentication, WebRequest can just refer to this new module automatically.

Conclusions

                HTTP authentication can take many forms, and .NET includes support for several standard schemes. When these are not sufficient, it is easy to add custom schemes for many other services with minimal changes to your existing code.

Notes & references

- WebClient: https://msdn.microsoft.com/en-us/library/system.net.webclient.aspx

- WebRequest: https://msdn.microsoft.com/en-us/library/system.net.webrequest.aspx

- IAthenticationModule: https://msdn.microsoft.com/en-us/library/system.net.iauthenticationmodule.aspx 

- Register a new custom module in the App.Config file without modifying existing code: https://msdn.microsoft.com/en-us/library/y9b82x09.aspx

- CredentialCache: https://msdn.microsoft.com/en-us/library/system.net.credentialcache.aspx

- GoogleLogin API: https://code.google.com/apis/accounts/docs/AuthForInstalledApps.html

- RFC 2617 – HTTP Basic & Digest authentication - https://www.ietf.org/rfc/rfc2617.txt

- RFC 2616 – HTTP 1.1 - https://www.ietf.org/rfc/rfc2616.txt

- VB example: https://support.microsoft.com/default.aspx/kb/331501

~Chris Ross

System.Net

Comments

• Anonymous
  June 30, 2009
  The comment has been removed

• Anonymous
  June 30, 2009
  Sorry for the terrible formatting, wasn't sure how to fix that!

• Anonymous
  July 02, 2009
  The comment has been removed

• Anonymous
  July 03, 2009
  The comment has been removed

• Anonymous
  December 15, 2009
  I'm having some issues with this and was hoping for a little help.  I am calling the above method for hte following URL : http://www.google.com/m8/feeds/profiles/domain/REMOVED.com/full Is there something I am missing ?  I am passing an admin username and password.  Any help is GREATLY appreciated!! Thanks, Mike

• Anonymous
  January 05, 2010
  From what I can see in their documentation for hosted domains, this approach should work.  Unfortunately I don't have access to a hosted domain to verify.

• Anonymous
  May 02, 2010
  I am consuming a webservice in asp.net(c#) which is hosted by some third party which grant access to webservice using Http Basic thentication(RFC2617). I am not using wcf. plz send me some working code for how to call webservice giving username and password using Http Basic authentication(RFC2617). my emailid is kunaltilak@gmail.com thanks in advance

• Anonymous
  February 19, 2014
  if i enter login credentials in my username & passwords fields and click on submit i want redirect to gmail inbox ie gmail inbox