Partilhar via

Why URLScan ignores the querystring for DenyUrlSequences

  • Artigo

Users frequently want to use URLScan's DenyUrlSequences feature to scan the incoming URL, including querystring, and reject based on various arbitrary criteria, such as potential SQL injection characters, banned characters for directories/files, etc.

However, one quickly finds that URLScan only applies its checks, including DenyUrlSequences, against the URI-stem portion but NOT the querystring. So, as the following user asks, why? Is this just a bug in URLScan, or is there another reason...

Question:

Does urlscan ignore the rest of the querystring after "?" and if so, is there a way to get it to process the entire querystring? I can include invalid characters in the querystring after the ? and urlscan allows them - sql injection becomes quite a problem then : )

Answer:

This behavior is actually by design. URLScan does not apply its checks against the querystring because of one fundamental uncertainty - it is not clear how to decode and normalize the querystring to be able to make the logical character comparisons that the user configured.

Why Normalize?

Now, some users question why URLScan needs to normalize a string for logical character comparison. After all, isn't just a simple strstr() of the raw data good enough for both URI-stem and querystring? Well... not exactly.

Consider the simple case of detecting the .. (directory traversal) sequence. According to URL encoding rules, web servers must consider the following raw data sequence sent by clients logically as .. :

  • ..
  • %2E%2E
  • .%2E
  • %2E.

Thus, if you are trying to deny directory traversal but making character comparisons against raw data, you need to declare all permutations of possible encodings which logically map to .. (because the bad guy can try any combination). Clearly, this will easily get out of hand since there are many possible encodings for every logical character and for security, you must give all possible permutations. This is why logical character comparisons MUST be made against normalized strings, not raw data.

The Case Against Querystring

Now, HTTP clearly defines possible encoding semantics for the URI-stem, so URLScan can confidently decode and normalize the URI-strem to make logical character comparisons and take action. But what about the querystring?

RFC 2396 defines "query" in section 3.4 as (I have snipped out all of the relevant BNF definition of the terms in question):

    The query component is a string of information to be interpreted by
   the resource.

      query         = *uric

   Within a query component, the characters ";", "/", "?", ":", "@",
   "&", "=", "+", ",", and "$" are reserved.

      uric          = reserved | unreserved | escaped

      reserved    = ";" | "/" | "?" | ":" | "@" | "&" | "=" | "+" |
                    "$" | ","

      unreserved  = alphanum | mark

      alphanum = alpha | digit

      alpha    = lowalpha | upalpha

      lowalpha = "a" | "b" | "c" | "d" | "e" | "f" | "g" | "h" | "i" |
                 "j" | "k" | "l" | "m" | "n" | "o" | "p" | "q" | "r" |
                 "s" | "t" | "u" | "v" | "w" | "x" | "y" | "z"

      upalpha  = "A" | "B" | "C" | "D" | "E" | "F" | "G" | "H" | "I" |
                 "J" | "K" | "L" | "M" | "N" | "O" | "P" | "Q" | "R" |
                 "S" | "T" | "U" | "V" | "W" | "X" | "Y" | "Z"

      digit    = "0" | "1" | "2" | "3" | "4" | "5" | "6" | "7" |
                 "8" | "9"

      mark        = "-" | "_" | "." | "!" | "~" | "*" | "'" | "(" | ")"

      escaped     = "%" hex hex
      hex         = digit | "A" | "B" | "C" | "D" | "E" | "F" |
                            "a" | "b" | "c" | "d" | "e" | "f"

Notice that the definition of "query" states that it is a string of information to be interpreted by the resource (underlined for emphasis).

In other words, the encoding and meaning of the querystring is completely arbitrary and determined by the targeted resource itself. Intermediaries like URLScan have no idea how the querystring is encoded nor interpreted.

This means that it is impossible for automated tools like URLScan to generically decode and normalize the querystring with 100% accuracy and apply a logical character scan (such as for SQL Injection character sequences). How can URLScan figure out how your web application decodes the querystring and falls victim to SQL Injection?

Quite simply, it cannot. This is the fundamental reason why generic character sequence scanning tools against the querystring is not 100% reliable.

But, but, but...

Now, some users scoff at the harsh stance of URLScan - why not have the ability to only apply some relaxed querystring scanning to certain URL extensions and do the obvious - no decoding whatsoever since applications tend to do this? I mean, other IIS Security tools offers this sort of security "feature", so why doesn't URLScan?

Well, yes, if you constrain the problem, you can find localized solutions that require tweaking, but URLScan is a general purpose tool. Anyone can write a custom ISAPI Filter which works only for their particular situation, but that does not make it a redistributable solution for the masses.

Besides, if you knew which URLs to apply the scan against and you knew how it interprets the querystring, why don't you just fix the code itself? And if you did NOT know the URL nor how the querystring is interpreted, then how do you suppose an external tool like URLScan can figure it out?

Thus, I suggest that you approach solving security issues by fixing the vulnerable source code itself. Generic tools to scan/reject requests simply cannot confidently work 100% of the time.

//David

Comments

  • Anonymous
    July 21, 2005
    I have seen this issue catch people unaware, as they assume that URLScan does do this. This entry will be good to reference when I have to explain why it doesn't.

  • Anonymous
    July 22, 2005
    Thanks.

    Yeah, there are a couple of other URLScan features that are easily misunderstood. I will eventually get around to discussing them as well. Such as granular DenyExtensions and per-site/app filtering (why?!?).

    //David

  • Anonymous
    July 24, 2005

    Windows Vista - formerly Windows code name "Longhorn".
    Rick Strahl has a low-level look at ASP.NET...

  • Anonymous
    October 31, 2007
    David, how to write a custom ISAPI filter which looks for various deny url sequences in a querystring?

  • Anonymous
    December 10, 2007
    David, I have recently seen this: http://www.codeplex.com/IIS6SQLInjection/Release/ProjectReleases.aspx?ReleaseId=8764 Have you heard of it? What is your opinion about it? I have recently built a webshop which uses parametised stored procedures (ASP/VB with MS SQL2000). I wanted to be 100% sure that my site was safe, so was wondering if this ISAPI would provide that extra peace of mind.

  • Anonymous
    December 20, 2007
    Dooza - if you want to be 100% sure that your site was safe, then security code review your web application. That is the only way. The same reason that URLScan ignores the querystring for DenyUrlSequences applies to SQL Injection detection. You cannot perform static security analysis to catch 100% issues involving runtime application context. //David

  • Anonymous
    January 16, 2008
    The comment has been removed

  • Anonymous
    January 17, 2008
    Alberto - Ignoring Querystring is the correct thing to do as I explained in the blog entry. Trying to scan the querystring is a no-win situation. If you want proprietary scanning tools which offer useless "snap on" security, that is your right. //David

  • Anonymous
    March 14, 2008
    Hi Dooza, The ISAPI filter is an open source project and it works to prevent SQL Injection in an intelligent way (i.e. using patterns instead of keywords). Reviewing code as David said is always the best alternative, but this ISAPI was designed to resolve problems of legacy applications which is virtually impossible to find all gaps. For instance, I first developed this filter to save a client with a huge enterprise application written in asp and stored procedure. There were SQL Injection security flaws in both asp code and stored procedures which is really difficult to solve. Test for yourself and let me know in the discussion forum if you find any way of passing the filter with destructive SQL Injection. The project is here: http://www.codeplex.com/IIS6SQLInjection Thanks, Rodney

  • Anonymous
    May 19, 2008
    The problem isn't that I don't want to fix my code, the problem is that the website and DB MUST be up NOW, and so I need something to hold off the current attacks until I can fix the code. And this statement "Anyone can write a custom ISAPI Filter which works only for their particular situation". Well, I'm anyone and I can't. That was a pretty arrogant statement. Either the tool should be more versatile or there should a different tool that IS more versatile.

  • Anonymous
    May 21, 2008
    Julie - sorry you have to endure the stress of a 0 Day security attack which threatens your application, with little warning nor alternatives. I understand your desire to find anything that holds off the attack, but that hardly justifies an unreasonable demand for an external solution that does not and cannot exist. Even if there IS a PERFECTLY versatile tool, I can guarantee you that the attacker can still get their attack through -- that is the nature of canonicalization attacks. So, I'm sorry to say that there is no external defense -- if you find such a tool which can "hold off the current attack", and I can show you how an attacker can easily bypass your tool's defenses. This is a cat-and-mouse game where you cannot win. The safest thing for you to do is to take the application offline and fix it as quickly as possible. That's the problem with a 0-day attack -- it is highly disruptive. //David

  • Anonymous
    May 29, 2008
    Why dont you make a global.asa filtering URL? if u are using asp global.asa stop the attack Be cool

  • Anonymous
    June 18, 2008
    Magul > An URL filtering in the global.asa file can help sometimes. However, it can be easily bypassed for session-based sites: For instance, if a single request pass through your filter, all requests of the same session are accepted ! This is the type of attack my web site is currently under. As far as I know, I have no injection flaw in my pages, however such attacks can consume a lot of memory. The only solution I have found so far is to include an asp filter in every page of my site that do a "Session.Abandon" + "Response.End" in case of injection attempts.

  • Anonymous
    June 26, 2008
    Good news: today you can download URLSCAN 3 beta. It has new features:    * Deny rules can now be independently applied to query string, all headers, a particular header, URL or a combination of these.    * A global DenyQueryString section in configuration lest you add deny rules for query strings with the option of checking the un-escaped version of the query string as well.    * Using escape sequences (like %0A%0D) can now be used in deny rules so it is possible to deny CRLF and other sequences involving non-printable characters.    * Multiple UrlScan instances can now be installed as site filters, each with its own configuration and rules (urlscan.ini).    * Configuration (urlscan.ini) change notifications will be propagated to IIS worker processes so you won’t have to recycle your worker processes after making a configuration change. Logging settings are the only exception to this.    * Enhanced logging to give descriptive configuration errors. http://learn.iis.net/page.aspx/473/using-urlscan

  • Anonymous
    July 07, 2008
    I'm glad to see that Querystring scanning was added into the beta. As a hosting provider I run URLScan on all my servers, but I can't fix my customers broken code and I'm getting a bit tired of restoring db's that have had sql injection attacks.  A couple of pattern matches to the urlscan would have prevented alot of headache trying to explaing to non technical customers why thier website was "hacked" and why it wasn't our fault but thier programmers.

  • Anonymous
    July 15, 2008
    The comment has been removed