HOWTO: ISAPI Filter which Logs original Client IP for Load Balanced IIS Servers
Invariably, when you run IIS servers that are load-balanced or forwarded requests behind some other network device, you will find that IIS logs the IP of the network device and not the original client that made the request.
Technically, there is no standard describing how to address/fix this situation, so IIS does not have built-in support for this issue. However, there are some popular private hacks used by several network devices, and here is an ISAPI Filter that allows IIS to easily work with those devices to fix this issue and log the original client IP in IIS.
Question:
Hi,
I need some help!
I have a situtation where I am using a hardware web load balancing device (Netscaler) that changes the IP address that the client HTTP request appears to come from. It also adds an HTTP header to the request (X-Client-IP) that contains the "real" IP addres that the client HTTP request came from.
This is all well and good except that the IIS log files do not contain the right client IP address. I would like to "fix" the log file so that it contains the true client IP.
What I think I need is to write an ISAPI filter that gets the X-Client-IP from the HTTP request and then replace the HTTP IP (pszClientHostName ??) before it gets logged.
Am I on the right track? Is there good example code on how to do something like this? My solution would only need to work for IIS6.
Thanks,
Answer:
Ok, this is one of the more popular behavior requests of IIS which can be easily accomplished with a simple ISAPI Filter which extends the functionality of IIS. So, I went ahead and implemented it, and it is attached at the end of this entry. If you need help compiling it, check out my "sample code" RSS feed for more assistance. My assumption is that if you are able to copy sample code, then you are able to compile it as well as support it yourself.
Behaviorally, this ISAPI Filter is very straight forward.
- In SF_NOTIFY_PREPROC_HEADERS, use GetHeader() to retrieve the named request header that is supposed to contain the value of the "original" client IP.
- This named request header is actually configurable via an INI file directive, as shown through the GetModuleFileName() and GetPrivateProfileString() Win32 calls. It defaults to "X-Client-IP:" (trailing ':' is important to GetHeader() ).
- If the request header exists, then allocate memory using AllocMem() and save it in pFilterContext for retrieval later on in that request's SF_NOTIFY_LOG event.
- In SF_NOTIFY_LOG, if pFilterContext != NULL, then it means that the named header was found on the request and pFilterContext contains the new value, so replace pszClientHostName with this value to change what IIS logs. Otherwise, do nothing.
Here are some of the subtle decision points in this filter:
- I chose to retrieve the request header containing the original client IP using GetHeader() in SF_NOTIFY_PREPROC_HEADERS instead of GetServerVariable() in SF_NOTIFY_LOG because of two reasons:
GetHeader() is able to retrieve any valid request header (including both '-' and '_' characters) on all IIS versions.
GetServerVariable() is NOT able to retrieve valid request headers that contain '-' UNLESS you are running IIS6 and use the documented special HEADER_ prefix
My goal for this sample filter is for maximum compatibility with all IIS versions. Since I lose nothing with my approach and still retain compatibility, I am choosing the win-win approach. :-)
- The memory used in changing a IIS log entry is allocated with AllocMem() and NOT new/delete. The reason is because IIS requires this memory to stay valid until it has logged the changed value to disk. This means a stack-based buffer is insufficient because it goes out of scope as soon as SF_NOTIFY_LOG finishes, and managing the lifetime of a new/delete buffer is actually icky (on a per-connection basis, you need to delete this memory after SF_NOTIFY_LOG completes but BEFORE SF_NOTIFY_PREPROC_HEADERS fires again. I cannot think of an event to consistently catch this state). AllocMem() is nice because IIS tracks this memory for you and automatically frees it when its associated connection ends. This matches perfectly with the requirements of modifying log entries and eliminates management of that memory buffer's lifetime. Once again, a win-win situation.
- pfc->pFilterContext is useful to convey a value generated from one event (SF_NOTIFY_PREPROC_HEADERS) and consumed by a later event (SF_NOTIFY_LOG)
If you want to configure the name of the request header that contains the original client's IP, put the following text into an INI file located in the same directory and named the same as the name of the ISAPI Filter DLL module (i.e. if the Filter DLL is named "LoadBalancedIP.dll", then the text needs to be in a file named "LoadBalancedIP.ini" in that directory). You need to change the MODULE_NAME #define to match the LIBRARY directive in your .def file if you want to control the name of the ISAPI Filter DLL (I am assuming it is called LoadBalancedIP.dll). Yes, this behaves just like URLScan.
[config]
ClientHostName=New-Header_Name:
This will cause the value of the request header "New-Header_Name:" to be logged as the client IP in the IIS log file, if it exists.
To have changes in the INI file take effect, you need to cause the Filter DLL to reload (i.e. go through GetFilterVersion() again). For IIS6, it means to recycle that worker process; for prior IIS versions, it means to restart the W3SVC service.
Enjoy.
//David
#include <windows.h>
#include <httpfilt.h>
#define MAX_CLIENT_IP_SIZE 256
#define MODULE_NAME "LoadBalancedIP"
TCHAR g_szHeaderName[ MAX_PATH + 1 ];
TCHAR g_szIniPathName[ MAX_PATH + 1 ];
DWORD
OnPreprocHeaders(
IN HTTP_FILTER_CONTEXT * pfc,
IN HTTP_FILTER_PREPROC_HEADERS * pPPH
);
DWORD
OnLog(
IN HTTP_FILTER_CONTEXT * pfc,
IN HTTP_FILTER_LOG * pLog
);
BOOL
WINAPI
GetFilterVersion(
HTTP_FILTER_VERSION * pVer
)
/*++
Purpose:
Required entry point for ISAPI filters. This function
is called when the server initially loads this DLL.
Arguments:
pVer - Points to the filter version info structure
Returns:
TRUE on successful initialization
FALSE on initialization failure
--*/
{
CHAR * pCursor = NULL;
//
// Locate the config file, if it exists
//
GetModuleFileName(
GetModuleHandle( MODULE_NAME ),
g_szIniPathName,
MAX_PATH );
pCursor = strrchr( g_szIniPathName, '.' );
if ( pCursor )
{
*(pCursor + 1) = '\0';
}
//
// Config file is located with DLL with extension INI
//
strcat( g_szIniPathName, "ini" );
if ( !GetPrivateProfileString( "config",
"ClientHostName",
"X-Client-IP:",
g_szHeaderName,
MAX_PATH,
g_szIniPathName ) )
{
SetLastError( ERROR_INVALID_PARAMETER );
return FALSE;
}
pVer->dwFilterVersion = HTTP_FILTER_REVISION;
lstrcpyn( pVer->lpszFilterDesc,
"ISAPI Filter to twiddle log entry fields based "
"on parameterized sources (like request header)",
SF_MAX_FILTER_DESC_LEN );
//
// Technically, I could retrieve the request header from
// SF_NOTIFY_LOG using GetServerVariable, but that loses
// flexibility because GetServerVariable cannot retrieve
// headers that include "_" (underscore) without using
// IIS6 specific syntax of HEADER_(header-name-as-is).
//
// For maximum compatibility (this filter will work from
// IIS4 on up), I am retrieving the request header from
// SF_NOTIFY_PREPROC_HEADERS using GetHeader (which can
// retrieve headers as-is) and changing the log entry
// in SF_NOTIFY_LOG.
//
// This allows me to illustrate common usage case for
// pFilterContext and pfc->AllocMem(), especially when
// modifying log fields (the memory must stay valid!!!)
//
pVer->dwFlags =
SF_NOTIFY_ORDER_HIGH |
SF_NOTIFY_PREPROC_HEADERS |
SF_NOTIFY_LOG
;
return TRUE;
}
DWORD
WINAPI
HttpFilterProc(
IN HTTP_FILTER_CONTEXT * pfc,
DWORD dwNotificationType,
LPVOID pvNotification
)
/*++
Purpose:
Required filter notification entry point. This function is called
whenever one of the events (as registered in GetFilterVersion) occurs.
Arguments:
pfc - A pointer to the filter context for this notification
NotificationType - The type of notification
pvNotification - A pointer to the notification data
Returns:
One of the following valid filter return codes:
- SF_STATUS_REQ_FINISHED
- SF_STATUS_REQ_FINISHED_KEEP_CONN
- SF_STATUS_REQ_NEXT_NOTIFICATION
- SF_STATUS_REQ_HANDLED_NOTIFICATION
- SF_STATUS_REQ_ERROR
- SF_STATUS_REQ_READ_NEXT
--*/
{
switch ( dwNotificationType )
{
case SF_NOTIFY_PREPROC_HEADERS:
return OnPreprocHeaders(
pfc,
(HTTP_FILTER_PREPROC_HEADERS *) pvNotification );
case SF_NOTIFY_LOG:
return OnLog(
pfc,
(HTTP_FILTER_LOG *) pvNotification );
}
return SF_STATUS_REQ_NEXT_NOTIFICATION;
}
DWORD
OnPreprocHeaders(
HTTP_FILTER_CONTEXT * pfc,
HTTP_FILTER_PREPROC_HEADERS * pPPH
)
{
DWORD dwRet = SF_STATUS_REQ_NEXT_NOTIFICATION;
BOOL fRet = FALSE;
CHAR * pszBuf = NULL;
DWORD cbBuf = 0;
SetLastError( NO_ERROR );
if ( pfc == NULL ||
pPPH == NULL )
{
SetLastError( ERROR_INVALID_PARAMETER );
goto Finished;
}
//
// Make sure to reset the filter context to NULL for every
// request so that if we failed to find a named header, we
// do nothing in SF_NOTIFY_LOG
//
pfc->pFilterContext = NULL;
fRet = pPPH->GetHeader( pfc, g_szHeaderName, pszBuf, &cbBuf );
if ( fRet == FALSE )
{
if ( GetLastError() == ERROR_INVALID_INDEX )
{
//
// The header was not found
// Ignore and do nothing
//
OutputDebugString( "Named header is not found. Do nothing.\n" );
SetLastError( NO_ERROR );
goto Finished;
}
else if ( GetLastError() == ERROR_INSUFFICIENT_BUFFER &&
cbBuf < MAX_CLIENT_IP_SIZE )
{
//
// The header was found and fit size requirements.
//
// Let's allocate from IIS's ACache memory which
// is guaranteed to live as long as the connection,
// so it is perfect for log entry modification.
//
pszBuf = (CHAR *)pfc->AllocMem( pfc, cbBuf, NULL );
if ( pszBuf == NULL )
{
SetLastError( ERROR_NOT_ENOUGH_MEMORY );
goto Finished;
}
fRet = pPPH->GetHeader( pfc, g_szHeaderName, pszBuf, &cbBuf );
if ( fRet == FALSE )
{
goto Finished;
}
OutputDebugString( "Named header value is: " );
OutputDebugString( pszBuf );
OutputDebugString( "\n" );
}
else
{
goto Finished;
}
}
//
// At this point, pszBuf points to the value of named header
// Just save it and move on.
//
pfc->pFilterContext = pszBuf;
SetLastError( NO_ERROR );
Finished:
if ( GetLastError() != NO_ERROR )
{
OutputDebugString( "Error!\n" );
dwRet = SF_STATUS_REQ_ERROR;
}
return dwRet;
}
DWORD
OnLog(
IN HTTP_FILTER_CONTEXT * pfc,
IN HTTP_FILTER_LOG * pLog
)
{
//
// If the named header was found, set the pszClientHostName
// log field with its value
//
if ( pfc->pFilterContext != NULL )
{
pLog->pszClientHostName = (CHAR *)pfc->pFilterContext;
}
return SF_STATUS_REQ_NEXT_NOTIFICATION;
}
Comments
Anonymous
October 14, 2005
Hey Dadid,
I have a question about how to compile the project. It's been a while since I've worked with VisualStudio and C.
Should I create a Dynamic Link Library or use the ISAPI Extension wizard in Visual Studio 6?
I tried to compile it as a dll and it seems to load properly (No error messages), but it doesn't write anything to the log.
Thanks in advance,
TImoAnonymous
October 15, 2005
The comment has been removedAnonymous
October 21, 2005
OK, I managed to compile the code and figured out how to install and debug it.
It didn't seem to work.
I found a bug (atleast I think it was a bug) in the code, but I got it working anyway.
So, thanks for the code. A great bloggAnonymous
October 21, 2005
Timo - hey, you can't just "claim" there is a bug and not show any evidence... that's not very nice to me nor future readers. I mean, your compilation/linking skills are far more suspect... ;-)
In any case:
No proof = no bug
When I wrote and tested it, it worked perfectly as advertised. :-)
//DavidAnonymous
December 14, 2005
Thanks for the great code.
Boss said "See if you can get this to work". I am the only developer in the group with VS C++ 6.0 experience (over 5 years ago since I last used it) so I am the lucky guy.
Couple of very fundemental questions about this. What sort of project do I start with? Started off with a ISAPI Extension project and only selected Filter. Looked at the code, it looks like you have already written most of the IIS hooks. Should I just copy over all that code or do I just create a blank DLL project copy/paste & compile? Point IIS to the new dll and viola a filter?
Or am I missing something fundemental about this that I should be embarassed about?
thanks for the help,Anonymous
December 19, 2005
The comment has been removedAnonymous
December 24, 2005
David,
Great job!
I was able to get the ISAPI filter to work and it records source IP infromation to the IIS log, however is it possible also to modify REMOTE_ADDR server variable, so source IP values can be used in the ASP or .Net applications.
If I run Request.ServerVariables("REMOTE_ADDR") it still returns load balancer IP address.
Is it ISAPI priority issue? Currently your ISAPI filter is listed on the very bottom in the ISAPI filter list with the priority Unknown
Thanks!Anonymous
December 24, 2005
The comment has been removedAnonymous
January 26, 2006
Hey,
First of all I'd like to thank you for your blog about "ISAPI Filter which Logs original Client IP." I learned a lot about ISAPI Filters and IIS.
I'd also like to apologize for my comment about finding an aledged bug.
But, I can't seem to get the filter to work. It seems to load proparly but it doesn't appear to write anything in the log. When I debug it, the GetHeader seems to return "ERROR_INVALID_INDEX." The header I'm trying to retrieve is "HTTP_IPREMOTEADDR:" I've also tried to retrieve "HEADER_HTTP_IPREMOTEADDR:" but without any luck.
I tried to code it using MFC and creating an ISAPI Extension Wizard. Then it seemed to work fine on a Windows 2000 Server with IIS5 but not on a Windows 2003 Server with IIS6.
I've also tried to modify your code so that OnPreProdHeaders anly returns "SF_STATUS_REQ_NEXT_NOTIFICATION" and use GetServerVariable in "OnLog-function." Even that seems to work on Windows 2000 but again not on Windows 2003.
Any ideas what the problem could be?
Perhaps you could email me your version of the .DLL, so I could try that one? I'm using MS Visual Studio 6 on a Win2k machine.
Thank you in advanceAnonymous
January 26, 2006
Timo - Actually, your problems are due to improper syntax.
I presume the request header you want to retrieve is:
IPRemoteAddr: 1.2.3.4rn
Now, before you make any code changes, please realize that my filter already supports logging from a custom request header. As I mentioned in this blog entry, create LoadBalancedIP.ini in the same directory as the DLL (if you customized the DLL name, you must change the #define MODULE_NAME or else the INI file won't work) with the following contents and after recycling the Application Pool, it should start logging IPRemoteAddr's header value as ClientIP:
[config]
ClientHostName=IPRemoteAddr:
When you use GetHeader(), the syntax to use is the exact header name with trailing colon.
i.e. GetHeader( "IPRemoteAddr:" ).
"HTTP_IPREMOTEADDR:" and "HEADER_HTTP_IPREMOTEADDR:" are incorrect and you have verified that. You cannot simply mix syntax between GetHeader() and GetServerVariable() and hope that it works...
GetHeader() Documentation:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iissdk/html/c54bd6db-16cb-4e8e-b140-e4f937bf97a9.asp
When you use GetServerVariable(), the syntax depends on the prefix you prepend to the header name.
- If you use HTTP_ (i.e. GetServerVariable( "HTTP_IPREMOTEADDR" ), you follow the CGI specification to retrieve header values -- you use the exact header name except "-" are substituted with "" (i.e. If the real header name is My-Header:, you use "HTTP_MY_HEADER"). The flaw in the CGI specification is that you cannot retrieve real header names that include "".
- If you use HEADER_ (i.e. GetServerVariable( "HEADER_IPREMOTEADDR" ), you can retrieve any headername, including those with "-" and "_" (i.e. "HEADER_MY_HEADER" retrieves My_Header: and "HEADER_MY-HEADER" retrieves My-Header:). This syntax was introduced in IIS6. See documentation for explanation.
GetServerVariable() Documentation:
http://msdn.microsoft.com/library/default.asp?url=/library/en-us/iissdk/html/0f8a163e-ef8b-40de-a4a7-219207614c42.asp
The reason that your GetServerVariable() notation worked on IIS5 for "HTTP_IPREMOTEADDR:" is because IIS5 variable name parsing is looser than IIS6. You will succeed on both IIS5 and IIS6 if you pass in syntactically correct names. If you pass in syntactically incorrect names, IIS5 may accept but IIS6 will reject it.
FYI: I do not privately email and certainly cannot email a binary DLL.
//DavidAnonymous
January 27, 2006
Thank you!
I feel a bit stupid for not realizing that you dont need the HTTP_ when using GetHeader().
It works fine!!!Anonymous
January 27, 2006
Hi,
Thanks for the code, I'm new to ISAPI filters and it's great to learn. I have one trouble though and I thought someone could help me out.
In OnPreProcHeaders, after the call of GetHeader, the return is false and I fall in the "else" part of the if. I debugged the values and GetLastError == 0 (NO ERROR). Also, the cbBuf variable is equal to 0 after the call. However, I managed to make it work if I call GetHeader with a pre-sized CHAR* and a buffer size equal to that of my char array. Any idea why it's not working with your way? Thank you very much.Anonymous
January 27, 2006
Alex - What IIS version and can you give me the exact RAW HTTP request you used.
I had no problems running this code on IIS5 and IIS6, so I am interested in the IIS version as well as the request that triggers the condition. The behavior you described, if reproduced, would be a bug in IIS.
//DavidAnonymous
January 30, 2006
IIS 5.0, I got the following from IEWatch
10:02:37.136 188 ms 11445 GET 200 text/html; charset=utf-8 http://localhost/Test/page1.aspx
I activated the trace on-page in my web app to confirm that the header I'm reading exists and contains the good value: it does.
Is this the info you were asking for?
Thanks,Anonymous
January 30, 2006
By the way, I compiled it in Visual Studio .Net 2003 using the configuration you specified for visual studio 2005 express in your other blog post.Anonymous
January 30, 2006
Alex - I need the exact request you made which results in GetHeader() returning FALSE and GetLastError() returning NO_ERROR. It should look something like:
GET /Test/page1.aspx HTTP/1.1rn
Host: localhostrn
X-ClientIP: 1.2.3.4rn
Accept: */*rn
rn
Now, you said that you observed GetHeader() returning FALSE but GetLastError() returning NO_ERROR on IIS5 - which would be considered a bug in IIS. There is nothing wrong with the filter code, so your proposed change is a work-around for your specific issue.
To verify, I would need:
1. the exact request
2. what version of IIS5 you are using
3. What LoadBalancedIP.ini you are using
... because I do not see what you claim on my W2KSP4 IIS5 machine.
I need this information because without it reproduced independently of your machine, the issue basically does not exist and cannot be addressed.
//DavidAnonymous
January 31, 2006
I totally agree that so far the problem looks to be on my side and I'm far from claiming there is a bug in either IIS or your code :)
I am on w2k advanced server, IIS 5, my LoadBalancedIP.ini contains:
[config]
ClientHostName=X-Forwarded-For:
This HTTP header is forwarded by the GIsapi filter (http://www.s0nic.hostinguk.com/wiki/ow.asp?GIsapiFilter) we installed on our ISA server. But since I'm not a hardened c++ developer (I usually work in c# or vb.net in the .net framework), I do not intend to take all your time to fix my problems :)
I'm looking for a tool to get the raw http request.Anonymous
January 31, 2006
The comment has been removedAnonymous
January 31, 2006
Alex - hehe, no worries... I have used IIS5 long enough to know that the bug could be in there somewhere; just need a special request to get it. For example, request headers that do not have terminating rn (hex 0D 0A) have caused problems in the past.
Thanks for telling me that there is a proxy server between the client and IIS... because that means that we need the modified request by GIsapi. Easiest way to get this is to use a Network Monitoring tool like Microsoft Network Monitor or Ethereal to capture incoming traffic right before IIS server gets it. And it's important to post the header termination characters (that was why I put in rn explicitly) because they can affect behavior.
//DavidAnonymous
February 01, 2006
Finally got it, here is the raw GET request
00000030 47 45 54 20 2F 54 65 73 74 49 GET./TestI
00000040 53 41 2F 70 61 67 65 31 2E 61 73 70 78 20 48 54 SA/page1.aspx.HT
00000050 54 50 2F 31 2E 30 0D 0A 56 69 61 3A 20 31 2E 31 TP/1.0..Via:.1.1
00000060 20 4C 44 37 30 33 35 34 35 0D 0A 48 6F 73 74 3A .LD703545..Host:
00000070 20 65 72 69 63 0D 0A 55 73 65 72 2D 41 67 65 6E .eric..User-Agen
00000080 74 3A 20 4D 6F 7A 69 6C 6C 61 2F 34 2E 30 20 28 t:.Mozilla/4.0.(
00000090 63 6F 6D 70 61 74 69 62 6C 65 3B 20 4D 53 49 45 compatible;.MSIE
000000A0 20 36 2E 30 3B 20 57 69 6E 64 6F 77 73 20 4E 54 .6.0;.Windows.NT
000000B0 20 35 2E 30 3B 20 2E 4E 45 54 20 43 4C 52 20 31 .5.0;..NET.CLR.1
000000C0 2E 31 2E 34 33 32 32 3B 20 2E 4E 45 54 20 43 4C .1.4322;..NET.CL
000000D0 52 20 31 2E 30 2E 33 37 30 35 29 0D 0A 43 6F 6F R.1.0.3705)..Coo
000000E0 6B 69 65 3A 20 41 53 50 2E 4E 45 54 5F 53 65 73 kie:.ASP.NET_Ses
000000F0 73 69 6F 6E 49 64 3D 6B 68 72 6D 70 30 6D 61 67 sionId=khrmp0mag
00000100 6D 6E 72 68 6D 35 35 6B 70 67 68 67 6E 35 35 0D mnrhm55kpghgn55.
00000110 0A 41 63 63 65 70 74 3A 20 2A 2F 2A 0D 0A 41 63 .Accept:./..Ac
00000120 63 65 70 74 2D 4C 61 6E 67 75 61 67 65 3A 20 66 cept-Language:.f
00000130 72 2D 63 61 2C 65 6E 2D 75 73 3B 71 3D 30 2E 35 r-ca,en-us;q=0.5
00000140 0D 0A 41 63 63 65 70 74 2D 45 6E 63 6F 64 69 6E ..Accept-Encodin
00000150 67 3A 20 67 7A 69 70 2C 20 64 65 66 6C 61 74 65 g:.gzip,.deflate
00000160 0D 0A 58 2D 46 4F 52 57 41 52 44 45 44 2D 46 4F ..X-FORWARDED-FO
00000170 52 3A 20 31 37 32 2E 32 35 2E 35 32 2E 31 36 35 R:.172.25.52.165
00000180 0D 0A 58 2D 43 45 52 54 43 4F 4F 4B 49 45 3A 20 ..X-CERTCOOKIE:.
00000190 0D 0A 58 2D 43 45 52 54 53 45 52 49 41 4C 4E 55 ..X-CERTSERIALNU
000001A0 4D 42 45 52 3A 20 0D 0A 58 2D 43 45 52 54 53 55 MBER:...X-CERTSU
000001B0 42 4A 45 43 54 3A 20 0D 0A 58 2D 43 45 52 54 49 BJECT:...X-CERTI
000001C0 53 53 55 45 52 3A 20 0D 0A 58 2D 49 53 44 45 42 SSUER:...X-ISDEB
000001D0 55 47 3A 20 46 41 4C 53 45 0D 0A 43 6F 6E 6E 65 UG:.FALSE..Conne
000001E0 63 74 69 6F 6E 3A 20 4B 65 65 70 2D 41 6C 69 76 ction:.Keep-Aliv
000001F0 65 0D 0A 0D 0A e....
I hope it displays ok :)Anonymous
February 01, 2006
Here's a cleaner version :)
GET /TestISA/page1.aspx HTTP/1.0
Via: 1.1 LD703545
Host: eric
User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; .NET CLR 1.1.4322; .NET CLR 1.0.3705)
Cookie: ASP.NET_SessionId=khrmp0magmnrhm55kpghgn55
Accept: /
Accept-Language: fr-ca,en-us;q=0.5
Accept-Encoding: gzip, deflate
X-FORWARDED-FOR: 172.25.52.165
X-CERTCOOKIE:
X-CERTSERIALNUMBER:
X-CERTSUBJECT:
X-CERTISSUER:
X-ISDEBUG: FALSE
Connection: Keep-AliveAnonymous
February 01, 2006
Alex - Thanks for the details. I don't immediately have time but I will be looking at (and around) the repro request that you provided along with the ISAPI Filter and INI file configuration and figure out what is going on.
//DavidAnonymous
February 19, 2006
Hi,
Thanks for the code. I'm new with the IIS application and it seems that when I tried to used your code, I still see the load balancer IP in the IIS logs.
Currently I'm using win 2k with IIS version 5. I like to ask if this will work as well in a Cisco loadbalancer? Also, is there a temp file created where the IIS reads and filters it to the IIS logs. We are just curious how the IIS logs were been created.
Please advise.Anonymous
February 21, 2006
Paul - Honestly, I have no idea whether it works for you. It all depends on how your Cisco loadbalancer works.
The ISAPI Filter illustrates how to modify the Client-Host log field using the value from a customizable request header name. It should be good enough to solve this problem.
I intentionally did not study any popular load balancers to pre-bake a series of request headers such that the ISAPI Filter "automatically" works. Why? Because doing that simply trivializes me into merely providing product support with no relevance to ISAPI when the greater value is for me to illustrate how to provide a solution with ISAPI.
In other words, if I provided such an ISAPI solution, then I would be compelled to provide support to your question of "I tried to use your code but it does not work". Clearly, I cannot do that since there is only one of me. It is far better for me to provide information to help you figure out your own answers... so this is why I provided an ISAPI that can be configured to solve the issue... but you are responsible for figuring out and supporting that ISAPI configuration.
You need to determine whether your Cisco loadbalancer even forwards the original client IP and if so, using what request header name... and then configure the ISAPI Filter accordingly.
//DavidAnonymous
February 21, 2006
The comment has been removedAnonymous
February 21, 2006
Hi,
Thanks for the info. I will try to check the documentation of our load balancer maybe we have not setup it correctly. I have no question or problem with the code. It's a great help for us to understand ISAPI.
Honestly, I don't have any idea the details of our load balancer and I don't even have access to check it.
Thanks.Anonymous
February 21, 2006
Paul - ah... well, I've never worked with Cisco before but I imagine there is a documented admin port available internally that hopefully advertises the hardware's version/etc.
You can also use a network sniffer to pragmatically view the requests being forwarded from the load balancer to see if any particular request header looks like an added Client-IP - and if it is, then configure the ISAPI Filter accordingly.
If you don't see a request header... then time to check the Cisco side of things to see if your version supports it, a firmware upgrade available, etc.
//DavidAnonymous
February 22, 2006
The comment has been removedAnonymous
February 22, 2006
Hi David,
Any new development regarding my issue?
Thanks,
AlexAnonymous
February 22, 2006
Alex - hmm... I took your exact request, compiled this blog entry, and used the same INI file contents... and unfortunately, I am not seeing the issue reproduce for me.
I looked through the hex dump, toggled compression on/off, started adding/removing spaces and other characters (I see the 0D 0A for headers in what you gave, so I don't play with that), removed X-Forwarded-For:, ... but I could not get similar behavior. I just can't think of a way to have GetHeader() return FALSE and GetLastError() = NO_ERROR - which is what causes the filter to abort any actions and not modify the log field.
//DavidAnonymous
March 01, 2006
Hi David,
Could I be compiling against older ISAPI headers in which the GetHeader function behaves differently? Anything I should check to make sure I'm using the right stuff?
Thanks,
AlexAnonymous
March 01, 2006
Alex - "older" ISAPI header issues would be found at compile time, not runtime.
There is no LIB file for ISAPI to link against, so difference in behavior comes from different runtime configuration, including IIS version, request, and IIS configuration.
Since I've tried similar IIS version and request without reproducing it... could you have different list of ISAPI Filters configured - what filters at the global and website level do you have configured and in what order?
//DavidAnonymous
March 16, 2006
Hi David,
I started a new MFC ISAPI Filter in visual studio 2003, got down to code approximatively the same logic as your filter (two calls to GetHeader to call with the exact buffer size, the first call being with a NULL pointer and a zeroed buffer size) and it works perfectly.
I just thought I'd let you know. Thank you for the support though, it's been appreciated :)
AlexAnonymous
April 15, 2006
Question:
I'm trying to write a Filter that handles writing a W3C-compliant log file based on a special...Anonymous
May 05, 2006
hey david..
i m new to isapi and i found this blog quite helpful..
i m trying to make an isapi filter which logs the ip addresses and the urlzz accessed and other details of the users which access my website having extension .CBS..
can u help me in dat..
i m in real need of this coz my project is on hold..
looking fwd to an early response..
TalhaAnonymous
May 05, 2006
talha - if you have a specific question about ISAPI/IIS then I would try to answer it.
Based on your description, I believe that everything you need has already been described by my various blog entries as well as related MSDN documentation, all linked from my blog.
I suggest you do the necessary homework to figure this out.
//DavidAnonymous
May 05, 2006
david i have made a filter which redirects all the request to pages having extension .cbs..now i m having probss in the ONLOG functionzz usage.. :-(
it wud be nice if u cud kindly put the link of the blog entries u mentioned...Anonymous
May 12, 2006
talha - if you have a specific question about ISAPI/IIS or a specific design question, then I can try to answer it.
However, since I have little real information on what you trying to do or having problems with, I have no idea what links you need.
I suggest that you search my blog or MSDN for relevant samples and documentation. It's all written down and publicly available..
//DavidAnonymous
May 27, 2006
i am trying to access a database through ISAPI filter.
i am using the cdatabase thingie and i am not able to conect to teh database and add a row in the database.
hoping to get a replyAnonymous
June 09, 2006
I got sent to this link thanks to google and fastian's comment. I'm trying to find an IIS5 ISAPI-filter code sample to connect to a SQL server database... Can anyone help?Anonymous
June 09, 2006
The comment has been removedAnonymous
June 26, 2006
David, thanks for this example. One thing I've been noticing (and correct me if I'm wrong here...) is that an ISAPI filter that hooks into the SF_NOTIFY_LOG event notification seems to make IIS 6 believe the filter is not "cache friendly" and therefore prevents items from being inserted into the http.sys cache. FilterEnableCache is definitely enabled in the metabase for the filter, but whenever I have it installed I get no "Kernel: URI Cache Hits" increments.
So, I guess my question is what can I do to stick an X-Forwarded-For header in as the Client host without losing the http.sys cache? I'm hoping that we dont have to try to manually insert the items into the cache... Any suggestions? Thanks.Anonymous
June 26, 2006
Mike - Yes, SF_NOTIFY_LOG event is considered "cache unfriendly" and turns off the kernel response cache.
Here is the dilemma:
1. HTTP.SYS writes the log file for a given parsed request
2. ISAPI Filter listening for SF_NOTIFY_LOG forces HTTP.SYS to ask IIS in user mode for any changes to the log entry fields (like Client host)
3. which means it is possible to not record the correct Client host in the cache-hit case - HTTP.SYS cannot run custom code in KERNEL MODE to determine the correct Client-host for such a forwarded request.
Thus, you cannot keep the kernel response cache and modify the log entry.
//DavidAnonymous
July 17, 2006
The comment has been removedAnonymous
July 18, 2006
Visitor - Make sure that F5 Big IP actually forwards the original client IP on SSL requests, and if so, with what request header name.
My ISAPI Filter allows configurable header name so it is easy to read "X-Forwarded-For" - just edit a INI config file as specified in the blog entry.
Regarding your conjectures about how LSASS, ISAPI Filters, and SSL interact - I'll just say up front that it all works logically and correctly; the verification is an exercise best left to the reader.
Now, ISAPI Filter has to be configured to listen and act on SSL traffic via the SF_NOTIFY_SECURE_PORT flag. I did not set it in the sample because the point of the sample was to show how to log original client IP. I presume that interested users can come up with the necessary code change for their situation - you have the source code.
//DavidAnonymous
August 02, 2006
Visitor -
SSL encrypts the whole HTTP request, including headers.
Unless the Big IP itself is doing the encryption (and it can, with the proper add-on module), it will not be able to add a header to the encrypted request. Moving SSL to the Big IP should solve the problem.
I currently do just that to save on certificate licensing costs and it works wonderfully, X-Forwarded-For headers and all.
-NickAnonymous
August 17, 2006
The comment has been removedAnonymous
September 10, 2006
Hello,
I have IIS and ISA installed. On ISA I have turned out the logging for web pulishing and for the filters which are related with it. For web publishing I have also selected "request appear to come from the original client" option under ISA. I have php installed on this system and when I open our web based, ssl e-mail client page, my ip appears in the log files of IIS. As soon as I open any other php or html pages on our server the logfiles contains only the ip address of the server for client ip. Unfortunately, it all happens with or without this filter. Obviously, ISA logs the correct ip addresses of the clients, but for web usage statistics I need to use IIS.
Could you advise me something more to check?
Thanks,
JozsefAnonymous
October 06, 2006
The comment has been removedAnonymous
November 09, 2007
If you are looking to use X-Forwarded-For on ISA Server take a look at http://www.winfrasoft.com/X-Forwarded-For.htmAnonymous
February 05, 2008
Why doesn't IIS support this directly? To what extent has this been tested or completely transparent to applications that might need to leverage it? As a hosting provider, we might have little predictability as to what applications customers might be installing. I'm trying to gauge what extent we'd have to validate each implementation and whether Microsoft would officially support us if we called in an IIS or other product case. Thanks.Anonymous
May 01, 2008
Thanks for great information. We are network imbeciles. We are using haproxy. We read the code where you can have the ClientHostName in the INI or in the code where it attempts to read the ini in GetPrivateProfileString. We modified the X-Client-IP: to X-Forwarded-For: We copied the code you have listed, and compiled it. We gave permissions to everyone on the server. When attempting to hit the web site from an outside the network web connection the page fails to load. We receive an error in application event viewer states: Failed to load dll. The data is the error. Any ideas?Anonymous
May 02, 2008
The comment has been removedAnonymous
May 02, 2008
mark m - Unfortunately, there is no standard mechanism to convey this information, so it is not possible to produce an IIS feature. Furthermore, this problem is really caused by the Load balancer, and I consider IIS good enough that it can be extended to resolve another product's issue. But to have a "feature" just to deal with another product's bug -- quite unreasonable. //DavidAnonymous
August 29, 2008
Does this solution work with 2008/IIS7?Anonymous
September 08, 2008
James - this should work with IIS7 which has the ISAPI Filter Feature support installed. //DavidAnonymous
January 22, 2009
The comment has been removedAnonymous
January 28, 2009
First, thanks for the valuable resource. Any suggestions why adapting this for SM_USER, or HEADER_SM_USER, or HTTP_SM_USER would not be working, given it's compiled and that is loaded into IIS 6 fine? -KyleAnonymous
January 28, 2009
I see the problem, in our environment there is no SM_USER, at least available to WebScarab. Looks like I need to find out if isapi can get at encrypted cookie values...Anonymous
January 29, 2009
Kyle - ISAPI can get at any value on the HTTP request. You will have to unwind any data transformations (like encryption) and formats to obtain the data you want. //DavidAnonymous
January 29, 2009
ken - ISAPI can only change the IP in the log file. It is not possible to change REMOTE_ADDR for the request. Thus, you will not be able to use IIS's IP Restriction behind a load balancer since the load balancer re-issues the request and loses the original client's IP in the TCP packet header. //DavidAnonymous
February 19, 2009
David, I read whole post and don’t understand the idea. If Load Balanced configured properly – it’ll pass real client IP, not SNATed one. There are situations when you need SNAT client IP to work around routing issues, but they are quite rare. I admit, the code is useful to log clients behind with proxy servers (but sure if it legal to do ;-) And not all proxy servers insert client tracking headers ether. The real problem is how NOT to log health check requests from load balancer itself? I.e. we can detect request coming from LB easily, but what to tell IIS to prevent log from littering?Anonymous
March 23, 2009
The comment has been removedAnonymous
March 28, 2009
Someshar - the ISAPI filter works with SSL. When a load balancer acts as endpoint termination for a client's HTTP and SSL connections, it needs to transmit that IP address on the load-balanced request because without it, NO downstream web server will be able to determine nor log that IP address. All traffic will look like it came from the load balancer. Right now, it sounds like the problem is with WFE terminating SSL connections from the client but failing to pass onward that original client IP address in an HTTP header. Since this behavior is not specified in any RFC specification for HTTP, you have no expectation to get that original client IP address. You have to read documentation or contact support personel for WFE to find if it retransmits the original client IP address on the load balanced request and if so, what is the header name. You can then configure this ISAPI Filter to use it, and things will work. But, if WFE does not pass that client IP address onward, it will be impossible to log the original client IP. //DavidAnonymous
March 28, 2009
Apokrif - IIS supports Request Logging configuration down to the URL scope. It is possible to configure IIS to not log requests for the specific URLs used as health check by the load balancer. However, it is not possible to dynamically decide whether a request should be logged or not. If the URL is configured to be logged, it will always be logged; and vice versa. When it comes to log files, any analysis program should have sufficient aggregation and filtering features to make sense of a raw log file. If you expect to open up a raw log file and see exactly what you want, then that expectation is flawed. A web server can receive requests from any client for any URL with any response status. What you should do is use a tool like LogParser.exe to aggregate and filter your IIS log file so that you see what you want. //DavidAnonymous
April 08, 2009
Will this work on IIS running in 64 Bit on 2003 x64? Or is there a different version required?Anonymous
April 20, 2009
Joe - the filter code will work on any IIS version and any bitness. You will need to ensure that you compile the source code with a x64 target, which is possible using the Windows 2003 SDK (it comes with 64bit compilers, linkers, LIB, and header files) as well as Visual C++ Express, though I do not have the step-by-step. It should not be hard to figure it out. //DavidAnonymous
June 29, 2009
Hello David, thanks for this post, is what i really need, i followed all the steps exactly as you mentioned also i followed the tutorial to compile your isapi filters examples, i uploaded the LoadBalance.dll to my win2008 server IIS7, i added to my site, and then i checked the logs, i am still getting the LoadBalancer IP address, do you think is not working because the different server and IIS versions? what could i did wrong? how can i debug it? also i compiled the dll on my personal computer then i uploaded to the web server. do you think this could affect it? Thanks in advance Have a nice day!Anonymous
August 28, 2009
I have done all the necesaory things and its loaded successfully but there is no log in IIS log file "C:WINDOWSsystem32Logfiles" after requesting the aspx page. (IIS version 5.1). Please help...Anonymous
October 21, 2009
I have an isapi filter which registers with SF_NOTIFY_LOG. Before the filter is registered, the blank space URL gets encoded as '+'. After the filter is registered the space gets encoded as '%20'. Any idea why this would chage? I can repro this by just registering the filter and returning SF_STATUS_REQ_NEXT_NOTIFICATION. (IIS 6,W3C extended log)Anonymous
April 15, 2010
This works for ISA server working as web publishing? Thanks.Anonymous
April 28, 2010
The comment has been removed