Can requests appear to come from the original client?RSS

36 replies

Last post Nov 27, 2011 09:28 PM by HCamper

  • Can requests appear to come from the original client?

    Feb 02, 2009 05:14 PM|Jinkinz|LINK

    In the proxy settings of a server farm there is the option to preserve the client IP in a custom header.  My question is can requests appear to come from the original client?  The reason I would like this functionality is because we have log parsers that have already been set up to run reports on each server and IP address but the IP addresses for all the requests are the ARR server's IP address, not the client's.  In the CTP1 version of the feature, this was possible.  See the following screenshot: http://images.ddti.net/capture.jpg

    ARR logging

  • Re: Can requests appear to come from the original client?

    Feb 02, 2009 06:22 PM|anilr|LINK

    You would have to run some code in the destination server to replace the information being logged with the client ip in the header you chose in ARR - same as TP1 - we are planning to release a helper module you can install on the destination server to make this possible.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Feb 03, 2009 02:49 PM|Jinkinz|LINK

    anilr

    You would have to run some code in the destination server to replace the information being logged with the client ip in the header you chose in ARR - same as TP1 - we are planning to release a helper module you can install on the destination server to make this possible.

    I've started a new module and I was hoping I could overwrite the Request.UserHostAddress with context.Request.Headers["X-Forwarded-For"] and let the logging module pick it up later in the pipeline.  But it is a read-only property.  How do you suggest I proceed here?  Will your helper module be available soon?  Thanks for your time.

    Kelly

  • Re: Can requests appear to come from the original client?

    Feb 04, 2009 01:44 PM|anilr|LINK

    I don't think it is possible from a managed module - you would have to write a native module.  The helper module will be available soon.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Feb 24, 2009 03:46 PM|Jinkinz|LINK

    Any update on this now that we've got a RTW?  Thanks so much.

  • Re: Can requests appear to come from the original client?

    Feb 25, 2009 07:20 PM|anilr|LINK

    It will be very soon.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Mar 03, 2009 08:25 PM|anilr|LINK

    The helper module is released here.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Mar 03, 2009 10:41 PM|Jinkinz|LINK

    Works great...thanks for your support.

  • Re: Can requests appear to come from the original client?

    Apr 13, 2009 12:04 PM|Rolle|LINK

    Hi,
    The ARR Helper Module is just what I was looking for to replace an old ISAPI Filter.

    While testing the module on our development system (IIS 7) I have come across a problem.
    When using the X-Forwarded-For functionality to capture the client IP from the Loadbalancer all is well as long as the entered value from the Loadbalancer is a valid IP, which can also be split using a Comma.

    When the IP is no longer valid, for Example (192.168.1 or unknown) the ARR Helper Module seems to still try and pass this value to the IIS Core. In this case the IIS Web Core rejects with an Exception (500; Internal Server Error) and therefore the entire Request fails.

    Now I am not using the ARR Helper Module in conjunction with the IIS ARR but as stated in the Post any Loadbalancer can be used and in this scenario the Loadbalancer passes the value “unknown” in the X-Fowarded-For Header when no client IP can be determined.

    Is this a problem which can be fixed in the ARR Helper Module maybe using a RegEx to filter the value or is this filtering not to be supported by the ARR Helper? Or am I just missing something?

    Thanks in advance for any help

  • Re: Can requests appear to come from the original client?

    Apr 13, 2009 02:57 PM|anilr|LINK

    I can look at fixing this issue and re-releasing the helper module - btw, in what case can the load-balancer not determine the client IP address?  The client IP address is part of the tcp handshake and no dns query is needed for that.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Apr 14, 2009 12:35 PM|Rolle|LINK

    Hi,

    I know that the client IP should always be passed and thats why this is quite interesting. Some requests to the Loadbalancer are passed directly to the Backend IIS Servers, this is due to the current rebuild of the system that we are in at the moment. Due to this I started a Trace and found Requests coming from upstream Proxies directly to the Backendservers and setting the X-Forwarded-For Header to uknown due to probably having some sort of acl's set on IP level. In this case the ARR Helper is doing what it is supposed to...

    Therefore an update of the ARR Helper would be of help for such a situation without having to create new sites excluding the Module in the IIS Servers, but strictly spoken this is also not the required functionality of the ARR Helper Module so I suppose an update to include such a filter would be purely good will :-)

     

    Thanks again for the quick reply.

     

  • Re: Can requests appear to come from the original client?

    Apr 15, 2009 04:36 AM|Rolle|LINK

    Another scenario I have been able to reproduce this problem in, is as follows:

    Using a squid proxy for example and disabling the client IP forwarding, the proxy sets the X-Forwarded-For to unknown in the request header. The load balancer then sends the IP of the requesting proxy and if set the value of the X-Forwarded-For in the request header comma separated to the IIS backend server. In this case the ARR Helper extracts the last value after the comma being unknown and sends this to the IIS Web Core which causes an Exception and terminates the entire request.

  • Re: Can requests appear to come from the original client?

    Apr 19, 2009 07:48 PM|ShqTth|LINK

    Sometimes squid likes to make one http get request on behalf of many client requests.

     So X-FORWARDED-FOR contrains the ips of the many clients seperated by ",".

     So I get that a lot when I used F5XForwardedFor.dll. But that doesnt work with Vista.

     Also, it would be nice it your module checked the remote address to verify its from a specific source before using the X-FORWARDED-FOR header.

     

    The reason being, is a client can fake X-FORWARDED-FOR header to fool a script.

    If your module checked the REMOTE_ADDR or REMOTE_HOST to make sure it matched a specific value or if the REMOTE_ADDR is of the same network as LOCAL_ADDR (you can tell if local addr is 192.168.x.x or 127.0.0.1 etc) then HTTP_X-FORWARDED-FOR header would be used and trusted, if its not trusted then only REMORE_ADDR is used.

     the HTTP_VIA indicated that a proxy server is used, but that can be spoofed too. Usually that header contains the name of the pc and port of the proxy server. So the machine name of the proxy has to resolve to the REMOTE_ADDR.

     

    Anyways my IIS server can be accessed with or without a proxy. (proxy runs on a different PC), so it would be good to make sure the IP in X-FORWARDED-FOR is really the IP of the client and not a spoofed ip

  • Re: Can requests appear to come from the original client?

    Apr 20, 2009 02:20 PM|anilr|LINK

    How can you verify that the X-Forwarded-For is not spoofed?  It is never going to be the IP of the client, but an arbitrary IP that connected to the load-balancer.  Any validation of the client should be done by the load-balancer connecting to ARR.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Apr 20, 2009 02:48 PM|ShqTth|LINK

    1)You can check to make sure REMOTE_ADDR matches one of the ip address defined in the config file.

     If there is a match, then you know the request came from the load balancer / proxy srrver in reverse mode meaning X-FORWARDED-FOR can be trusted.

     - This option should be an option to turn off or on.
     - if LOCAL_ADDR matches REMOTE_ADDR then its trusted.
     - network range or netmask would be nice such as y.y.y.x or x.x.x.x/y (example 192.168.1. or 192.168.1.0/24)

     2) Or if REMOTE_ADDR matches 127.0.0.1 or 192.168.X.X or LOCAL_ADDR you pretty much can assume that X-FORWARDED-FOR can be trusted as the request came from a local server.

     

    3) Advanced option (will be slower as it requires a lookup, but result can be cached):
     Or the VIA header cantains the name of the proxy server. oviously the name much resolve to the REMOTE_ADDR to be valid.

    4) REMOTE_HOST can be matched. it is usually the ip addres or name of the proxy server. But information in REMOTE_HOST isn't always reliable. (I never seen it set to an actual host name to date)

    Anyways

    options 1 & 2 seem like a good way to make sure X-FORWARDED-FOR can be trusted or not. but if an option like that is implemented it should be optional as some people wont need it.

  • Re: Can requests appear to come from the original client?

    Apr 20, 2009 02:57 PM|ShqTth|LINK

    When using squid,

    HTTP_VIA=1.1 sheldows-vista:80 (squid/2.7.STABLE4)
    HTTP_X_FORWARDED_FOR=96.48.192.227
    LOCAL_ADDR=127.0.0.1
    REMOTE_ADDR=127.0.0.1
    REMOTE_HOST=127.0.0.1

    HTTP_X_FORWARDED_FOR is the ip address of the client connected to the proxy, or if squid made a request for multiple clients, then sometime X-FORWARDED-FOR may contain multiple ip addresses seperated by "," such as:
    HTTP_X_FORWARDED_FOR = x.x.x.x,y.y.y.y, z.z.z.z

    REMOTE_ADDR will be the ip address of the proxy
    REMOTE_HOST will be either the ip address of the proxy or the name of the proxy machine

  • Re: Can requests appear to come from the original client?

    May 04, 2009 11:27 AM|niik|LINK

    Hey, great module,exactly what we where looking for!

    One question though, is there any way of enabling/disabling the ARRHelper module for specific websites?

    Some of our websites are portmapped directly through our firewall and some go through our ssl-offload/reverse-proxy and I'd like the ARRHelper to only operate on the latter.

    I've tried removing the element from the modules-section of applicationHost.config without any luck. I've also tried adding a element to the applicationHost and the Web.config file, also without any luck. It seems to as if the module gets activated once it has been included in the globalModules section.

    Perhaps you could add an enabled="(true|false") attribute to the configuration schema?

    Thanks!
  • Re: Can requests appear to come from the original client?

    May 04, 2009 12:36 PM|anilr|LINK

    I will look at that - it may be a while (also for the request to ignore invalid X-Forwarded-For headers), I am currently pretty busy with beta2 of ARRv2.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    May 05, 2009 10:37 AM|Mike Ayling|LINK

    I would think that you could add conditions to the rewrite rule that sends traffic to a defined webfarm. For example, if you only want the request rewritten for ARR if a specific host header is specified, then add a condition for {HTTP_HOST} in the rewrite rule.

  • Re: Can requests appear to come from the original client?

    May 05, 2009 01:04 PM|anilr|LINK

    Mike, I think niik is talking about the application server and not the load-balancing server (he is probably not using ARR at all) - he wants the ARRHelper module to only do its work on certain websites on the application server.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    May 13, 2009 08:28 AM|niik|LINK

    @anilr: you're absolutely right, we're not using ARR yet the helper module was simple and effective way of solving our problem! thanks a bunch btw =)
  • Re: Can requests appear to come from the original client?

    Jul 15, 2009 08:53 PM|anilr|LINK

    I have re-released the ARR helper module on my blog with fix for ignoring invalid X-Forwarded-For header.  The feature to only accept X-Forwarded-For headers from trusted proxies is still under consideration.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Jul 20, 2009 11:36 AM|Rolle|LINK

    Great! It's working on our dev area...

    Thanks!

  • Re: Can requests appear to come from the original client?

    Jul 22, 2009 04:12 PM|JohnGalt1|LINK

    This definitely does appear to work great. Thanks much anilr!

    I'll have to throw my hat in the ring for a little more security on it though. Assuming a chain of proxies before the web server, and therefore a chain of X-Forwarded-For addresses in the header, which IP is picked to be the client IP in your module? Is it always the first IP in the chain?

     If you are interested, here's some interesting info on how the problem has been addressed in Apache's mod_extract_forwarded module http://www.openinfo.co.uk/apache/index.html.

     

     

  • Re: Can requests appear to come from the original client?

    Jul 26, 2009 09:41 AM|anilr|LINK

    Thanks, I will give that a look.  Not sure exactly when I will update the ARRHelper module though.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Aug 26, 2009 01:55 AM|Brian Adams|LINK

    Just wanted to second everyone else that says the version of the ARRHelper that lets us provide and update trusted proxies would be very helpful.  The problem I have is that with managed code there does not appear to be a way to get in front of the ARRHelper before it executes in the pipeline.  If I could do that then I could have my own logic that checks for trusted proxy chains and I could blank out the x-forward-for header before the AARHelper even has a chance to process it.

  • Re: Can requests appear to come from the original client?

    Oct 11, 2009 12:44 PM|stever@bitshop.com|LINK

    Have you considered just making this a CodePlex project so these enhancements could be made? I would think some of the people interested in these enhancements could contribute them..

     

  • Re: Can requests appear to come from the original client?

    Nov 05, 2009 06:21 PM|anilr|LINK

    ARR helper has now been updated with the suggestions above.

    Anil Ruia
    Software Design Engineer
    IIS Core Server
  • Re: Can requests appear to come from the original client?

    Oct 13, 2011 01:18 PM|Robert.Colbert|LINK

    Anil,

    My company is running both 32bit and 64bit app pools on IIS 7.5. I tried to install the x86 package on my Win 2008 R2 server but it blocked me because it isn't a 32bit OS.

    It would be helpful if the 64bit installer contained both 32bit & 64bit installers and that it set the proper bitness precondition in IIS on install.

    -Robert 

    AppPool ARR Helper 64-bit 32-bit AppPool 64-bit server

  • Re: Can requests appear to come from the original client?

    Oct 19, 2011 05:21 AM|HCamper|LINK

    Hello Robert,

    You should check both downloads

     http://blogs.iis.net/anilr/archive/2009/03/03/client-ip-not-logged-on-content-server-when-using-arr.aspx

    for the Module:

     intel x86 32-bit http://blogs.iis.net/blogs/anilr/arr_helper_x86.zip

     amd X64 64-bit http://blogs.iis.net/blogs/anilr/arr_helper_x64.zip

    For your problem "I tried to install the x86 package on my Win 2008 R2 server but it blocked me because it isn't a 32bit OS"

    download the amd X64 64bit installer and do the install.

    Then use the configuration steps:

    It will install the module under "%PROGRAMFILES%\IIS\ARR Helper\"  and register the configuration section it uses. 

    It allows configuration of a few parameters in IIS configuration - you can find them in %windir%\system32\inetsrv\config\schema\arr_helper_schema.xml - they should be pretty self descriptive. 

    You can use appcmd/AHAdmin/MWA/Config-Editor etc to edit those configuration parameters.

    For your requests & questions

     "It would be helpful if the 64bit installer contained both 32bit & 64bit installers and that it set the proper bitness precondition in IIS on install.

    Anil Ruia
    Senior Software Design Engineer
    IIS Core Server

    <div class=ForumPostSignature>
    </div> <div class=ForumPostSignature> </div> <div class=ForumPostSignature>Anil left the IIS Team some time in case your waiting on a reply.</div> <div class=ForumPostSignature> </div> <div class=ForumPostSignature>Martin</div> <div class=ForumPostSignature> </div>
    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: Can requests appear to come from the original client?

    Oct 19, 2011 08:29 AM|Robert.Colbert|LINK

    Hi Martin,

    Thank you for the reply. If Anil left the company, are there any plans to release the ARR Helper source on Codeplex? His module did one thing that the one created by F5 Networks did not: It changed the source IP early enough in the pipeline that it appeared to the .NET code running (i.e. SharePoint 2010)

    The F5 code only changes the IP when it is written out to the log files which is good for log based analytics but doesn't help if you need to do anything with the real source IP earlier in the request.

    -Robert 

  • Re: Can requests appear to come from the original client?

    Oct 19, 2011 03:47 PM|HCamper|LINK

    Hi Robert,

    I am not sure of the plans for the ARR Helper Module ?

    I am not sure the source code could be released or where CodePlex or MSDN Archives ? 

    Just was making sure you were able to get the ARR Helper Module installed.

    I suggest you contact the IIS ARR Team  to get the Status.

    I do not use the ARR Helper Module.

    You should include the features items in your contact to the IIS ARR Team.

    Martin

     

    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: Can requests appear to come from the original client?

    Nov 27, 2011 03:15 AM|ShqTth|LINK

    I have a load balancing server that serves requests data from mulple computers. And this module works great.

    But it would be nice if your module supported cascading proxy servers (secondary proxy server trust list) 

    However, I am trying out Cloudflare, it has a certain ip range it uses for requests. The problem is that its requests show up in my log file instead of hte ip's of the people connecting. Cloudflare is a proxy/cloud that helps accelerate my server all over the world, since cloudflare has its own network of computers all over the world. It works great, and is free. Also its uses an RBL to block bad ips such as bad crowlers or form spammers etc.

     

    Ok my problem is that I iuse a load balancing server (proxy) to server up website and to send requests to my proper back end servers, and adding cloudflare makes its like a proxy threw a proxy. So your module works and knows i have a proxy, but it doesn't do the same for cloudflare.

     LOCAL_ADDR
    REMOTE_ADDR=199.27.128.183   <--- this is cloudflares IP. REMOTE_ADDR should be 1xx.1.9x.1xx not 199.27.128.183
    HTTP_X_FORWARDED_FOR=1xx.1.9x.1xx, 199.27.128.183 <- this correct but199.27.128.183  is the secondary proxy that i trust.

    I talked about making a trust list of ip addresses to trust the HTTP_X_FORWARDED_FOR header if REMOTE_ADDR equals one of these ip addresses or REMOTE_ADDR equals LOCAL_ADDR, or LOCAL_ADDR/REMOTE_ADDR equals 127.0.0.1. (or REMOTE_ADDR is on the same network as LOCAL_ADDR ex 192.168.x if LOCAL_ADDR is a class A,B,C private network range) list could be a list of ip addresses or also includes a range.

    Well, if HTTP_X_FORWARDED_FOR header is trusted, then the ip addresses should be scanned from right to left, and if a secondary proxy thats tusted or ip match to the list, then its removed in your process of HTTP_X_FORWARDED_FOR for figuring out what is the clients connecting ip. Normally your module uses the first IP address to the right.

     

    The problem I experience, is becuase I use a dynamic filter to block ip addresses to connect to my website too much to prevent leeching or unfair usage, well cloudflares IP addresses get blocked by mistake. Also my cutom ASPX programs need to know the ip address of the remote client, and the software for my log files needs to have the proper remote ip tracked or else the tracking information in the log files is useless.

     

    All it would take is a for loop that analyzes each ip from right to left of  HTTP_X_FORWARDED_FOR, and stops if no match, and proceeds to the next ip if a match is found. HTTP_X_FORWARDED_FOR should be preserved so logging software knows if a secondary proxy was used, and maybe add the header REMOTE_ADDR_PROXY/REMOTE_ADDR_PROXY(x) etc one for each so its possible to track what proxy servers. REMOTE_ADDR_PROXY for the proimary proxy server that initially set HTTP_X_FORWARDED_FOR, and REMOTE_ADDR_PROXY(x) for each additional secondary proxy server that is trusted.

    It may be nice to know if the client accessed data threw a secondary proxy server.

  • Re: Can requests appear to come from the original client?

    Nov 27, 2011 03:24 AM|ShqTth|LINK

    Just to note, that cloudflare does add headers:

    HTTP_CF_CONNECTING_IP=1xx.1.9x.1xx
    HTTP_CF_IPCOUNTRY=CA

    and also sets HTTP_X_FORWARDED accordingly.

    But that does not help with the issue of the wrong REMOTE_ADDR being tracked. And people may use other secondary proxy servers.

     

     

    So you know, I accept connections threw my primary proxy server, also, I accept connections from websites that I host with cloudflare enabled, or I accept connections from websites that have both my ip address listed and a secondary proxy server listed to help with load balancing.

  • Re: Can requests appear to come from the original client?

    Nov 27, 2011 08:16 AM|HCamper|LINK

    Hi,

    Just in case you missed the note:

    <div class=ForumPostSignature>Anil left the IIS Team some time in case your waiting on a reply.</div> <div class=ForumPostSignature> </div> <div class=ForumPostSignature>Martin</div> <div class=ForumPostSignature> </div>
    Windows and Linux work Together IT-Pros
    Community Member Award 2011
  • Re: Can requests appear to come from the original client?

    Nov 27, 2011 02:21 PM|ShqTth|LINK

    I didnt notice. I just saw he hasn't posted in a long time. Is someone else taking over this module or we out of luck?

  • Re: Can requests appear to come from the original client?

    Nov 27, 2011 09:28 PM|HCamper|LINK

    Hi,

    Last post long time ago. 

    Not sure of status check with the Team  http://forums.iis.net/t/1176889.aspx .

    Martin

     

    Windows and Linux work Together IT-Pros
    Community Member Award 2011