We are excited to announce that the IIS.NET Forums are moving to the new Microsoft Q&A experience. Learn more >

Logging issue with X-Forwarded-For, wrong locationRSS

11 replies

Last post Jun 20, 2021 03:37 PM by lextm

‹ Previous Thread|Next Thread ›
  • Logging issue with X-Forwarded-For, wrong location

    May 11, 2021 02:58 PM|IISUserLondon|LINK

    I have a really strange issue with logging the x-forwarded-for header in IIS logs on a few particular servers.

    The log looks like this:

    #Software: Microsoft Internet Information Services 8.5
    #Version: 1.0
    #Date: 2021-05-04 00:00:12
    #Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken
    2021-04-20 11:24:51 2000000a3+X-Forwarded-For+-+ serverxxx x.x.x.x GET /path/path/paths.png - 443 - x.x.x.x HTTP/1.1 Mozilla/5.0+(Windows+NT+6.2;+WOW64)+AppleWebKit/538.1+(KHTML,+like+Gecko)+PhantomJS/2.1.1+Safari/538.1 ASPSESSIONIDSXXXXX=XXXXXX https://website.company.net/path/path/paths.asp sitename.company.net 200 0 0 51636 465 45
    

    As you can see, the x-forwarded-for info looks all kinds of weird and it appears after the date/time. I have 5 x Windows Server 2012 R2 IIS servers which all have the problem. I have another ~50 servers which are absolutely fine. On the 50 servers that are OK, the x-forwarded-for field appears in the #fields of the log and correctly as the last entry of a line.

    I have the logging field setup uniformly across all the servers on the IIS root as it was done via a script.

    The IIS log file name doesn't get the _x at the end of the filename like the other servers do.

    I can see in the applicationHost.config we have the following, which is exactly the same between a working server and a non working server. There are no other entries in there for <LogFile>

                <siteDefaults>
                    <logFile logExtFileFlags="Date, Time, ClientIP, UserName, SiteName, ComputerName, ServerIP, Method, UriStem, UriQuery, HttpStatus, Win32Status, BytesSent, BytesRecv, TimeTaken, ServerPort, UserAgent, Cookie, Referer, ProtocolVersion, Host, HttpSubStatus" logFormat="W3C" directory="%SystemDrive%\inetpub\logs\LogFiles">
                        <customFields>
                            <clear />
                            <add logFieldName="X-Forwarded-For" sourceName="X-Forwarded-For" sourceType="RequestHeader" />
                        </customFields>
                    </logFile>
                    <traceFailedRequestsLogging directory="%SystemDrive%\inetpub\logs\FailedReqLogFiles" />
                </siteDefaults>

    Does anyone have any idea why this is happening?

  • Re: Logging issue with X-Forwarded-For, wrong location

    May 12, 2021 08:57 AM|samwu|LINK

    Hi IISUserLondon,

    IISUserLondon

    As you can see, the x-forwarded-for info looks all kinds of weird and it appears after the date/time. I have 5 x Windows Server 2012 R2 IIS servers which all have the problem. I have another ~50 servers which are absolutely fine. On the 50 servers that are OK, the x-forwarded-for field appears in the #fields of the log and correctly as the last entry of a line.

    Have you checked whether the corresponding log file in logFiels is correct? or you can try to reset iis and try again.

    Right-click on cmd.exe and select run as administrator, at the command prompt, type IISRESET, press Enter.

    Best regards,

    Sam

    IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
  • Re: Logging issue with X-Forwarded-For, wrong location

    May 12, 2021 03:34 PM|IISUserLondon|LINK

    samwu

    Have you checked whether the corresponding log file in logFiels is correct? or you can try to reset iis and try again.
     

    Unfortunately I'm not quite sure what you mean.

    A working server has the following headers in the log and the log file itself has _x appended to the filename.

    #Software: Microsoft Internet Information Services 8.5
    #Version: 1.0
    #Date: 2021-04-25 06:36:35
    #Fields: date time s-sitename s-computername s-ip cs-method cs-uri-stem cs-uri-query s-port cs-username c-ip cs-version cs(User-Agent) cs(Cookie) cs(Referer) cs-host sc-status sc-substatus sc-win32-status sc-bytes cs-bytes time-taken X-Forwarded-For

    The broken servers do not have the _x appended to the log filename, nor do they have "X-Forwarded-For" in the log header field (#fields)

    Re the iis restart - we've restarted these server many times since the log fields were added to IIS.

  • Re: Logging issue with X-Forwarded-For, wrong location

    May 13, 2021 07:59 AM|samwu|LINK

    Hi IISUserLondon,

    IISUserLondon

    The broken servers do not have the _x appended to the log filename, nor do they have "X-Forwarded-For" in the log header field (#fields)

    I tested X-Forwarded-For in my iis logs and it works normally. Please check your logFieldName and source spelling for errors.

    Or you can also try to add X-Forwarded-For header  by PowerShell command at the server level.

    Add-WebConfigurationProperty -pspath 'MACHINE/WEBROOT/APPHOST'  -filter "system.applicationHost/sites/siteDefaults/logFile/customFields" -name "." -value @{logFieldName='X-Forwarded-For';sourceName='X-Forwarded-For';sourceType='RequestHeader'}

    Best regards,

    Sam

    IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
  • Re: Logging issue with X-Forwarded-For, wrong location

    May 13, 2021 08:29 AM|IISUserLondon|LINK

    There are definitely no spelling errors and that's the exact piece of PowerShell I used to set up all ~50 servers, including the ones with the issue.

  • Re: Logging issue with X-Forwarded-For, wrong location

    May 13, 2021 09:51 AM|samwu|LINK

    Hi IISUserLondon,

    IISUserLondon

    There are definitely no spelling errors and that's the exact piece of PowerShell I used to set up all ~50 servers, including the ones with the issue.

    Have you tried using PowerShell commands in broken servers? Have you compared whether the configuration of this part of the web.config file between the two servers is the same?

    Best regards,

    Sam

    IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
  • Re: Logging issue with X-Forwarded-For, wrong location

    May 13, 2021 11:29 AM|IISUserLondon|LINK

    Yes, I used the powershell command to setup the broken servers.

    I've compared the applicationHost.config (as seen in the first post), which has identical logFile settings - I've compared with Notepad++. The section is identical.

  • Re: Logging issue with X-Forwarded-For, wrong location

    May 14, 2021 08:54 AM|samwu|LINK

    Hi IISUserLondon,

    IISUserLondon

    I've compared the applicationHost.config (as seen in the first post), which has identical logFile settings - I've compared with Notepad++. The section is identical.

    Is your log folder correct? The number after W3SVC corresponds to your website id.

    Best regards,

    Sam

    IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
  • Re: Logging issue with X-Forwarded-For, wrong location

    May 14, 2021 09:51 AM|IISUserLondon|LINK

    Yes the folders are correctly named as you describe. They are auto created by IIS itself.

    Thanks

  • Re: Logging issue with X-Forwarded-For, wrong location

    May 18, 2021 08:15 AM|samwu|LINK

    Hi IISUserLondon,

    IISUserLondon

    Yes the folders are correctly named as you describe. They are auto created by IIS itself.

    Have you tried other custom fields in broken servers?

    Best regards,

    Sam

    IIS.NET forums are moving to a new home on Microsoft Q&A, we encourage you to go to Microsoft Q&A for .NET for posting new questions and get involved today. Learn more >
  • Re: Logging issue with X-Forwarded-For, wrong location

    Jun 20, 2021 03:00 PM|IISUserLondon|LINK

    OK I've discovered something interesting.

    If I set any custom field in the root server level log setting. I get the same problem. I removed the x-forwarded-for header and entered a test header:

    https://ibb.co/qYw2Fch

    iis

    The log has the same problem:

    2021-06-20 14:10:50 2000000cc+X-Forwarded-For+-+Test-Header+-+ v152 x.x.x.x POST /x/x/x/x - 443 - x.x.x.x HTTP/1.1 - - - xxxxxxx 200 0 0 586 583 3198

    But if I put the header into the log setting but at site level the header works correctly and gets entered into the log and the _x gets appended to the file name.

    Unfortunately the servers I have this problem on have around 300-600 sites on each so setting the log on a site level basis would be crazy!

  • Re: Logging issue with X-Forwarded-For, wrong location

    Jun 20, 2021 03:37 PM|lextm|LINK

    IISUserLondon

    Unfortunately the servers I have this problem on have around 300-600 sites on each so setting the log on a site level basis would be crazy!

    If you utilize a little PowerShell, then that's not a crazy task.

    Lex Li
    Want to have a chat on the issues you meet? Book an appointment at https://buy.stripe.com/cN24ia0yi7sAdIA7sv
    ---------------------------
    This posting is provided "AS IS" with no warranties, and confers no rights.
‹ Previous Thread|Next Thread ›