Why some sites return HTTP 404 some don't? [Answered]RSS

4 replies

Last post Jun 14, 2018 11:00 AM by flaneder

  • Why some sites return HTTP 404 some don't?

    Jun 01, 2018 02:43 PM|flaneder|LINK

    I want to add a load balancer to an existing asp.net project using Application Request Routing. So I made myself familiar with the concepts and created a local test-setup:

    • IIS locally running on Windows 10:
      • Installed Application Request Routing 3.0 with Windows Platform Installer
      • Created server farm with following servers:
        1. <test-server-name>.de (Microsoft 2012 R2 Server: contains the asp.net project)
        2. www.google.com (just to see if load balancing and url rewriting works because I don't have two test servers available)

    URL-Rewriting rule: url rewrite rule

    After typing localhost multiple times in any browser, I can see that load balancing (weighted round robin) is working fine. It's alternating between 1. and 2. website.

    The problem I'm facing is a 404 Error on both websites. enter image description here

    I already tried the following:

    • Installing and enabling Failed Request Tracing Rules (on local IIS): URL Rewriting is working properly i think.
      Failed Request Log for www.google.com: google drive, unzip and open xml in e.g. IE for better view

    • Create Server Farm without automatic creation of URL Rewrite rulesenter image description here

      (selecting No and create own URL Rewrite rule)

    • Change "Managed Pipeline Mode"-setting of Applcation Pool from Integrated to Classic

    • Healthcheck on other Websitesenter image description here I have absolutly no clue why it's working on Git-websites and why facebook is returning a 400 error code.

    • Enabling/disabling proxy (IIS-Manager -> Application Request Routing Cache -> Server Proxy Settings...)

    I don't know what i could do next, so I appreciate any help. Thanks.

  • Rovastar Rovastar

    5469 Posts

    MVP

    Moderator

    Re: Why some sites return HTTP 404 some don't?

    Jun 01, 2018 11:56 PM|Rovastar|LINK

    That is not an effective test.

    What you are doing is sending the hostname of your request to the third party servers. Like Google.

    So if your request is say http://mysite.com you are sending this to say www.google.com and the Google servers will likely reject this as you can see

    Web server admins generally don't let themselves receive traffic from domain thet do not host.

    If you sent a request to my servers IP with mysite.com I too would likely reject it. (Things get complex if you have wildcard sites and you allow all traffic through)

    But simply showing that 404 page from Google means tour request hit there server so that implies ARR is working.

    If you really wanted to test it this way have a local host file with www.google.com resolving to your servers IP. Set up a site with www.google.com as the hostheader and then you should see the correct info hitting Google. But there is no accounting for what 3rd party admins do on their side.
    Troubleshoot IIS in style
    https://www.leansentry.com/
  • Re: Why some sites return HTTP 404 some don't?

    Jun 04, 2018 08:51 AM|flaneder|LINK

    Thanks for the fast reply! That's the actual problem i have. I totally misunderstood how ARR works under the hood.

    Instead of following the instruction to your solution (which would probably work) i was able to find a advanced setting when adding a server to the server farm. You can specify a host name.



    Now all health checks for all websites pass (i changed nothing else). But when i enter the same URL which i used in the health check (http://localhost/) in any browser i still get a 404 error.

    Did i misunderstood something again? Or is this just a bug?

  • Re: Why some sites return HTTP 404 some don't?

    Jun 07, 2018 07:14 AM|deepakpanchal10|LINK

    Hi Flender,

    Instead of using Google.com as a server, Try to make a test with the servers on your work place.

    If you just type the URL with localhost in browser to access the page then it will not able to find it because no such page existed.

    This is the reason you get 404 error.

    Regards

    Deepak

    MSDN Community Support
    Please remember to click "Mark as Answer" the responses that resolved your issue.
    If you have any compliments or complaints to MSDN Support, feel free to contact MSDNFSF@microsoft.com.
  • Re: Why some sites return HTTP 404 some don't?

    Jun 14, 2018 11:00 AM|flaneder|LINK

    Hi Deepak,

    thanks for pointing out my issue in simple words.
    But is there no option to map a server to a hostname?

    Example:
    1. A request comes in
    2. The loadbalancer of ARR chooses google.com as next server
    3. ARR saved a host for google.com (which is www.google.com) before and changes host in HTTP message from "localhost" to "www.google.com"
    4. ARR forwards request to google.com server
    5. No 404 Error occurs


    I'm just curious and don't know if this would have any use in a real server setup.