We are excited to announce that the IIS.NET Forums are moving to the new Microsoft Q&A experience. Learn more >

Intermittent 401 and 302 Errors on Static FilesRSS

21 replies

Last post Jun 22, 2012 02:13 PM by jim.b

  • Intermittent 401 and 302 Errors on Static Files

    Jun 06, 2012 06:00 PM|jim.b|LINK

    I'm developing a website for a client of ours and have been having a nagging issue for weeks. The problem is intermittent and might happen twice in a day or once in 2 weeks.

    When it happens, the client will start getting 401 "Unauthorized" errors and corresponding 302 redirects back to the login.aspx page when trying to load static script or style tags on the page. Most of the time it is a .css or .js file that gets denied and setting `<modules runAllManagedModulesForAllRequests="false" />` hasn't fixed the problem. When the problem happens it effectively blocks the file until the application pool is restarted.

    The site is using Forms Authentication with the stock "AspNetSqlMembershipProvider". The site has protected files in an /Accounts folder, styles in /Styles, and scripts in /Scripts. The site has anonymous access enabled and has a separate web.config in the /Accounts folder which deny's access to anonymous users.

    I've looked through the event logs and I see this:
    2012-06-06 14:23:32 [ipaddress] GET /subfolder/Styles/Site.css ver=11 443 - [ipaddress] Mozilla/5.0+(Windows+NT+6.1;+WOW64;+rv:12.0)+Gecko/20100101+Firefox/12.0 https://thesitename.net/subfolder/Account/Login.aspx 401 3 5 1493 429 62

    Sometimes after a short period of time it fixes itself or if I restart the application pool I get this:
    2012-06-06 14:30:30 [ipaddress] GET /subfolder/Styles/Site.css ver=11 443 - [ipaddress] Mozilla/5.0+(Windows+NT+6.1;+WOW64;+rv:12.0)+Gecko/20100101+Firefox/12.0 https://thesitename.net/subfolder/Account/Login.aspx 200 0 0 2533 455 78

    I finally got the hosting provider to enable failed request tracing and finally got a trace of the error:

    failed request tracing log

    It seems that the AnonymouseAuthenticationModule returned an AUTH_SUCCEEDED but then the IsapiFilterModule returned a 401.3. We never got a call on this problem and I only found it by browsing through the failed request tracing logs. It would seem that sometimes this "IsapiFilterModule" deny's access but then some minutes later it lets it through again???

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 06, 2012 10:54 PM|owjeff|LINK

    Are you using forms authentication and an integrated AppPool? You can either switch to a classic mode AppPool or you're on the right track, but I think you need to remove and re-add the modules:

    http://www.codingbeaver.com/myblog/index.php/2012/04/11/asp-net-forms-authentication-blocks-images-and-css-files/

    Jeff Graves

    OrcsWeb: Managed Windows Hosting Solutions
    "Remarkable Service. Remarkable Support."
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 07, 2012 01:17 PM|jim.b|LINK

    Yes it's integrated and yes I'm using Forms Auth. Looking at the article you sent, it seems that you would only remove and then re-add them if you wanted all the files to be run through the modules that you re-add. I want aspx, asmx, etc. to run through them like normal but I don't give a hoot about any other files. And I've already got the <location> sections in there too.

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 07, 2012 02:17 PM|owjeff|LINK

    The <allow users="*" /> in the location tag should be changed to <allow users="?" /> for the sections which you don't want to protect with forms auth.
    Jeff Graves

    OrcsWeb: Managed Windows Hosting Solutions
    "Remarkable Service. Remarkable Support."
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 07, 2012 03:10 PM|jim.b|LINK

     Thanks! I'll give that a try. However, that doesn't explain why suddenly it would require authentication and stay that way until an AppPool restart...

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 13, 2012 12:56 PM|jim.b|LINK

     Unfortunately this didn't help. Any other suggestions?

  • Rovastar Rovastar

    5495 Posts

    MVP

    Moderator

    Re: Intermittent 401 and 302 Errors on Static Files

    Jun 13, 2012 03:21 PM|Rovastar|LINK

    Have a read of this article. It is old but still relevant

    http://blogs.msdn.com/b/david.wang/archive/2005/07/14/howto-diagnose-iis-401-access-denied.aspx

    I suspect that an individual user access persimmons are getting messed up maybe something in the code after multiple times.

    Use procmon to see the file permissions when it is broken to see what is happening

    http://technet.microsoft.com/en-us/sysinternals/bb896645

    Warning: it can get a bit heavy.

    Also run Failed request tracing for the 401s to see what IIS thinks is going on.

    Also a long shot try removing compression for these static files. Strange things can happen at times with compression.

    Troubleshoot IIS in style
    https://www.leansentry.com/
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 13, 2012 03:51 PM|jim.b|LINK

     Unfortunately this is a hosted server and I can't reproduce it anywhere else so I can't run procmon. The next time it happens I'll ask them to run procmon for me and give me the results, if they will...

    I think you can run procmon unattended but the problem is intermittent so it could be logging for days or weeks before it will pop up again.

    I didn't see any way to enable failed request tracing through this hosted provider but I'll ask them, I don't think it's installed...

  • Rovastar Rovastar

    5495 Posts

    MVP

    Moderator

    Re: Intermittent 401 and 302 Errors on Static Files

    Jun 13, 2012 04:08 PM|Rovastar|LINK

    This is a problem with managed hosting providers.

    I presume once it happens it staying broken (for that client at least) therefore it is trappable and you can run these tools.

    For the failed request tracing set it up for 401 errors and have the default directory for them in an area where you have access. That way you should be able to see them

    Troubleshoot IIS in style
    https://www.leansentry.com/
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 14, 2012 01:39 PM|jim.b|LINK

     Okay, they've enabled failed request tracing for me (wasn't easy to accomplish). I'll post back any results I find from it.

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 06:20 PM|jim.b|LINK

    I added more information to the original post. I'm still trying to get the hostmon trace though.

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 06:27 PM|owjeff|LINK

    Are you able to replicate the issue locally?

    Jeff Graves

    OrcsWeb: Managed Windows Hosting Solutions
    "Remarkable Service. Remarkable Support."
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:04 PM|jim.b|LINK

     I've been trying hard but can't. I finally got it to be reproduced this one time in QA on the hosted site but I'll bet I can't do it again. Maybe it's a timing bug where there's an exception at the exact time when IIS is serving the file, then IIS gets shut down at the same time because of the exception and then locks out the URL? I don't have any idea...

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:17 PM|owjeff|LINK

    I re-read your original post and what may be happening is that the application pool is recycling and you're storing session state in proc. Because the worker process is dying, your client is submitting a request using an authenticated cookie but the server's already lost the session. The problem is 3-fold:

    First, unless necessary, you probably don't need to protect the CSS and Javascript content. If you set authorization to allow unathenticated users for that content, you shouldn't see those errors on that content type.

    Second, if you want your user sessions to survive an AppPool recycle, you'll either need to use State Server (provided by your hosting company) or SQL session state. http://msdn.microsoft.com/en-us/library/ms178586.aspx

    Third, you'll want to find out why your AppPool is recycling. I suspect because of the Linq error you provided, it could be that you're retreiving data from a DB and it's storing it in memory and that the worker process is exceeding it's memory limits. If that's the case, you'll either need to increase the memory limits on the AppPool or address the memory leak (assuming it is a memory leak). Having your provider run Debug Diagnostics on the worker process can provide you with a memory dump that you can then analyze to see what's happening when the worker process fails.

    Jeff Graves

    OrcsWeb: Managed Windows Hosting Solutions
    "Remarkable Service. Remarkable Support."
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:18 PM|jim.b|LINK

     I finally got the procmon trace from the hosting provider but I don't think they filtered it right, not sure. I sent hundreds of requests to global.js but in the few minutes they seem to have monitored there's only a couple dozen...

    https://dl.dropbox.com/u/421021/procmon.jpg

     

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:21 PM|jim.b|LINK

     Nope, the session state info is stored in SQL server and anonymous access is enabled to those files. In fact i set <modules runAllManagedModulesForAllRequests="false" /> explicitly so that IIS should bypass all managed modules (including authentication and authorization modules) completely.

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:32 PM|owjeff|LINK

    If that's configuring correctly (and working) you should be able to clear your browser cache and access https://thesitename.com/Styles/site.css and https://thesitename.com/Scripts/global.js without being redirected to the login page. Does that work?
    Jeff Graves

    OrcsWeb: Managed Windows Hosting Solutions
    "Remarkable Service. Remarkable Support."
  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:56 PM|jim.b|LINK

     Unfortunately no, it gets redirected to the login page. It doesn't make any sense!

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 07:58 PM|jim.b|LINK

     I even reproduced the exact same Linq error on my dev machine but can't reproduce this problem.

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 15, 2012 08:13 PM|jim.b|LINK

     Thank you, your post made me realize that I've been chasing my tail for 5 hours! The file is indeed getting processed through the ASP.NET modules and I specifically denied this file as a test...to make sure the denials are getting logged to the database. I guess I forgot about it in QA. So it remains that I have never been able to reproduce this problem and will have to wait until it happens again in production.

    On the bright side at least the hosting company figured out procmon and got failed requests working. The next time it happens I'll be ready. I'll post back here the next time it happens...

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 18, 2012 12:20 PM|jim.b|LINK

     Okay, it happened again this morning and I was able to get the failed request tracing logs. See my modified original post showing the UrlAuthorizationModule failing.

  • Re: Intermittent 401 and 302 Errors on Static Files

    Jun 22, 2012 02:13 PM|jim.b|LINK

     Okay, I FINALLY got a good trace of it denying global.js. However, sometime later it started working again by itself...