IIS 7 and Above
Timeout for some of websites after they stopped for any reason while...
Last post Mar 21, 2021 08:07 PM by PureEnergy
May 18, 2017 10:54 AM|Webio|LINK
I have very weird situation happening from time to time. Scenario: IIS 8.5 with about 1.5k websites. App pool after 20-30 minutes is being terminated. They all work under 32bit. From time to time (once in a month maybe?) I'm experiencing a situation:
As far as I can see there are no errors in Event Log System or Application except entries mentioned above. There is a big chance that most other websites where working just fine because they where not restarted because of being actively used or not crashing
because of application errors. IIS logs just have gaps between moment when application stopped and iisreset command.
Does someone experienced similar issue or have any idea what could cause that problem?
May 18, 2017 05:17 PM|Rovastar|LINK
May 19, 2017 08:40 AM|Webio|LINK
It looks like httperr log contained entries with "Client_Reset" status. For 10466 lines there was 1269 entries with this status. In previous httperr file for 10467 lines there was 835 Client_Reset entries and in previous httperr file 210 entries.
EDIT: Yes application pools runs under separate threads. Resources should be sufficient for handling all websites.
May 19, 2017 09:16 AM|Yuk Ding|LINK
Have you monitored the memory and CPU usage? You could even check whether there is something wrong with Check whether CPU usage throttling action has been enabled in application pool advanced setting. Besides, Please make sure there is no memory leak and
session will never be closed.
This link also provide the difference between recycling and IIS RESET:
Jul 03, 2017 04:21 PM|Webio|LINK
Issue occured again. Take a look at example httperr logs:
2017-07-03 14:10:25 REMOTE IP 29697 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 10047 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 38351 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 38361 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 8612 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 5083 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 10890 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 11012 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 11318 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
2017-07-03 14:10:25 REMOTE IP 11991 LOCAL IP 80 HTTP/1.1 GET / - 503 1419 Disabled LOCAL APP POOL NAME
this just a fraction of httperr log which contained Disabled errors for multiple websites. Not all websites where down only part of them. Interesting thing that even if log states that there was 503 Disabled website status website was running and app pool
was also running (according to IIS Manager). When I was trying to access "Disabled" website browser was loading page indifinetely but w3wp process for this website was not running on Windows Server side (again website and app pool had running status). Trying
to stop and run or recycle website and app pool was not helping. Only solution is to perform iisreset which is fixing issue right away but is restarting all websites which for obvious reasons is not any close to solution.
System hosting websites has been recently updated to Windows Server 2016 (Standard) but this issue occured also on Windows Server 2012.
All app pools are set to recycle on 0:30 and they are not being suspended but Terminated after idle time-out. They are also set to ThrottleUnderLoad
P.S. Website which was first with multiple Disabled errors in httperr was not using almost none of cpu and its memory usage was at about 100-150mb.
Jul 07, 2017 08:46 AM|Yuk Ding|LINK
If IIS specific website return 503. It could probably be caused by the concurrent connection. It depend on how did you set the concurrent connection, queue length in IIS. If the concurrent connection larger than queue length, it could return 503 error. Besides.
if the connection in worker process doesn't get disconnected, it could return 503 as well. You may need to check the application event in event viewer. It could provide the error message. Besides, you also need to debug diagnose tool to analyze the dump file.
Just ensure there is no memory leak. Also if the max thread in thread pool get stuck. The 503 error could occur.
Jul 13, 2017 08:27 AM|Webio|LINK
When you mention concurrent connections you are talking about this params:
<applicationPool maxConcurrentRequestsPerCPU="5000" maxConcurrentThreadsPerCPU="0" requestQueueLimit="5000"/>
from aspnet.config file?
Jul 13, 2017 08:45 AM|Rovastar|LINK
Jul 13, 2017 08:56 AM|Webio|LINK
Ok but why it is not starting? Applications are set as running for website and its dedicated application pool but when trying to reach them browser is loading indefinitely on client side and on server side no w3wp process is being started for this website.
Jul 13, 2017 11:24 AM|Rovastar|LINK
Jul 13, 2017 11:34 AM|Webio|LINK
For now only thing which comes to my mind is some kind of connection limit on some level. When checking current connections in perf mon value is between 550 and 830 so maybe when it is reaching some higher values problem occurs. Issue is impossible to replicate
using any steps and occurs on random time so I will have to wait for another occurence.
Jul 13, 2017 11:42 AM|Rovastar|LINK
Those connection amounts small overall (I regularly see 10,000 connections on a low overhead site that is an API and 5,000 on complex sites)
Maybe it is the spread of the connections over the amount of different sites/SSLs/etc
Jul 13, 2017 11:55 AM|Webio|LINK
This is a shared hosting server. There are some higher usage sites but there are also almost no user sites too. Perfmon counter Web Service\Current Connections _Total shows IMHO proper values (1500/750 current users gives 2 active users in moment of reading
per site which is realistic value). System has 2 quad core CPUs and 96GB of memory. Task manager shows memory and cpu usage mostly at 50-65% level, disk queue length from average 0.1-0.3 so basically everything looks normal as it should.
Mar 21, 2021 08:07 PM|PureEnergy|LINK
Did this issue ever get resolved? We are having exactly the same issue and nothing seems to fix it.