We run a large JScript.NET page, which is precompiled to a DLL for ASP.NET 2.0. If the DLL is under cc. 2mb in size, it works as expected. As soon as we add some more code/data, it crashes W3WP on Windows Server 2003 and IIS 6 with the following event:
Error Properties
Type: Error
Source: .NET Runtime 2.0 Error
Category: None
Event ID: 1000
Description:
Faulting application w3wp.exe, version 6.0.3790.3959, stamp 45d6968e, faulting module kernel32.dll, version 5.2.3790.3959, stamp 45d70ad8, debug? 0, fault address 0x0000bee7.
For more information, see Help and Support Center at http://go.microsoft.com/fwlink/events.asp.
I used DebugDiag to generate the following report:
The crash occurs on IIS6 and IIS7. We checked multiple configurations, tested the code locally and on several hosting companies' servers. Again, it seems to be related to the amount of the code size, since there's no problem if we slim down the DLL to under
~2mb in size by removing some code (does not seem to matter which code parts we remove).
The crash never occurs with IIS5.1 or with Visual Studio's ASP.NET Development Server (Cassini).
This is probably not an application error. The application catches exceptions, again it's JScript.NET and it generally appears to be error free. However, debugging the code on IIS6 is impossible (since Visual Studio is also unable to load it on IIS6).
Any help is appreciated...
iis 6.0w3wpwindows server 2003ASP.NET 2.0windows server 2008
My guess it is a application error maybe something from the .net library is not loading correctly and causing the whole process to fail. The error doesn't mean IIS has done anything wrong more than likely the .nET framework is causing it to be mashed up.
I would make sure you servers are fully patched up.
I managed to make VS debugger run in IIS6 (for the record: our ZoneAlarm firewall blocked the debugger, making Windows hang - now the firewall is disabled and the VS debugger works with IIS6 as well, and not just with Cassini).
I set a breakpoint on the very first line of our code, but execution does not even get there. The following happens:
An unhandled exception of type 'System.StackOverflowException' occurred in Microsoft.JScript.dll System.StackOverflowException {Cannot evaluate expression because the current thread is in a stack overflow state.}
This is a large .js file with 27600 lines (total file size: 1.5mb).
In the WS2003 test environment, the only change I make is that in VS, Property Pages, Start Options, Server, I switch from "Use default Web server" (Cassini) to "Use custom server" (IIS6) and I type the path (http://localhost/TestApp). On Cassini it's error
free, on IIS6 there's this exception.
If I don't run the debugger but I request the .aspx (not precompiled) on IIS6 from the browser, the same event 1000 happens. This means that this stack overflow is 99% likely the same problem as the one in my first comment.
As it runs good on IIS5.1 and Cassini but not on IIS6 and IIS7, can this be due to some change (e.g. new IIS setting) introduced with IIS6? (Or, is this a bug or limitation of JScript.Dll?)
Please comment... we're desperate. This is a mission critical application for us.
It works with Cassini the reason is it is working in Debug mode and Cassini is development server through Visual Studio. So when you run like that, the exceptions are going handled by VS Debugger and process won't crash.
In case of IIS 6, it crashes w3wp - the reason is - Application is throwing exceptions which are not handled and those basically needs to be handled in the application code.
Check this KB for more information why w3wp.exe crashes:
Simple resoultion is mentioned in above KB (method 2). This will stop w3wp.exe from crashing but it's not a good idea. Better handle the exceptions in the app code.
In order to find out what those exceptions are you could check event logs or take a Crash dump using DebugDiag tool and analyze the dump.
HTH.
~ Ganesh
Please: Don't forget to click "Mark as Answer" on the post that helped you. That way future readers will know which post solved your issue.
Ganesh, thanks for your post. Indeed we should check whether it's about unhandled exceptions. I set the legacyUnhandledExceptionPolicy to enabled="true", but the problem persists. So in our case, the problem does not seem to be about unhandled exceptions.
Do you have any other ideas? As you can see from my second post, the stack overflow occurs during initialization in
Microsoft.JScript.dll (i.e. not in our code). What is the next step I should try?
Download DebugDiag tool from IIS.net downloads section.
Set up a Crash rule. While setting it up in Advanced Settings>Exceptions>Add Exception and select the 1st one (CLR .NET exception) > Here under .NET exception Type - mention the exact exception. -
'System.StackOverflowException'
Repro the issue and we should get a dump. Once you have dumps, you could analyze it or I may help you with that.
~ Ganesh
Please: Don't forget to click "Mark as Answer" on the post that helped you. That way future readers will know which post solved your issue.
Ganesh, I managed to extract more info from DebugDiag than what I wrote in my first post. I read the exception codes from the DebugDiag log and set up crash rules for them as you suggested.
If I request the .aspx page in a browser:
If I deactivate DebugDiag, this is when event 1000 is thrown.
There are multiple exceptions at kernel32!RaiseException+3c. All but one of them start from
mscorwks!ClrCreateManagedInstance+4d01 (code = 0XE06D7363, i.e. a C++ exception) and there is one exception from mscorwks!GetCompileInfo+1d896 (code = 0XE053534F).
If I start the .aspx page in Visual Studio with F5:
If I deactivate DebugDiag, this is when System.StackOverflowException is displayed.
It creates exceptions 0x80000003 (Breakpoint) and 0xC0000005 (Access Violation) besides the errors mentioned above (multiple 0XE06D7363 + one 0XE053534F).
I don't yet know what's going on, but I will continue debugging.
I'm also trying to create a synthetic test which reproduces this bug.
Attached is a small test that creates a crash:
test code
This compiles error free with aspnet_compiler, works well with Cassini and IIS5.1, and crashes on IIS6 and IIS7.
The crash on IIS6 is a System.StackOverflowException in the JScript.NET precompiled DLL (App_Web_xxxxxx.dll).
Can you guys reproduce the crash?
It needs a virtual directory on IIS6, say /test17 for "c:\Code\test17". Then in Visual Studio, Open Web Site "c:\Code\test17", press F5 and it should display "success" (the default server is Cassini). Now click Property Pages, Start Options, Use custom server,
Url: http://localhost/test17, Ok, and press F5. Now IIS should crash. The crash occurs here 100% of the time (if not precompiled).
Gernet
7 Posts
W3WP Crash
Nov 24, 2008 10:37 PM|LINK
We run a large JScript.NET page, which is precompiled to a DLL for ASP.NET 2.0. If the DLL is under cc. 2mb in size, it works as expected. As soon as we add some more code/data, it crashes W3WP on Windows Server 2003 and IIS 6 with the following event:
I used DebugDiag to generate the following report:
The crash occurs on IIS6 and IIS7. We checked multiple configurations, tested the code locally and on several hosting companies' servers. Again, it seems to be related to the amount of the code size, since there's no problem if we slim down the DLL to under ~2mb in size by removing some code (does not seem to matter which code parts we remove).
The crash never occurs with IIS5.1 or with Visual Studio's ASP.NET Development Server (Cassini).
This is probably not an application error. The application catches exceptions, again it's JScript.NET and it generally appears to be error free. However, debugging the code on IIS6 is impossible (since Visual Studio is also unable to load it on IIS6).
Any help is appreciated...
iis 6.0 w3wp windows server 2003 ASP.NET 2.0 windows server 2008
Rovastar
3321 Posts
MVP
Moderator
Re: W3WP Crash
Nov 24, 2008 11:59 PM|LINK
My guess it is a application error maybe something from the .net library is not loading correctly and causing the whole process to fail. The error doesn't mean IIS has done anything wrong more than likely the .nET framework is causing it to be mashed up.
I would make sure you servers are fully patched up.
and run a debugger (DbgCLR.exe (?))
Gernet
7 Posts
Re: W3WP Crash
Nov 25, 2008 10:39 AM|LINK
Thanks for your comment Rovastar!
I managed to make VS debugger run in IIS6 (for the record: our ZoneAlarm firewall blocked the debugger, making Windows hang - now the firewall is disabled and the VS debugger works with IIS6 as well, and not just with Cassini).
I set a breakpoint on the very first line of our code, but execution does not even get there. The following happens:
An unhandled exception of type 'System.StackOverflowException' occurred in Microsoft.JScript.dll System.StackOverflowException {Cannot evaluate expression because the current thread is in a stack overflow state.}Call stack:
> App_Web_8msifzxq.dll!ASP.server_aspx.__Render__control1(System.Web.UI.HtmlTextWriter __w = {System.Web.UI.HtmlTextWriter}, System.Web.UI.Control parameterContainer = {ASP.server_aspx}) + 0x223 bytes UnknownThis is a large .js file with 27600 lines (total file size: 1.5mb).
In the WS2003 test environment, the only change I make is that in VS, Property Pages, Start Options, Server, I switch from "Use default Web server" (Cassini) to "Use custom server" (IIS6) and I type the path (http://localhost/TestApp). On Cassini it's error free, on IIS6 there's this exception.
If I don't run the debugger but I request the .aspx (not precompiled) on IIS6 from the browser, the same event 1000 happens. This means that this stack overflow is 99% likely the same problem as the one in my first comment.
As it runs good on IIS5.1 and Cassini but not on IIS6 and IIS7, can this be due to some change (e.g. new IIS setting) introduced with IIS6? (Or, is this a bug or limitation of JScript.Dll?)
Please comment... we're desperate. This is a mission critical application for us.
ganeshanekar
551 Posts
Re: W3WP Crash
Nov 25, 2008 12:57 PM|LINK
It works with Cassini the reason is it is working in Debug mode and Cassini is development server through Visual Studio. So when you run like that, the exceptions are going handled by VS Debugger and process won't crash.
In case of IIS 6, it crashes w3wp - the reason is - Application is throwing exceptions which are not handled and those basically needs to be handled in the application code.
Check this KB for more information why w3wp.exe crashes:
911816 Unhandled exceptions cause ASP.NET-based applications to unexpectedly quit in the .NET Framework 2.0
http://support.microsoft.com/default.aspx?scid=kb;EN-US;911816
Simple resoultion is mentioned in above KB (method 2). This will stop w3wp.exe from crashing but it's not a good idea. Better handle the exceptions in the app code.
In order to find out what those exceptions are you could check event logs or take a Crash dump using DebugDiag tool and analyze the dump.
HTH.
~ Ganesh
tanmayathopa...
31 Posts
Re: W3WP Crash
Nov 26, 2008 02:07 AM|LINK
Hi,
Hope this small suggestion on the worker process failure will help you. You will need to debug the code causing its failure,
why application pool/worker process crashes?
Gernet
7 Posts
Re: W3WP Crash
Nov 26, 2008 09:15 AM|LINK
Ganesh, thanks for your post. Indeed we should check whether it's about unhandled exceptions. I set the legacyUnhandledExceptionPolicy to enabled="true", but the problem persists. So in our case, the problem does not seem to be about unhandled exceptions.
Do you have any other ideas? As you can see from my second post, the stack overflow occurs during initialization in
Microsoft.JScript.dll(i.e. not in our code). What is the next step I should try?ganeshanekar
551 Posts
Re: W3WP Crash
Nov 26, 2008 09:51 AM|LINK
Download DebugDiag tool from IIS.net downloads section.
Set up a Crash rule. While setting it up in Advanced Settings>Exceptions>Add Exception and select the 1st one (CLR .NET exception) > Here under .NET exception Type - mention the exact exception. - 'System.StackOverflowException'
Repro the issue and we should get a dump. Once you have dumps, you could analyze it or I may help you with that.
~ Ganesh
Gernet
7 Posts
Re: W3WP Crash
Nov 26, 2008 04:57 PM|LINK
Ganesh, I managed to extract more info from DebugDiag than what I wrote in my first post. I read the exception codes from the DebugDiag log and set up crash rules for them as you suggested.
If I request the .aspx page in a browser:
If I start the .aspx page in Visual Studio with F5:
I don't yet know what's going on, but I will continue debugging.
I'm also trying to create a synthetic test which reproduces this bug.
Meanwhile, I appreciate any suggestions.
ganeshanekar
551 Posts
Re: W3WP Crash
Nov 27, 2008 01:01 PM|LINK
Hi,
Check these blogs which might be at some help from Debugging perspective.
.NET Case Study: Stackoverflow Exception when using a complex rowfilter
http://blogs.msdn.com/tess/archive/2008/03/31/net-case-study-stackoverflow-exception-when-using-a-complex-rowfilter.aspx
ASP.NET Crash: StackOverflowException with Server.Transfer
http://blogs.msdn.com/tess/archive/2008/05/06/asp-net-crash-stackoverflowexception-with-server-transfer.aspx
HTH.
~ Ganesh
Gernet
7 Posts
Re: W3WP Crash
Dec 02, 2008 05:04 PM|LINK
Attached is a small test that creates a crash: test code
This compiles error free with aspnet_compiler, works well with Cassini and IIS5.1, and crashes on IIS6 and IIS7.
The crash on IIS6 is a System.StackOverflowException in the JScript.NET precompiled DLL (App_Web_xxxxxx.dll).
Can you guys reproduce the crash?
It needs a virtual directory on IIS6, say /test17 for "c:\Code\test17". Then in Visual Studio, Open Web Site "c:\Code\test17", press F5 and it should display "success" (the default server is Cassini). Now click Property Pages, Start Options, Use custom server, Url: http://localhost/test17, Ok, and press F5. Now IIS should crash. The crash occurs here 100% of the time (if not precompiled).