Re: Debugging Pool Crash

Mike426 — Fri, 26 Jun 2009 21:11:48 GMT

Can you add some logging to your ASP application code so you can see what it was doing right before the crash? You would want to include the local process id in the logging statements so you can match it up to the message in the event viewer log.

Re: Debugging Pool Crash

murtaza_t — Tue, 23 Jun 2009 18:31:17 GMT

I am not sure if you have sorted this problem however if not, then can you paste the Warning message in Event Viewer > System logs, that you get before the pool crashes.

Re: Debugging Pool Crash

lextm — Fri, 19 Jun 2009 04:29:32 GMT

About this question I think you need to make a decision. Do you plan to manually analyze the dump alone or you would like an expert to help besides?

Debug Diagnostics is a free utility which has technical limitations on dump analysis, as not everything comes automatically. So for most cases, manual analysis with Visual Studio or WinDbg is required.

Re: Debugging Pool Crash

tyes — Fri, 19 Jun 2009 03:36:05 GMT

lextm:

If you need asistance on dump analysis, you can open a case via http://support.microsoft.com

Hi Lex - I am guessing I will have to pay for this?

Cheers

tyes

Re: Debugging Pool Crash

lextm — Fri, 19 Jun 2009 02:57:16 GMT

If you need asistance on dump analysis, you can open a case via http://support.microsoft.com

Debugging Pool Crash

tyes — Fri, 19 Jun 2009 02:52:47 GMT

Hi. I have classic ASP application which seems to work happily on a number of implementations (All Win 2K3). On one server however the application pool crashes randomly (up to 10 times per day) and therefore forces session resets for the users. The event log error reported is always:

Source: W3SVC
Type: Warning
Event ID: 1011
Description: "A process serving application pool 'myPoolName' suffered a fatal communication error with the World Wide Web Publishing Service. The process id was '6896'"

I have implemented debug watches using the debug diagnostic tool and it has captured this crash now several times. Analysing the dump data doesn't reveal anything obviously to me (not that I know what I am looking for).

Here is the exception summary for the last few crashes:

19/6/09 11:17:

eax=4a77ba08 ebx=00080000 ecx=00000008 edx=0c830003 esi=4a77ba58 edi=4a77ba50
eip=7c82a0d0 esp=0340fc54 ebp=0340fe70 iopl=0         nv up ei pl nz ac po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00010212
ntdll!RtlAllocateHeap+0xfa:
7c82a0d0 884706          mov     byte ptr [edi+6],al        ds:0023:4a77ba56=90

19/6/09 10:41

eax=78000010 ebx=00020001 ecx=00020001 edx=7c8285ec esi=000d70c8 edi=000e7560
eip=4a77c5ec esp=0421fef4 ebp=0421ff04 iopl=0         nv up ei ng nz ac pe cy
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00010297
comsvcs!DllUnregisterServer+0xa7a:
4a77c5ec ff5028          call    dword ptr [eax+28h] ds:0023:78000038=????????

19/6/09 10:14

eax=002e0031 ebx=0728fd24 ecx=0728fd24 edx=7c8285ec esi=0728fca0 edi=07286800
eip=4a77c5ec esp=00d8fef4 ebp=00d8ff04 iopl=0         nv up ei ng nz ac pe cy
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00010297
comsvcs!DllUnregisterServer+0xa7a:
4a77c5ec ff5028          call    dword ptr [eax+28h] ds:0023:002e0059=????????

18/6/09 2:05pm

eax=4a77ba0c ebx=00080000 ecx=00000004 edx=399b0005 esi=4a77ba58 edi=4a77ba50
eip=7c82a0d0 esp=0340fc54 ebp=0340fe70 iopl=0         nv up ei pl nz ac pe nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00010216
ntdll!RtlAllocateHeap+0xfa:
7c82a0d0 884706          mov     byte ptr [edi+6],al        ds:0023:4a77ba56=90

17/6/09 10:00am

eax=7771f908 ebx=00080000 ecx=00000008 edx=25f20006 esi=7771f9f1 edi=7771f9e9
eip=7c82a0d0 esp=0340fc54 ebp=0340fe70 iopl=0         nv up ei pl nz ac po nc
cs=001b ss=0023 ds=0023 es=0023 fs=003b gs=0000             efl=00010212
ntdll!RtlAllocateHeap+0xfa:
7c82a0d0 884706          mov     byte ptr [edi+6],al        ds:0023:7771f9ef=90

Here are the details from the last full crash using the IIS crash anaylsys tool:

Report for w3wpPID6476Date06_19_2009__Time_11_17_29AM687Second_Chance_Exception_C0000005.dmp

Type of Analysis Performed	Crash Analysis
Machine Name	[ServerName]
Operating System	Windows Server 2003 Service Pack 2
Number Of Processors	8
Process ID	6476
Process Image	c:\WINDOWS\system32\inetsrv\w3wp.exe
System Up-Time	1 day(s) 13:14:48
Process Up-Time	00:36:02

Thread 22 - System ID 10176

Entry point	msvcrt!_endthreadex+2f
Create time	6/19/2009 10:41:27 AM
Time spent in user mode	0 Days 0:0:0.0
Time spent in kernel mode	0 Days 0:0:0.0

Function	Arg 1	Arg 2	Arg 3
ntdll!RtlAllocateHeap+1f5	00080000	00000000	00000008
ole32!CRetailMalloc_Alloc+16	77796784	00000008	0340fec4
ole32!CoTaskMemAlloc+13	00000008	00000001	7c8276cb
comsvcs!com_ff_allocator<CWaitFreeQueueNode<CSTAWork > >::allocate+16	00000002	00000000	0340ff58
comsvcs!std::vector<IWorkQueue ,com_ff_allocator<IWorkQueue > >::insert+51	00103dc4	00000001	0340ff30
comsvcs!std::vector<IWorkQueue ,com_ff_allocator<IWorkQueue > >::insert+1b	00103dc4	0340ff30	00000000
comsvcs!std::priority_queue<CSTAThread ,std::vector<CSTAThread ,com_ff_allocator<CSTAThread > >,CSTAPoolStatistics::indirect_greater<CSTAThread > >::push+13	0340ff30	4a8098d0	0340ff78
comsvcs!CSTAPoolStatistics::operator+=+34	000de3b0	02183c98	00000000
comsvcs!CSTAThreadPool::BalanceLoad+5e	02183dc8	0340ffb8	77bcb530
comsvcs!CSTAThreadPool::LoadBalanceThreadControlLoop+34	4a8098d0	00000000	00000000
msvcrt!_endthreadex+a3	02183c98	00000000	00000000
kernel32!BaseThreadStart+34	77bcb4bc	02183c98	00000000

NTDLL!RTLALLOCATEHEAP+1F5

Detailed Info For Corrupt Heap

Heap 1 - 0x00080000

Heap Name	Default process heap
Heap Description	This heap is created by default and shared by all modules in the process
Reserved memory	1,024.00 KBytes
Committed memory	872.00 KBytes (85.16% of reserved)
Uncommitted memory	152.00 KBytes (14.84% of reserved)
Number of heap segments	1 segments
Number of uncommitted ranges	2 range(s)
Size of largest uncommitted range	148.00 KBytes
Calculated heap fragmentation	2.63%

Segment Information

Base Address	Reserved Size	Committed Size	Uncommitted Size	Number of uncommitted ranges	Largest uncommitted block	Calculated heap fragmentation
0x00080640	1,024.00 KBytes	872.00 KBytes	152.00 KBytes	2	148.00 KBytes	2.63%

Top 5 allocations by size

Allocation Size - 6260

Allocation Size - 32

Allocation Size - 520

Allocation Size - 20000

Allocation Size - 4168

79.47 KBytes

58.94 KBytes

27.93 KBytes

19.53 KBytes

16.28 KBytes

Top 5 allocations by count

Allocation Size - 32

Allocation Size - 8

Allocation Size - 80

Allocation Size - 64

Allocation Size - 16

1886 allocation(s)

145 allocation(s)

122 allocation(s)

120 allocation(s)

118 allocation(s)

Back to Top

In w3wp__PID__6476__Date__06_19_2009__Time_11_17_29AM__687__Second_Chance_Exception_C0000005.dmp the assembly instruction at ntdll!RtlAllocateHeap+1f5 in C:\WINDOWS\system32\ntdll.dll from Microsoft Corporation has caused an access violation exception (0xC0000005) when trying to write to memory location 0x4a77ba56 on thread 22

Heap corruption was detected in heap 0x00080000, however pageheap was not enabled in this dump. Please follow the instructions in the recommendation section for troubleshooting heap corruption issues.

Current NTGlobalFlags value: 0x0

Module Information
Image Name:	C:\WINDOWS\system32\ntdll.dll	Symbol Type:	PDB
Base address:	0x7c800000	Time Stamp:	Sun Feb 18 00:02:00 2007
Checksum:	0x000bd6f9	Comments:
COM DLL:	False	Company Name:	Microsoft Corporation
ISAPIExtension:	False	File Description:	NT Layer DLL
ISAPIFilter:	False	File Version:	5.2.3790.3959 (srv03_sp2_rtm.070216-1710)
Managed DLL:	False	Internal Name:	ntdll.dll
VB DLL:	False	Legal Copyright:	© Microsoft Corporation. All rights reserved.
Loaded Image Name:	ntdll.dll	Legal Trademarks:
Mapped Image Name:		Original filename:	ntdll.dll
Module name:	ntdll	Private Build:
Single Threaded:	False	Product Name:	Microsoft® Windows® Operating System
Module Size:	768.00 KBytes	Product Version:	5.2.3790.3959
Symbol File Name:	c:\symcache\ntdll.pdb\93E72E109DC84F16AA54797E4DA8C1682\ntdll.pdb	Special Build:	&

Any thoughts or suggestions?

Classic ASP

Re: Debugging Pool Crash

Re: Debugging Pool Crash

Re: Debugging Pool Crash

Re: Debugging Pool Crash

Re: Debugging Pool Crash

Debugging Pool Crash

Report for w3wp__PID__6476__Date__06_19_2009__Time_11_17_29AM__687__Second_Chance_Exception_C0000005.dmp

Thread 22 - System ID 10176

Detailed Info For Corrupt Heap

Heap 1 - 0x00080000

Segment Information

Top 5 allocations by size

Top 5 allocations by count

Module Information

Report for w3wpPID6476Date06_19_2009__Time_11_17_29AM687Second_Chance_Exception_C0000005.dmp