w3wp process at 100% - problem with rewrite?

[Bids Profile]

Hi,

We're seeing a problem on both our (load balanced) production webservers where w3wp.exe will go to 100% on one or more CPUs for about 40-60 seconds at a time. Using the Debug Diagnostics tool we believe that the culprit is ISAPI_Rewrite.

Consistently when the server is pegged, at least one of the threads in the culprit w3wp.exe process will have this stack trace:

Function
ISAPI_RewriteProxy!TerminateExtension+27cf6
ISAPI_RewriteProxy!TerminateExtension+27d65
ISAPI_RewriteProxy!TerminateExtension+525e
winhttp!InternetIndicateStatus+211
winhttp!WinHttpQueryDataAvailable+166
ISAPI_RewriteProxy+1c48
w3core!W3_ISAPI_HANDLER::IsapiOnCompletion+71
w3core!W3_ISAPI_HANDLER::OnCompletion+8d
w3core!W3_HANDLER::MainOnCompletion+52
w3core!W3_CONTEXT::ExecuteHandlerCompletion+23
w3core!W3_MAIN_CONTEXT::DoWork+91
w3core!W3_MAIN_CONTEXT::OnIoCompletion+37
w3dt!UL_NATIVE_REQUEST::DoStateProcess+48
w3dt!UL_NATIVE_REQUEST::DoWork+7f
w3dt!OverlappedCompletionRoutine+1a
w3tp!THREAD_POOL_DATA::ThreadPoolThread+73
w3tp!THREAD_POOL_DATA::ThreadPoolThread+24
w3tp!THREAD_MANAGER::ThreadManagerThread+39
kernel32!BaseThreadStart+34

Has anyone seen this? Any ideas how to fix it?

This crawl is from 3.1.0.76, we get the same problem in 3.1.0.89 though.

This is IIS 6.0 on Windows 2003 Server.

Cheers,

Steve

[Anton]

Hello, Steve

Thanks for your detailed explanation.
But let me ask some more questions:
- how often does the problem occur?
- could you notice any events after which it happens? (e.g. recycling pool, performing backup or other)
- are there any records in Windows Application Event log around the time of failure?
- "w3wp.exe will go to 100% on one or more CPUs for about 40-60 seconds at a time"
does it mean that after going up to 100% for 40-60 seconds it then goes down to normal?