I am receiving an Error Number 233 (0x800700e9) error on a certain set of pages almost consistently, but I also receive the Err Number 233 at other times elsewhere in the application. Yesterday my production server exhibited this behavior for the first time,
in the past it's only been my development machine that experienced this issue.
The frustrating part is that when this happens I have clue why it has happened. When we inspect the Ruby on Rails log, it reports that the POST action has completed and it had issued a 302 redirect. The fastcgi.crash.log doesn't report anything useful,
only the all too common Unsupported signal USR1.
I tried debugging using the IIS DebugToolkit by using NTSD, but when I get the 233 error, the NTSD command window closes.
I have tried tweaking my fcgiext.ini file to no end. I've also played with the IIS Application Pool settings (Web garden 1, 2, 3, No pinging, etc.), but none of these changes have an effect.
I also wondered how I could keep the RUBY.exe processes active? I thought FastCGI would load the processes once and reuse them. But when I watch task manager, I see the ruby processes ending (disappearing) after some period of time (short and long). When
I watch new requests come in, sometimes a RUBY is reused, but most often I see a new RUBY launching and crunching until it reaches about 50mb of memory, then the page processes and process goes idle.
Here is the screen shot of the dead page:
FastCGI Error
The FastCGI Handler was unable to process the request.
--------------------------------------------------------------------------------
Error Details:
Error Number: 233 (0x800700e9). Error Description: No process is on the other end of the pipe. HTTP Error 500 - Server Error. Internet Information Services (IIS)
Any ideas or thoughts on how to get this configuration to be more stable would be appreciated.
My Virtual directory does have a configuration entry for .rap3 (the ionic rewrite extension) to fastcgi.dll. It is marked script engine (Verify file exists is not checked).
I do have the fastcgi extension set to allowed in IIS Web Server extensions.
This is the last line in the development.log that shows the completion:
Jul 17 08 08:05:14 -- Completed in 3.54700 (0 reqs/sec) | DB: 2.12900 (60%) | 302 Found [http://server/watch/events/4/updates/6]
Here is the fastcgi.crash.log (not helpful at all).
I am having the exact same behavior. I am using Microsoft's own Fastcgi ISAPI module, IIS 6 in Windows 2003, configured as a wildcard filter and no Rewrite module. I am using Apache Bench to stress test it, with just 4 concurrent processes loaded up. IIS loads
them up but after a few hundred requests from ab, some of the processes just die and restarts with this in the fastcgi crash log: Ignoring unsupported signal USR1 I tried to replace ruby's own fcgi.so module with this one: http://avhelp.ucc.usyd.edu.au/james/fcgi-vc6.zip
Still no luck. I tried to tweak fcgiext.ini a lot with no different results either. Anyone got any insights on that?
Same issue here. Processes stop earlier then configured. At some POSTS (especially uploads) and the following redirect the ruby.exe processes die.
Running IIS7 on Vista SP1, FastCGI & URL Rewriter
Suspect problem in fcgi.so or internal ruby.exe. Process exits with exit-code 0.
Now switched to IIS7 with Application Request Routing and 4 mongrels as Windows Service. No troubles from there. So the issue must be in the fcgi.rb part of Ruby itself.
It appears that Rails fcgi does not always wait for an EOF on STDIN. Which is permissible, but introduces a race condition with it sending
FCGI_END_REQUEST
. Specifically, it is possible for it to end and free the request and then receive the zero-length record that FCGIEXT had already sent before noticing the (premature) end.
When this happens, the record needs to be ignored. This is what libfcgi/fcgiapp.c:1724 (from 12/2001!) tries to do. But since the content length is zero, what it really does is skip part of the next
FCGI_BEGIN_REQUEST
. This in turn leads to the
TryAgain
logic in
FCGX_Accept_r
, which closes the named pipe (believing that the other end has stopped and expecting to accept another one). As soon as that happens, FCGIEXT does
TerminateProcess
(and reports the broken pipe to the browser as 233 / 0x800700e9: No process is on the other end of the pipe), so the logs are no help. (With FastCGI 1.5 and
SignalBeforeTerminateSeconds
, the
_FCGI_SHUTDOWN_EVENT_
would get set and
ShutdownRequestThread
would exit from deep inside libfcgi, which is not much better.)
All this is particularly likely with a POST (sends data over STDIN) + redirect (new request immediately follows).
If my diagnosis is correct, the fix appears to be to simply add
Please see the following post on Perl Monks. The fix involves changes to libfcgi. A workaround can be achieved by reading in all of STDIN between each request. http://www.perlmonks.org/?node_id=862506 Btw, thanks for the info in this thread, it helped us eventually
track down the issue.
rvisuvasam, can we have the libfcgi fix? This issue has been driving me insane and it's great to hear someone has figured it out. It is not in the fastcgi.com cvs repository (not yet anyway).
sfigart
1 Post
FastCGI Error Number 233 for Ruby on Rails Configuration
Jul 17, 2008 06:38 PM|LINK
I have a production site (closed internal network) using Microsoft Windows Server 2003, IIS 6, Microsoft FastCGI, Ionic Rewrite, and Ruby on Rails (1.2.3) configured using the instructions at http://en.wikipedia.org/wiki/Installing_Ruby_on_Rails_on_IIS_with_FastCgi.
I am receiving an Error Number 233 (0x800700e9) error on a certain set of pages almost consistently, but I also receive the Err Number 233 at other times elsewhere in the application. Yesterday my production server exhibited this behavior for the first time, in the past it's only been my development machine that experienced this issue.
The frustrating part is that when this happens I have clue why it has happened. When we inspect the Ruby on Rails log, it reports that the POST action has completed and it had issued a 302 redirect. The fastcgi.crash.log doesn't report anything useful, only the all too common Unsupported signal USR1.
I tried debugging using the IIS DebugToolkit by using NTSD, but when I get the 233 error, the NTSD command window closes.
http://blogs.iis.net/rickjames/archive/2007/10/16/fastcgi-debugging-quot-the-fastcgi-process-exited-unexpectedly-quot.aspx
I have tried tweaking my fcgiext.ini file to no end. I've also played with the IIS Application Pool settings (Web garden 1, 2, 3, No pinging, etc.), but none of these changes have an effect.
I also wondered how I could keep the RUBY.exe processes active? I thought FastCGI would load the processes once and reuse them. But when I watch task manager, I see the ruby processes ending (disappearing) after some period of time (short and long). When I watch new requests come in, sometimes a RUBY is reused, but most often I see a new RUBY launching and crunching until it reaches about 50mb of memory, then the page processes and process goes idle.
Here is the screen shot of the dead page:
Here is my fcgiext.ini file
My IONIC Rewrite File
iis ror fastcgi
akitaonrails
1 Post
Re: FastCGI Error Number 233 for Ruby on Rails Configuration
Sep 25, 2008 03:33 PM|LINK
Sourceror
3 Posts
Re: FastCGI Error Number 233 for Ruby on Rails Configuration
Oct 14, 2008 07:51 PM|LINK
Same issue here. Processes stop earlier then configured. At some POSTS (especially uploads) and the following redirect the ruby.exe processes die.
Running IIS7 on Vista SP1, FastCGI & URL Rewriter
Suspect problem in fcgi.so or internal ruby.exe. Process exits with exit-code 0.
Now switched to IIS7 with Application Request Routing and 4 mongrels as Windows Service. No troubles from there. So the issue must be in the fcgi.rb part of Ruby itself.
MMcM
1 Post
Re: FastCGI Error Number 233 for Ruby on Rails Configuration
Nov 28, 2009 02:34 PM|LINK
It appears that Rails fcgi does not always wait for an EOF on STDIN. Which is permissible, but introduces a race condition with it sending
. Specifically, it is possible for it to end and free the request and then receive the zero-length record that FCGIEXT had already sent before noticing the (premature) end.When this happens, the record needs to be ignored. This is what libfcgi/fcgiapp.c:1724 (from 12/2001!) tries to do. But since the content length is zero, what it really does is skip part of the next . This in turn leads to the logic in , which closes the named pipe (believing that the other end has stopped and expecting to accept another one). As soon as that happens, FCGIEXT does (and reports the broken pipe to the browser as 233 / 0x800700e9: No process is on the other end of the pipe), so the logs are no help. (With FastCGI 1.5 and , the would get set and would exit from deep inside libfcgi, which is not much better.)
All this is particularly likely with a POST (sends data over STDIN) + redirect (new request immediately follows).
If my diagnosis is correct, the fix appears to be to simply add
if(data->contentLen > 0) { ... }around thervisuvasam
2 Posts
Re: FastCGI Error Number 233 for Ruby on Rails Configuration
Oct 04, 2010 03:12 PM|LINK
rbrowne
1 Post
Re: FastCGI Error Number 233 for Ruby on Rails Configuration
Oct 05, 2010 06:54 AM|LINK
rvisuvasam, can we have the libfcgi fix? This issue has been driving me insane and it's great to hear someone has figured it out. It is not in the fastcgi.com cvs repository (not yet anyway).