fastcgi berzerking

You may have noticed that Slash7 has been down for a few days. Well, the downtime was (I thought) precipitated by too much load on the server, thanks to my Rails apps. I was notified by my friend, who lets me share his server, that he had to kill my FastCGI processes and shut me down! The load had crept up to a whopping 6. This meant I had to figure out why the load was so high, and my solution to that broke everything… or so I thought.

In the process, I’ve learned more about FastCGI and troubleshooting Rails apps than I ever really cared to. And now, I’m going to force that knowledge on you! Since I’m constrained for time, this one is going to be a short multi-part series at this point, and probably recombined into an article on troubleshooting Rails at a later date (e.g., when those pesky cows get home).

        <span id="more-3609"></span>

Before I go any further—yes, the stylesheet/design has slightly reverted and I am not done with the design. So, hold your kvetchin’ til later.

She Can’t Take Much More, Captain!

So, first to tackle the load issue. With the help of the wonderful folks in freenode’s #rubyonrails IRC channel, I found that my two Typo installs were running in development instead of production. D’oh. Here’s how you can fix that (in your httpd.conf—I’m assuming Apache, here):

Find your IfModule directive:

<IfModule mod_fastcgi.c>
      FastCgiIpcDir /tmp/fcgi_ipc/
      AddHandler fastcgi-script .fcgi .fcg .fpl .rb
    </IfModule>

Add the following:

      FastCgiConfig  -idle-timeout 300 -initial-env RAILS_ENV=production

This adds an idle time-out, which is not a default setting, and sets the RAILS_ENV as you can see. Most Rails apps default to development if you haven’t set the RAILS_ENV elsewhere, and that results in much, much heavier load on the server.

Out of Date, Out of Mind

The other problem is that we were running version 0.8.5 of the FastCGI bindings, which are known to have a memory leak. Now, what I have yet to divine is whether the memory leak is confined to the rubygems version of 0.8.5, or all versions—we weren’t running the gem, we were running the package from Debian’s apt-get. But we upgraded anyway! And that, I thought for some time, is what broke things. (As you’ll find out in the “next installment,” I was incorrect. Sadly.)

If you are running the rubygems FastCGI bindings (called fcgi), then you should definitely uninstall it and install from either your operating system’s package manager—if it has the up-to-date version—or from the source. Unfortunately, I don’t have a link to the source on hand and Google is being remarkably recalcitrant, but you should be looking for the phrase “fcgi bindings” or “libfcgi,” and make sure to get version 0.8.6.

Gem? Wha?

Don’t know if you have fcgi installed as a gem? Open up ye olde trusty command-line interface and type:

gem list

If you see the following:

fcgi (0.8.5)
    FastCGI ruby binding.

You’ll know you have some cleanin’ to do. You can just type sudo gem uninstall fcgi and be on your way (assuming you’re not already root).