PDA

View Full Version : Stop runaway error logging


jenolan
02-16-2007, 01:55 AM
We had a problem with a plugin that everytime it ran would log 43,000 log entries, we ended up with a syslog of over 12 million entries and the system started to feel it :(

So to stop this from happening...

Open /includes/globalfunctions.php
go to the bottom and find;
function ErrorHandler($errno, $message, $filename, $line)replace all of the function up to the closing ?>

function ErrorHandler($errno, $message, $filename, $line)
{
static $ErrorHandlerCount = 0;
if( $ErrorHandlerCount > 100 )
{
Watchdog( 'run-away', "More than 100 error calls made, bailing out", WATCHDOG_ERROR );
die( "System Runaway" );
}
if ($errno & (ERROR_LOG_LEVEL))
{
$ErrorHandlerCount++;
$types = array(1 => 'error', 2 => 'warning', 4 => 'parse error', 8 => 'notice', 16 => 'core error', 32 => 'core warning', 64 => 'compile error', 128 => 'compile warning', 256 => 'user error', 512 => 'user warning', 1024 => 'user notice', 2048 => 'strict warning');
$entry = $types[$errno] .': '. $message .' in '. $filename .' on line '. $line .'.';

Watchdog('php', $entry, ($errno == 1) ? WATCHDOG_ERROR : WATCHDOG_WARNING);
}
}


Changed to use static as per suggestion.

thomas
02-16-2007, 02:01 AM
I think you should post this in subdreamer.com's feature request area. Sounds like a good idea

jessenco
02-16-2007, 03:07 AM
It is to prevent such logging loop
...but that doesn't solve the plugin problem.
It would be nice to know which plugin, what version, etc...

thomas
02-16-2007, 03:12 AM
Can you copy in one of the errors that occured?

Tobias
02-16-2007, 07:22 AM
Sorry, but the above code change can't work as it is - the function has to have as the first line "global $ErrorHandlerCount;" in it.

jenolan
02-16-2007, 09:21 AM
I am positive I made that change earlier .. bugger!

It is there now. So far as which plugin had the problem is was the comic one can't remember the id.

Larry

looked it up p63_Comics

Tobias
02-16-2007, 03:23 PM
Larry: please find an updated Comics frontend file here:
http://www.subdreamer.org/forum/showthread.php?t=2072

And btw, your code change for the SysLog still misses a detail:
you're mixing global and local scopes of the counter variable.
Please change the function to this:

function ErrorHandler($errno, $message, $filename, $line)
{
static $ErrorHandlerCount = 0;
if ($errno & (ERROR_LOG_LEVEL))
{
$ErrorHandlerCount++;
$types = array(1 => 'error', 2 => 'warning', 4 => 'parse error', 8 => 'notice', 16 => 'core error', 32 => 'core warning', 64 => 'compile error', 128 => 'compile warning', 256 => 'user error', 512 => 'user warning', 1024 => 'user notice', 2048 => 'strict warning');
$entry = $types[$errno] .': '. $message .' in '. $filename .' on line '. $line .'.';

if($ErrorHandlerCount >= 100)
{
if($ErrorHandlerCount == 100)
{
Watchdog( 'run-away', "More than 100 error calls made, bailing out", WATCHDOG_ERROR);
}
}
else
Watchdog('php', $entry, ($errno == 1) ? WATCHDOG_ERROR : WATCHDOG_WARNING);
}
}

This will not just abort the script upon the 100th error, but make a final syslog entry and continue the script.

Regards,
Tobias

jenolan
02-16-2007, 10:30 PM
But the run away will continue, as in the comic issue it will call error after error until php times out. If the system has generated 100 errors then I want the script terminated.

Tobias
02-16-2007, 10:49 PM
Does your SysLog contain also "Notices" or only "Warnings" etc.?
I take it you tried the Comic plugin being the only plugin within a category?
The Comic plugin alone itself is to my mind not capable to cause hundreds of SysLog entries per page call.

Personally I wouldn't pursue to terminate a script *if* it generates warnings/notices since that page might be processing important data - I'd rather discontinue the plugin in question till it's improved. ;)
With the ErrorCountHandler you already avoid the SysLog fill-up.

jenolan
02-17-2007, 03:17 AM
The code was a while/read loop that could never end as it was testing a resource that didn't exist. Each time it was called it logged about 43,000 times before the system pulled the plug (php timeout).

I will probably improve it later so that it is smarter .. but if the system is generating 100+ errors per display page there is something seriously wrong.

Tobias
02-17-2007, 07:36 AM
Ah...ok, that file loop.
That was also fixed in the revised version (http://www.subdreamer.org/forum/showthread.php?t=2072) I posted earlier, please have a try. :)