Ra - filter syntax error

[elof2 at sentor.se]

Apropos this old thread.

If ra 3.0.8 still take too long to compile the filter, the logged message 
should be cropped.
Currently, my *long* filter is more than 992 characters, and apparently 
that is some limit. Either in ra or in syslog, because the syslog message
looks like this:

"2015-01-22 15:54:33 +01:00 foobar ra[36131]: 1421938473.058216 src 
net (10.0.0.0/8 or ... or ... ... ..  . . . . . . . . . and not host 
10.123.123.100 and not host 10.123.123.123 and not host"

Where the message is 992 characters long.

In reality it should continue with more "and not host" and after that 
probably give some error-message about the filer compilation taking too 
long. However, this information is not shown due to the length of my 
*long* filterstring.
So, either crop the filterstring, or put the error-message in front of the 
filterstring so that people understand why ra is complaining.

/Elof


On Mon, 20 May 2013, Carter Bullard wrote:

> Hey Elof,
> Great, and thanks for the info.  While the clients will be blocked waiting for the filter to be compiled, because this is structured as a deadman timer, if the compiler returns quickly, there won't be any delay.
>
> For remote accesses, we don't connect until we've compiled the filter.  The logic is if we can't compile the filter, the remote won't be able to either...so a long time wait for the local compiler won't over run us with data.
>
> I'll make it 1sec in the code base until I hear otherwise.
> Carter
>
>
> On May 20, 2013, at 7:41 AM, elof2 at sentor.se wrote:
>
>>
>> Carter and I have discussed a very unusual error message; "filter syntax error", which may show up if the machine is HEAVILY burdened, like swapping a lot and/or receiving tons of interrupts, while feeding a long and complex filter expression to a ra* process.
>>
>>> I haven't seen this problem before, but there maybe a simple fix.
>>> In order to get around a problem with using flex and bison to compile
>>> multiple filters at a time, we fork a process to compile a filter, and wait
>>> for that process to write the filter back to argus through a pipe.  We
>>> have specific error messages for the pipe creation, but we use a
>>> generic syntax error message if the forked process doesn't
>>> return a binary filter back to us in a timeout period, which is now set
>>> at 200 milliseconds.
>>
>> I suggest that Carter add a specific error message for this particular scenario, logging the message "filter compilation timeout" instead of the generic "filter syntax error".
>>
>>> In the argus-clients file ./common/argus_code.c, in the routine ArgusFilterCompile(),
>>> on line 389, we set the timer value.  Increase the time to, what, a second ?
>>> Here is a patch for the 3.0.6.2 code…..
>>> thoth:common carter$ diff argus_code.c.orig argus_code.c
>>> 389,390c389,390
>>> <       wait.tv_sec  = 0;
>>> <       wait.tv_usec = 200000;
>>> ---
>>>>       wait.tv_sec  = 1;
>>>>       wait.tv_usec = 0;
>>
>>
>> During the weekend I've had cron start ra with a long and complex filter string every 5 minutes.
>> With a timeout of 500ms, I had 9 "filter syntax error" in 34 hours.
>> (on an old Intel Xeon 3050 @ 2.13GHz machine from 2008)
>> (with 200ms I suspect I would have had approx 300 warnings)
>>
>>
>> So to prevent this message to appear "too often" on choked machines, you could increase the wait.tv_usec to 750000  ...or even higher if a higher value doesn't introduce any overall negative impact on ra*.
>>
>>
>>
>> I now upped the timeout to 900000 just to see if that is enough to quell even those 9 warnings.
>>
>> /Elof
>