Ra - filter syntax error

[elof2 at sentor.se]

Hi Carter!

Sure, a configurable timeout sounds like the most professional solution.
A ra.conf option is enough in my opinion. No need to also add a new 
commandline option to 'ra'.

Perhaps the errormessage should also hint about this ra.conf option.

/Elof


On Mon, 1 Jun 2015, Carter Bullard wrote:

> Hey /Elof,
> There are 2 parts to this ... Length of compiler timeout and error message format.
> Do we want a configurable timer ???
> Carter
>
>
>> On Jan 26, 2015, at 5:22 AM, elof2 at sentor.se wrote:
>>
>>
>> Actually, my error is not a syntax error - I use the same filter in a cronjob every 5 minutes. Only sometimes, like once in three weeks, I get the error below.
>>
>> So there's nothing wrong with the syntax, but I suspect that the filter compilation sometimes takes too long, and that this somehow triggers the "filter syntax error" when it should trigger a timeout error instead.
>> (I haven't had time to recompile and test yet, to verify)
>>
>>
>> If you put the errormessage first, no truncation is needed I think.
>> If syslogd has a maximum messagesize, then *it* will truncate instead but we will still see the actual error message.
>>
>> /Elof
>>
>>
>>> On Thu, 22 Jan 2015, Carter Bullard wrote:
>>>
>>> Hey /Elof,
>>> Error messages where the filter comes before the error type,
>>> are all syntax errors, and are found in the ./common/argus_util.c
>>> file.
>>>
>>> Try this patch to move the filter to the end of the error messages …
>>>
>>> ==== //depot/argus/clients/common/argus_util.c#390 - /Volumes/Users/carter/argus/clients/common/argus_util.c ====
>>> 1553c1553
>>> <          ArgusLog (LOG_ERR, "%s filter syntax error", parser->ArgusRemoteFilter);
>>> ---
>>>>         ArgusLog (LOG_ERR, "filter syntax error: %s", parser->ArgusRemoteFilter);
>>> 1557c1557
>>> <          ArgusLog (LOG_ERR, "%s filter syntax error", parser->ArgusLocalFilter);
>>> ---
>>>>         ArgusLog (LOG_ERR, "filter syntax error: %s", parser->ArgusLocalFilter);
>>> 1561c1561
>>> <          ArgusLog (LOG_ERR, "%s filter syntax error", parser->ArgusDisplayFilter);
>>> ---
>>>>         ArgusLog (LOG_ERR, "filter syntax error: %s", parser->ArgusDisplayFilter);
>>>
>>>
>>> If we truncated the string length, say using %.xxxs where xxx is the number of bytes,
>>> what do you think would be a good length ???
>>>
>>> Carter
>>>
>>>> On Jan 22, 2015, at 12:14 PM, elof2 at sentor.se wrote:
>>>>
>>>>
>>>> Apropos this old thread.
>>>>
>>>> If ra 3.0.8 still take too long to compile the filter, the logged message should be cropped.
>>>> Currently, my *long* filter is more than 992 characters, and apparently that is some limit. Either in ra or in syslog, because the syslog message
>>>> looks like this:
>>>>
>>>> "2015-01-22 15:54:33 +01:00 foobar ra[36131]: 1421938473.058216 src net (10.0.0.0/8 or ... or ... ... ..  . . . . . . . . . and not host 10.123.123.100 and not host 10.123.123.123 and not host"
>>>>
>>>> Where the message is 992 characters long.
>>>>
>>>> In reality it should continue with more "and not host" and after that probably give some error-message about the filer compilation taking too long. However, this information is not shown due to the length of my *long* filterstring.
>>>> So, either crop the filterstring, or put the error-message in front of the filterstring so that people understand why ra is complaining.
>>>>
>>>> /Elof
>>>>
>>>>
>>>>> On Mon, 20 May 2013, Carter Bullard wrote:
>>>>>
>>>>> Hey Elof,
>>>>> Great, and thanks for the info.  While the clients will be blocked waiting for the filter to be compiled, because this is structured as a deadman timer, if the compiler returns quickly, there won't be any delay.
>>>>>
>>>>> For remote accesses, we don't connect until we've compiled the filter.  The logic is if we can't compile the filter, the remote won't be able to either...so a long time wait for the local compiler won't over run us with data.
>>>>>
>>>>> I'll make it 1sec in the code base until I hear otherwise.
>>>>> Carter
>>>>>
>>>>>
>>>>>> On May 20, 2013, at 7:41 AM, elof2 at sentor.se wrote:
>>>>>>
>>>>>>
>>>>>> Carter and I have discussed a very unusual error message; "filter syntax error", which may show up if the machine is HEAVILY burdened, like swapping a lot and/or receiving tons of interrupts, while feeding a long and complex filter expression to a ra* process.
>>>>>>
>>>>>>> I haven't seen this problem before, but there maybe a simple fix.
>>>>>>> In order to get around a problem with using flex and bison to compile
>>>>>>> multiple filters at a time, we fork a process to compile a filter, and wait
>>>>>>> for that process to write the filter back to argus through a pipe.  We
>>>>>>> have specific error messages for the pipe creation, but we use a
>>>>>>> generic syntax error message if the forked process doesn't
>>>>>>> return a binary filter back to us in a timeout period, which is now set
>>>>>>> at 200 milliseconds.
>>>>>>
>>>>>> I suggest that Carter add a specific error message for this particular scenario, logging the message "filter compilation timeout" instead of the generic "filter syntax error".
>>>>>>
>>>>>>> In the argus-clients file ./common/argus_code.c, in the routine ArgusFilterCompile(),
>>>>>>> on line 389, we set the timer value.  Increase the time to, what, a second ?
>>>>>>> Here is a patch for the 3.0.6.2 code…..
>>>>>>> thoth:common carter$ diff argus_code.c.orig argus_code.c
>>>>>>> 389,390c389,390
>>>>>>> <       wait.tv_sec  = 0;
>>>>>>> <       wait.tv_usec = 200000;
>>>>>>> ---
>>>>>>>>     wait.tv_sec  = 1;
>>>>>>>>     wait.tv_usec = 0;
>>>>>>
>>>>>>
>>>>>> During the weekend I've had cron start ra with a long and complex filter string every 5 minutes.
>>>>>> With a timeout of 500ms, I had 9 "filter syntax error" in 34 hours.
>>>>>> (on an old Intel Xeon 3050 @ 2.13GHz machine from 2008)
>>>>>> (with 200ms I suspect I would have had approx 300 warnings)
>>>>>>
>>>>>>
>>>>>> So to prevent this message to appear "too often" on choked machines, you could increase the wait.tv_usec to 750000  ...or even higher if a higher value doesn't introduce any overall negative impact on ra*.
>>>>>>
>>>>>>
>>>>>>
>>>>>> I now upped the timeout to 900000 just to see if that is enough to quell even those 9 warnings.
>>>>>>
>>>>>> /Elof
>>>
>