Search filter, UDP, INT/CON, primitives + exit status

[elof2 at sentor.se]

On Mon, 9 May 2011, Carter Bullard wrote:
> Regarding exit status for ra* programs.
> Currently, we exit with 0 for success and 1 for failure.  This is the Unix way, and I believe
> that we're doing ok with this. Errors, are real errors, no file, can't get a socket, no
> remote host, something like that.

Yepp, doing things the UNIX way seem like a good approach. :-)

(
Apropos errors and exit values:
Ra v3.0.5.8 don't complain and don't give any exit status to 
indicate failure if no argus log is found.
   $ ra -r foo
   0
This command will exit with status 0 indicating success when it should 
have returned 1 to indicate failure locating the non-existing file foo.
)


> You are suggesting that we go the grep.1 route for return codes, where the expression
> is any filter that may be provided.
> This is doable, at least in the way the logic is done, but many of the client programs have
> their own idea of success and failure.  Simply adding a filter shouldn't change the meaning
> of the return code for those programs.
> We can implement this for ra(), 0 when there is a match, 1 when there is no error, but no
> matching records, and 2 when there is an error.  This will require some interpretation, as
> ra() has a number of options/modes of operation that bend this simple approach.
>
> Biggest question is what does it mean to match records when there is no filter.  A strict
> interpretation suggests that you should get a 0, if you open a file or socket, and get an initial
> management record, indicating that its a legitimate argus data file/stream, but the stream
> could be empty, i.e. no data records.

In my opinion, the management records should be excluded from the "grep 
style exit value". Almost as if management records are equal to stderr and 
actual traffic records are stdout. Compare this with the normal behavior 
of 'grep' - grep don't look for matches on stderr.

The answer to your question would then be that 'ra' with no filter will 
exit with 0 if there were *any* actual traffic (llc, arp, ip...) in the 
argus logfile and exit with 1 if no actual traffic were seen (but 
management records could exist and be printed on screen, just as stderr).

> With a filter, you still get the management record, but the proposal is that you have to
> have a record that matches the filter.  This seems logical, so I can buy that.
> If you use ra() to read  a non-argus file, file is good, we can open it, etc... so no failure.
> But we don't see any argus management records, so return 1.  We currently return 1 now,
> for this condition, as it is a failure.
> ra() has a mode, "-M poll", where it attaches to a remote argus data source, and if the source
> responds with a management record, it exits and returns 0.  A poll for data source availability.
> This bends the idea a bit, but can be rationalized as "a management record is a match, if
> you don't have a filter".  "-M poll" invalidates any type of filter.

All of this could be disregarded if treating management records as 
"stderr".

> When using the "-e <regex>" option to ra(), this return strategy is already implemented,
> but it is independant of any filter matching.  When the "-e" option is used, we won't care
> about whether there is a filter match.

Yes. The -e option should override the default grep.1 exit status. 
...and real Errors should override both.


> Any other condition that I haven't thought about?

There might be a scenario where the user want to set the exit status based 
on matches on traffic OR management records. Perhaps there should be a new 
option to include these otherwise excluded management records when 
calculating the exit value?
Lets say we use "-m" (as in management records). Adding this option 
to ra would be kind of simillar of doing a 2>&1 before a grep.

I hope you follow my drift. :-)


> Is this going in the direction that you suggested?
> If so, I'll have to have a RETURN VALUES section to explain this, no problem.
> But I don't know what we are going to break with this.  May not be terrible.

Yes, definately moving in the right direction.
Yes, a man-section will be neccessary to explain how the exit value is 
calculated. :-)

/Elof



> On May 4, 2011, at 6:30 AM, elof2 at sentor.se wrote:
>
>>
>> Regarding search filters and primitives...
>>
>> I would like the possibility to use all the filter primitives on a per flow-direction basis, like "src est", "src syn" or "dst synack".
>>
>>
>> Some examples of how I'd like to be able to match:
>> "PA_PA" - src push and src ack and not src syn and not src fin and not src reset and dst push and dst ack and not dst syn and not dst fin and not dst
>> reset
>>
>> "FSPA_" - src est and not dst est (detect incorrect SPAN setup where only
>>                                 one direction of the traffic is mirrored)
>> "_FSPA" - not src est and dst est (detect incorrect SPAN setup)
>>
>> "*R_SPA*" - src est and src reset and dst est (detect connections that are reset (RST) by the client rather than terminated using FIN)
>>
>> ...and so on. Currently, this kind of detailed granularity can't be achieved internally in ra, so I have to use an external command like grep.
>>
>>
>>
>>
>> If the ra* tools also set an exit status depending on if there were any match or not, that would be fantastic! Then you could use something like this:
>>
>> if ra -r huge_argus.log -qN1 - port 12345; then
>>  echo "Yes, port 12345 exist"
>> else
>>  echo "Nope, no port 12345 in huge_argus.log"
>> fi
>>
>> /Elof
>>
>>
>> On Mon, 2 May 2011, Jesper Skou Jensen wrote:
>>
>>> Excellent, it works Carter.
>>>
>>> Thank you for the help.
>>>
>>>
>>> --
>>>
>>> Jesper S. Jensen
>>> UNI-C - Århus, Danmark
>>>
>>> On 29-04-2011 18:12, Carter Bullard wrote:
>>>> Hey gentle people,
>>>> OK, so I have fixed the 'con' filter item, and 'init' is now equivalent to 'start'.
>>>> There is some cleanup work to do on all these states, so if you see something that is amiss,
>>>> don't hesitate to send to the list.
>>>> This has uncovered a bug in argus, however.  argus() is reporting most records as 'status',
>>>> even thought they may be starting records or status records.  Which means, that currently,
>>>> some argus data is incomplete, and I have a bug to fix in argus().
>>>> The clients should be fine now.
>>>> I'll upload new client software in an hour or so.
>>>> Carter
>>>
>
>