Nondeterministic output

[Carter Bullard]

Hey George,
Well, there are a number of things going on at the same time, but I'm getting a grip
on this issue.  When writing to a file, argus seems to be very consistent.  When
writing to stdout, it seems to be having some issues with getting all the records out
of the engine, and it seems that when we zero out a record (when we write a flow
status record, we maintain the cache, but zero out the metrics) things may not behave
as we would like, ....., but only when we have to queue records for output.  We do this
when we are delivering flow records to the output socket/file descriptor faster than they
can be written out the device.  When we have partially written a record, and we are
still queuing outgoing flow records, we get into a bad situation where we only clear
one record every turn, and we have a turn every 0.020 seconds, so we get really slow.
That is where the trouble then begins.

So the short story is, when you write to disk, all is good, when you write to stdout, all
is not.  I'm working this now, but it is a head scratcher and so it may take a few days.

Sorry for the inconvenience,

Carter

On Sep 21, 2010, at 11:05 AM, George Jones wrote:

> The following command produces different output:
>  
>   cat foo.pcap | argus -U 64 -r - -w /tmp/foo.ar
>   cat foo.pcap | argus -U 64 -r - -w /tmp/bar.ar
>  
> cksum(1) shows the content differs (but byte count is the same).
>  
> More disturbing is different numbers of records output from identical runs on the same input, etc.
>  
>   cat foo.pcap | argus -U 64 -r - -w - | racluster -r - -w - | ra -r - | tee /tmp/1.out
>   cat foo.pcap | argus -U 64 -r - -w - | racluster -r - -w - | ra -r - | tee /tmp/2.out
>  
> results in slightly different output.  Sometimes there are slight differences in the flgs (packet ordering, I think),
> but in a file of several thousand records, I'm getting 6 or so additional records in one output file vs the other.
>  
> Confused,
> ---George Jones


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100930/681ef04f/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100930/681ef04f/attachment.bin>