Misc problems with argus-clients v. 3.0.5.34

[Carter Bullard]

Hey Markku,
Thanks for the bug reports, and I'll work on them today.
Please don't wait to report the next set.  No need to be frustrated,
for any extended period of time.

Your #1 and #2 are definitely things that need to be fixed.

I think the problems you're getting from rabins.1 may not be bugs, but
its definitely not the right results.   When processing time based bins,
rabins.1 will reject data if its not within its working time range.  If you
don't tell rabins.1 what time range to use, with the -t option, rabins.1 will
have to figure out what the time range is.  It does this when reading
from a file by reading the file twice.  Once to figure out the time ranges,
then another run to process the data into its preallocated bins.

When you pipe data into rabins.1, it has to guess what the time range
is going to be, and so there is potential to throw data away if its all over
the calendar, so to speak.    This doesn't solve your problem, but it just
explains why I think you're getting unpredicted results.

I can address this, but I need to understand a bit more about the original
data.  Can you share the data file that you are using, so we can converge on
what you think is the right result?


I'm concerned about your last example, which definitely needs attention. If
I can reproduce it, I can fix it.

Carter


On Feb 29, 2012, at 6:12 AM, Markku Parviainen wrote:

> Hi,
> 
> I'm reporting some bugs and (annoying) features in argus-clients
> version 3.0.5.34. The host is 64bit linux, gcc 4.1.2, kernel 2.6.18.
> 
> 
> 1. Aborting rabins/racluster with ctrl+c
> 
> There's a feature at least in racluster and rabins that I would want
> to be removed: aborting the command with ctrl+c does not always work,
> especially when the process is becoming a memory hog. Please fix it so
> that ctrl+c would always instantly abort whatever the command was
> doing, trashing everything from the memory without printing anything
> it didn't had time to. Now it seems that it receives the command, but
> ignores it while doing "just one more thing for next 5 minutes".
> Sometimes when racluster starts outputting lines to console and I
> can't stop it with ctrl+c at all.
> Now I typically have to stop the process with ctrl+z and then kill it
> with kill -9 %2, or similar. That works, but kinda sucks.
> 
> 
> 
> 2. Segmentation fault when writing to a protected directory
> 
> All argus clients crash with Segmentation fault when trying to write
> to a file (i.e. create it) in a directory where the current user has
> no write permission.
> 
> $ ra -D6 -r t.arg -w t2.arg
> ...
> ra[23826.e025c773a62b0000]: 2012-02-28T11:54:22 ArgusReadFileStream() starting
> ra[23826.e025c773a62b0000]: 2012-02-28T11:54:22 ArgusReadStreamSocket
> (0x2ba673db6010) read 600 bytes
> ra[23826.e025c773a62b0000]: 2012-02-28T11:54:22 ArgusGenerateRecord
> (0x2ba673db6620, 0) len 200
> Segmentation fault
> 
> 
> 
> 3. Wrong packet counts from rabins
> 
> Rabins reports wrong and somewhat random packet counts. It is expected
> that rabins averages the values, but the sum of packets should not
> change. Am I doing something wrong here?
> I tried pre-sorting the data by stime, but it does not help on all
> cases, depending on the size of the time bins.
> And as it seems, when the file was written by argus and then grouped
> by racluster (with -m saddr daddr proto dport), the file typically is
> not ordered by stime on disk. The test file here (t.arg) contains data
> for one hour, and obviously, is not changed in the middle:
> 
> This is correct:
> $ racluster -m proto dport -r t.arg -s proto dport pkts -n - 'tcp and
> dst port (0 or 80 or 22)'
>   tcp 0          8903
>   tcp 22         1640
>   tcp 80       136659
> 
> The same directly with rabins causes garbage:
> $ rabins -m proto dport -M hard time 5m -r t.arg -w - - 'tcp and dst
> port (0 or 80 or 22)' | racluster -m proto dport -s proto dport pkts
> -n
>   tcp 0          2001
>   tcp 22           88
>   tcp 80        29865
> 
> Pre-sorting fixes the problem here:
> $ rasort -m stime -r t.arg -w - - 'tcp and dst port (0 or 80 or 22)' |
> rabins -m proto dport -M hard time 5m -w - | racluster -m proto dport
> -s proto dport pkts -n
>   tcp 0          8903
>   tcp 22         1640
>   tcp 80       136659
> 
> But not here when using 5s bins:
> $ rasort -m stime -r t.arg -w - - 'tcp and dst port (0 or 80 or 22)' |
> rabins -m proto dport -M hard time 5s -w - | racluster -m proto dport
> -s proto dport pkts -n
>   tcp 0          9378
>   tcp 22         1165
>   tcp 80       136659
> 
> 
> What's even more odd here is that another racluster in chain changes
> the values again.
> This is correct as it should be:
> $ racluster -m saddr daddr proto dport -r t.arg -w - - 'tcp and dst
> port (0 or 80 or 22)' | rasort -m stime  -w - | rabins -m proto dport
> -M hard time 5m -w - | racluster -m proto dport -s proto dport pkts -n
>   tcp 0          8903
>   tcp 22         1640
>   tcp 80       136659
> 
> But this is not. The values are not even the same as with 5s bins before.
> $ racluster -m saddr daddr proto dport -r t.arg -w - - 'tcp and dst
> port (0 or 80 or 22)' | rasort -m stime  -w - | rabins -m proto dport
> -M hard time 5s -w - | racluster -m proto dport -s proto dport pkts -n
>   tcp 0          9882
>   tcp 22          661
>   tcp 80       136659
> 
> Random results makes a network analyst very frustrated...

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120229/d1bf0f8e/attachment.bin>