Misc problems with argus-clients v. 3.0.5.34

[Markku Parviainen]

Hi,

2012/2/29 Carter Bullard <carter at qosient.com>:
> I think the problems you're getting from rabins.1 may not be bugs, but
> its definitely not the right results.   When processing time based bins,
> rabins.1 will reject data if its not within its working time range.  If you
> don't tell rabins.1 what time range to use, with the -t option, rabins.1 will
> have to figure out what the time range is.  It does this when reading
> from a file by reading the file twice.  Once to figure out the time ranges,
> then another run to process the data into its preallocated bins.
>
> When you pipe data into rabins.1, it has to guess what the time range
> is going to be, and so there is potential to throw data away if its all over
> the calendar, so to speak.    This doesn't solve your problem, but it just
> explains why I think you're getting unpredicted results.

Actually on my original script where I noticed this problem, I already
was using -t option. It doesn't affect to the results. But thanks for
pointing out that rabins will read the data twice without it.

Related to this problem, rabins also seems to generate new records.
Look at the port 0 (and 3074) below.

This is what the data looks like:
# racluster -m proto dport -r test.arg.gz -w - - udp | rasort -m pkts
-s proto dport pkts -n  | head -5
   udp 161      353877
   udp 137      133430
   udp 3074     108009
   udp 53        89020
   udp 8612      69988

Rabins produces port 0 and changes the packet counter of port 3074:
# rasort -m stime -r test.arg.gz -w - - udp | time rabins -M hard time
5s -m proto dport -w - | racluster -m proto dport -w - | rasort -m
pkts -s proto dport pkts -n  | head -5
   udp 161      353877
   udp 137      133430
   udp 3074     105026
   udp 0        103673
   udp 53        89020

But the individual search for 3074 produces the correct result:
# rasort -m stime -r test.arg.gz -w - - udp and dst port 3074 | rabins
-M hard time 5s -m proto dport -w - | racluster -m proto dport -w - |
rasort -m pkts -s proto dport pkts -n  | head -5
   udp 3074     108009

# racluster -m srcid -r test.arg.gz -s pkts - udp and dst port 3074
  108009

There are no really many port 0 packets there:
# rasort -m stime -r test.arg.gz -w - - udp and dst port 0 | rabins -M
hard time 5s -m proto dport -w - | racluster -m proto dport -w - |
rasort -m pkts -s proto dport pkts -n  | head -5
   udp 0           107
# racluster -m srcid -r test.arg.gz -s pkts - udp and dst port 0
     107

# ratimerange -r test.arg.gz -u
1327489126.869327 - 1327492741.717686

I tried all commands also with -t 1327489126-1327492741 (with every
rasort, rabins and racluster). It didn't change the results, nor did
it make those commands any faster.

I also noticed this:

# ranonymize -r test.arg.gz -w anon.arg
# racluster -m srcid -r anon.arg -s pkts -  udp and dst port 3074
       1
# racluster -m srcid -r anon.arg -s pkts - udp and dst port 0
     107
# racluster -m srcid -r anon.arg -s pkts
 4046512
# racluster -m srcid -r test.arg.gz -s pkts
 4046512

There are packets in the anonymized file, but the filter does not find
all of them?

I'll send the original packet file separately for your analysis.