Strange rasqlinsert problem

[David Edelman]

Using racluster I get the results that I expect. One additional comment
based on your explaination of matrix, the redacted X is 153 making  that
address larger than 5.161.164.145  so that address should be the destination
not the source. It looks as if all the swaps are happening except for the
addresses themselves.

--Dave

-----Original Message-----
From: Carter Bullard [mailto:carter at qosient.com] 
Sent: Sunday, June 30, 2013 11:00 AM
To: David Edelman
Cc: Argus
Subject: Re: Strange rasqlinsert problem

Best way to compare is to use racluster() against the original data,
using the same " -m keyFields "  option to see if the aggregation goes well.
Compare your database entry with:

   racluster -r /tmp/funnySource.argus  -m srcid matrix proto -s \
        ltime dur srcid saddr daddr proto bytes - host 'X.Y.35.220' and
'5.161.164.145

Do you get the weird single line?

You are doing " matrix " aggregation, which is much different than " saddr
daddr "
aggregation.  Matrix is directionless  We sort the addresses, making the
numerically
lesser one the src and the other the dst.   When we do that we flip the
metrics and
other attributes as well.

Carter

On Jun 27, 2013, at 10:43 PM, "David Edelman" <dedelman at iname.com> wrote:

> I've managed to get this down to a very simple example of the problem and
I
> can make the files available if necessary. I have manually redacted some
of
> the IP addresses.
> 
> The background: I have Argus flow records that were created by netflow
> feeding radium 3.0.7.3.  I used rasqlinsert  3.0.7.10  (taken from the
> website this morning) to insert these data into a MySQL database. When I
saw
> the problem, I went back to the original flow records and extracted two
> flows into an output file so that I could minimize the number of moving
> parts:
> 
> The output of ra using the test file as a source is exactly what I expect
to
> see:
> ra -r /tmp/funnySource.argus -Zb
>                        StartTime      Flgs  Proto         TcpOpt
> SrcAddr        Sport   Dir            DstAddr        Dport          State
> Trans  TotPkts   TotBytes
>      Wed 2013-06-26 16:06:16.121 Ne           tcp
> X.Y.35.220.60243           ->      5.161.164.145.20830
S_
> 1        3        144
>      Wed 2013-06-26 16:06:16.281 Ne           tcp
> X.Y.35.220.60432           ->      5.161.164.145.20898
SA_
> 1        3        144
> 
> The command to insert these records was:
> rasqlinsert -M time 1d -r /tmp/funnySource.argus  -w
> mysql://argus@localhost/argus/funnyMatrix_%Y_%m_%d -m srcid matrix proto
-s
> ltime dur srcid saddr daddr proto bytes - ip
> 
> The rasql output is not what I expect to see:
> rasql -t -3d -r mysql://argus@localhost/argus/funnyMatrix_%Y_%m_%d  -M
time
> 1d -M sql=" saddr = '5.161.164.145' or daddr = '5.161.164.145'" -w - \
> | ra -s stime dur proto saddr sport sco dir daddr dport dco state trans
pkts
> bytes -L0 -Zb
> 
>                        StartTime        Dur  Proto            SrcAddr
> Sport sCo   Dir            DstAddr        Dport dCo          State  Trans
> TotPkts   TotBytes
>      Wed 2013-06-26 16:06:16.121      9.156    tcp     X.Y.35.220.0
> US    -       5.161.164.145.0             IR             _S      2
6
> 288
> 
> What happened to the TCP flags?  it seems like one is missing and one
> crossed to the other side of the road.
> 
> --Dave
> 
>