RA_CIDR_ADDRESS_FORMAT="yes" and rasqlinsert -S [radium source] may be a problem
Dave Edelman
dedelman at iname.com
Thu May 9 23:14:12 EDT 2013
I have a single instance of argus that has been running for years and
creating hourly files of the flow data. On a daily basis, I copy a day's
worth of the flow files to a second system where I run rqasqlinsert in
various flavors to create several different tables.
I finally decided to use radium and rastream to do this the right way (but I
didn't stop the local file creation, just to be safe.)
The details follow but it looks like there is a toxic interaction between
RA_CIDR_ADDRESS_FORMAT="yes" in ~/.rarc and rasqlinsert using -S from a
radium instance in client version 3.0.7.8 and possibly earlier.
--------- Danger beyond this point are the gory details
--------------------------------
The original method worked very well:
A typical MySQL table queried for all 10.1.1.10 activity gave reasonable
results (I learned not to select the record blob :-) )
mysql> select ltime, dur, saddr, daddr, proto, bytes from matrix_2013_04_09
where saddr = '10.1.1.10' or daddr = '10.1.1.10';
+-------------------+--------------+------------+-----------------+-------+-
------------+
| ltime | dur | saddr | daddr | proto |
bytes |
+-------------------+--------------+------------+-----------------+-------+-
------------+
| 1365551889.364000 | 86287.953000 | 10.1.1.10 | 10.1.1.60 | tcp |
843282 |
| 1365551948.803000 | 86347.195000 | 10.1.1.10 | 10.1.1.46 | tcp |
783536 |
| 1365551993.490000 | 86391.336000 | 10.1.1.10 | 10.1.1.45 | tcp |
16050742461 |
| 1365551986.978000 | 86381.195000 | 10.1.1.10 | 10.1.1.50 | tcp |
3962458654 |
| 1365551957.703000 | 86340.031000 | 10.1.1.10 | 255.255.255.255 | udp |
1346514 |
| 1365551992.462000 | 86374.555000 | 10.1.1.10 | 10.1.1.50 | udp |
1667967 |
| 1365551992.462000 | 86369.734000 | 10.1.1.10 | 10.1.1.50 | icmp |
1426656 |
| 1365551938.461000 | 86280.000000 | 10.1.1.45 | 10.1.1.10 | arp |
157312 |
| 1365551964.909000 | 86302.922000 | 10.1.1.50 | 10.1.1.10 | arp |
152960 |
| 1365551953.810000 | 86274.445000 | 10.1.1.46 | 10.1.1.10 | arp |
107904 |
| 1365551889.353000 | 86206.047000 | 10.1.1.60 | 10.1.1.10 | arp |
108544 |
| 1365551868.423000 | 86055.141000 | 10.1.1.10 | 10.1.1.12 | udp |
30672 |
| 1365551873.421000 | 86055.141000 | 10.1.1.10 | 10.1.1.12 | arp |
36352 |
| 1365551639.982000 | 85736.523000 | 10.1.1.10 | 10.1.1.127 | udp |
61560 |
| 1365551920.751000 | 85978.414000 | 10.1.1.10 | 10.1.1.50 | arp |
49408 |
| 1365551560.220000 | 77317.164000 | 10.1.1.10 | 10.1.1.101 | udp |
115032 |
| 1365551560.220000 | 77310.164000 | 10.1.1.10 | 10.1.1.101 | arp |
29696 |
| 1365550944.567000 | 76398.391000 | 10.1.1.10 | 224.0.0.251 | udp |
3703 |
| 1365529023.282000 | 54396.770000 | 10.1.1.10 | 10.1.1.101 | tcp |
1032139 |
| 1365475483.795000 | 786.242000 | 10.1.1.101 | 10.1.1.10 | arp |
896 |
| 1365501012.119000 | 24844.955000 | 10.1.1.10 | 10.1.1.45 | udp |
2130 |
| 1365501012.119000 | 24844.953000 | 10.1.1.10 | 10.1.1.45 | icmp |
2718 |
| 1365501007.827000 | 24840.254000 | 10.1.1.10 | 10.1.1.126 | arp |
384 |
| 1365501012.354000 | 24844.787000 | 10.1.1.10 | 167.206.245.130 | udp |
5766 |
| 1365501008.507000 | 24840.812000 | 10.1.1.10 | 113.37.91.61 | icmp |
612 |
| 1365501012.895000 | 24844.477000 | 10.1.1.10 | 113.37.91.61 | tcp |
37293 |
| 1365502197.518000 | 24840.180000 | 10.1.1.126 | 10.1.1.10 | arp |
384 |
+-------------------+--------------+------------+-----------------+-------+-
------------+
27 rows in set (0.00 sec)
rasql gave happy results:
rasql -u -r mysql://argus:argus@localhost/argus/matrix_2013_04_09 -M sql="
saddr = '10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1365551889.364 86287.953 69.113.13.218 10.1.1.60
10.1.1.10 tcp 843282
1365551948.803 86347.195 69.113.13.218 10.1.1.46
10.1.1.10 tcp 783536
1365551993.490 86391.336 69.113.13.218 10.1.1.45
10.1.1.10 tcp 160507424*
1365551986.978 86381.195 69.113.13.218 10.1.1.50
10.1.1.10 tcp 3962458654
1365551957.703 86340.031 69.113.13.218 10.1.1.10
255.255.255.255 udp 1346514
1365551992.462 86374.555 69.113.13.218 10.1.1.10
10.1.1.50 udp 1667967
1365551992.462 86369.734 69.113.13.218 10.1.1.10
10.1.1.50 icmp 1426656
1365551938.461 86280.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 157312
1365551964.909 86302.922 69.113.13.218 10.1.1.50
10.1.1.10 arp 152960
1365551953.810 86274.445 69.113.13.218 10.1.1.46
10.1.1.10 arp 107904
1365551889.353 86206.047 69.113.13.218 10.1.1.60
10.1.1.10 arp 108544
1365551868.423 86055.141 69.113.13.218 10.1.1.10
10.1.1.12 udp 30672
1365551873.421 86055.141 69.113.13.218 10.1.1.10
10.1.1.12 arp 36352
1365551639.982 85736.523 69.113.13.218 10.1.1.10
10.1.1.127 udp 61560
1365551920.751 85978.414 69.113.13.218 10.1.1.10
10.1.1.50 arp 49408
1365551560.220 77317.164 69.113.13.218 10.1.1.10
10.1.1.101 udp 115032
1365551560.220 77310.164 69.113.13.218 10.1.1.10
10.1.1.101 arp 29696
1365550944.567 76398.391 69.113.13.218 10.1.1.10
224.0.0.251 udp 3703
1365529023.282 54396.770 69.113.13.218 10.1.1.101
10.1.1.10 tcp 1032139
1365475483.795 786.242 69.113.13.218 10.1.1.101
10.1.1.10 arp 896
1365501012.119 24844.955 69.113.13.218 10.1.1.10
10.1.1.45 udp 2130
1365501012.119 24844.953 69.113.13.218 10.1.1.45
10.1.1.10 icmp 2718
1365501007.827 24840.254 69.113.13.218 10.1.1.10
10.1.1.126 arp 384
1365501012.354 24844.787 69.113.13.218 10.1.1.10
167.206.245.130 udp 5766
1365501008.507 24840.812 69.113.13.218 10.1.1.10
113.37.91.61 icmp 612
1365501012.895 24844.477 69.113.13.218 10.1.1.10
113.37.91.61 tcp 37293
1365502197.518 24840.180 69.113.13.218 10.1.1.126
10.1.1.10 arp 384
And a confirmation from the original flow files checked out well
racluster -m srcid matrix protocol -r * -u -p 3 -s ltime dur srcid saddr
daddr proto bytes - host 10.1.1.10
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1365551992.462 86374.555 69.113.13.218 10.1.1.10
10.1.1.50 udp 1667967
1365551560.220 77317.164 69.113.13.218 10.1.1.10
10.1.1.101 udp 115032
1365551957.703 86340.031 69.113.13.218 10.1.1.10
255.255.255.255 udp 1346514
1365501012.354 24844.787 69.113.13.218 10.1.1.10
167.206.245.130 udp 5766
1365551868.423 86055.141 69.113.13.218 10.1.1.10
10.1.1.12 udp 30672
1365551639.982 85736.523 69.113.13.218 10.1.1.10
10.1.1.127 udp 61560
1365501012.119 24844.955 69.113.13.218 10.1.1.10
10.1.1.45 udp 2130
1365550944.567 76398.391 69.113.13.218 10.1.1.10
224.0.0.251 udp 3703
1365551986.978 86389.391 69.113.13.218 10.1.1.10
10.1.1.50 tcp 3962486779
1365529023.282 54396.770 69.113.13.218 10.1.1.10
10.1.1.101 tcp 1032139
1365501012.895 24844.477 69.113.13.218 10.1.1.10
113.37.91.61 tcp 37293
1365551993.490 86391.336 69.113.13.218 10.1.1.10
10.1.1.45 tcp 160507424*
1365551948.803 86347.195 69.113.13.218 10.1.1.10
10.1.1.46 tcp 783536
1365551889.364 86287.953 69.113.13.218 10.1.1.10
10.1.1.60 tcp 843282
1365501012.119 24844.953 69.113.13.218 10.1.1.10
10.1.1.45 icmp 2718
1365551992.462 86369.734 69.113.13.218 10.1.1.10
10.1.1.50 icmp 1426656
1365501008.507 24840.812 69.113.13.218 10.1.1.10
113.37.91.61 icmp 612
1365551873.421 86055.141 69.113.13.218 10.1.1.10
10.1.1.12 arp 36352
1365551920.751 85978.414 69.113.13.218 10.1.1.10
10.1.1.50 arp 49408
1365551560.220 77310.164 69.113.13.218 10.1.1.10
10.1.1.101 arp 29696
1365501007.827 24840.254 69.113.13.218 10.1.1.10
10.1.1.126 arp 384
1365551938.461 86340.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 157440
1365551953.810 86274.445 69.113.13.218 10.1.1.46
10.1.1.10 arp 107904
1365551964.909 86302.922 69.113.13.218 10.1.1.50
10.1.1.10 arp 152960
1365551889.353 86206.047 69.113.13.218 10.1.1.60
10.1.1.10 arp 108544
1365475483.795 786.242 69.113.13.218 10.1.1.101
10.1.1.10 arp 896
1365502197.518 24840.180 69.113.13.218 10.1.1.126
10.1.1.10 arp 384
Then I set these three running on the machine with the database
(argus-clients-3.0.7.8)
/usr/local/bin/radium -f /usr/local/argus/SNKradium.conf -d
/usr/local/bin/rastream -S localhost:9603 -f /usr/local/argus/SNKstream.sh
-M time 1h -B 15 -w /data/argus/%Y/%m/%d/argus.%Y.%m.%d.%H -d
/usr/local/bin/rasqlinsert -M time 1d -M cache -S localhost:9603 -w
mysql://argus@localhost/argus/matrix_%Y_%m_%d -m srcid matrix proto -s ltime
dur srcid saddr daddr proto bytes -d
# cat /usr/local/argus/SNKradium.conf
RADIUM_DAEMON=no
RADIUM_CLASSIFIER_FILE=/usr/local/argus/SNKlabel.conf
RADIUM_ACCESS_PORT=9603
RADIUM_ARGUS_SERVER=rodnel-new:561
The SNKstream.sh file doesn't do anything but gzip the file.
Now I get these results:
The MySQL table is a bit unusual but not absolutely awful:
mysql> select ltime,dur,srcid,saddr, daddr, proto, bytes from
matrix_2013_05_09 where saddr = '10.1.1.10' or daddr = '10.1.1.10';
+-------------------+--------------+---------------+------------+-----------
------+-------+------------+
| ltime | dur | srcid | saddr | daddr
| proto | bytes |
+-------------------+--------------+---------------+------------+-----------
------+-------+------------+
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.50
| udp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.50
| icmp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.50 | 10.1.1.10
| arp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.50
| tcp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 |
255.255.255.255 | udp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.60
| tcp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.45
| tcp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.60 | 10.1.1.10
| arp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.45 | 10.1.1.10
| arp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.50
| arp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.127
| udp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.45
| udp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.45
| icmp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.126
| arp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 |
167.206.245.130 | udp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 |
113.37.91.61 | icmp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 |
113.37.91.61 | tcp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.126 | 10.1.1.10
| arp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.10 | 10.1.1.71
| udp | 4609938798 |
| 1368144006.774000 | 86377.883000 | 69.113.13.218 | 10.1.1.71 | 10.1.1.10
| arp | 4609938798 |
+-------------------+--------------+---------------+------------+-----------
------+-------+------------+
20 rows in set (0.00 sec)
The files created by rastream look correct:
racluster -m srcid matrix protocol -r * -u -p 3 -s ltime dur srcid saddr
daddr proto bytes - host 10.1.1.10
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368143927.352 86340.555 69.113.13.218 10.1.1.10
255.255.255.255 udp 1346514
1368143957.087 86335.609 69.113.13.218 10.1.1.10
10.1.1.50 udp 1674059
1368106993.711 44648.406 69.113.13.218 10.1.1.10
167.206.245.130 udp 5766
1368143651.470 85679.195 69.113.13.218 10.1.1.10
10.1.1.127 udp 61560
1368106993.205 44648.055 69.113.13.218 10.1.1.10
10.1.1.45 udp 2130
1368110304.637 0.001 69.113.13.218 10.1.1.10
10.1.1.71 udp 188
1368106995.036 44648.059 69.113.13.218 10.1.1.10
113.37.91.61 tcp 35811
1368143944.782 86342.789 69.113.13.218 10.1.1.10
10.1.1.45 tcp 173423905*
1368143942.334 86354.383 69.113.13.218 10.1.1.10
10.1.1.50 tcp 3945119719
1368143931.073 86329.680 69.113.13.218 10.1.1.10
10.1.1.60 tcp 429295
1368106993.205 44648.055 69.113.13.218 10.1.1.10
10.1.1.45 icmp 2718
1368143957.087 86335.609 69.113.13.218 10.1.1.10
10.1.1.50 icmp 1439680
1368106987.571 44641.789 69.113.13.218 10.1.1.10
113.37.91.61 icmp 612
1368143870.011 85965.906 69.113.13.218 10.1.1.10
10.1.1.50 arp 60032
1368106986.534 44640.969 69.113.13.218 10.1.1.10
10.1.1.126 arp 384
1368143949.756 86280.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 160128
1368143942.065 86316.039 69.113.13.218 10.1.1.50
10.1.1.10 arp 158848
1368143875.921 86209.273 69.113.13.218 10.1.1.60
10.1.1.10 arp 109440
1368110304.635 0.000 69.113.13.218 10.1.1.71
10.1.1.10 arp 128
1368108176.467 44641.297 69.113.13.218 10.1.1.126
10.1.1.10 arp 384
Then we come to the output of rasql which for some reason informs me way too
many times that something on my network (10.1.1.0/25) sent a bunch of
traffic to somewhere NB: this is the one place where I see CIDR notation and
that might be is a clue.
rasql -u -r mysql://argus@localhost/argus/matrix_2013_05_09 -M sql=" saddr =
'10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0/25
0.0.0.0/4 ip 4609938798
So I copied one day's files from the system running argus to a clean
directory on the MySQL machine and ran the rasqlinsert incantation that I
always used to use:
rasqlinsert -M time 1d -r * -w
mysql://argus@localhost/argus/testMatrix_%Y_%m_%d -m srcid matrix proto -s
ltime dur srcid saddr daddr proto bytes
and got results like the ones I used to get:
rasql -u -r mysql://argus@localhost/argus/testMatrix_2013_05_09 -M sql="
saddr = '10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368143991.233 86389.844 69.113.13.218 10.1.1.60
10.1.1.10 tcp 429589
1368143944.782 86342.789 69.113.13.218 10.1.1.45
10.1.1.10 tcp 173086923*
1368143957.087 86335.609 69.113.13.218 10.1.1.50
10.1.1.10 udp 1674059
1368143957.087 86335.609 69.113.13.218 10.1.1.10
10.1.1.50 icmp 1439680
1368143942.065 86316.039 69.113.13.218 10.1.1.50
10.1.1.10 arp 158848
1368143942.334 86313.445 69.113.13.218 10.1.1.50
10.1.1.10 tcp 3945115585
1368143987.364 86340.555 69.113.13.218 10.1.1.10
255.255.255.255 udp 1346514
1368143996.241 86329.594 69.113.13.218 10.1.1.60
10.1.1.10 arp 109568
1368143949.756 86280.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 160128
1368143870.011 85965.906 69.113.13.218 10.1.1.10
10.1.1.50 arp 60032
1368143651.470 85679.195 69.113.13.218 10.1.1.10
10.1.1.127 udp 61560
1368106993.205 44648.055 69.113.13.218 10.1.1.10
10.1.1.45 udp 2130
1368106993.205 44648.055 69.113.13.218 10.1.1.45
10.1.1.10 icmp 2718
1368106986.534 44640.969 69.113.13.218 10.1.1.10
10.1.1.126 arp 384
1368106993.711 44648.406 69.113.13.218 10.1.1.10
167.206.245.130 udp 5766
1368106987.571 44641.789 69.113.13.218 10.1.1.10
113.37.91.61 icmp 612
1368106995.036 44648.059 69.113.13.218 10.1.1.10
113.37.91.61 tcp 35811
1368108176.467 44641.297 69.113.13.218 10.1.1.126
10.1.1.10 arp 384
1368110304.635 0.000 69.113.13.218 10.1.1.71
10.1.1.10 arp 128
1368110304.637 0.001 69.113.13.218 10.1.1.71
10.1.1.10 udp 188
Then I used the files created by rastream to do the same thing (remember
that these files came from the same radium feed as fed the rasqlinsert that
wasn't so good)
cd /data/argus/2013/05/09
rasqlinsert -M time 1d -r * -w
mysql://argus@localhost/argus/test2Matrix_%Y_%m_%d -m srcid matrix proto -s
ltime dur srcid saddr daddr proto bytes
and got the results that I expected:
rasql -u -r mysql://argus@localhost/argus/test2Matrix_2013_05_09 -M sql="
saddr = '10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368143931.073 86329.680 69.113.13.218 10.1.1.60
10.1.1.10 tcp 429295
1368143944.782 86342.789 69.113.13.218 10.1.1.45
10.1.1.10 tcp 173423905*
1368143957.087 86335.609 69.113.13.218 10.1.1.50
10.1.1.10 udp 1674059
1368143957.087 86335.609 69.113.13.218 10.1.1.10
10.1.1.50 icmp 1439680
1368143942.065 86316.039 69.113.13.218 10.1.1.50
10.1.1.10 arp 158848
1368143942.334 86313.445 69.113.13.218 10.1.1.50
10.1.1.10 tcp 3945115585
1368143927.352 86280.539 69.113.13.218 10.1.1.10
255.255.255.255 udp 1345782
1368143875.921 86209.273 69.113.13.218 10.1.1.60
10.1.1.10 arp 109440
1368143949.756 86280.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 160128
1368143870.011 85965.906 69.113.13.218 10.1.1.10
10.1.1.50 arp 60032
1368143651.470 85679.195 69.113.13.218 10.1.1.10
10.1.1.127 udp 61560
1368106993.205 44648.055 69.113.13.218 10.1.1.10
10.1.1.45 udp 2130
1368106993.205 44648.055 69.113.13.218 10.1.1.45
10.1.1.10 icmp 2718
1368106986.534 44640.969 69.113.13.218 10.1.1.10
10.1.1.126 arp 384
1368106993.711 44648.406 69.113.13.218 10.1.1.10
167.206.245.130 udp 5766
1368106987.571 44641.789 69.113.13.218 10.1.1.10
113.37.91.61 icmp 612
1368106995.036 44648.059 69.113.13.218 10.1.1.10
113.37.91.61 tcp 35811
1368108176.467 44641.297 69.113.13.218 10.1.1.126
10.1.1.10 arp 384
1368110304.635 0.000 69.113.13.218 10.1.1.71
10.1.1.10 arp 128
1368110304.637 0.001 69.113.13.218 10.1.1.71
10.1.1.10 udp 188
Just in case the -M cache is making a difference, I included it in a test
and it didn't break anything:
rasqlinsert -M time 1d -r * -M cache -w
mysql://argus@localhost/argus/test3Matrix_%Y_%m_%d -m srcid matrix proto -s
ltime dur srcid saddr daddr proto bytes
rasql -u -r mysql://argus@localhost/argus/test3Matrix_2013_05_09 -M sql="
saddr = '10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368143991.233 86389.844 69.113.13.218 10.1.1.60
10.1.1.10 tcp 429589
1368143944.782 86342.789 69.113.13.218 10.1.1.45
10.1.1.10 tcp 173086923*
1368143957.087 86335.609 69.113.13.218 10.1.1.50
10.1.1.10 udp 1674059
1368143957.087 86335.609 69.113.13.218 10.1.1.10
10.1.1.50 icmp 1439680
1368143942.065 86316.039 69.113.13.218 10.1.1.50
10.1.1.10 arp 158848
1368143942.334 86313.445 69.113.13.218 10.1.1.50
10.1.1.10 tcp 3945115585
1368143987.364 86340.555 69.113.13.218 10.1.1.10
255.255.255.255 udp 1346514
1368143996.241 86329.594 69.113.13.218 10.1.1.60
10.1.1.10 arp 109568
1368143949.756 86280.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 160128
1368143870.011 85965.906 69.113.13.218 10.1.1.10
10.1.1.50 arp 60032
1368143651.470 85679.195 69.113.13.218 10.1.1.10
10.1.1.127 udp 61560
1368106993.205 44648.055 69.113.13.218 10.1.1.10
10.1.1.45 udp 2130
1368106993.205 44648.055 69.113.13.218 10.1.1.45
10.1.1.10 icmp 2718
1368106986.534 44640.969 69.113.13.218 10.1.1.10
10.1.1.126 arp 384
1368106993.711 44648.406 69.113.13.218 10.1.1.10
167.206.245.130 udp 5766
1368106987.571 44641.789 69.113.13.218 10.1.1.10
113.37.91.61 icmp 612
1368106995.036 44648.059 69.113.13.218 10.1.1.10
113.37.91.61 tcp 35811
1368108176.467 44641.297 69.113.13.218 10.1.1.126
10.1.1.10 arp 384
1368110304.635 0.000 69.113.13.218 10.1.1.71
10.1.1.10 arp 128
1368110304.637 0.001 69.113.13.218 10.1.1.71
10.1.1.10 udp 188
I kill CIDR notation in my ~/.rarc file to see what happens (I dropped the
current table and restarted the clients) and it is looking much better
rasql -u -r mysql://argus:argus@localhost/argus/matrix_2013_05_10 -M sql="
saddr = '10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368153616.837 60.160 69.113.13.218 10.1.1.60
10.1.1.10 tcp 588
1368154260.248 700.975 69.113.13.218 10.1.1.50
10.1.1.10 udp 13899
1368154260.248 700.975 69.113.13.218 10.1.1.10
10.1.1.50 icmp 10688
1368153868.734 307.102 69.113.13.218 10.1.1.50
10.1.1.10 tcp 425755
1368154283.605 721.920 69.113.13.218 10.1.1.60
10.1.1.10 arp 1408
1368154247.544 675.886 69.113.13.218 10.1.1.10
255.255.255.255 udp 12078
1368153964.784 360.043 69.113.13.218 10.1.1.45
10.1.1.10 tcp 10108
1368154172.306 566.982 69.113.13.218 10.1.1.50
10.1.1.10 arp 1280
1368154209.756 600.000 69.113.13.218 10.1.1.45
10.1.1.10 arp 1408
1368153742.552 0.000 69.113.13.218 10.1.1.10
10.1.1.127 udp 513
1368153908.043 67.891 69.113.13.218 10.1.1.10
10.1.1.50 arp 256
The fix is not retroactive, NB: the testMatrix, test2Matrix, and test3Matrix
tables were all generated by rasqlinsert with the .rarc containing
RA_CIDR_ADDRESS_FORMAT="yes" and they were fine so it looks like an
interaction between CIDR notation and rasqlinsert -S from a radium source
rasql -u -r mysql://argus:argus@localhost/argus/matrix_2013_05_09 -M sql="
saddr = '10.1.1.10' or daddr = '10.1.1.10'"
LastTime Dur SrcId SrcAddr
DstAddr Proto TotBytes
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
1368144006.774 86377.883 69.113.13.218 10.1.1.0
0.0.0.0 ip 4609938798
More information about the argus
mailing list