A couple of labeling questions/issues...
Dave Edelman
dedelman at iname.com
Sat Mar 9 20:34:21 EST 2013
The quality of the data in the files that you collect from the RIRs has
always been a bit awful. That isn't likely to be the reason that you don't
see ZZ as the country code for the RFC1918 address space.
I get this without any problem from my home network (times are UTC):
ra -S localhost:561 -s stime:32 proto saddr sport sco dir dco daddr dport -
not man
StartTime Proto SrcAddr Sport sCo
Dir dCo DstAddr Dport
Sun 2013-03-10 01:20:53.302 tcp 10.1.1.50.4427 ZZ
-> ZZ 10.1.1.45.22
Sun 2013-03-10 01:20:53.303 tcp 10.1.1.46.35417 ZZ
-> ZZ 10.1.1.45.561
Sun 2013-03-10 01:20:53.506 tcp 10.1.1.101.56123 ZZ
-> ZZ 10.1.1.8.8080
Sun 2013-03-10 01:20:53.626 tcp 2001:470:8d5c:1:3*.56122
-> 2001:470:8d5c:1:7*.8080
Sun 2013-03-10 01:20:53.670 udp 10.1.1.101.49647 ZZ
<-> ZZ 10.1.1.8.161
Sun 2013-03-10 01:20:54.166 arp 10.1.1.92 ZZ
who ZZ 10.1.1.68
Sun 2013-03-10 01:20:54.173 udp 10.1.1.92.59373 ZZ
<-> ZZ 10.1.1.68.53
Sun 2013-03-10 01:20:54.386 udp 2001:470:8d5c:1:5*.123
<-> 2001:470:bce7:3::*.123
Sun 2013-03-10 01:20:54.545 udp 10.1.1.101.54272 ZZ
<-> ZZ 10.1.1.8.161
Sun 2013-03-10 01:20:54.945 udp 10.1.1.50.137 ZZ
-> ZZ 10.1.1.126.137
Sun 2013-03-10 01:20:55.204 ipv6-* 2001:470:8d5c:1:3*.135
<-> 2001:470:8d5c:1:7*.0
Sun 2013-03-10 01:20:55.297 arp 10.1.1.68 ZZ
who ZZ 10.1.1.126
Sun 2013-03-10 01:20:55.307 arp 10.1.1.68 ZZ
who ZZ 10.1.1.101
Sun 2013-03-10 01:20:56.975 arp 10.1.1.50 ZZ
who ZZ 10.1.1.10
Sun 2013-03-10 01:20:58.346 tcp 10.1.1.46.35417 ZZ
-> ZZ 10.1.1.45.561
Sun 2013-03-10 01:20:58.347 tcp 10.1.1.50.4427 ZZ
-> ZZ 10.1.1.45.22
Sun 2013-03-10 01:20:58.523 tcp 10.1.1.101.56124 ZZ
-> ZZ 10.1.1.8.8080
Sun 2013-03-10 01:20:58.563 tcp 10.1.1.101.56123 ZZ
-> ZZ 10.1.1.8.8080
Sun 2013-03-10 01:20:59.173 arp 10.1.1.68 ZZ
who ZZ 10.1.1.92
Sun 2013-03-10 01:20:59.386 ipv6-* fe80::5246:5dff:f*.135
<-> fe80::212:1eff:fe*.0
Sun 2013-03-10 01:20:59.507 tcp fe80::3e07:54ff:f*.56125
-> fe80::7aac:c0ff:f*.8080
Sun 2013-03-10 01:20:59.539 tcp 10.1.1.101.56126 ZZ
-> ZZ 10.1.1.8.8080
Sun 2013-03-10 01:20:59.844 tcp 10.1.1.68.647 ZZ
<?> ZZ 10.1.1.45.46741
Sun 2013-03-10 01:21:00.163 tcp 10.1.1.50.5109 ZZ
-> US 209.177.145.148.22
Sun 2013-03-10 01:21:00.431 udp 10.1.1.45.123 ZZ
<-> US 192.5.41.40.123
There are two things to check before anything else: in ~/.rarc what value do
you have for RA_DELEGATED_IP (and is it active or commented out) and what
are the full contents of your f /usr/local/argus/ralabel.conf file. I guess
that the third thing to ask is the version of ralabel that you are using,
get it directly from the program by invoking it with -h (I seem to remember
a similar problem with an older version of the code.)
--Dave
From: Craig Merchant [mailto:cmerchant at responsys.com]
Sent: Saturday, March 09, 2013 7:54 PM
To: Dave Edelman; 'Argus'
Subject: RE: [ARGUS] A couple of labeling questions/issues...
Dave.
Here is the line in the delegated-iana-latest file for the 10.0.0.0/8
network:
delegated-iana-latest:iana|ZZ|ipv4|10.0.0.0|16777216|19940301|ietf
16777216 is 256x256x256, so that should be the correct number of IPs in a
class A network. Yet none of the flows for that network range have a "ZZ"
country code.
When you talked about the data quality, were you referring to the data in
the file or to Argus' ability to apply the proper country code based on that
file?
Thanks.
Craig
From: Dave Edelman [mailto:dedelman at iname.com]
Sent: Saturday, March 09, 2013 4:24 PM
To: Craig Merchant; 'Argus'
Subject: RE: [ARGUS] A couple of labeling questions/issues...
Craig,
I have way too much experience with these files so I can probably provide an
answer. The columns are from left to right:
RIR name
Country Code
Resource Type
First IP address in the range
Total number of IP addresses in the range
Date of last change
Status
There have always been issues with the data quality of these files that have
nothing to do with Argus. There is also a convention that has caused more
than a little confusion. The base address and address count mechanism can
and frequently does lead to a situation where you cannot create a single
CIDR to represent the allocation. You typically end up creating two or more
CIDRs much like the pre-VLSM days where the mask could have discontiguous
bits set.
Save yourself a bit of pain and look at the iana-address-file that should be
in the same location as the delegated-ipv4-latest file and use that. The
instructions are pretty clear and the format is carbon life form friendly.
--Dave
From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu
[mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On
Behalf Of Craig Merchant
Sent: Saturday, March 09, 2013 6:15 PM
To: Argus (argus-info at lists.andrew.cmu.edu)
Subject: [ARGUS] A couple of labeling questions/issues...
Hey, Carter. I've been testing the labeling functionality and trying to
investigate further the country code issue I posted about earlier in the
week.
I'm currently using the delegated-ipv4-latest file that is included with the
3.0.7.5 client. I've noticed that Argus isn't finding a country code match
for our public IP range. I'd like to add those ranges to the
delegated-ipv4-latest, but I'm not sure what the format is. Is there any
way to make the file recognize a subnet mask? I notice that the entry for
10.0.0.0 isn't causing the country codes for my internal networks to show up
as "ZZ". What do the numbers in the second and third to last columns
represent?
delegated-afrinic-latest:afrinic|ZA|ipv4|41.73.32.0|8192|20100112|allocated
Is it possible to use the MaxMind GeoIP database for country codes instead
of the iana file? I compiled the clients with GeoIP support. The
ralabel.conf file says the GEOIP labels will be saved as scity,dcity,icity.
I've tried adding those fields to my searches and nothing shows up. I
looked through rarc.print.all.conf to see if I could find a field in there
that looked related to GeoIP data, but nothing popped out at me. I'm using
the same GeoLiteCity.dat file that we use on our Splunk server. The GeoIP
part of my ralabel.conf is:
RALABEL_GEOIP_CITY="saddr,daddr,inode:cco,cname"
RALABEL_GEOIP_CITY_FILE="/usr/local/argus/GeoIPCity.dat"
I've tried using the -M label="regex" option, but the regex never seems to
match.
ralabel -S 10.230.174.40:561 -f /usr/local/argus/ralabel.conf -n -u -c ","
-s +sco,+dco,+label:200 -M label="blacklisted", for example, doesn't match
the following events:
1362868911.000236, e s ,tcp,199.7.204.127,39935,
->,82.98.86.167,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
1362868911.000282, e s ,tcp,199.7.204.127,56477,
->,82.98.86.161,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
1362868912.000211, e s ,tcp,199.7.202.248,44177,
->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
1362868913.000465, e s ,tcp,199.7.204.127,42282,
->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
The events above also show the issue with country codes. I'm noticing a few
things. The erroneous country codes are always a pair of letters found
somewhere in the label, but I don't see any pattern as to which letters are
used. If I disable the iana label file in ralabel.conf, country codes
display normally. If a country code is found for both a source and
destination, one of the country codes is always incorrect, but the country
code is properly displayed in the label field:
1362869594.000036, e ,udp,12.130.136.11,53,
->,69.154.227.43,53,1,70,INT,pu,US,saddr=US,public,DC1,mta-external:daddr=US
1362869594.000036, e ,udp,12.130.136.12,53,
->,200.26.226.6,53,1,68,INT,pu,AN,saddr=US,public,DC1,mta-external:daddr=AN
In the first set of events, both country codes are incorrect (dest code not
shown). In the events above, the destination code is shown correctly. I
have noticed that if, as above, the 12.130.136.12 has its sco field
displayed incorrectly, it will also have its dco field displayed incorrectly
when it is the destination of a flow.
If there is no label for a source or destination, the country code for that
host will be empty:
1362866671.000438, e
,udp,74.125.17.82,42492,<->,192.168.30.41,53,4,1260,CON,,dn,daddr=internal,D
C4,SP1--Apache,dns,ext-DC4
I enabled label support and radium and ran the same query as above, except
with ra. The issue with the country codes is the same as when I run the
searches with ralabel. Radium has a fairly significant memory leak when
labels are enabled.
Let me know if you would like me to send you any binary flow files or my
iana label file.
Thanks!
Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130309/3a9a6bb5/attachment.html>
More information about the argus
mailing list