A couple of labeling questions/issues...

[Dave Edelman]

Craig,
 
I have way too much experience with these files so I can probably provide an
answer. The columns are from left to right:
RIR name
Country Code
Resource Type
First IP address in the range
Total number of IP addresses in the range
Date of last change
Status
 
There have always been issues with the data quality of these files that have
nothing to do with Argus. There is also a convention that has caused more
than a little confusion. The base address and address count mechanism can
and frequently does lead to a situation where you cannot create a single
CIDR to represent the allocation. You typically end up creating two or more
CIDRs much like the pre-VLSM days where the mask could have discontiguous
bits set.
 
Save yourself a bit of pain and look at the iana-address-file that should be
in the same location as the delegated-ipv4-latest file and use that. The
instructions are pretty clear and the format is carbon life form friendly.
 
--Dave
 
 
 
From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu
[mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On
Behalf Of Craig Merchant
Sent: Saturday, March 09, 2013 6:15 PM
To: Argus (argus-info at lists.andrew.cmu.edu)
Subject: [ARGUS] A couple of labeling questions/issues...
 
Hey, Carter.  I've been testing the labeling functionality and trying to
investigate further the country code issue I posted about earlier in the
week.
 
I'm currently using the delegated-ipv4-latest file that is included with the
3.0.7.5 client.  I've noticed that Argus isn't finding a country code match
for our public IP range.  I'd like to add those ranges to the
delegated-ipv4-latest, but I'm not sure what the format is.  Is there any
way to make the file recognize a subnet mask?  I notice that the entry for
10.0.0.0 isn't causing the country codes for my internal networks to show up
as "ZZ".    What do the numbers in the second and third to last columns
represent?  
 
delegated-afrinic-latest:afrinic|ZA|ipv4|41.73.32.0|8192|20100112|allocated
 
Is it possible to use the MaxMind GeoIP database for country codes instead
of the iana file?  I compiled the clients with GeoIP support.  The
ralabel.conf file says the GEOIP labels will be saved as scity,dcity,icity.
I've tried adding those fields to my searches and nothing shows up.  I
looked through rarc.print.all.conf to see if I could find a field in there
that looked related to GeoIP data, but nothing popped out at me.  I'm using
the same GeoLiteCity.dat file that we use on our Splunk server.  The GeoIP
part of my ralabel.conf is:
 
RALABEL_GEOIP_CITY="saddr,daddr,inode:cco,cname"
RALABEL_GEOIP_CITY_FILE="/usr/local/argus/GeoIPCity.dat"
 
I've tried using the -M label="regex" option, but the regex never seems to
match.
 
ralabel -S 10.230.174.40:561 -f /usr/local/argus/ralabel.conf -n -u -c ","
-s +sco,+dco,+label:200 -M label="blacklisted", for example, doesn't match
the following events:
 
1362868911.000236, e s      ,tcp,199.7.204.127,39935,
->,82.98.86.167,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
1362868911.000282, e s      ,tcp,199.7.204.127,56477,
->,82.98.86.161,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
1362868912.000211, e s      ,tcp,199.7.202.248,44177,
->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
1362868913.000465, e s      ,tcp,199.7.204.127,42282,
->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blackl
isted
 
The events above also show the issue with country codes.  I'm noticing a few
things.  The erroneous country codes are always a pair of letters found
somewhere in the label, but I don't see any pattern as to which letters are
used.  If I disable the iana label file in ralabel.conf, country codes
display normally.  If a country code is found for both a source and
destination, one of the country codes is always incorrect, but the country
code is properly displayed in the label field:
 
1362869594.000036, e        ,udp,12.130.136.11,53,
->,69.154.227.43,53,1,70,INT,pu,US,saddr=US,public,DC1,mta-external:daddr=US
1362869594.000036, e        ,udp,12.130.136.12,53,
->,200.26.226.6,53,1,68,INT,pu,AN,saddr=US,public,DC1,mta-external:daddr=AN
 
In the first set of events, both country codes are incorrect (dest code not
shown).  In the events above, the destination code is shown correctly.  I
have noticed that if, as above, the 12.130.136.12 has its sco field
displayed incorrectly, it will also have its dco field displayed incorrectly
when it is the destination of a flow.
 
If there is no label for a source or destination, the country code for that
host will be empty:
 
1362866671.000438, e
,udp,74.125.17.82,42492,<->,192.168.30.41,53,4,1260,CON,,dn,daddr=internal,D
C4,SP1--Apache,dns,ext-DC4
 
I enabled label support and radium and ran the same query as above, except
with ra.  The issue with the country codes is the same as when I run the
searches with ralabel.  Radium has a fairly significant memory leak when
labels are enabled.  
 
Let me know if you would like me to send you any binary flow files or my
iana label file.
 
Thanks!
 
Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130309/0b37a392/attachment.html>