A couple of labeling questions/issues...

[Craig Merchant]

Dave, thanks for taking time out of your Saturday to explain the label file format!

I'm already using an iana-address file to apply labels to the hosts/networks in my environment.  I don't see any files in the argus client package that contains country code to CIDR mappings.  I might be able to find one on the web somewhere, but then those country codes are going to show up in the label field, not the dco and sco fields, right?  I'd like my country codes in a separate field if possible.  It will save on post-processing work later down the line.

Thx.

Craig


From: Dave Edelman [mailto:dedelman at iname.com]
Sent: Saturday, March 09, 2013 4:24 PM
To: Craig Merchant; 'Argus'
Subject: RE: [ARGUS] A couple of labeling questions/issues...

Craig,

I have way too much experience with these files so I can probably provide an answer. The columns are from left to right:
RIR name
Country Code
Resource Type
First IP address in the range
Total number of IP addresses in the range
Date of last change
Status

There have always been issues with the data quality of these files that have nothing to do with Argus. There is also a convention that has caused more than a little confusion. The base address and address count mechanism can and frequently does lead to a situation where you cannot create a single CIDR to represent the allocation. You typically end up creating two or more CIDRs much like the pre-VLSM days where the mask could have discontiguous bits set.

Save yourself a bit of pain and look at the iana-address-file that should be in the same location as the delegated-ipv4-latest file and use that. The instructions are pretty clear and the format is carbon life form friendly.

--Dave



From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu<mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu> [mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On Behalf Of Craig Merchant
Sent: Saturday, March 09, 2013 6:15 PM
To: Argus (argus-info at lists.andrew.cmu.edu<mailto:argus-info at lists.andrew.cmu.edu>)
Subject: [ARGUS] A couple of labeling questions/issues...

Hey, Carter...  I've been testing the labeling functionality and trying to investigate further the country code issue I posted about earlier in the week.

I'm currently using the delegated-ipv4-latest file that is included with the 3.0.7.5 client.  I've noticed that Argus isn't finding a country code match for our public IP range.  I'd like to add those ranges to the delegated-ipv4-latest, but I'm not sure what the format is.  Is there any way to make the file recognize a subnet mask?  I notice that the entry for 10.0.0.0 isn't causing the country codes for my internal networks to show up as "ZZ".    What do the numbers in the second and third to last columns represent?

delegated-afrinic-latest:afrinic|ZA|ipv4|41.73.32.0|8192|20100112|allocated

Is it possible to use the MaxMind GeoIP database for country codes instead of the iana file?  I compiled the clients with GeoIP support.  The ralabel.conf file says the GEOIP labels will be saved as scity,dcity,icity.  I've tried adding those fields to my searches and nothing shows up.  I looked through rarc.print.all.conf to see if I could find a field in there that looked related to GeoIP data, but nothing popped out at me.  I'm using the same GeoLiteCity.dat file that we use on our Splunk server.  The GeoIP part of my ralabel.conf is:

RALABEL_GEOIP_CITY="saddr,daddr,inode:cco,cname"
RALABEL_GEOIP_CITY_FILE="/usr/local/argus/GeoIPCity.dat"

I've tried using the -M label="regex" option, but the regex never seems to match.

ralabel -S 10.230.174.40:561 -f /usr/local/argus/ralabel.conf -n -u -c "," -s +sco,+dco,+label:200 -M label="blacklisted", for example, doesn't match the following events:

1362868911.000236, e s      ,tcp,199.7.204.127,39935, ->,82.98.86.167,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted
1362868911.000282, e s      ,tcp,199.7.204.127,56477, ->,82.98.86.161,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted
1362868912.000211, e s      ,tcp,199.7.202.248,44177, ->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted
1362868913.000465, e s      ,tcp,199.7.204.127,42282, ->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted

The events above also show the issue with country codes.  I'm noticing a few things...  The erroneous country codes are always a pair of letters found somewhere in the label, but I don't see any pattern as to which letters are used.  If I disable the iana label file in ralabel.conf, country codes display normally.  If a country code is found for both a source and destination, one of the country codes is always incorrect, but the country code is properly displayed in the label field:

1362869594.000036, e        ,udp,12.130.136.11,53, ->,69.154.227.43,53,1,70,INT,pu,US,saddr=US,public,DC1,mta-external:daddr=US
1362869594.000036, e        ,udp,12.130.136.12,53, ->,200.26.226.6,53,1,68,INT,pu,AN,saddr=US,public,DC1,mta-external:daddr=AN

In the first set of events, both country codes are incorrect (dest code not shown).  In the events above, the destination code is shown correctly.  I have noticed that if, as above, the 12.130.136.12 has its sco field displayed incorrectly, it will also have its dco field displayed incorrectly when it is the destination of a flow.

If there is no label for a source or destination, the country code for that host will be empty:

1362866671.000438, e        ,udp,74.125.17.82,42492,<->,192.168.30.41,53,4,1260,CON,,dn,daddr=internal,DC4,SP1--Apache,dns,ext-DC4

I enabled label support and radium and ran the same query as above, except with ra.  The issue with the country codes is the same as when I run the searches with ralabel.  Radium has a fairly significant memory leak when labels are enabled.

Let me know if you would like me to send you any binary flow files or my iana label file.

Thanks!

Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130310/24db9a14/attachment.html>