A couple of labeling questions/issues...

[Craig Merchant]

Dave...

Here is the line in the delegated-iana-latest file for the 10.0.0.0/8 network:

delegated-iana-latest:iana|ZZ|ipv4|10.0.0.0|16777216|19940301|ietf

16777216 is 256x256x256, so that should be the correct number of IPs in a class A network.  Yet none of the flows for that network range have a "ZZ" country code.

When you talked about the data quality, were you referring to the data in the file or to Argus' ability to apply the proper country code based on that file?

Thanks.

Craig

From: Dave Edelman [mailto:dedelman at iname.com]
Sent: Saturday, March 09, 2013 4:24 PM
To: Craig Merchant; 'Argus'
Subject: RE: [ARGUS] A couple of labeling questions/issues...

Craig,

I have way too much experience with these files so I can probably provide an answer. The columns are from left to right:
RIR name
Country Code
Resource Type
First IP address in the range
Total number of IP addresses in the range
Date of last change
Status

There have always been issues with the data quality of these files that have nothing to do with Argus. There is also a convention that has caused more than a little confusion. The base address and address count mechanism can and frequently does lead to a situation where you cannot create a single CIDR to represent the allocation. You typically end up creating two or more CIDRs much like the pre-VLSM days where the mask could have discontiguous bits set.

Save yourself a bit of pain and look at the iana-address-file that should be in the same location as the delegated-ipv4-latest file and use that. The instructions are pretty clear and the format is carbon life form friendly.

--Dave



From: argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu<mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu> [mailto:argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu] On Behalf Of Craig Merchant
Sent: Saturday, March 09, 2013 6:15 PM
To: Argus (argus-info at lists.andrew.cmu.edu<mailto:argus-info at lists.andrew.cmu.edu>)
Subject: [ARGUS] A couple of labeling questions/issues...

Hey, Carter...  I've been testing the labeling functionality and trying to investigate further the country code issue I posted about earlier in the week.

I'm currently using the delegated-ipv4-latest file that is included with the 3.0.7.5 client.  I've noticed that Argus isn't finding a country code match for our public IP range.  I'd like to add those ranges to the delegated-ipv4-latest, but I'm not sure what the format is.  Is there any way to make the file recognize a subnet mask?  I notice that the entry for 10.0.0.0 isn't causing the country codes for my internal networks to show up as "ZZ".    What do the numbers in the second and third to last columns represent?

delegated-afrinic-latest:afrinic|ZA|ipv4|41.73.32.0|8192|20100112|allocated

Is it possible to use the MaxMind GeoIP database for country codes instead of the iana file?  I compiled the clients with GeoIP support.  The ralabel.conf file says the GEOIP labels will be saved as scity,dcity,icity.  I've tried adding those fields to my searches and nothing shows up.  I looked through rarc.print.all.conf to see if I could find a field in there that looked related to GeoIP data, but nothing popped out at me.  I'm using the same GeoLiteCity.dat file that we use on our Splunk server.  The GeoIP part of my ralabel.conf is:

RALABEL_GEOIP_CITY="saddr,daddr,inode:cco,cname"
RALABEL_GEOIP_CITY_FILE="/usr/local/argus/GeoIPCity.dat"

I've tried using the -M label="regex" option, but the regex never seems to match.

ralabel -S 10.230.174.40:561 -f /usr/local/argus/ralabel.conf -n -u -c "," -s +sco,+dco,+label:200 -M label="blacklisted", for example, doesn't match the following events:

1362868911.000236, e s      ,tcp,199.7.204.127,39935, ->,82.98.86.167,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted
1362868911.000282, e s      ,tcp,199.7.204.127,56477, ->,82.98.86.161,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted
1362868912.000211, e s      ,tcp,199.7.202.248,44177, ->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted
1362868913.000465, e s      ,tcp,199.7.204.127,42282, ->,82.98.86.171,25,2,148,REQ,pu,bl,saddr=public,DC2,Outbound:daddr=DE,blacklisted

The events above also show the issue with country codes.  I'm noticing a few things...  The erroneous country codes are always a pair of letters found somewhere in the label, but I don't see any pattern as to which letters are used.  If I disable the iana label file in ralabel.conf, country codes display normally.  If a country code is found for both a source and destination, one of the country codes is always incorrect, but the country code is properly displayed in the label field:

1362869594.000036, e        ,udp,12.130.136.11,53, ->,69.154.227.43,53,1,70,INT,pu,US,saddr=US,public,DC1,mta-external:daddr=US
1362869594.000036, e        ,udp,12.130.136.12,53, ->,200.26.226.6,53,1,68,INT,pu,AN,saddr=US,public,DC1,mta-external:daddr=AN

In the first set of events, both country codes are incorrect (dest code not shown).  In the events above, the destination code is shown correctly.  I have noticed that if, as above, the 12.130.136.12 has its sco field displayed incorrectly, it will also have its dco field displayed incorrectly when it is the destination of a flow.

If there is no label for a source or destination, the country code for that host will be empty:

1362866671.000438, e        ,udp,74.125.17.82,42492,<->,192.168.30.41,53,4,1260,CON,,dn,daddr=internal,DC4,SP1--Apache,dns,ext-DC4

I enabled label support and radium and ran the same query as above, except with ra.  The issue with the country codes is the same as when I run the searches with ralabel.  Radium has a fairly significant memory leak when labels are enabled.

Let me know if you would like me to send you any binary flow files or my iana label file.

Thanks!

Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130310/38d98d66/attachment.html>