ralabel and geolocation data

[Will Urbanski]

Carter, thanks! What a difference using ragetcountrycodes.sh makes. I
believe the ARIN file I was generating was in an incorrect format and that's
why ralabel was spinning it's wheels. This is fixed now, and yes, you're
right, the ARIN files are much faster than MaxMind.

Thanks again,

Will

On Fri, May 6, 2011 at 9:42 AM, Carter Bullard <carter at qosient.com> wrote:

> Hey Will,
> There is a script in ./support/Config called ragetcountrycodes.sh.  This
> uses 'wget' to grab all the address allocation files from the registries,
> and cat them together to generate a single file for ipv4 addresses.  The big
> file, ./support/Config/delegated-ipv4-latest is generated using this script.
>   I've almost finished the support for ipv6, so that should be here soon.
> When I use these types of files, they perform very well, at least for me.
>  This uses the same logic that ra() uses to print the country codes, and it
> is faster than the MaxMind interface, for me at least.
>
> If your ralabel() is taking so long, it is possible that you are labeling
> records with the DNS names for the IP addresses?
> Is "RALABEL_BIND_NAME='all'" turned on in your ralabel.conf file?  Just a
> guess.


> Send your ralabel.conf file, and I'll try to debug.
>
> To get ralabel() to go fast with ARIN files, use the delegated-ipv4-latest
> file in the distribution, create a ralabel.conf file that has only this in
> it:
>
> RALABEL_ARIN_COUNTRY_CODES=yes
> RA_DELEGATED_IP="/path/to/your/delegated-ipv4-latest"
>
> and then run this:
>   ralabel -f ralabel.conf -r argus.data.file -w argus.data.file.co
>
> or something like that.  It should go pretty fast?
>
> Carter
>
> On May 6, 2011, at 9:21 AM, Will Urbanski wrote:
>
> > I have been experimenting using ralabel to add geolocation data to argus
> feeds. I have been using the ARIN delegations file and the MaxMind GeoIP
> library. The ARIN delegations file is nice because it adds the country code
> directly the argus file, but the MaxMind GeoIP library has much more detail
> regarding the location.
> >
> > I have noticed a couple things about the ARIN file. First, the ARIN,
> LACNIC, RIPENCC, APNIC, etc all have a very similar delegation file format.
> I have tried cating all these delegation files together and using this with
> ralabel but it is VERY slow (takes over 18 hours to process a 147MB file,
> unsuccessfully). Next, you can get similar country code data from MaxMind,
> and the MaxMind API is extremely fast, but unfortunately ralabel appends
> this data to the scity= and dcity= tags in the label and does not use the
> internal sco and dco fields.
> >
> > My questions are:
> >
> > 1) Are there any efficient ways to put county code information in a file
> with ralabel using a combination of all the delegations files (ARIN, LACNIC,
> RIPENCC, APNIC, etc)
> >
> > 2) Are there any ways to put county code information (sco, dco) in a file
> with ralabel using the MaxMind GeoIP database?
> >
> > Thanks in advance,
> >
> > -Will
> >
>
>


-- 
Will Urbanski
Information Technology Security Office and Lab
Virginia Polytechnic Institute and State University
(540) 231-9230
will dot urbanski at vt dot edu
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110506/c650f14e/attachment.html>