MaxMind GeoIP support in 3.0.8.2
Kevin Branch
kevin at branchnetconsulting.com
Sat Sep 29 11:56:36 EDT 2018
Hi Carter and David,
I just tried the following with the same ralabel.conf as before:
ra -r /argus/today/unt-01.arg -w - | ralabel -f /etc/ralabel.conf -r - -s
label:200 -N20
and saw this. Looks like I was using the wrong syntax which I am glad to
know better now, but the GeoIP lookups are still not happening:
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
"scity=:dcity="
I know the data file is good because it works with this:
# geoiplookup -f /usr/local/share/GeoIP/GeoIPCity.dat 207.48.48.19
GeoIP City Edition, Rev 1: US, MO, Missouri, Chesterfield, 63017,
38.650002, -90.533401, 609, 314
In case it helps, I rebuilt argus-clients with the .debug flag file present
and then ran this command limited to only a single argus record that I
independently confirmed has a srcip value known to geoiplookup
# ra -r /argus/today/unt-01.arg -N1 -w - | ralabel -f /etc/ralabel.conf -r
- -s label:50 -N200 -D3
ralabel[44570.40b753ff8e7f0000]: 15:52:57.950161 ArgusNewLabeler
(0x7f8eff3f7010, 0) returning 0x1b057c0
ralabel[44570.40b753ff8e7f0000]: 15:52:57.959656 RaLabelParseResourceFile
(/etc/ralabel.conf) returning 0
ralabel[44570.40b753ff8e7f0000]: 15:52:57.959730 ArgusReadConnection() read
16 bytes
ralabel[44570.40b753ff8e7f0000]: 15:52:57.959748 ArgusReadConnection() read
112 bytes
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961133 ArgusInitAddrtoname
(0x7f8eff3f7010, 0xa9fe6700, 0xffffff00)
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961145
ArgusParseInit(0x7f8eff3f7010 0x7f8eff386010
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961155
ArgusReadConnection(0xff386010, 1) returning 1
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961329 ArgusAddToRecordLabel
(0x7f8eff3f7010, 0x1b188c0, scity=:dcity=) returning 0
Label
scity=:dcity=
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961385
ArgusCloseInput(0xff386010) closing
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961408
ArgusCloseInput(0xff386010) done
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961415 main: ArgusReadFileStream
(-) done
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961426 main: reading files
completed
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961432 ArgusShutDown (0)
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961439 RaParseComplete (0)
returning
ralabel[44570.40b753ff8e7f0000]: 15:52:57.961444 RaParseComplete(caught
signal 0)
root at nsm.wycliffe.org:~/argus-clients-3.0.8# ra -r /argus/today/unt-01.arg
-N1 -w - | ralabel -f /etc/ralabel.conf -r - -s label:50 -N200 -D8
ralabel[4462.40472429a37f0000]: 15:53:24.374433 ArgusFree (0x26b2250)
ralabel[4462.40472429a37f0000]: 15:53:24.374461 ArgusFree (0x26b22b0)
ralabel[4462.40472429a37f0000]: 15:53:24.374469 ArgusFree (0x26b2310)
ralabel[4462.40472429a37f0000]: 15:53:24.374474 ArgusFree (0x26b2370)
ralabel[4462.40472429a37f0000]: 15:53:24.374479 ArgusFree (0x26b23d0)
ralabel[4462.40472429a37f0000]: 15:53:24.374484 ArgusFree (0x26b2430)
ralabel[4462.40472429a37f0000]: 15:53:24.374489 ArgusFree (0x26b2490)
ralabel[4462.40472429a37f0000]: 15:53:24.374494 ArgusFree (0x26b24f0)
ralabel[4462.40472429a37f0000]: 15:53:24.374499 ArgusFree (0x26b2550)
ralabel[4462.40472429a37f0000]: 15:53:24.374504 ArgusFree (0x26b25b0)
ralabel[4462.40472429a37f0000]: 15:53:24.374509 ArgusFree (0x26b2610)
ralabel[4462.40472429a37f0000]: 15:53:24.374533 ArgusCalloc (1, 80)
returning 0x26b2610
ralabel[4462.40472429a37f0000]: 15:53:24.374544 ArgusCalloc (1, 296)
returning 0x26b27c0
ralabel[4462.40472429a37f0000]: 15:53:24.374566 ArgusCalloc (1, 112)
returning 0x26b2950
ralabel[4462.40472429a37f0000]: 15:53:24.374577 ArgusCalloc (1, 80)
returning 0x26b25b0
ralabel[4462.40472429a37f0000]: 15:53:24.374595 ArgusNewQueue () returning
0x26b25b0
ralabel[4462.40472429a37f0000]: 15:53:24.374613 ArgusCalloc (65536, 8)
returning 0x2849a010
ralabel[4462.40472429a37f0000]: 15:53:24.374619 ArgusNewLabeler
(0x7fa329100010, 0) returning 0x26b27c0
ralabel[4462.40472429a37f0000]: 15:53:24.384352 RaLabelParseResourceFile
(/etc/ralabel.conf) returning 0
ralabel[4462.40472429a37f0000]: 15:53:24.384387 ArgusCalloc (1, 560)
returning 0x26b2250
ralabel[4462.40472429a37f0000]: 15:53:24.384411 ArgusCalloc (1, 112)
returning 0x26b2e80
ralabel[4462.40472429a37f0000]: 15:53:24.384418 ArgusCalloc (1, 80)
returning 0x26b2f00
ralabel[4462.40472429a37f0000]: 15:53:24.384424 ArgusNewQueue () returning
0x26b2f00
ralabel[4462.40472429a37f0000]: 15:53:24.384430 ArgusCalloc (1, 56)
returning 0x26b2f60
ralabel[4462.40472429a37f0000]: 15:53:24.384454 ArgusCalloc (65536, 8)
returning 0x27358010
ralabel[4462.40472429a37f0000]: 15:53:24.384461 ArgusNewHashTable (65536)
returning 0x26b2f60
ralabel[4462.40472429a37f0000]: 15:53:24.384483 ArgusReadConnection() read
16 bytes
ralabel[4462.40472429a37f0000]: 15:53:24.384498 ArgusReadConnection() read
112 bytes
ralabel[4462.40472429a37f0000]: 15:53:24.384514 ArgusCalloc (1, 4194304)
returning 0x26f57010
ralabel[4462.40472429a37f0000]: 15:53:24.384523 ArgusCalloc (1, 262144)
returning 0x26f16010
ralabel[4462.40472429a37f0000]: 15:53:24.385934 ArgusInitAddrtoname
(0x7fa329100010, 0xa9fe6700, 0xffffff00)
ralabel[4462.40472429a37f0000]: 15:53:24.385947
ArgusParseInit(0x7fa329100010 0x7fa32908f010
ralabel[4462.40472429a37f0000]: 15:53:24.385956
ArgusReadConnection(0x2908f010, 1) returning 1
ralabel[4462.40472429a37f0000]: 15:53:24.385969 ArgusReadFileStream()
starting
ralabel[4462.40472429a37f0000]: 15:53:24.385981 ArgusReadStreamSocket
(0x7fa32908f010) read 380 bytes
ralabel[4462.40472429a37f0000]: 15:53:24.386015 ArgusCalloc (1, 384)
returning 0x26c58c0
ralabel[4462.40472429a37f0000]: 15:53:24.386025 ArgusCalloc (1, 12)
returning 0x26c5a50
ralabel[4462.40472429a37f0000]: 15:53:24.386031 ArgusCalloc (1, 80)
returning 0x26c5a70
ralabel[4462.40472429a37f0000]: 15:53:24.386037 ArgusCalloc (1, 36)
returning 0x26c5ad0
ralabel[4462.40472429a37f0000]: 15:53:24.386042 ArgusCalloc (1, 52)
returning 0x26c5b00
ralabel[4462.40472429a37f0000]: 15:53:24.386047 ArgusCalloc (1, 80)
returning 0x26c5b40
ralabel[4462.40472429a37f0000]: 15:53:24.386053 ArgusCalloc (1, 120)
returning 0x26c5ba0
ralabel[4462.40472429a37f0000]: 15:53:24.386058 ArgusCalloc (1, 8)
returning 0x26c5c20
ralabel[4462.40472429a37f0000]: 15:53:24.386063 ArgusCalloc (1, 164)
returning 0x26c5c40
ralabel[4462.40472429a37f0000]: 15:53:24.386070 ArgusCalloc (1, 20)
returning 0x26c5cf0
ralabel[4462.40472429a37f0000]: 15:53:24.386075 ArgusCalloc (1, 20)
returning 0x26c5d10
ralabel[4462.40472429a37f0000]: 15:53:24.386080 ArgusCalloc (1, 12)
returning 0x26c5d30
ralabel[4462.40472429a37f0000]: 15:53:24.386092 ArgusCalloc (1, 16)
returning 0x26c5d50
ralabel[4462.40472429a37f0000]: 15:53:24.386131 ArgusCalloc (1, 12)
returning 0x26c5e80
ralabel[4462.40472429a37f0000]: 15:53:24.386234 ArgusAddToRecordLabel
(0x7fa329100010, 0x26c58c0, scity=:dcity=) returning 0
Label
scity=:dcity=
ralabel[4462.40472429a37f0000]: 15:53:24.386280 ArgusFree (0x26c5a50)
ralabel[4462.40472429a37f0000]: 15:53:24.386287 ArgusFree (0x26c5a70)
ralabel[4462.40472429a37f0000]: 15:53:24.386292 ArgusFree (0x26c5ad0)
ralabel[4462.40472429a37f0000]: 15:53:24.386297 ArgusFree (0x26c5b00)
ralabel[4462.40472429a37f0000]: 15:53:24.386301 ArgusFree (0x26c5b40)
ralabel[4462.40472429a37f0000]: 15:53:24.386304 ArgusFree (0x26c5ba0)
ralabel[4462.40472429a37f0000]: 15:53:24.386308 ArgusFree (0x26c5c20)
ralabel[4462.40472429a37f0000]: 15:53:24.386312 ArgusFree (0x26c5c40)
ralabel[4462.40472429a37f0000]: 15:53:24.386316 ArgusFree (0x26c5cf0)
ralabel[4462.40472429a37f0000]: 15:53:24.386322 ArgusFree (0x26c5d10)
ralabel[4462.40472429a37f0000]: 15:53:24.386359 ArgusFree (0x26c5d30)
ralabel[4462.40472429a37f0000]: 15:53:24.386369 ArgusFree (0x26c5e80)
ralabel[4462.40472429a37f0000]: 15:53:24.386377 ArgusFree (0x26c5d50)
ralabel[4462.40472429a37f0000]: 15:53:24.386384 ArgusFree (0x26c58c0)
ralabel[4462.40472429a37f0000]: 15:53:24.386392 RaProcessRecord
(0x2908f630) returning
ralabel[4462.40472429a37f0000]: 15:53:24.386400 RaScheduleRecord
(0x7fa329100010, 0x7fa32908f630) scheduled
ralabel[4462.40472429a37f0000]: 15:53:24.386408 ArgusHandleRecord
(0x7fa326f57010, 0x7fa329221800) returning 380
ralabel[4462.40472429a37f0000]: 15:53:24.386417 ArgusReadStreamSocket
(0x7fa32908f010) returning 0
ralabel[4462.40472429a37f0000]: 15:53:24.386432 ArgusReadStreamSocket
(0x7fa32908f010) read 0 bytes
ralabel[4462.40472429a37f0000]: 15:53:24.386439 ArgusReadStreamSocket
(0x7fa32908f010) returning 1
ralabel[4462.40472429a37f0000]: 15:53:24.386447 ArgusCloseInput(0x2908f010)
closing
ralabel[4462.40472429a37f0000]: 15:53:24.386463 ArgusFree (0x7fa326f57010)
ralabel[4462.40472429a37f0000]: 15:53:24.386473 ArgusFree (0x7fa326f16010)
ralabel[4462.40472429a37f0000]: 15:53:24.386492 ArgusCloseInput(0x2908f010)
done
ralabel[4462.40472429a37f0000]: 15:53:24.386500 ArgusReadFileStream()
returning
ralabel[4462.40472429a37f0000]: 15:53:24.386508 main: ArgusReadFileStream
(-) done
ralabel[4462.40472429a37f0000]: 15:53:24.386522 ArgusFree (0x7fa32908f010)
ralabel[4462.40472429a37f0000]: 15:53:24.386529 main: reading files
completed
ralabel[4462.40472429a37f0000]: 15:53:24.386536 ArgusShutDown (0)
ralabel[4462.40472429a37f0000]: 15:53:24.386546 ArgusFree (0x26b2190)
ralabel[4462.40472429a37f0000]: 15:53:24.386553 ArgusDeleteQueue
(0x26b2190) returning
ralabel[4462.40472429a37f0000]: 15:53:24.386561 ArgusFree (0x26b21f0)
ralabel[4462.40472429a37f0000]: 15:53:24.386569 ArgusDeleteQueue
(0x26b21f0) returning
ralabel[4462.40472429a37f0000]: 15:53:24.386599 RaParseComplete (0)
returning
ralabel[4462.40472429a37f0000]: 15:53:24.386608 RaParseComplete(caught
signal 0)
Thanks for looking at this,
Kevin
On Sat, Sep 29, 2018 at 11:18 AM Carter Bullard <carter at qosient.com> wrote:
> Hey Kevin,
> Any success ???
> Carter
>
> [image: QoSient] <http://qosient.com/>
> Carter Bullard <carter at qosient.com>• CTO
> 150 E 57th Street Suite 12D
> New York, New York 10022-2795
> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>
> On Sep 28, 2018, at 9:40 PM, David Edelman <dedelman at iname.com> wrote:
>
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Please try this:
>
> # ra -r /argus/today/unt-01.arg -w - | ralabel -f /etc/ralabel.conf -r -
> -s label:200 -N20
>
>
>
> From: Argus-info <
> argus-info-bounces+dedelman=iname.com at lists.andrew.cmu.edu> On Behalf Of
> Kevin Branch
> Sent: Friday, September 28, 2018 5:25 PM
> To: Carter Bullard <carter at qosient.com>
> Cc: Argus <argus-info at lists.andrew.cmu.edu>
> Subject: Re: [ARGUS] MaxMind GeoIP support in 3.0.8.2
>
>
>
> Looks like GeoIP is linked in already:
>
>
>
> # ldd `which ralabel`
> linux-vdso.so.1 => (0x00007ffddaf8a000)
> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f5fe67c9000)
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0
> (0x00007f5fe65ab000)
> libGeoIP.so.1 => /usr/lib/x86_64-linux-gnu/libGeoIP.so.1
> (0x00007f5fe637c000)
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f5fe5fb3000)
> /lib64/ld-linux-x86-64.so.2 (0x00007f5fe6acf000)
>
>
>
> I switched to just "lat,lon" like you suggested in ralabel.conf:
>
>
>
> # cat /etc/ralabel.conf
>
> RALABEL_ARIN_COUNTRY_CODES=no
>
>
>
> RALABEL_GEOIP_ASN=yes
>
> RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
>
>
>
> RALABEL_GEOIP_CITY="lat,lon"
>
> RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"
>
>
>
> But I don't see anything but ASNs getting added in:
>
>
>
> # ra -r /argus/today/unt-01.arg -w - | ralabel -f /etc/ralabel.conf -r -
> -s sas,das,scity,dcity,icity | head -n20
>
> sAS dAS
>
> 4323
>
> 4323
>
> 4323
>
> 22927 4323
>
> 6582 4323
>
> 7018 4323
>
> 4323 3
>
> 4323
>
> 4323 15169
>
> 4323
>
> 4323
>
> 4323 15169
>
> 4323 21928
>
> 2828 4323
>
>
>
> Thoughts?
>
> Kevin
>
>
>
> On Fri, Sep 28, 2018 at 2:36 PM Carter Bullard <carter at qosient.com <
> mailto:carter at qosient.com <carter at qosient.com>> > wrote:
>
> Hey Kevin,
>
> If you do a ./configure —help you’ll see all the options that
> ./configure supports. If you specify —with-GeoIP it wants the directory
> where the GeoIP library is.
>
> —with-GeoIP=DIR. compile with GeoIP in <dir>
>
>
>
> You can check if the library is bound to the client binaries with
> ldd …
>
> ldd `which ralabel`
>
>
>
> If you’re already bound, the city stuff should work …. I’m not
> familiar with “reg,cco” as GeoIP City objects off the top of my head. I’m
> familiar with “off,cont,lat,lon,region,city,cname”. Maybe try lat,lon as a
> test, as I use that all the time ..
>
>
>
> Carter
>
>
>
>
>
>
>
>
> On Sep 28, 2018, at 1:59 PM, Kevin Branch <
> kevin at branchnetconsulting.com <mailto:kevin at branchnetconsulting.com
> <kevin at branchnetconsulting.com>> > wrote:
>
>
>
> Hi Carter,
>
>
>
> I was just trying today to make ralabel on argus 3.0.8.2
> do GeoIP labeling with GeoIPCity.dat, but even though the file referred to
> in ralabel.conf for that purpose exists, no GeoIP labeling takes place.
> ASN lookups work great, but ralabel never adds any scity or dcity fields.
>
>
>
> My ralabel.conf:
>
>
>
> RALABEL_ARIN_COUNTRY_CODES=no
>
>
>
> RALABEL_GEOIP_ASN=yes
>
>
> RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
>
>
>
> RALABEL_GEOIP_CITY="reg,cco"
>
>
> RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIPCity.dat"
>
>
>
> No errors are thrown by ralabel. I thought maybe I needed
> to specially compile in GeoIP support like I read about here:
> https://qosient.com/argus/geolocation.shtml
>
> but when I run "./configure --with-GeoIP=yes" it throws
> this error:
>
> configure: WARNING: unrecognized options:
> --with-GeoIP
>
> However, I presume the fact that ASN lookups work means
> that MaxMind GeoIP support libraries are already installed, and presumably
> installed by default in the latest argus now.
>
>
>
> # ra -r /argus/today/unt-01.arg -w - | ralabel -f
> /etc/ralabel.conf -r - -s sas,das,scity,dcity,icity | head -n20
>
> sAS dAS
>
> 4323
>
> 4323
>
> 4323
>
> 22927 4323
>
> 6582 4323
>
> 7018 4323
>
> 4323 3
>
> 4323
>
> 4323 15169
>
> 4323
>
> 4323
>
> 4323 15169
>
> 4323 21928
>
> 2828 4323
>
>
>
> Any thoughts on what I am missing or how I might further
> debug this issue?
>
>
>
> Thanks!
>
> Kevin
>
>
> -----BEGIN PGP SIGNATURE-----
>
> iF0EARECAB0WIQQP+UHquEepll566aqXCCyZOY1FIQUCW67X6gAKCRCXCCyZOY1F
> IdLwAKDv6tPpe7OwLBEEfM1YTnAD8MxZCgCeOlGEEuEIUvqlyIMh8CPBWctmBd0=
> =p/tg
> -----END PGP SIGNATURE-----
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20180929/65544560/attachment.html>
More information about the argus
mailing list