Why sas das feature in rasqlinsert doesnot work?
Carter Bullard
carter at qosient.com
Wed Jul 24 10:40:45 EDT 2013
If you were using the supplied rarc file, then your output would look like this:
StartTime Dur SrcAddr DstAddr Label
10:21:07.266905 0.000000 172.20.11.172 172.20.15.255
10:21:07.309697 0.000000 172.20.3.142 172.20.15.255
10:21:07.313182 0.000000 172.20.3.201 172.20.15.108
10:21:07.314350 0.002182 172.20.3.201 172.20.15.255
10:21:07.341980 0.000000 172.20.0.201 224.0.0.251
Because the supplied rarc file has this entry in it.
# Most ra* clients are designed to print argus records out in ASCII,
# with each client supporting its own output formats. For ra() like
# clients, this variable will generate column headers as labels.
# The number is the number of lines between repeated header output.
# Setting this value to zero (0) will cause the labels to be printed
# once. If you don't want labels, then comment this line out or
# delete it.
#
#
RA_PRINT_LABELS=0
It is good to see that your client programs have linked the GeoIP libraries.
> root at debian:/home/star# ldd /usr/local/bin/ralabel
> [snip]
> libGeoIP.so.1 => /usr/local/lib/libGeoIP.so.1 (0x00007f362d263000)
> [snip]
You need to ./configure and compile your client programs with debug support.
In your client distribution's root directory, do this:
% touch .debug .devel
% ./configure
% make clean
% make
Now when you run a client program with a "-D n" option, you'll see a
lot of debugging information.
osiris:~ carter$ ra -S localhost -s stime dur saddr daddr das sas label -D2 -N5
ra[2419.80b15572ff7f0000]: 10:30:31.050569 main: reading files completed
ra[2419.80b15572ff7f0000]: 10:30:31.050640 Trying ::1 port 561 Expecting Argus records
Jul 24 10:30:31 osiris.local ra[2419] <Warning>: 10:30:31.050769 connect to ::1:561 failed 'Connection refused'
ra[2419.80b15572ff7f0000]: 10:30:31.051120 Trying 127.0.0.1 port 561 Expecting Argus records
ra[2419.80b15572ff7f0000]: 10:30:31.051328 connected
ra[2419.80b15572ff7f0000]: 10:30:31.051424 ArgusGetServerSocket (0x105d72000) returning 5
ra[2419.80b15572ff7f0000]: 10:30:31.054161 ArgusReadConnection() read 16 bytes
ra[2419.80b15572ff7f0000]: 10:30:31.054863 ArgusInitAddrtoname (0x105c4e000, 0x0, 0x0)
ra[2419.80b15572ff7f0000]: 10:30:31.054879 ArgusParseInit(0x105c4e000 0x105d72000
StartTime Dur SrcAddr DstAddr dAS sAS Label
12:12:01.10204* 109032166* 0 0
10:30:29.993535 0.000000 172.20.6.126 172.20.15.255
10:30:30.072218 0.771288 172.20.15.141 172.20.15.255
10:30:30.222117 0.750039 172.20.12.109 172.20.15.255
10:30:30.315995 0.000000 fe80::5a55:caff:f* ff02::1
10:30:30.329070 0.000000 172.20.2.202 172.20.15.255
ra[2419.80b15572ff7f0000]: 10:30:31.528874 ArgusShutDown (0)
ra[2419.80b15572ff7f0000]: 10:30:31.528928 RaParseComplete(caught signal 0)
ra[2419.80b15572ff7f0000]: 10:30:31.528958 RaParseComplete(caught signal 0)
ra[2419.80b15572ff7f0000]: 10:30:31.528979 RaParseComplete(caught signal 0)
ra[2419.80b15572ff7f0000]: 10:30:31.529000 ArgusShutDown (0)
Run with D as high as 15.
You are showing us output of data that only has private and multicast
addresses, which will not generate labels, based on your ralabel.conf
file. When I run ralabel() here, most of the records don't have any labels.
Do you have any examples where any of the IP addresses are
public addresses?
Carter
On Jul 23, 2013, at 3:34 PM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
> I have coped the rarc file already.
> the labels are shown,
>
> ralabel -f ralabel.conf -r /usr/kv2.argus -s stime dur saddr daddr label:64 -D 12
>
> stime dur saddr daddr
>
> 25054 0.000000 fe80::e8ac:92cc:8* ff02::1:3
> 14:06:43.825181 0.000000 192.168.2.1 224.0.0.252
> 14:06:43.902277 0.000000 192.168.2.1 0.0.0.1
> 14:06:49.800547 4.600892 192.168.2.1 192.168.2.2
>
> Yes I already installed the library GeoIP
> I compiled the argus client with this parameters:
>
> ./configure --with-libft=/usr/local/flow-tools/lib --with-GeoIP=yes
>
> Did you re-configure and recompile your client code after you installed the library ?no , what should I do?
> Did you demonstrate to yourself, that the configure found the GeoIP library successfully ? the all config files have right directory of GeoIP
> Did you demonstrate to yourself that the linker uses the -lGeoIP option ?? I didn't get your question, I do all thing that have been said at site
> What is the output of " ldd /usr/local/bin/ralabel " ?? Is there mention of the GeoIP library ?
> root at debian:/home/star# ldd /usr/local/bin/ralabel
> linux-vdso.so.1 => (0x00007fff86c84000)
> libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f362d6ae000)
> libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f362d492000)
> libGeoIP.so.1 => /usr/local/lib/libGeoIP.so.1 (0x00007f362d263000)
> libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f362d04c000)
> libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f362ccc2000)
> /lib64/ld-linux-x86-64.so.2 (0x00007f362d946000)
>
> Please let me know where is my work was wrong?
> Thanks in advance
>
>
>
> On Tue, Jul 23, 2013 at 7:59 PM, Carter Bullard <carter at qosient.com> wrote:
> No, no no no no !!!!
> Have you not done even the simplest of things to get started?
> Why aren't you getting column labels in your printout?
> Don't you have a .rarc file installed? ( cp ./support/Config/rarc ~/.rarc )
>
> Where is the label ?
> Why aren't you showing what command you actually ran ??
>
> Have you installed the GeoIP C language libraries ??
> Did you read the descriptions of how to use GeoIP from the web site?
> Did you re-configure and recompile your client code after you installed the library ?
> Did you demonstrate to yourself, that the configure found the GeoIP library successfully ?
> Did you demonstrate to yourself that the linker uses the -lGeoIP option ??
> What is the output of " ldd /usr/local/bin/ralabel " ?? Is there mention of the GeoIP library ?
>
> Did you run with the " -D 12 " option ????
>
> Please, do a bit more than simply state " it doesn't work ".
>
> Carter
>
> On Jul 23, 2013, at 11:12 AM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
>
>> When I run this command gives me like:
>>
>> 13:23:44.038524 0.000000 192.168.2.158 46.203.170.143
>> 13:23:45.425982 2.959139 192.168.2.157 173.194.71.16
>> 13:23:48.721902 0.000000 192.168.2.160 99.181.217.139
>> 13:23:48.894962 0.000401 192.168.2.158 192.168.2.254
>> 13:23:48.895565 0.000318 192.168.2.158 192.168.2.254
>>
>> all commands is good but the sas, das filed just don't work.
>>
>>
>> On Tue, Jul 23, 2013 at 7:36 PM, Carter Bullard <carter at qosient.com> wrote:
>> You are not providing the information requested.
>> Your ralabel.conf file has a large number of operations. Geolocation data, AS labeling,
>> and flow labeling.
>>
>> What is the label that is being generated by ralabel() when you run it.
>>
>> ralabel -f ralabel.conf -r argus.data -s stime dur saddr daddr label:64
>>
>> If you don't think ralabel is doing the right thing, run it with the "-D 12" option,
>> assuming you have turned on debugging, to see what it thinks is going on.
>>
>> Carter
>>
>> On Jul 23, 2013, at 11:02 AM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
>>
>>> Thank you very much indeed Carter.
>>> I test the simple command like ralabel -f ralabel.conf -r argus.data -s stime dur saddr sas daddr das, at that time it gives just saddr, daddr, stime and dur.
>>>
>>> Yes I download the file "GeoIPASNum.dat" and have coped that directory.
>>> I don't know where is the problem?
>>>
>>>
>>> On Tue, Jul 23, 2013 at 7:18 PM, Carter Bullard <carter at qosient.com> wrote:
>>> Please. Use the available tools to demonstrate to yourself that you can generate useful data.
>>> Then use programs like rasqlinsert() to push the data into a database table.
>>>
>>> argus -r packet.data -w argus.data
>>>
>>> now you can use the tools to educate yourself on how the tools work.
>>>
>>> ralabel -f ralabel.conf -r argus.data -s stime dur saddr sas daddr das
>>>
>>> Set one method in ralabel() at a time, until you understand how the tools work.
>>> Your ralabel.conf file references the file /usr/local/share/GeoIP/GeoIPASNum.dat.
>>> Does it exist ?
>>>
>>> What do your labels look like?
>>>
>>> ralabel -f ralabel.conf -r argus.data -s stime dur saddr daddr label:64
>>>
>>>
>>> Carter
>>>
>>>
>>> On Jul 23, 2013, at 10:39 AM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
>>>
>>>> At first, I appreciate for your time and apologize for bothering you with my questions. I really need it for my works.
>>>> I'm so sorry, but this post for yesterday, not a few weeks ago,
>>>> I have red several time the manual of commands, but I think somewhere I make mistake.
>>>>
>>>> This is peice of my database. The feature is not work
>>>>
>>>> stime ltime dur srcid flgs proto saddr sport dir daddr dport pkts bytes state spkts dpkts sbytes dbytes das sas record
>>>> 1.37E+09 1.37E+09 103.668 0.0.0.0 e sD tcp 192.168.2.159 1066 -> 74.125.143.16 465 23616 12965059 FIN 8022 15594 12026652 938407 0 0 ...
>>>> 1.37E+09 1.37E+09 71.71558 0.0.0.0 e dS tcp 74.125.143.16 465 <?> 192.168.2.157 1047 12027 7356979 FIN 7368 4659 442223 6914756 0 0 ...
>>>> 1.37E+09 1.37E+09 49.26319 0.0.0.0 e dS tcp 74.125.143.16 465 <?> 192.168.2.160 1043 7924 4842419 FIN 4869 3055 292283 4550136 0 0 ...
>>>> 1.37E+09 1.37E+09 38.95642 0.0.0.0 e dS tcp 74.125.143.16 465 <?> 192.168.2.156 1047 6129 3729166 FIN 3768 2361 226225 3502941 0 0 ...
>>>>
>>>> Again I thanks for your helps, and wish the best for you.
>>>>
>>>>
>>>>
>>>> On Tue, Jul 23, 2013 at 6:53 PM, Carter Bullard <carter at qosient.com> wrote:
>>>> I apologize, but why is it that you show up only a few weeks ago, and now everything is urgent ?
>>>> This is a developers mailing list, not a " I can't read the manual " list.
>>>>
>>>> Please try to learn how to use the tools before bombarding the list with your requests for training.
>>>>
>>>> You did not show what IP addresses have 0 AS numbers, I will presume that the feature works.
>>>>
>>>> Carter
>>>>
>>>>
>>>> On Jul 23, 2013, at 10:06 AM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
>>>>
>>>>> Is there any Idea for solving it???
>>>>> I need urgently
>>>>>
>>>>> Thanks in advance
>>>>>
>>>>>
>>>>> On Tue, Jul 23, 2013 at 5:21 PM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
>>>>> My ralabel.conf file the same below: and I copy it to /etc/ and /usr/local/argus/ directories.
>>>>> The all of Ip address are 0,
>>>>>
>>>>> #
>>>>> # Argus Client Software
>>>>> # Copyright (c) 2000-2013 QoSient, LLC
>>>>> # All rights reserved.
>>>>> #
>>>>> #
>>>>> # RaLabel Configuration
>>>>> #
>>>>> # Carter Bullard
>>>>> # QoSient, LLC
>>>>> #
>>>>> # This configuration is a ralabel(1) configuration file.
>>>>> #
>>>>> # The concept is to provide a number of labeling strategies
>>>>> # with configuration capabilities for each of the labelers.
>>>>> # This allows the user to specify the order of the labeling,
>>>>> # which is provided to support hierarchical labeling.
>>>>> #
>>>>> # Here is a valid and simple configuration file. It doesn't do
>>>>> # anything in particular, but it is one that is used at some sites.
>>>>> #
>>>>>
>>>>> # Supported Labeling Strategies
>>>>> # Addresss Based Classification
>>>>> # Address based classifications involve building a patricia tree
>>>>> # that we can hang labels against. The strategy is to order the
>>>>> # address label configuration files, to develop a hierarchical
>>>>> # label scheme.
>>>>> #
>>>>>
>>>>> # IANA IPv4 and IPv6 Address Classification Labeling
>>>>> #
>>>>> # The type of IP network address can be used by many analysis
>>>>> # programs to make decisions. While IANA standard classifications
>>>>> # don't change, this type of classification should be extendable
>>>>> # to allow local sites to provide additional labeling capabilities.
>>>>>
>>>>> #RALABEL_IANA_ADDRESS=yes
>>>>> #RALABEL_IANA_ADDRESS_FILE="/usr/local/argus/iana-address-file"
>>>>>
>>>>>
>>>>> # Addresss Based Country Code Classification
>>>>> # Address based country code classification leverages the feature
>>>>> # where ra* clients cant print country codes for the IP addresses
>>>>> # that are in a flow record. Country codes are generated from the ARIN
>>>>> # delegated address space files. Specify the location of your
>>>>> # DELEGATED_IP file here, or in your .rarc file (which is default).
>>>>> #
>>>>> # Unlike the GeoIP based country code labeling, these codes can be sorted
>>>>> # filtered and aggregated, so if you want to do that type of operations
>>>>> # with country codes, enable this feature here.
>>>>> #
>>>>>
>>>>> #RALABEL_ARIN_COUNTRY_CODES=yes
>>>>> #RA_DELEGATED_IP="/usr/local/argus/delegated-ipv4-latest"
>>>>>
>>>>>
>>>>> # BIND Based Classification
>>>>> # BIND services provide address to name translations, and these
>>>>> # reverse lookup strategies can provide FQDN labels, or domain
>>>>> # labels that can be added to flow. The IP addresses that can be
>>>>> # 'labeled' are the saddr, daddr, or inode. Keywords "yes" and "all"
>>>>> # are synonomous and result in labeling all three IP addresses.
>>>>> #
>>>>> # Use this strategy to provide transient semantic enhancement based
>>>>> # on ip address values.
>>>>> #
>>>>>
>>>>> #RALABEL_BIND_NAME="all"
>>>>>
>>>>> #
>>>>> # When labelers provide names, they can use blocking or non-blocking
>>>>> # resolvers to perform the lookups. Blocking, the default, will cause
>>>>> # the labeler to wait for resolutions to return. This ensures that the
>>>>> # label will have the best answer in every flow record process, however
>>>>> # blocking resolvers can cause performance issues. Non-blocking will
>>>>> # queue lookups and establish its name resolution cache, in a lazy
>>>>> # manner.
>>>>>
>>>>> #RALABEL_BIND_NON_BLOCKING="no""
>>>>>
>>>>> #
>>>>> # When labelers provide names, they can prit the FQDN, the host portion
>>>>> # or just the domain name, depending on your uses of the name label.
>>>>> #
>>>>>
>>>>> #RALABEL_PRINT_DOMAINONLY="no"
>>>>> #RALABEL_PRINT_LOCALONLY="no"
>>>>>
>>>>> #
>>>>> # All name resolutions are cached, to improve performance. This provides
>>>>> # the best performance, however, for long lived labeling daemons, a timeout
>>>>> # or TTL, can be placed on the name table, so that the labeler will
>>>>> # periodically requery for resolutions.
>>>>> #
>>>>> # The default is -1, which disables cache timeouts.
>>>>> # Zero (0) will turn off any caching and will have a performance impact.
>>>>>
>>>>> #RALABEL_DNS_NAME_CACHE_TIMEOUT=-1
>>>>>
>>>>>
>>>>>
>>>>> # Port Based Classification
>>>>> # Port based classifications involves simple assignment of a text
>>>>> # label to a specific port number. While IANA standard classifications
>>>>> # are supported throught the Unix /etc/services file assignments,
>>>>> # and the basic "src port" and "dst port" ra* filter schemes,
>>>>> # this scheme is used to enhance/modify that labeling strategy.
>>>>> # The text associated with a port number is placed in the metadata
>>>>> # label field, and is searched using the regular expression searching
>>>>> # strategies that are available to label matching.
>>>>> #
>>>>> # Use this strategy to provide transient semantic enhancement based
>>>>> # on port values.
>>>>> #
>>>>>
>>>>> #RALABEL_IANA_PORT=yes
>>>>> #RALABEL_IANA_PORT_FILE="/usr/local/argus/iana-port-numbers"
>>>>>
>>>>>
>>>>> # Flow Filter Based Classification
>>>>> # Flow filter based classification uses the standard flow
>>>>> # filter strategies to provide a general purpose labeling scheme.
>>>>> # The concept is similar to racluster()'s fall through matching
>>>>> # scheme. Fall through the list of filters, if it matches, add the
>>>>> # label. If you want to continue through the list, once there is
>>>>> # a match, add a "cont" to the end of the matching rule.
>>>>> #
>>>>>
>>>>> RALABEL_ARGUS_FLOW=yes
>>>>> RALABEL_ARGUS_FLOW_FILE="/usr/local/argus/argus-flow-file"
>>>>>
>>>>>
>>>>> # GeoIP Based Labeling
>>>>> # The labeling features can use the databases provided by MaxMind
>>>>> # using the GeoIP LGPL libraries. If your code was configured to use
>>>>> # these libraries, then enable the features here.
>>>>> #
>>>>> # GeoIP provides a lot of support for geo-location, configure support
>>>>> # by enabling a feature and providing the appropriate binary data files.
>>>>> # ASN reporting is done from a separate set of data files, obtained from
>>>>> # MaxMind.com, and so enabling this feature is independent of the
>>>>> # traditional city data available.
>>>>> #
>>>>>
>>>>> RALABEL_GEOIP_ASN=yes
>>>>> RALABEL_GEOIP_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNum.dat"
>>>>> RALABEL_GEOIP_V6_ASN_FILE="/usr/local/share/GeoIP/GeoIPASNumv6.dat"
>>>>>
>>>>> #
>>>>> # Data for city relevant data is enabled through enabling and configuring
>>>>> # the city database support. The types of data available are:
>>>>> # country_code, country_code3, country_name, region, city, postal_code,
>>>>> # latitude, longitude, metro_code, area_code and continent_code.
>>>>> # time_offset is also available.
>>>>> #
>>>>> # The concept is that you should be able to add semantics for any
>>>>> # IP address that is in the argus record. Support addresses are:
>>>>> #
>>>>> # saddr, daddr, inode
>>>>> #
>>>>> # The labels provided will be tagged as:
>>>>> # scity, dcity, icity
>>>>> #
>>>>> # To configure what you want to have placed in the label, use the list of
>>>>> # objects, in whatever order you like, as the RALABEL_GEOPIP_CITY string
>>>>> # using these keywords:
>>>>> # cco - country_code
>>>>> # cco3 - country_code3
>>>>> # cname - country_name
>>>>> # reg - region
>>>>> # city - city
>>>>> # pcode - postal_code
>>>>> # lat - latitude
>>>>> # long - longitude
>>>>> # metro - metro_code
>>>>> # area - area_code
>>>>> # cont - continent_code
>>>>> # off - GMT time offset
>>>>> #
>>>>> # Working examples could be:
>>>>> # RALABEL_GEOIP_CITY="saddr,daddr:lat/lon"
>>>>> # RALABEL_GEOIP_CITY="*:city,region,cname,lat,lon"
>>>>> #
>>>>> RALABEL_GEOIP_CITY="saddr,daddr,inode:off,cont,lat,lon"
>>>>> RALABEL_GEOIP_CITY_FILE="/usr/local/share/GeoIP/GeoIP.dat"
>>>>> RALABEL_GEOIP_V6_CITY_FILE="/usr/local/share/GeoIP/GeoIPv6.dat"
>>>>>
>>>>>
>>>>> On Tue, Jul 23, 2013 at 5:03 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>> what are the contents of your ralabel.conf file, and what addresses are reporting 0?
>>>>> simply stating that something is not working is very impolite.
>>>>>
>>>>> Carter
>>>>>
>>>>> On Jul 23, 2013, at 8:28 AM, Rahimeh Khodadadi <rahimeh.khodadadi at gmail.com> wrote:
>>>>>
>>>>>> I solve the problem by this command, but still the value of sas, dasare zero?????
>>>>>>
>>>>>> argus -r pcaped.pcap -F /dev/null -w - | ralabel -f ralabel.conf -r - -w - -s +sas +das | rasqlinsert -r - -w mysql://root@localhost/argus/a -s stime ltime dur srcid flgs proto saddr sport dir daddr dport pkts bytes state spkts dpkts sbytes dbytes das sas
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, Jul 23, 2013 at 10:59 AM, Mike Iglesias <iglesias at uci.edu> wrote:
>>>>>> On 07/22/2013 10:54 PM, Rahimeh Khodadadi wrote:
>>>>>> > Thank you very much indeed Matt, but when I run the command gives such a erorr:
>>>>>>
>>>>>> If you're not using the latest code that Carter put up today, try that and see
>>>>>> if it fixes this error. http://qosient.com/argus/dev/
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Mike Iglesias Email: iglesias at uci.edu
>>>>>> University of California, Irvine phone: 949-824-6926
>>>>>> Office of Information Technology FAX: 949-824-2270
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> With Best Regards
>>>>>> Rahimeh Khodadadi
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> With Best Regards
>>>>> Rahimeh Khodadadi
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> With Best Regards
>>>>> Rahimeh Khodadadi
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> With Best Regards
>>>> Rahimeh Khodadadi
>>>>
>>>
>>>
>>>
>>>
>>> --
>>> With Best Regards
>>> Rahimeh Khodadadi
>>>
>>
>>
>>
>>
>> --
>> With Best Regards
>> Rahimeh Khodadadi
>>
>
>
>
>
> --
> With Best Regards
> Rahimeh Khodadadi
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130724/49133581/attachment.bin>
More information about the argus
mailing list