rasqlinsert memory consumption

Mon Sep 27 19:05:35 EDT 2010

Hey Javier,
I've been pondering your problem, and I'm making some changes to the way rasqlinsert() processes files.
I've been working under the assumption that files are different from reading from a continuous stream of
argus records, but your files are so large, that I shouldn't make that assumption.  

I should have something to test later today/tomorrow.
Sorry for the huge delay.

Carter

On Aug 17, 2010, at 11:35 AM, Javier Almillategui wrote:

> I'm using a single file.
> 
> Javier Almillategui
> Center for Secure Information Systems
> George Mason University
> Mobile: (703)309-2060
> Email: jalmilla at gmu.edu
> 
> On Aug 17, 2010, at 9:50, carter at qosient.com wrote:
> 
>> Hey Javier,
>> OK, and that is for a single file or multiple files in a run?
>> Carter
>> Sent from my Verizon Wireless BlackBerry
>> 
>> From: Javier Almillategui <jalmilla at gmu.edu>
>> Date: Tue, 17 Aug 2010 08:41:32 -0500
>> To: carter at qosient.com<carter at qosient.com>
>> Cc: Argus<argus-info at lists.andrew.cmu.edu>
>> Subject: Re: [ARGUS] rasqlinsert memory consumption
>> 
>> Hi Carter,
>> 
>> I have been using argus files of 100+ GB in size.  With smaller files, the memory consumption is not noted, because it doesn't run long enough.
>> 
>> Best, 
>> 
>> Javier Almillategui
>> 
>> Cell:(703)309-2060
>> E-mail: jalmillategui at me.com
>> Sent from my iPhone
>> 
>> On Aug 17, 2010, at 6:11, carter at qosient.com wrote:
>> 
>>> Hey Javier,
>>> Does this happen on any file? Particularly large files?
>>> I'll look at this tonight, want to make sure I have all the information.
>>> 
>>> Carter 
>>> Sent from my Verizon Wireless BlackBerry
>>> 
>>> From: Javier Almillategui <jalmilla at gmu.edu>
>>> Date: Sun, 15 Aug 2010 12:51:26 -0500
>>> To: Carter Bullard<carter at qosient.com>
>>> Cc: argus-info at lists.andrew.cmu.edu<argus-info at lists.andrew.cmu.edu>
>>> Subject: Re: [ARGUS] rasqlinsert memory consumption
>>> 
>>> Hi Carter,
>>> 
>>> I tested the development version and it shows the same behavior as 3.0.2
>>> 
>>> Best,
>>> 
>>> Javier Almillategui
>>> 
>>> Cell:(703)309-2060
>>> E-mail: jalmillategui at me.com
>>> 
>>> On Aug 13, 2010, at 7:54, Carter Bullard <carter at qosient.com> wrote:
>>> 
>>>> Hey Javier,
>>>> The odd release numbers are development versions, so there will be a few compile warnings, so no worries.
>>>> However, the buffer overflow warnings you highlighted seem to be bogus.  The first one is in a system
>>>> include file, which of course could happen, but in this case isn't, and the one in rasqlinsert.c is obviously
>>>> an error.  We're using snprintf and the number of bytes copied (BUFSIZ, which should be 1K) is 1/4 the size
>>>> of the array (4K).  These buffers are the full pathname of the archive variable, which we're not using at this
>>>> time.
>>>> 
>>>> Go ahead and run the programs to see if all is better.
>>>> 
>>>> Carter
>>>> 
>>>> 
>>>> On Aug 13, 2010, at 6:52 AM, Javier Almillategui wrote:
>>>> 
>>>>> Hi Carter,
>>>>> 
>>>>> I have compiled version 3.0.3 on my system and got some interesting warnings regarding buffer overflows:
>>>>> 
>>>>> making in ./common
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/common'
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_main.c
>>>>> rm -f ../lib/argus_parse.a; ar qc ../lib/argus_parse.a argus_main.o
>>>>> ranlib ../lib/argus_parse.a
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_code.c
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_filter.c
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_util.c
>>>>> ./argus_util.c: In function ‘ArgusParseResourceFile’:
>>>>> ./argus_util.c:1680: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result
>>>>> ./argus_util.c: In function ‘ArgusLog’:
>>>>> ./argus_util.c:18124: warning: format not a string literal and no format arguments
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_auth.c
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_parser.c
>>>>> flex -Pargus_ -t scanner.l > $$.scanner.c; mv $$.scanner.c scanner.c
>>>>> bison -y -p argus_ -d grammar.y
>>>>> mv y.tab.c grammar.c
>>>>> mv y.tab.h tokdefs.h
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./scanner.c
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -Dyylval=argus_lval -c grammar.c
>>>>> sed -e 's/.*/char version[] = "&";/' ./../VERSION > version.c
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./version.c
>>>>> rm -f ../lib/argus_common.a; ar qc ../lib/argus_common.a argus_code.o argus_filter.o argus_util.o argus_auth.o argus_parser.o scanner.o grammar.o version.o
>>>>> ranlib ../lib/argus_common.a
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_client.c
>>>>> ./argus_client.c: In function ‘ArgusParseAggregator’:
>>>>> ./argus_client.c:11758: warning: ignoring return value of ‘strtol’, declared with attribute warn_unused_result
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_label.c
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_grep.c
>>>>> rm -f ../lib/argus_client.a; ar qc ../lib/argus_client.a argus_client.o argus_label.o argus_grep.o
>>>>> ranlib ../lib/argus_client.a
>>>>> gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_event.c
>>>>> rm -f ../lib/argus_event.a; ar qc ../lib/argus_event.a argus_event.o
>>>>> ranlib ../lib/argus_event.a
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/common'
>>>>> making in ./include
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/include'
>>>>> make[1]: Nothing to be done for `all'.
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/include'
>>>>> making in ./clients
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/clients'
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ra.c
>>>>> gcc -O3 -o ../bin/ra ra.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./racount.c
>>>>> gcc -O3 -o ../bin/racount racount.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rasort.c
>>>>> gcc -O3 -o ../bin/rasort rasort.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rasplit.c
>>>>> gcc -O3 -o ../bin/rasplit rasplit.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rabins.c
>>>>> gcc -O3 -o ../bin/rabins rabins.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./racluster.c
>>>>> gcc -O3 -o ../bin/racluster racluster.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./raconvert.c
>>>>> gcc -O3 -o ../bin/raconvert raconvert.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rastrip.c
>>>>> gcc -O3 -o ../bin/rastrip rastrip.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rahisto.c
>>>>> gcc -O3 -o ../bin/rahisto rahisto.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ralabel.c
>>>>> gcc -O3 -o ../bin/ralabel ralabel.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rapolicy.c
>>>>> gcc -O3 -o ../bin/rapolicy rapolicy.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ranonymize.c
>>>>> gcc -O3 -o ../bin/ranonymize ranonymize.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rapath.c
>>>>> gcc -O3 -o ../bin/rapath rapath.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rastream.c
>>>>> gcc -O3 -o ../bin/rastream rastream.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratree.c
>>>>> gcc -O3 -o ../bin/ratree ratree.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rafilteraddr.c
>>>>> gcc -O3 -o ../bin/rafilteraddr rafilteraddr.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratimerange.c
>>>>> gcc -O3 -o ../bin/ratimerange ratimerange.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> sed 's+ at PERLBIN@+/usr/bin/perl+' raports.pl > ../bin/raports
>>>>> chmod +x ../bin/raports
>>>>> sed 's+ at PERLBIN@+/usr/bin/perl+' rahosts.pl > ../bin/rahosts
>>>>> chmod +x ../bin/rahosts
>>>>> sed 's+ at PERLBIN@+/usr/bin/perl+' radark.pl > ../bin/radark
>>>>> chmod +x ../bin/radark
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rauserdata.c
>>>>> gcc -O3 -o ../bin/rauserdata rauserdata.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./raservices.c
>>>>> gcc -O3 -o ../bin/raservices raservices.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./raevent.c
>>>>> gcc -O3 -o ../bin/raevent raevent.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ragrep.c
>>>>> gcc -O3 -o ../bin/ragrep ragrep.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratemplate.c
>>>>> gcc -O3 -o ../bin/ratemplate ratemplate.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/clients'
>>>>> making in ./ragraph
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ragraph'
>>>>> sed 's+ at BINDIR@+/usr/local/bin+' ragraph.pl | \
>>>>> 	sed 's+ at PERLBIN@+/usr/bin/perl+' > ../bin/ragraph
>>>>> chmod +x ../bin/ragraph
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ragraph'
>>>>> making in ./ratop
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ratop'
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratop.c
>>>>> gcc -O3 -o ../bin/ratop ratop.o  ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP -lncurses 
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ratop'
>>>>> making in ./radium
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radium'
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./radium.c
>>>>> ./radium.c: In function ‘RadiumParseResourceFile’:
>>>>> ./radium.c:818: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result
>>>>> gcc -O3 -o radium radium.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lwrap -lnsl -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radium'
>>>>> making in ./radump
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radump'
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./radump.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-aodv.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-arp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-beep.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-bfd.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-bgp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-bootp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-domain.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-krb.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-ldp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-l2tp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-lmp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-msdp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-ntp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-pim.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-pptp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-snmp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-syslog.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-rip.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-telnet.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-tftp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-isoclns.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-ether.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-timed.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-radius.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-rx.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-smb.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-stp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-isakmp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-igmp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-dvmrp.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./smbutil.c
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./gmpls.c
>>>>> gcc -O3 -o ../bin/radump radump.o print-aodv.o print-arp.o print-beep.o print-bfd.o print-bgp.o print-bootp.o print-domain.o print-krb.o print-ldp.o print-l2tp.o print-lmp.o print-msdp.o print-ntp.o print-pim.o print-pptp.o print-snmp.o print-syslog.o print-rip.o print-telnet.o print-tftp.o print-isoclns.o print-ether.o print-timed.o print-radius.o print-rx.o print-smb.o print-stp.o print-isakmp.o print-igmp.o print-dvmrp.o smbutil.o gmpls.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radump'
>>>>> making in ./ramysql
>>>>> make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ramysql'
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -c ./rasql.c
>>>>> In file included from /usr/include/string.h:640,
>>>>>                  from ./rasql.c:42:
>>>>> In function ‘bzero’:
>>>>> /usr/include/bits/string3.h:100: warning: call to__builtin___memset_chk will always overflow destination buffer
>>>>> gcc -O3 -o ../bin/rasql rasql.o -L/usr/lib/mysql -lmysqlclient_r  ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -c ./rasqlinsert.c
>>>>> In file included from /usr/include/stdio.h:910,
>>>>>                  from ../include/argus_util.h:195,
>>>>>                  from ./rasqlinsert.c:53:
>>>>> In function ‘snprintf’,
>>>>>     inlined from ‘ArgusProcessCommand’ at ./rasqlinsert.c:2531:
>>>>> /usr/include/bits/stdio2.h:65: warning: call to__builtin___snprintf_chk will always overflow destination buffer
>>>>> gcc -O3 -o ../bin/rasqlinsert rasqlinsert.o -L/usr/lib/mysql -lmysqlclient_r  ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP -lncurses 
>>>>> gcc -O3 -I. -I../include -I../common -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -c ./rasqltimeindex.c
>>>>> gcc -O3 -o ../bin/rasqltimeindex rasqltimeindex.o -L/usr/lib/mysql -lmysqlclient_r  ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz   -lpthread  -L/usr/lib -lGeoIP
>>>>> make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ramysql'
>>>>> 
>>>>> I will look into this later and see what alternatives there are...  I will also test the compiled binaries later today.
>>>>> 
>>>>> best,
>>>>> 
>>>>> Javier Almillategui
>>>>> 
>>>>> On Aug 12, 2010, at 5:35 PM, Carter Bullard wrote:
>>>>> 
>>>>>> Hey Javier,
>>>>>> The current client version to use that may fix this bug is argus-clients-3.0.3.17, which you can get here:
>>>>>>   http://qosient.com/argus/dev/argus-clients-3.0.3.17.tar.gz
>>>>>> 
>>>>>> I believe that there was a memory leak that we fixed.  If you try the new code, could you please send
>>>>>> some email if the problem was fixed?  That would be most helpful.  If its not fixed, send more email.
>>>>>> 
>>>>>> Carter
>>>>>> 
>>>>>> On Aug 12, 2010, at 4:56 PM, Javier Almillategui wrote:
>>>>>> 
>>>>>>> Hi All,
>>>>>>> 
>>>>>>> I have noticed a strange behavior in rasqlinsert (using argus client 3.0.2 compiled from source) in ubuntu 10.04 x86_64
>>>>>>> 
>>>>>>> after running the rasqlinsert to load a database using the following options:
>>>>>>> 
>>>>>>> rasqlinsert -w mysql://xxx:xxx@localhost/anondb/xxxtap_%Y_%m_%d -r xxx_tap_20100521_prepared.argus -s +2seq -s +3ltime -s +4dur -s +loss -s +1srcid -s -record -m none -M time 1d
>>>>>>> 
>>>>>>> the memory consumption of rasqlinsert continues to increase to the point of using a significant part of my systems memory.
>>>>>>> 
>>>>>>> it went from consuming ~16 MB to consuming 712 MB (at this point).
>>>>>>> 
>>>>>>> is there any known memory leak in rasqlinsert??
>>>>>>> 
>>>>>>> best,
>>>>>>> 
>>>>>>> Javier Almillategui
>>>>>>> 
>>>>>> 
>>>>> 
>>>> 

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100927/1ef5073c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100927/1ef5073c/attachment.bin>