rasqlinsert memory consumption
Javier Almillategui
jalmilla at gmu.edu
Fri Aug 13 06:52:37 EDT 2010
Hi Carter,
I have compiled version 3.0.3 on my system and got some interesting warnings regarding buffer overflows:
making in ./common
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/common'
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_main.c
rm -f ../lib/argus_parse.a; ar qc ../lib/argus_parse.a argus_main.o
ranlib ../lib/argus_parse.a
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_code.c
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_filter.c
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_util.c
./argus_util.c: In function ‘ArgusParseResourceFile’:
./argus_util.c:1680: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result
./argus_util.c: In function ‘ArgusLog’:
./argus_util.c:18124: warning: format not a string literal and no format arguments
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_auth.c
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_parser.c
flex -Pargus_ -t scanner.l > $$.scanner.c; mv $$.scanner.c scanner.c
bison -y -p argus_ -d grammar.y
mv y.tab.c grammar.c
mv y.tab.h tokdefs.h
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./scanner.c
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -Dyylval=argus_lval -c grammar.c
sed -e 's/.*/char version[] = "&";/' ./../VERSION > version.c
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./version.c
rm -f ../lib/argus_common.a; ar qc ../lib/argus_common.a argus_code.o argus_filter.o argus_util.o argus_auth.o argus_parser.o scanner.o grammar.o version.o
ranlib ../lib/argus_common.a
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_client.c
./argus_client.c: In function ‘ArgusParseAggregator’:
./argus_client.c:11758: warning: ignoring return value of ‘strtol’, declared with attribute warn_unused_result
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_label.c
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_grep.c
rm -f ../lib/argus_client.a; ar qc ../lib/argus_client.a argus_client.o argus_label.o argus_grep.o
ranlib ../lib/argus_client.a
gcc -O3 -I. -I../include -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_event.c
rm -f ../lib/argus_event.a; ar qc ../lib/argus_event.a argus_event.o
ranlib ../lib/argus_event.a
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/common'
making in ./include
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/include'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/include'
making in ./clients
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/clients'
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ra.c
gcc -O3 -o ../bin/ra ra.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./racount.c
gcc -O3 -o ../bin/racount racount.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rasort.c
gcc -O3 -o ../bin/rasort rasort.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rasplit.c
gcc -O3 -o ../bin/rasplit rasplit.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rabins.c
gcc -O3 -o ../bin/rabins rabins.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./racluster.c
gcc -O3 -o ../bin/racluster racluster.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./raconvert.c
gcc -O3 -o ../bin/raconvert raconvert.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rastrip.c
gcc -O3 -o ../bin/rastrip rastrip.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rahisto.c
gcc -O3 -o ../bin/rahisto rahisto.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ralabel.c
gcc -O3 -o ../bin/ralabel ralabel.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rapolicy.c
gcc -O3 -o ../bin/rapolicy rapolicy.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ranonymize.c
gcc -O3 -o ../bin/ranonymize ranonymize.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rapath.c
gcc -O3 -o ../bin/rapath rapath.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rastream.c
gcc -O3 -o ../bin/rastream rastream.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratree.c
gcc -O3 -o ../bin/ratree ratree.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rafilteraddr.c
gcc -O3 -o ../bin/rafilteraddr rafilteraddr.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratimerange.c
gcc -O3 -o ../bin/ratimerange ratimerange.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
sed 's+ at PERLBIN@+/usr/bin/perl+' raports.pl > ../bin/raports
chmod +x ../bin/raports
sed 's+ at PERLBIN@+/usr/bin/perl+' rahosts.pl > ../bin/rahosts
chmod +x ../bin/rahosts
sed 's+ at PERLBIN@+/usr/bin/perl+' radark.pl > ../bin/radark
chmod +x ../bin/radark
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./rauserdata.c
gcc -O3 -o ../bin/rauserdata rauserdata.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./raservices.c
gcc -O3 -o ../bin/raservices raservices.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./raevent.c
gcc -O3 -o ../bin/raevent raevent.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ragrep.c
gcc -O3 -o ../bin/ragrep ragrep.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratemplate.c
gcc -O3 -o ../bin/ratemplate ratemplate.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/clients'
making in ./ragraph
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ragraph'
sed 's+ at BINDIR@+/usr/local/bin+' ragraph.pl | \
sed 's+ at PERLBIN@+/usr/bin/perl+' > ../bin/ragraph
chmod +x ../bin/ragraph
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ragraph'
making in ./ratop
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ratop'
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./ratop.c
gcc -O3 -o ../bin/ratop ratop.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP -lncurses
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ratop'
making in ./radium
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radium'
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./radium.c
./radium.c: In function ‘RadiumParseResourceFile’:
./radium.c:818: warning: ignoring return value of ‘fgets’, declared with attribute warn_unused_result
gcc -O3 -o radium radium.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lwrap -lnsl -lm -lz -lpthread -L/usr/lib -lGeoIP
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radium'
making in ./radump
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radump'
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./radump.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-aodv.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-arp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-beep.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-bfd.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-bgp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-bootp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-domain.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-krb.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-ldp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-l2tp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-lmp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-msdp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-ntp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-pim.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-pptp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-snmp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-syslog.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-rip.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-telnet.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-tftp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-isoclns.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-ether.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-timed.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-radius.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-rx.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-smb.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-stp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-isakmp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-igmp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./print-dvmrp.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./smbutil.c
gcc -O3 -I. -I../include -I../common -I/usr/include -DHAVE_CONFIG_H -c ./gmpls.c
gcc -O3 -o ../bin/radump radump.o print-aodv.o print-arp.o print-beep.o print-bfd.o print-bgp.o print-bootp.o print-domain.o print-krb.o print-ldp.o print-l2tp.o print-lmp.o print-msdp.o print-ntp.o print-pim.o print-pptp.o print-snmp.o print-syslog.o print-rip.o print-telnet.o print-tftp.o print-isoclns.o print-ether.o print-timed.o print-radius.o print-rx.o print-smb.o print-stp.o print-isakmp.o print-igmp.o print-dvmrp.o smbutil.o gmpls.o ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/radump'
making in ./ramysql
make[1]: Entering directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ramysql'
gcc -O3 -I. -I../include -I../common -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -c ./rasql.c
In file included from /usr/include/string.h:640,
from ./rasql.c:42:
In function ‘bzero’:
/usr/include/bits/string3.h:100: warning: call to __builtin___memset_chk will always overflow destination buffer
gcc -O3 -o ../bin/rasql rasql.o -L/usr/lib/mysql -lmysqlclient_r ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
gcc -O3 -I. -I../include -I../common -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -c ./rasqlinsert.c
In file included from /usr/include/stdio.h:910,
from ../include/argus_util.h:195,
from ./rasqlinsert.c:53:
In function ‘snprintf’,
inlined from ‘ArgusProcessCommand’ at ./rasqlinsert.c:2531:
/usr/include/bits/stdio2.h:65: warning: call to __builtin___snprintf_chk will always overflow destination buffer
gcc -O3 -o ../bin/rasqlinsert rasqlinsert.o -L/usr/lib/mysql -lmysqlclient_r ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP -lncurses
gcc -O3 -I. -I../include -I../common -I/usr/include -I/usr/include/mysql -DHAVE_CONFIG_H -c ./rasqltimeindex.c
gcc -O3 -o ../bin/rasqltimeindex rasqltimeindex.o -L/usr/lib/mysql -lmysqlclient_r ../lib/argus_parse.a ../lib/argus_common.a ../lib/argus_client.a -lm -lz -lpthread -L/usr/lib -lGeoIP
make[1]: Leaving directory `/home/jalmillategui/Downloads/argus-clients-3.0.3.17/ramysql'
I will look into this later and see what alternatives there are... I will also test the compiled binaries later today.
best,
Javier Almillategui
On Aug 12, 2010, at 5:35 PM, Carter Bullard wrote:
> Hey Javier,
> The current client version to use that may fix this bug is argus-clients-3.0.3.17, which you can get here:
> http://qosient.com/argus/dev/argus-clients-3.0.3.17.tar.gz
>
> I believe that there was a memory leak that we fixed. If you try the new code, could you please send
> some email if the problem was fixed? That would be most helpful. If its not fixed, send more email.
>
> Carter
>
> On Aug 12, 2010, at 4:56 PM, Javier Almillategui wrote:
>
>> Hi All,
>>
>> I have noticed a strange behavior in rasqlinsert (using argus client 3.0.2 compiled from source) in ubuntu 10.04 x86_64
>>
>> after running the rasqlinsert to load a database using the following options:
>>
>> rasqlinsert -w mysql://xxx:xxx@localhost/anondb/xxxtap_%Y_%m_%d -r xxx_tap_20100521_prepared.argus -s +2seq -s +3ltime -s +4dur -s +loss -s +1srcid -s -record -m none -M time 1d
>>
>> the memory consumption of rasqlinsert continues to increase to the point of using a significant part of my systems memory.
>>
>> it went from consuming ~16 MB to consuming 712 MB (at this point).
>>
>> is there any known memory leak in rasqlinsert??
>>
>> best,
>>
>> Javier Almillategui
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100813/e88ca2c4/attachment.html>
More information about the argus
mailing list