Radium memory leak

[Carter Bullard]

Hey Craig,
Here is a patch to fix the leak.  Sloppy on my part, but we made some changes
in the library, and missed the change here.  My bad.

Carter

osiris:common carter$ diff -c argus_label.c.orig argus_label.c      
*** argus_label.c.orig	Tue Mar 26 14:30:39 2013
--- argus_label.c	Tue Mar 26 14:31:03 2013
***************
*** 3072,3077 ****
--- 3072,3078 ----
                    if ((saddr = RaFetchIPv4AddressLabel(parser, &flow->ip_flow.ip_src)) != NULL) {
                       int slen = strlen(RaAddressLabelBuffer);
                       snprintf (&RaAddressLabelBuffer[slen], 1024 - slen, "saddr=%s", saddr);
+                      free(saddr);
                       found++;
                    }
                    if ((daddr = RaFetchIPv4AddressLabel(parser, &flow->ip_flow.ip_dst)) != NULL) {
***************
*** 3081,3086 ****
--- 3082,3088 ----
                          slen++;
                       }
                       snprintf (&RaAddressLabelBuffer[slen], 1024 - slen, "daddr=%s", daddr);
+                      free(daddr);
                       found++;
                    }
                 }


On Mar 26, 2013, at 11:57 AM, Craig Merchant <cmerchant at responsys.com> wrote:

> I restarted radium last night around 9 pm.  12 hours later, it was using about 44% of 128 GB.
>  
> Other than cleaning up some of the entries in the ralabel file, the only change I made recently was increasing the time interval of rastream from 5m to 15m to (hopefully) reduce the volume of flow records a bit when that data was fed to racluster.
>  
> We seem to be generating about 10-20 GB of binary flow records per hour.  Let me know how much data you need for testing…
>  
> Thanks.
> 
> Craig
>  
> From: Carter Bullard [mailto:carter at qosient.com] 
> Sent: Tuesday, March 26, 2013 4:54 AM
> To: Craig Merchant
> Cc: Argus (argus-info at lists.andrew.cmu.edu)
> Subject: Re: [ARGUS] Radium memory leak
>  
> Hmmmmm, I really thought we had fixed that.
> Any sense of the rate of memory loss ?
> I may need some flow data from you, if my data doesn't
> generate any leaks.
>  
> I'll valgrind it today.
>  
> Carter
> 
> On Mar 25, 2013, at 11:04 PM, Craig Merchant <cmerchant at responsys.com> wrote:
> 
> Hey, Carter…
>  
> I tried installing the latest 3.0.7.7 client build and I’m still seeing a memory leak in radium when label files are enabled.
>  
> These are the searches that we launch when radium starts up:
>  
>         /usr/local/bin/ra -S 10.10.10.10:561 -n -u -c "," -s "+0dnstroke" | egrep -v "(^,|^0,)" >> /ssd/argus/splunk/reverse_keystroke.csv &
>  
>         /usr/local/bin/ralabel -S 10.10.10.10:561 -f /usr/local/argus/ralabel.conf -n -u -c "," -M dsrs="+label" label="blacklisted" -s +1dur,+label:200 - not dst net 10.0.0.0/8 and not dst net 12.130.140.0/24 and not dst port 25 and not dst port 53 >> /ssd/argus/splunk/blacklisted.csv &
>  
>         /usr/local/bin/ra -S 10.10.10.10:561 -n -u -c "," -M dsrs="+duser,+suser,+label" -u -e "^SSH-" -s "+1dur,+suser,+duser,+label:200" - not dst net 10.0.0.0/8 and not dst net 12.130.140.0/24 | egrep -v "whitelisted" >> /ssd/argus/splunk/suspicious_outbound_ssh.csv &
>  
>         /usr/local/bin/rastream -S 10.10.10.10:561 -M time 15m -B 10s -w /ssd/argus/%s.argus -f /usr/local/argus/rastream.sh &
>  
> The /usr/local/argus/rastream.sh launches four searches (with different filters) that look like the following:
>  
> racluster -r $FILES -M correct -m proto saddr daddr dport -c "@" -p 3 -u -Z b -n                                     -s "+0ltime,+trans,+dur,+runtime,+mean,+stddev,+sum,+sco,+dco,+spkts,+dpkts,+sb                                    ytes,+dbytes,+load,+sload,+dload,+loss,+sloss,+dloss,+ploss,+sploss,+dploss,+rat                                    e,+srate,+drate,+appbytes,+sappbytes,+dappbytes,+label:200"
>  
> I sent you the iana label file we’re using a while back along with the ralabel.conf file.  Those are still pretty much the same (though I cleaned up some errors in the label file recently).  If you want the most recent version of the label file, let me know.  Or if you want some more samples of binary flow data.
>  
> Let me know how best to troubleshoot this…
>  
> Thx.
> 
> Craig

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130326/84268939/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2589 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130326/84268939/attachment.bin>