Ring Buffer Memory

[elof2 at sentor.se]

It sounds good to have an override option to manually set the bufsize.

If no bufsize is given, argus should use the system default.
For FreeBSD, this is: sysctl net.bpf.bufsize

When argus ask for a large buffersize, error handling should be present if 
requested buffer exceed sysctl net.bpf.maxbufsize.

/Elof


On Thu, 20 Nov 2014, Carter Bullard wrote:

> Hey John,
> Yes, it seems the hardest part is how to configure it ... command line option, argus.conf variable name ...
>
> so I'll do something like '-M pcapbufsiz=size'  allowing for K,M,G notation, and have an ARGUS_PCAP_BUF_SIZE variable.
>
> Maybe in a 2015 (4 weeks) ... ????
> Carter
>
>
>> On Nov 20, 2014, at 2:05 PM, John T. Myers <myersj0 at gmail.com> wrote:
>>
>> Carter, it would be very useful to have Argus have the ability to set the pcap_buffer_size.
>>
>> I had someone do a proof-of-concept and we got it working by adding the following code to ArgusSource.c.
>>
>> We just replaced the call to pcap_open_live around line 290 with the following:
>>
>> We also hard-coded a 2GB ring buffer, as we are processing lots of flows at high BW, but having that set in KB through the .conf file would be pretty useful.
>>
>> int buffer_error, pdstatus = 0;
>>
>> …. …. ….
>>
>> // Create a handle for capture
>>
>>    inf->ArgusPd = pcap_create(device->name, errbuf);
>>
>>    if (inf->ArgusPd != NULL) {
>>
>> // Set the buffer, in bytes. Hardcoded for example purposes, recommend this be a config setting
>>
>>       if((buffer_error = pcap_set_buffer_size(inf->ArgusPd, 2048000000))!=0) {
>>
>>          ArgusLog(LOG_ERR, "%s \n", pcap_statustostr(buffer_error));
>>
>>    }
>>
>> //Activate the handle for capture
>>
>>       pdstatus = pcap_activate(inf->ArgusPd);
>>
>>       if (pdstatus < 0) {
>>
>>          ArgusLog(LOG_ERR, “%s \n”,
>>
>>       }
>>
>>
>>> On Tue, Nov 18, 2014 at 11:27 AM, Carter Bullard <carter at qosient.com> wrote:
>>> Hey John,
>>> ARGUS_ENV is provided to support sub-systems that can use environment variables for configuration.  This is helpful for packet capture card device drivers, for the most part, but some of the ring buffer drivers can use environment variables, like the PCAP_PF_RING_DNA_RSS, PF_RING_ACTIVE_POLL, PCAP_PF_RING_USE_CLUSTER_PER_FLOW, etc…  What ring buffer driver are you using ???
>>>
>>> With the -B option, standard tcpdump.org tcpdump is not modifying the environment, its calling pcap_set_buffer_size(), which is kinda new (2011?? 2012??).
>>>
>>> Would you like argus to be able to set the pcap_buffer_size ???
>>> If we need to make a change, can we put it in the argus.conf file ???
>>>
>>> Carter
>>>
>>>
>>>> On Nov 17, 2014, at 8:29 PM, John T. Myers <myersj0 at gmail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> When trying to increase the libpcap ring buffer capacity in argus.conf, it does not appear to change anything on an Ubuntu system running argus.
>>>>
>>>> We tried doing the same 2GB size with tcpdump using the -B option and watched the memory usage climb dramatically as it should.
>>>>
>>>> However, when trying to set using ARGUS_ENV="PCAP_MEMORY=2000000" there is no increase in memory usage by argus at all.
>>>>
>>>> John
>>>>
>>>>
>>
>