radium - ArgusWriteOutSocket max queue exceeded messages

[Carter Bullard]

Hey Dave,
You are seeing the rasqlinsert() close its input connection..."client sent DONE", and
radium taking its time to remove the output queue list, which has accumulated 0.5M
records for output.  Looks like radium is doing the right thing in this case.

Changing the queue lengths smaller will get you to the point where radium() will
think that your clients aren't processing flow record fast enough, and you'll see
radium() drop the connection.  The system logs will show the condition, so just
as easy to see as this condition.

The output queue length is defined on line 1819 in the file ./common/argus_output.c.

   int ArgusMaxListLength = 500000;

You can take it down to 2-3X the flow rate, so if you're transmitting 1000 records per
second, 2-3000 may work???

Carter



On Oct 3, 2013, at 8:51 PM, "David Edelman" <dedelman at iname.com> wrote:

> I have a radium process that collects argus flow records from several collectors and makes the labeled and processed flow records available to several local processes [rastream and rasqlinsert]
> 
> I’ve noticed instance of rasqlinsert disappearing with no obvious trace. This is not a frequent occurrence but it is a problem when it happens. Figuring that I need to catch the rabbit before I can cook the rabbit, I left radium running in gdb (not optimal but it did work) I see these messages from the radium instance:
> 
> 
> radium[9083]: Fri 2013-10-04 00:38:01.283 ArgusCheckClientMessage: client localhost.localdomain sent DONE
> 
> radium[9083]: Fri 2013-10-04 00:39:01.050 ArgusWriteOutSocket(0x9d61b050) max queue exceeded 500001
> 
> radium[9083]: Fri 2013-10-04 00:39:01.050 ArgusWriteOutSocket(0x9d61b050) max queue exceeded 500001
> 
> radium[9083]: Fri 2013-10-04 00:39:12.122 connect from localhost.localdomain
> 
> 
> Is there some value that I can play with to accommodate the size of the data or is there some way that I can get the ArgusWriteOutSocket() process to work with smaller amounts of data?
> 
> --Dave
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131003/85f4cb78/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131003/85f4cb78/attachment.bin>