argus/radium problem

Fri Jun 8 11:50:08 EDT 2012

Hey CS Lee,
Just going through the messages you sent, so we have some dialog on what is going on.

All of these particular messages are indicating errors, but can be considered ok.  The write
errors are not fatal, as we will get these if the remote end of the socket has flow controlled us
or if the output queue is full.  We will queue these messages up on the output queue, waiting
for the client to start reading again.

We will only do these for a while, but in this case, we have run out of memory, as we've got
1M records queued up to go, so we will just ignore these new records (drop them on the floor).
We're still waiting for the client to wake up and start reading, but more than likely the client has
gone, or something else is wrong.  So while this all looks bad, it on its own is not indicative of
total failure.  But, this is what you would expect if something has gone really wrong.

At this point we should have probably kicked off the client, reclaimed all the list records, and
gone back to waiting for new clients, but this isn't happening.

Carter 

On Jun 7, 2012, at 9:19 PM, CS Lee wrote:

> hi Carter,
> 
> Bivio team has sent me the argus rpm with debug enabled, here's what I get from argus -D 5 when argus client sees the idle stream message -
> 
> argus[26916.48c93490]: 08 Jun 12 07:20:28.594670 ArgusWriteSocket: write (7, 0x4d200058, 48, ...) -1
> argus[26916.48c93490]: 08 Jun 12 07:20:28.594696 ArgusWriteSocket: write (7, 0x4d200058, 48, ...) -1
> argus[26916.48c93490]: 08 Jun 12 07:20:28.594723 ArgusWriteSocket: write (7, 0x4d200058, 48, ...) -1
> argus[26916.498dd490]: 08 Jun 12 07:20:28.600001 ArgusMallocList memory pool exhausted (1000000 : 1000000)
> argus[26916.48c93490]: 08 Jun 12 07:20:28.603302 ArgusWriteSocket: write (7, 0x4d200058, 48, ...) -1
> argus[26916.48c93490]: 08 Jun 12 07:20:28.603332 ArgusWriteSocket: write (7, 0x4d200058, 48, ...) -1
> 
> argus[26916.48c93490]: 08 Jun 12 07:20:33.554648 ArgusWriteSocket: write (7, 0x4d200058, 48, ...) -1
> argus[26916.498dd490]: 08 Jun 12 07:20:33.554671 ArgusMallocList memory pool exhausted (1000000 : 1000000)
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563056 ArgusMallocListRecord (904) returned NULL
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563076 ArgusMallocList memory pool exhausted (1000000 : 1000000)
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563092 ArgusMallocListRecord (904) returned NULL
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563108 ArgusMallocList memory pool exhausted (1000000 : 1000000)
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563124 ArgusMallocListRecord (904) returned NULL
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563140 ArgusMallocList memory pool exhausted (1000000 : 1000000)
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563157 ArgusMallocListRecord (904) returned NULL
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563173 ArgusMallocList memory pool exhausted (1000000 : 1000000)
> argus[26916.498dd490]: 08 Jun 12 07:20:33.563189 ArgusMallocListRecord (904) returned NULL
> 
> Argus keeps running however no client can connect to it and retrieve anything.
> 
> On Fri, Jun 8, 2012 at 2:54 AM, CS Lee <geek00l at gmail.com> wrote:
> hi Carter,
> 
> I able to get argus running in foreground now, however since the rpm they provide doesn't compile with debug enable, I can only see limited output, here's where it breaks.
> 
> Argus client on linux box
> ra[32718]: 01:07:11.134418 ArgusReadStream 10.0.0.1: idle stream: closing
> 
> Argus on bivio
> argus -D 3 -i bond:s0.e0,s0.e1 -mAJZRU 64 -B 10.0.0.1 -P 561
> argus[1611]: 08 Jun 12 01:02:46.781824 started
> argus[1611]: 08 Jun 12 01:02:46.800912 ArgusGetInterfaceStatus: interface s0.e1 is up
> argus[1611]: 08 Jun 12 01:02:46.801064 ArgusGetInterfaceStatus: interface s0.e0 is up
> argus[1611]: 08 Jun 12 01:04:15.489853 connect from 10.0.0.3
> argus[1611]: 08 Jun 12 01:05:07.432936 connect from 10.0.0.3
> 
> The following line shows up after I control+c
> argus[1611]: 08 Jun 12 01:14:23.524250 ArgusCloseModeler(0x100c7b00) ArgusGenerateListRecord failed
> 
> Currently I'm working with one of bivio team engineer to see how this issue can be fixed. Just keep this between us. Thank you.
> 
> 
> On Fri, Jun 8, 2012 at 2:40 AM, Carter Bullard <carter at qosient.com> wrote:
> Lets see if I find something in the review tonight.
> I don't have that much familiarity with Bivio debugging myself, I'll appeal to
> some of the Bivio engineers to help out, if  we need to go that far.
> But definitely need to get this fixed.
> 
> Carter
> 
> 
> On Jun 7, 2012, at 1:50 PM, CS Lee wrote:
> 
>> hi Carter,
>> 
>> I'm currently deploying this system for Myanmar government(they got out of US Sanction status since May).
>> 
>> If you can't find any site to have similar issue, I can give you the access of the bivio system and also one of the linux server, currently I don't even share the access with bivio team yet, they just support me via email.
>> 
>> What do you think? Only thing is I don't know how to fork argus to foreground to see the debug message in bivio system, on linux server everything is straightforward.
>> 
>> 
>> -- 
>> Best Regards,
>> 
>> CS Lee<geek00L[at]gmail.com>
>> 
>> http://geek00l.blogspot.com
>> http://defcraft.net
> 
> 
> 
> 
> -- 
> Best Regards,
> 
> CS Lee<geek00L[at]gmail.com>
> 
> http://geek00l.blogspot.com
> http://defcraft.net
> 
> 
> 
> -- 
> Best Regards,
> 
> CS Lee<geek00L[at]gmail.com>
> 
> http://geek00l.blogspot.com
> http://defcraft.net

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120608/8ece3b40/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120608/8ece3b40/attachment.bin>