argus/radium problem

CS Lee geek00l at gmail.com
Fri Jun 8 13:52:04 EDT 2012 

hi Carter,

Currently it is very hard for me to run argus in foreground in this bivio
box, if I push to one cpu and run argus in bivio, it got killed after I
connect argus client to it. I have sent you the message in previous email.
The argus keeps getting killed message once I attach argus client for a
little while.


On Sat, Jun 9, 2012 at 12:58 AM, Carter Bullard <carter at qosient.com> wrote:

> Hey CS Lee,
> Just going through the messages you sent, so we have some dialog on what
> is going on.
>
> OK, so this is the most important to figuring out the argus fatal issues.
>  After deleting the socket that the client
> was using, somehow the modeler looks to have been re-initialized, or
> deleted.  This is not good.
> The model->ArgusOutputList is the communication channel from the
> ArgusModeler to the ArgusOutput
> thread, so if this goes away, broken argus.
>
> The ArgusModeler->ArgusOutputList goes away only when the something calls
> ArgusCloseOutput().
> Any chance you can verify, running argus with a "-D 2" option, that in
> this situation, ArgusCloseOutput()
> is called?
>
> Carter
>
> On Jun 7, 2012, at 11:58 PM, CS Lee wrote:
>
> hi Carter,
>
> I run ra in follwoing way and after ra has retrieved 100 records, it
> exits, however I can't reconnect to argus again -
>
> ra -N 100 -Z b -S 10.0.0.1:561 -s saddr sport daddr flgs proto state
>
> This is the output I get from argus
>
> argus[27489.48c93490]: 08 Jun 12 10:23:07.144707 ArgusWriteSocket: write
> (7, 0x58300058, 280, ...) -1
> argus[27489.48c93490]: 08 Jun 12 10:23:07.144726 ArgusWriteSocket: write
> (7, 0x58300058, 280, ...) -1
> argus[27489.48c93490]: 08 Jun 12 10:23:07.144786 ArgusWriteSocket: write
> (7, 0x58300058, 280, ...) -1
> argus[27489.48c93490]: 08 Jun 12 10:23:07.144804 ArgusDeleteSocket: list
> not empty
> argus[27489.48c93490]: 08 Jun 12 10:23:07.144909 ArgusDeleteSocket
> (0x58300008) returning
> argus[27489.498dd490]: 08 Jun 12 10:26:18.708883 ArgusSendFlowRecord
> (0x1009b008, 0x57595818, 64) no model->ArgusOutputList
> argus[27489.498dd490]: 08 Jun 12 10:26:19.110344 ArgusSendFlowRecord
> (0x1009b008, 0x527033d8, 64) no model->ArgusOutputList
>
> If I reconnect to argus again, I get connection refused message
>
> ra[20656]: 10:26:56.109970 connect to 10.0.0.1:561 failed 'Connection
> timed out'
>
> Thank you.
>
>
>
>
> On Fri, Jun 8, 2012 at 10:12 AM, CS Lee <geek00l at gmail.com> wrote:
>
>> hi Carter,
>>
>> I don't know if bivio team has compiled it with threaded support, need to
>> ask them(joel in bivio) if you know him.
>>
>> Here's the argus output message when radium connects to it -
>>
>> ArgusUpdateUDTState(0x49dd65a8, 32) UDT_DATA_PACKET seq 0x32a5da00 msgnum
>> 0x66573422 tstmp 0x216c794 sockid 0x12ce55c8 loss 0
>> argus[27125.498dd490]: 08 Jun 12 08:26:57.178068
>> ArgusUpdateUDTState(0x49dd65a8, 32) UDT_DATA_PACKET seq 0x329ee15a msgnum
>> 0x31573422 tstmp 0x216c794 sockid 0x12ce55c8 loss 0
>> argus[27125.498dd490]: 08 Jun 12 08:26:57.196113
>> ArgusUpdateUDTState(0x49dd65a8, 32) UDT_DATA_PACKET seq 0x321c61e1 msgnum
>> 0xac573422 tstmp 0x216c794 sockid 0x12ce55c8 loss 0
>> argus[27125.498dd490]: 08 Jun 12 08:26:57.220138
>> ArgusUpdateUDTState(0x49dd65a8, 32) UDT_DATA_PACKET seq 0x3227377b msgnum
>> 0xba573422 tstmp 0x216c794 sockid 0x12ce55c8 loss 0
>> argus[27125.498dd490]: 08 Jun 12 08:26:57.293605
>> ArgusUpdateUDTState(0x49dd65a8, 32) UDT_DATA_PACKET seq 0x32ac4271 msgnum
>> 0x59573422 tstmp 0x216c794 sockid 0x12ce55c8 loss 0
>> argus[27125.48c93490]: 08 Jun 12 08:26:58.039943 ArgusLoadList
>> (0x1009b9c0, 0x1009b948) load 1 objects
>> argus[27125.48c93490]: 08 Jun 12 08:26:58.040039 ArgusFreeListRecord
>> (0x100f679c) returning
>> argus[27125.48c93490]: 08 Jun 12 08:26:59.243664 ArgusLoadList
>> (0x1009b9c0, 0x1009b948) load 1 objects
>> argus[27125.48c93490]: 08 Jun 12 08:26:59.243759 ArgusFreeListRecord
>> (0x100f6b44) returning
>> argus[27125.48c93490]: 08 Jun 12 08:26:59.413949 ArgusLoadList
>> (0x1009b9c0, 0x1009b948) load 1 objects
>> argus[27125.48c93490]: 08 Jun 12 08:26:59.414030 ArgusFreeListRecord
>> (0x100f6eec) returning
>> argus[27125.48c93490]: 08 Jun 12 08:26:59.767378 ArgusLoadList
>> (0x1009b9c0, 0x1009b948) load 1 objects
>> argus[27125.48c93490]: 08 Jun 12 08:26:59.767474 ArgusFreeListRecord
>> (0x100f7294) returning
>> argus[27125.48c93490]: 08 Jun 12 08:27:00.112948 ArgusLoadList
>> (0x1009b9c0, 0x1009b948) load 1 objects
>> argus[27125.48c93490]: 08 Jun 12 08:27:00.113028 ArgusFreeListRecord
>> (0x100f763c) returning
>>
>> Then after a while, this appears -
>>
>> ArgusGenerateInitialMar() returning
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083154 ArgusCheckClientStatus:
>> wrote 128 bytes to client
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083175 ArgusCheckClientStatus()
>> returning
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083192 ArgusLoadList
>> (0x1009b9c0, 0x1009b948) load 19823 objects
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083209 ArgusOutputProcess() 1
>> client(s) for record 0x4ce4e2a4
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083226 ArgusOutputProcess() 1
>> client(s) not ready fd 4 sock 0x4f500008 start 0
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083244 ArgusFreeListRecord
>> (0x4ce4e2a4) returning
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083260 ArgusOutputProcess() 1
>> client(s) for record 0x4ce4e64c
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083276 ArgusOutputProcess() 1
>> client(s) not ready fd 4 sock 0x4f500008 start 0
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083293 ArgusFreeListRecord
>> (0x4ce4e64c) returning
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083310 ArgusOutputProcess() 1
>> client(s) for record 0x4ce4e9f4
>> argus[27125.48c93490]: 08 Jun 12 08:27:09.083326 ArgusOutputProcess() 1
>> client(s) not ready fd 4 sock 0x4f500008 start 0
>>
>>
>> On Fri, Jun 8, 2012 at 10:02 AM, Carter Bullard <carter at qosient.com>wrote:
>>
>>> Well, if it's getting write errors we should be shutting down the
>>> connection and deallocating any queued records.  This is useful.  Let me
>>> see what I can do with this.
>>>
>>> Carter
>>>
>>> Sent from my iPad
>>>
>>> On Jun 7, 2012, at 9:19 PM, CS Lee <geek00l at gmail.com> wrote:
>>>
>>> hi Carter,
>>>
>>> Bivio team has sent me the argus rpm with debug enabled, here's what I
>>> get from argus -D 5 when argus client sees the idle stream message -
>>>
>>> argus[26916.48c93490]: 08 Jun 12 07:20:28.594670 ArgusWriteSocket: write
>>> (7, 0x4d200058, 48, ...) -1
>>> argus[26916.48c93490]: 08 Jun 12 07:20:28.594696 ArgusWriteSocket: write
>>> (7, 0x4d200058, 48, ...) -1
>>> argus[26916.48c93490]: 08 Jun 12 07:20:28.594723 ArgusWriteSocket: write
>>> (7, 0x4d200058, 48, ...) -1
>>> argus[26916.498dd490]: 08 Jun 12 07:20:28.600001 ArgusMallocList memory
>>> pool exhausted (1000000 : 1000000)
>>> argus[26916.48c93490]: 08 Jun 12 07:20:28.603302 ArgusWriteSocket: write
>>> (7, 0x4d200058, 48, ...) -1
>>> argus[26916.48c93490]: 08 Jun 12 07:20:28.603332 ArgusWriteSocket: write
>>> (7, 0x4d200058, 48, ...) -1
>>>
>>> argus[26916.48c93490]: 08 Jun 12 07:20:33.554648 ArgusWriteSocket: write
>>> (7, 0x4d200058, 48, ...) -1
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.554671 ArgusMallocList memory
>>> pool exhausted (1000000 : 1000000)
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563056 ArgusMallocListRecord
>>> (904) returned NULL
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563076 ArgusMallocList memory
>>> pool exhausted (1000000 : 1000000)
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563092 ArgusMallocListRecord
>>> (904) returned NULL
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563108 ArgusMallocList memory
>>> pool exhausted (1000000 : 1000000)
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563124 ArgusMallocListRecord
>>> (904) returned NULL
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563140 ArgusMallocList memory
>>> pool exhausted (1000000 : 1000000)
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563157 ArgusMallocListRecord
>>> (904) returned NULL
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563173 ArgusMallocList memory
>>> pool exhausted (1000000 : 1000000)
>>> argus[26916.498dd490]: 08 Jun 12 07:20:33.563189 ArgusMallocListRecord
>>> (904) returned NULL
>>>
>>> Argus keeps running however no client can connect to it and retrieve
>>> anything.
>>>
>>> On Fri, Jun 8, 2012 at 2:54 AM, CS Lee <geek00l at gmail.com> wrote:
>>>
>>>> hi Carter,
>>>>
>>>> I able to get argus running in foreground now, however since the rpm
>>>> they provide doesn't compile with debug enable, I can only see limited
>>>> output, here's where it breaks.
>>>>
>>>> Argus client on linux box
>>>> ra[32718]: 01:07:11.134418 ArgusReadStream 10.0.0.1: idle stream:
>>>> closing
>>>>
>>>> Argus on bivio
>>>> argus -D 3 -i bond:s0.e0,s0.e1 -mAJZRU 64 -B 10.0.0.1 -P 561
>>>> argus[1611]: 08 Jun 12 01:02:46.781824 started
>>>> argus[1611]: 08 Jun 12 01:02:46.800912 ArgusGetInterfaceStatus:
>>>> interface s0.e1 is up
>>>> argus[1611]: 08 Jun 12 01:02:46.801064 ArgusGetInterfaceStatus:
>>>> interface s0.e0 is up
>>>> argus[1611]: 08 Jun 12 01:04:15.489853 connect from 10.0.0.3
>>>> argus[1611]: 08 Jun 12 01:05:07.432936 connect from 10.0.0.3
>>>>
>>>> The following line shows up after I control+c
>>>> argus[1611]: 08 Jun 12 01:14:23.524250 ArgusCloseModeler(0x100c7b00)
>>>> ArgusGenerateListRecord failed
>>>>
>>>> Currently I'm working with one of bivio team engineer to see how this
>>>> issue can be fixed. Just keep this between us. Thank you.
>>>>
>>>>
>>>> On Fri, Jun 8, 2012 at 2:40 AM, Carter Bullard <carter at qosient.com>wrote:
>>>>
>>>>> Lets see if I find something in the review tonight.
>>>>> I don't have that much familiarity with Bivio debugging myself, I'll
>>>>> appeal to
>>>>> some of the Bivio engineers to help out, if  we need to go that far.
>>>>> But definitely need to get this fixed.
>>>>>
>>>>> Carter
>>>>>
>>>>>
>>>>> On Jun 7, 2012, at 1:50 PM, CS Lee wrote:
>>>>>
>>>>> hi Carter,
>>>>>
>>>>> I'm currently deploying this system for Myanmar government(they got
>>>>> out of US Sanction status since May).
>>>>>
>>>>> If you can't find any site to have similar issue, I can give you the
>>>>> access of the bivio system and also one of the linux server, currently I
>>>>> don't even share the access with bivio team yet, they just support me via
>>>>> email.
>>>>>
>>>>> What do you think? Only thing is I don't know how to fork argus to
>>>>> foreground to see the debug message in bivio system, on linux server
>>>>> everything is straightforward.
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>>
>>>>> CS Lee<geek00L[at]gmail.com>
>>>>>
>>>>> http://geek00l.blogspot.com
>>>>> http://defcraft.net
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>>
>>>> CS Lee<geek00L[at]gmail.com>
>>>>
>>>> http://geek00l.blogspot.com
>>>> http://defcraft.net
>>>>
>>>
>>>
>>>
>>> --
>>> Best Regards,
>>>
>>> CS Lee<geek00L[at]gmail.com>
>>>
>>> http://geek00l.blogspot.com
>>> http://defcraft.net
>>>
>>>
>>
>>
>> --
>> Best Regards,
>>
>> CS Lee<geek00L[at]gmail.com>
>>
>> http://geek00l.blogspot.com
>> http://defcraft.net
>>
>
>
>
> --
> Best Regards,
>
> CS Lee<geek00L[at]gmail.com>
>
> http://geek00l.blogspot.com
> http://defcraft.net
>
>
>


-- 
Best Regards,

CS Lee<geek00L[at]gmail.com>

http://geek00l.blogspot.com
http://defcraft.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120609/e024c7b7/attachment.html>

More information about the argus mailing list