ARGUS Binary Size

Wed Sep 24 10:46:51 EDT 2014

Hey James,
When you send bug/problem reports, you should send some
output of the command run, so we all can see what’s going
on.

The manpage is a good place to start, and you can use an argus.conf file.

  -i dup:dag0,dag1/2.3.4.5

If its not working, then send some email, and we can try
to figure it out.

You should see a printed statement as to whether the interfaces
are up, etc…  And you can always use a -D2 or -D3 option to
see what it thinks is going on with the packet readers.

Carter

On Sep 24, 2014, at 10:37 AM, James Grace <jgrac002 at fiu.edu> wrote:

> Carter, 
> How does one go about attach to two cards using one argus?  I've tried:
> 
> argus -d —i dag0 dag1 -P 561
> 
> 
> 
> But this doesn't seem to attach to dag0 as expected -- the drops on the Dag0 are increasing as if no process was attached to it. 
> 
> 
> 
> Thanks,
> 
> James
> 
> 
> 
> 
> On Tue, Sep 16, 2014 at 2:38 PM, Carter Bullard <carter at qosient.com> wrote:
> Definitely should be running 1 argus, opening both interfaces.
> You should set a little bit more in your /etc/argus.conf file,
> like the ARGUS_MONITOR_ID.
> 
> Many tests for whether your getting the data you want is to
> inspect the actual ‘primitive’ data from argus.  Do any of the
> flows in your collection files look like they have bi-directional
> counts, more than 1 packet in each one, that type of stuff.
> 
> racluster() will merge records together, which will correct for
> any problems, such as the status interval setting, which is the
> value that gets you the most data reduction.
> 
> Carter
> 
> On Sep 16, 2014, at 11:19 AM, James Grace <jgrac002 at fiu.edu> wrote:
> 
>> Hi Carter, 
>> 
>> In regards to our flow records, when I run 
>> racluster -m matrix -r argus.12.35 -w - | rasort -m pkts
>> 
>> During this 5 minutes period, there were ~500 flows with large packet counts, but then these packets/per flow dropped off severely — most being around 7 or 8pkts/flow. 
>> 
>> These 2 DAG cards are measuring the egress and ingress on a single circuit, and if I run the DAG tools (dagsnap, etc) I am seeing unique traffic over the wire on each card. 
>> 
>> This may just be chalked up to the way traffic if flowing over this circuit rather than anything to do with Argus. 
>> 
>> Thoughts?
>> 
>> -james
>> 
>> On Sep 15, 2014, at 4:01 PM, Carter Bullard <carter at qosient.com> wrote:
>> 
>>> Hey James,
>>> The src and dst labels for traffic in argus records does not
>>> equate to the number of packets we receive on the 2 DAG interfaces.
>>> These src and dst are flow src and dst, and refer to the flow
>>> source (the initiator of the flow transaction) and flow destination
>>> (the target of the flow transaction).
>>> 
>>> I suspect that if there is a problem, its that argus is getting
>>> packets from only one interface.
>>> 
>>> Carter
>>> 
>>> 
>>> On Sep 15, 2014, at 1:42 PM, James Grace <jgrac002 at fiu.edu> wrote:
>>> 
>>>> David, Carter
>>>> 
>>>> Thanks for the help so far.  I’ve changed some things around and the new scheme is:
>>>> 
>>>> Two argus instances listening to a DAG card:
>>>> 
>>>> argus -d -i dag0 -P 561
>>>> argus -d -i dag1 -P 562
>>>> 
>>>> I have only one option in my /etc/argus.conf:
>>>> 
>>>> # cat /etc/argus.confg 
>>>> ARGUS_CAPTURE_DATA_LEN=88
>>>> 
>>>> I have also configured the DAG cards to slen=88. 
>>>> 
>>>> I have radium connecting to the argi thusly:
>>>> radium -S localhost:561 -S localhost:562 -d -M dsrs=-net -encaps -P 563
>>>> 
>>>> And I have rastream connecting to radium to store and sort the argus binaries.
>>>> rastream -s 48 -S localhost:563 -M dsrs=-net -M time 5m -w /flows/argus/%Y/%m/%d/argus.%H.%M -D3
>>>> 
>>>> I understand that each of these binaries now contain two streams of traffic. Is that correct?  Even with all these flags and slen parameters, I am seeing massive binary sizes:
>>>> 
>>>> ls -l
>>>> total 57231320
>>>> -rw-r--r--. 1 root root 5790698448 Sep 15 13:13 argus.12.45
>>>> -rw-r--r--. 1 root root 8052120672 Sep 15 13:13 argus.12.50
>>>> -rw-r--r--. 1 root root 8416875168 Sep 15 13:13 argus.12.55
>>>> -rw-r--r--. 1 root root 7613404132 Sep 15 13:13 argus.13.00
>>>> -rw-r--r--. 1 root root 6660266172 Sep 15 13:13 argus.13.05
>>>> -rw-r--r--. 1 root root 6112649344 Sep 15 13:19 argus.13.15
>>>> -rw-r--r--. 1 root root 7716601984 Sep 15 13:24 argus.13.20
>>>> -rw-r--r--. 1 root root 8098926720 Sep 15 13:29 argus.13.25
>>>> -rw-r--r--. 1 root root  143270016 Sep 15 13:30 argus.13.30
>>>> 
>>>> 
>>>> If I run recount I’m seeing 5million records per 5 minute binary:
>>>> 
>>>> racount -r argus.13.25
>>>> racount   records     total_pkts     src_pkts       dst_pkts       total_bytes        src_bytes          dst_bytes
>>>>     sum   53181673    53191299       37622951       15568348       49381504197        28748639718        20632864479
>>>> 
>>>> I’m thinking there’s got to be some miscounts.  The link I’m measuring, both ingress and egress traffic sum up to about 2Gb/s with a peak of 4Gb/s (attached is a graph). 
>>>> 
>>>> I would expect the binary size to be roughly equivalent to an SFlow  record (~3MB)
>>>> <chart2.png>
>>>> Any suggestions?
>>>> 
>>>> Thanks
>>>> -james
>>>> 
>>>> On Aug 7, 2014, at 11:50 PM, David Edelman <dedelman at iname.com> wrote:
>>>> 
>>>>> There are strange things done in the midnight sun by config files that hang around waiting for a trusting soul.
>>>>> 
>>>>> I suggest that you try adding –X as the first item in your command line just to be sure that someone, sometime didn’t configure /etc/argus to capture 2048 bytes of user data or something else that takes up space.
>>>>> 
>>>>> —Dave
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> From: James Grace <jgrac002 at fiu.edu>
>>>>> Date: Monday, August 4, 2014 at 8:02 PM
>>>>> To: Carter Bullard <carter at qosient.com>
>>>>> Cc: Argus <argus-info at lists.andrew.cmu.edu>
>>>>> Subject: Re: [ARGUS] ARGUS Binary Size
>>>>> 
>>>>> Carter, 
>>>>> 
>>>>> Here is the flow records for a 5 minute trace:
>>>>> 
>>>>> /flows/South/2014/07/31$ racount -r argus.13.50
>>>>> racount   records     total_pkts     src_pkts       dst_pkts       total_bytes        src_bytes          dst_bytes
>>>>> 
>>>>> sum   1764263     35097518       24751551       10345967       43920187649        30233499247        13686688402   
>>>>> I'm currently not running argus off of an argus.conf -- I imagine it's using default values. 
>>>>> 
>>>>> Thanks, 
>>>>> -james
>>>>> 
>>>>> 
>>>>> 
>>>>> On Mon, Aug 4, 2014 at 3:58 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>>> Hey James,
>>>>>> Could be you’re seeing a lot of flows.  How many flow records are being stored?
>>>>>> 
>>>>>>    % racount -r argus.data.file.5m
>>>>>> 
>>>>>> Depending on the configuration, a flow record can be anywhere from 100-1K bytes
>>>>>> per record, on the average.  Argus output size is not sensitive to packet size,
>>>>>> except when its capturing user data. This is controlled by the
>>>>>> ARGUS_CAPTURE_DATA_LEN variable in the /etc/argus.conf file, if you have one...
>>>>>> 
>>>>>> What is that set to ????
>>>>>> 
>>>>>> Carter
>>>>>> 
>>>>>> On Aug 4, 2014, at 3:50 PM, James Grace <jgrac002 at fiu.edu> wrote:
>>>>>> 
>>>>>> > Good afternoon List,
>>>>>> >
>>>>>> > I've been collecting traces using argus and rastream off of a  DAG8.1SX.  The link is running right around 2.3Gb/s.  I've been looking into the argus.conf manpage to see if there is a way to limit the packet length stored by rastream or argus.
>>>>>> >
>>>>>> > Right now, if I run argus and rastream with these flags"
>>>>>> >
>>>>>> > #argus -d -i dag0 -P 561
>>>>>> >
>>>>>> > #rastream -S localhost -M time 5m -w /flows/South/%Y/%m/%d/argus.%H.%M -D 3&
>>>>>> >
>>>>>> > I'm getting rather large binaries for 5 minutes -- right around 1GB.  I have a feeling its grabbing all 9000bytes of the jumbo frame. Is there a workaround for this?
>>>>>> >
>>>>>> > Thanks a bunch!
>>>>>> >
>>>>>> > James
>>>>>> >
>>>>>> 
>>>>> 
>>>> 
>>> 
>> 
> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140924/c9f1c709/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140924/c9f1c709/attachment.sig>