ARGUS Binary Size

Tue Sep 16 14:38:29 EDT 2014

Definitely should be running 1 argus, opening both interfaces.
You should set a little bit more in your /etc/argus.conf file,
like the ARGUS_MONITOR_ID.

Many tests for whether your getting the data you want is to
inspect the actual ‘primitive’ data from argus.  Do any of the
flows in your collection files look like they have bi-directional
counts, more than 1 packet in each one, that type of stuff.

racluster() will merge records together, which will correct for
any problems, such as the status interval setting, which is the
value that gets you the most data reduction.

Carter

On Sep 16, 2014, at 11:19 AM, James Grace <jgrac002 at fiu.edu> wrote:

> Hi Carter, 
> 
> In regards to our flow records, when I run 
> racluster -m matrix -r argus.12.35 -w - | rasort -m pkts
> 
> During this 5 minutes period, there were ~500 flows with large packet counts, but then these packets/per flow dropped off severely — most being around 7 or 8pkts/flow. 
> 
> These 2 DAG cards are measuring the egress and ingress on a single circuit, and if I run the DAG tools (dagsnap, etc) I am seeing unique traffic over the wire on each card. 
> 
> This may just be chalked up to the way traffic if flowing over this circuit rather than anything to do with Argus. 
> 
> Thoughts?
> 
> -james
> 
> On Sep 15, 2014, at 4:01 PM, Carter Bullard <carter at qosient.com> wrote:
> 
>> Hey James,
>> The src and dst labels for traffic in argus records does not
>> equate to the number of packets we receive on the 2 DAG interfaces.
>> These src and dst are flow src and dst, and refer to the flow
>> source (the initiator of the flow transaction) and flow destination
>> (the target of the flow transaction).
>> 
>> I suspect that if there is a problem, its that argus is getting
>> packets from only one interface.
>> 
>> Carter
>> 
>> 
>> On Sep 15, 2014, at 1:42 PM, James Grace <jgrac002 at fiu.edu> wrote:
>> 
>>> David, Carter
>>> 
>>> Thanks for the help so far.  I’ve changed some things around and the new scheme is:
>>> 
>>> Two argus instances listening to a DAG card:
>>> 
>>> argus -d -i dag0 -P 561
>>> argus -d -i dag1 -P 562
>>> 
>>> I have only one option in my /etc/argus.conf:
>>> 
>>> # cat /etc/argus.confg 
>>> ARGUS_CAPTURE_DATA_LEN=88
>>> 
>>> I have also configured the DAG cards to slen=88. 
>>> 
>>> I have radium connecting to the argi thusly:
>>> radium -S localhost:561 -S localhost:562 -d -M dsrs=-net -encaps -P 563
>>> 
>>> And I have rastream connecting to radium to store and sort the argus binaries.
>>> rastream -s 48 -S localhost:563 -M dsrs=-net -M time 5m -w /flows/argus/%Y/%m/%d/argus.%H.%M -D3
>>> 
>>> I understand that each of these binaries now contain two streams of traffic. Is that correct?  Even with all these flags and slen parameters, I am seeing massive binary sizes:
>>> 
>>> ls -l
>>> total 57231320
>>> -rw-r--r--. 1 root root 5790698448 Sep 15 13:13 argus.12.45
>>> -rw-r--r--. 1 root root 8052120672 Sep 15 13:13 argus.12.50
>>> -rw-r--r--. 1 root root 8416875168 Sep 15 13:13 argus.12.55
>>> -rw-r--r--. 1 root root 7613404132 Sep 15 13:13 argus.13.00
>>> -rw-r--r--. 1 root root 6660266172 Sep 15 13:13 argus.13.05
>>> -rw-r--r--. 1 root root 6112649344 Sep 15 13:19 argus.13.15
>>> -rw-r--r--. 1 root root 7716601984 Sep 15 13:24 argus.13.20
>>> -rw-r--r--. 1 root root 8098926720 Sep 15 13:29 argus.13.25
>>> -rw-r--r--. 1 root root  143270016 Sep 15 13:30 argus.13.30
>>> 
>>> 
>>> If I run recount I’m seeing 5million records per 5 minute binary:
>>> 
>>> racount -r argus.13.25
>>> racount   records     total_pkts     src_pkts       dst_pkts       total_bytes        src_bytes          dst_bytes
>>>     sum   53181673    53191299       37622951       15568348       49381504197        28748639718        20632864479
>>> 
>>> I’m thinking there’s got to be some miscounts.  The link I’m measuring, both ingress and egress traffic sum up to about 2Gb/s with a peak of 4Gb/s (attached is a graph). 
>>> 
>>> I would expect the binary size to be roughly equivalent to an SFlow  record (~3MB)
>>> <chart2.png>
>>> Any suggestions?
>>> 
>>> Thanks
>>> -james
>>> 
>>> On Aug 7, 2014, at 11:50 PM, David Edelman <dedelman at iname.com> wrote:
>>> 
>>>> There are strange things done in the midnight sun by config files that hang around waiting for a trusting soul.
>>>> 
>>>> I suggest that you try adding –X as the first item in your command line just to be sure that someone, sometime didn’t configure /etc/argus to capture 2048 bytes of user data or something else that takes up space.
>>>> 
>>>> —Dave
>>>> 
>>>> 
>>>> 
>>>> 
>>>> From: James Grace <jgrac002 at fiu.edu>
>>>> Date: Monday, August 4, 2014 at 8:02 PM
>>>> To: Carter Bullard <carter at qosient.com>
>>>> Cc: Argus <argus-info at lists.andrew.cmu.edu>
>>>> Subject: Re: [ARGUS] ARGUS Binary Size
>>>> 
>>>> Carter, 
>>>> 
>>>> Here is the flow records for a 5 minute trace:
>>>> 
>>>> /flows/South/2014/07/31$ racount -r argus.13.50
>>>> racount   records     total_pkts     src_pkts       dst_pkts       total_bytes        src_bytes          dst_bytes
>>>> 
>>>> sum   1764263     35097518       24751551       10345967       43920187649        30233499247        13686688402   
>>>> I'm currently not running argus off of an argus.conf -- I imagine it's using default values. 
>>>> 
>>>> Thanks, 
>>>> -james
>>>> 
>>>> 
>>>> 
>>>> On Mon, Aug 4, 2014 at 3:58 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>> Hey James,
>>>>> Could be you’re seeing a lot of flows.  How many flow records are being stored?
>>>>> 
>>>>>    % racount -r argus.data.file.5m
>>>>> 
>>>>> Depending on the configuration, a flow record can be anywhere from 100-1K bytes
>>>>> per record, on the average.  Argus output size is not sensitive to packet size,
>>>>> except when its capturing user data. This is controlled by the
>>>>> ARGUS_CAPTURE_DATA_LEN variable in the /etc/argus.conf file, if you have one...
>>>>> 
>>>>> What is that set to ????
>>>>> 
>>>>> Carter
>>>>> 
>>>>> On Aug 4, 2014, at 3:50 PM, James Grace <jgrac002 at fiu.edu> wrote:
>>>>> 
>>>>> > Good afternoon List,
>>>>> >
>>>>> > I've been collecting traces using argus and rastream off of a  DAG8.1SX.  The link is running right around 2.3Gb/s.  I've been looking into the argus.conf manpage to see if there is a way to limit the packet length stored by rastream or argus.
>>>>> >
>>>>> > Right now, if I run argus and rastream with these flags"
>>>>> >
>>>>> > #argus -d -i dag0 -P 561
>>>>> >
>>>>> > #rastream -S localhost -M time 5m -w /flows/South/%Y/%m/%d/argus.%H.%M -D 3&
>>>>> >
>>>>> > I'm getting rather large binaries for 5 minutes -- right around 1GB.  I have a feeling its grabbing all 9000bytes of the jumbo frame. Is there a workaround for this?
>>>>> >
>>>>> > Thanks a bunch!
>>>>> >
>>>>> > James
>>>>> >
>>>>> 
>>>> 
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140916/01be4820/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140916/01be4820/attachment.sig>