ARGUS Binary Size

Mon Sep 15 15:42:06 EDT 2014

Hey James,
Just a point for correction.

By setting the ARGUS_CAPTURE_DATA_LEN=88, you are telling argus that
you are interested in 88 bytes more than the minimum argus packet
snap length, which is 96.  As a result, you need to set your DAG slen
to 184 bytes.

OK, just so we’re seeing the same stats, looks like your seeing 53M
records per 5 minute period ???  And looks like you’re seeing about
the same number of packets, so you’re not getting any flow 
monitoring, really.  What do your flow records look like ???  A lot
of flows with 1 packet only ???

Sflow is a statistical packet sampling technique.  Argus is a comprehensive
flow monitor, so we’re trying to process every packet.  You should get
on the average about 1:100 - 1:1000 data reduction, if all is working as
it should.  Doesn’t look like this is really working as it should...

Carter

On Sep 15, 2014, at 1:42 PM, James Grace <jgrac002 at fiu.edu> wrote:

> David, Carter
> 
> Thanks for the help so far.  I’ve changed some things around and the new scheme is:
> 
> Two argus instances listening to a DAG card:
> 
> argus -d -i dag0 -P 561
> argus -d -i dag1 -P 562
> 
> I have only one option in my /etc/argus.conf:
> 
> # cat /etc/argus.confg 
> ARGUS_CAPTURE_DATA_LEN=88
> 
> I have also configured the DAG cards to slen=88. 
> 
> I have radium connecting to the argi thusly:
> radium -S localhost:561 -S localhost:562 -d -M dsrs=-net -encaps -P 563
> 
> And I have rastream connecting to radium to store and sort the argus binaries.
> rastream -s 48 -S localhost:563 -M dsrs=-net -M time 5m -w /flows/argus/%Y/%m/%d/argus.%H.%M -D3
> 
> I understand that each of these binaries now contain two streams of traffic. Is that correct?  Even with all these flags and slen parameters, I am seeing massive binary sizes:
> 
> ls -l
> total 57231320
> -rw-r--r--. 1 root root 5790698448 Sep 15 13:13 argus.12.45
> -rw-r--r--. 1 root root 8052120672 Sep 15 13:13 argus.12.50
> -rw-r--r--. 1 root root 8416875168 Sep 15 13:13 argus.12.55
> -rw-r--r--. 1 root root 7613404132 Sep 15 13:13 argus.13.00
> -rw-r--r--. 1 root root 6660266172 Sep 15 13:13 argus.13.05
> -rw-r--r--. 1 root root 6112649344 Sep 15 13:19 argus.13.15
> -rw-r--r--. 1 root root 7716601984 Sep 15 13:24 argus.13.20
> -rw-r--r--. 1 root root 8098926720 Sep 15 13:29 argus.13.25
> -rw-r--r--. 1 root root  143270016 Sep 15 13:30 argus.13.30
> 
> 
> If I run recount I’m seeing 5million records per 5 minute binary:
> 
> racount -r argus.13.25
> racount   records     total_pkts     src_pkts       dst_pkts       total_bytes        src_bytes          dst_bytes
>     sum   53181673    53191299       37622951       15568348       49381504197        28748639718        20632864479
> 
> I’m thinking there’s got to be some miscounts.  The link I’m measuring, both ingress and egress traffic sum up to about 2Gb/s with a peak of 4Gb/s (attached is a graph). 
> 
> I would expect the binary size to be roughly equivalent to an SFlow  record (~3MB)
> <chart2.png>
> Any suggestions?
> 
> Thanks
> -james
> 
> On Aug 7, 2014, at 11:50 PM, David Edelman <dedelman at iname.com> wrote:
> 
>> There are strange things done in the midnight sun by config files that hang around waiting for a trusting soul.
>> 
>> I suggest that you try adding –X as the first item in your command line just to be sure that someone, sometime didn’t configure /etc/argus to capture 2048 bytes of user data or something else that takes up space.
>> 
>> —Dave
>> 
>> 
>> 
>> 
>> From: James Grace <jgrac002 at fiu.edu>
>> Date: Monday, August 4, 2014 at 8:02 PM
>> To: Carter Bullard <carter at qosient.com>
>> Cc: Argus <argus-info at lists.andrew.cmu.edu>
>> Subject: Re: [ARGUS] ARGUS Binary Size
>> 
>> Carter, 
>> 
>> Here is the flow records for a 5 minute trace:
>> 
>> /flows/South/2014/07/31$ racount -r argus.13.50
>> racount   records     total_pkts     src_pkts       dst_pkts       total_bytes        src_bytes          dst_bytes
>> 
>> sum   1764263     35097518       24751551       10345967       43920187649        30233499247        13686688402   
>> I'm currently not running argus off of an argus.conf -- I imagine it's using default values. 
>> 
>> Thanks, 
>> -james
>> 
>> 
>> 
>> On Mon, Aug 4, 2014 at 3:58 PM, Carter Bullard <carter at qosient.com> wrote:
>>> Hey James,
>>> Could be you’re seeing a lot of flows.  How many flow records are being stored?
>>> 
>>>    % racount -r argus.data.file.5m
>>> 
>>> Depending on the configuration, a flow record can be anywhere from 100-1K bytes
>>> per record, on the average.  Argus output size is not sensitive to packet size,
>>> except when its capturing user data. This is controlled by the
>>> ARGUS_CAPTURE_DATA_LEN variable in the /etc/argus.conf file, if you have one...
>>> 
>>> What is that set to ????
>>> 
>>> Carter
>>> 
>>> On Aug 4, 2014, at 3:50 PM, James Grace <jgrac002 at fiu.edu> wrote:
>>> 
>>> > Good afternoon List,
>>> >
>>> > I've been collecting traces using argus and rastream off of a  DAG8.1SX.  The link is running right around 2.3Gb/s.  I've been looking into the argus.conf manpage to see if there is a way to limit the packet length stored by rastream or argus.
>>> >
>>> > Right now, if I run argus and rastream with these flags"
>>> >
>>> > #argus -d -i dag0 -P 561
>>> >
>>> > #rastream -S localhost -M time 5m -w /flows/South/%Y/%m/%d/argus.%H.%M -D 3&
>>> >
>>> > I'm getting rather large binaries for 5 minutes -- right around 1GB.  I have a feeling its grabbing all 9000bytes of the jumbo frame. Is there a workaround for this?
>>> >
>>> > Thanks a bunch!
>>> >
>>> > James
>>> >
>>> 
>> 
> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140915/19f763af/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 455 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140915/19f763af/attachment.sig>