[ARGUS] the packet and byte count are unreasonably high

Wed Nov 13 12:08:19 EST 2024

Hey Ming,
Well, I found few bugs because you’re running v3, so no problems at all …
Argus clients should do the right thing regardless of version …

I still need to look at how argus works with sequence number roll-over, as that is the condition I believe where we are getting into trouble.
The data in the records is good, except for the metrics ... because this is a little-endian machine, there is an opportunity to mess up the host to network byte conversions we need to do to export the records … there is supplemental information for each TCP, like window performance, out of order tracking,  gaps, loss, and retransmission rates, and all that data looks ok for the flow records that are reporting wild pkts and bytes … so I’m still working it …

Carter

> On Nov 13, 2024, at 11:48 AM, Ming Fu <Ming.Fu at esentire.com> wrote:
> 
> Hi Carter,
>  
> You are right. The affected machine is still running version 3, not sure how I missed this. Sorry for the confusion of the versions. I will upgrade and test.
>  
> Thanks,
> Ming
>  
> From: Carter Bullard <carter at qosient.com>
> Sent: Wednesday, November 13, 2024 11:09 AM
> To: Ming Fu <Ming.Fu at esentire.com>
> Cc: Argus <argus-info at lists.andrew.cmu.edu>
> Subject: Re: [ARGUS] the packet and byte count are unreasonably high
>  
> Hey Ming,
> With regards to our bug … is the argus that is generating the issue an argus-3.0 sensor ??
>  
> Carter
>  
> 
> 
> On Nov 11, 2024, at 3:29 PM, Ming Fu <Ming.Fu at esentire.com <mailto:Ming.Fu at esentire.com>> wrote:
>  
> Hi Carter,
>  
> We use the plain default configure command to run the auto configure.
>  
> Regards,
> Ming
>  
> From: Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>>
> Sent: Monday, November 11, 2024 1:12 PM
> To: Ming Fu <Ming.Fu at esentire.com <mailto:Ming.Fu at esentire.com>>
> Cc: Argus <argus-info at lists.andrew.cmu.edu <mailto:argus-info at lists.andrew.cmu.edu>>
> Subject: Re: [ARGUS] the packet and byte count are unreasonably high
>  
> @ 3.5Gbps we'll tickle the 64-bit counters in argus with a 30s flow duration ... should not be a problem but .... very interesting regarding LBL_ALIGN not being defined ... maybe a real hint ...
>  
> Carter
> <image001.png> <http://qosient.com/>	
>  
> Carter Bullard  <mailto:carter at qosient.com>• CEO
> 250 E 53rd Street Suite 501
> New York, New York 10022
> Phone +1.212.588.9133 • Mobile +1.917.497.9494
> 
> 
> 
> On Nov 11, 2024, at 1:08 PM, Ming Fu <Ming.Fu at esentire.com <mailto:Ming.Fu at esentire.com>> wrote:
> 
> Hi Carter,
> 
> It happens on machine with 10G and machine with multiple 1G interfaces, however, the total throughput is no more than 3.5G at peak.
> 
> Regards,
> Ming
> -----Original Message-----
> From: Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>>
> Sent: Monday, November 11, 2024 11:36 AM
> To: Ming Fu <Ming.Fu at esentire.com <mailto:Ming.Fu at esentire.com>>
> Cc: Argus <argus-info at lists.andrew.cmu.edu <mailto:argus-info at lists.andrew.cmu.edu>>
> Subject: Re: [ARGUS] the packet and byte count are unreasonably high
> 
> And one last question … what is the max bandwidth of the links you're monitoring ?  (10 Gbps ??)
> Carter
> 
> 
> 
> 
> On Nov 11, 2024, at 11:26 AM, Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>> wrote:
>  
> Hey Ming,
> Important question … what is the status of the LBL_ALIGN variable in your ./include/argus_config.h file ???
> On my systems it is defined …
>  
>  % grep LBL_ALIGN ./include/argus_config.h
>  include/argus_config.h:#define LBL_ALIGN /**/
>  
> Carter
>  
> On Nov 11, 2024, at 10:42 AM, Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>> wrote:
>  
> Hey Ming,
> Based on your earlier email … this should work to generate an argus file with about 24 records in it that would include errant flows as a well as reasonable flows for the same flow ??
>  
> % ra -w /tmp/argus.big.counter.flow.out -r argus.vsniff1.2024-10-11-22* - src host 10.61.6.12 and port 62275
>  
> If you can grab even tighter times, if you can get the specific flow between 2024-10-11.22:15:06 - 2024-10-11.22:21:15
> That should catch normal -> errant -> normal for a single flow …
>  
> All ra* programs can write its output to an argus data file, so by using the filter, you can grab the flows you want and create a manageable file ...
>  
> Carter
>  
> On Nov 11, 2024, at 10:23 AM, Ming Fu <Ming.Fu at esentire.com <mailto:Ming.Fu at esentire.com>> wrote:
>  
> Hi Carter,
>  
> The problem does not happen often, so unless we search for it on purpose across a large set of archives, we may not see it. We notice the problem mostly because we hit it during a query. I can't reproduce the problem in testing environment.
>  
> Is there a command to extract just the affected connection from the original archive file into a smaller archive? There are barriers other than just the size to share the full archive.
>  
> Regards
> Ming
>  
> -----Original Message-----
> From: Carter Bullard <carter at qosient.com <mailto:carter at qosient.com>>
> Sent: Monday, November 11, 2024 10:12 AM
> To: Ming Fu <Ming.Fu at esentire.com <mailto:Ming.Fu at esentire.com>>
> Cc: Argus <argus-info at lists.andrew.cmu.edu <mailto:argus-info at lists.andrew.cmu.edu>>
> Subject: Re: [ARGUS] the packet and byte count are unreasonably high
>  
> Hey Ming,
> We were working on this issue last year about this same time …. And in June/July (?) you thought we had fixed the problem …
> No problem, just wanting to know if it went away and then came back ??  Or maybe we were just lucky ??
>  
> Can you share a recent binary file of a record that is tooooo big ??
>  
> Carter

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20241113/63688c44/attachment-0001.htm>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 1385 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20241113/63688c44/attachment-0001.bin>