[ARGUS] Attribute clarification

[carter at qosient.com]

I think that there maybe some information in knowing the distance between argus records, but the variables that impact the record distance is pretty independent of anything that I would think of interest.  Traffic mix, argus configuration, etc ….  And in high volume traffic sources, the counter rolls over regularly so long term behavioral trends would be confused by the number, I suspect, but that can be dealt with, without much issue ….

I think that some want to use the sequence number as a quasi key for the transaction, but we have other transaction ids that are designed for that … and of course the flow key, plus time is a much better discriminator for flow identification.

Hope all is most excellent,
Carter
 

> On Sep 6, 2019, at 10:47 AM, Giampaolo Bovenzi <giampaolo.bovenzi at unina.it> wrote:
> 
> Hi Carter,
>  
> thank for the rapidity and precision of your response.
> I have got no problem with the attribute itself.
>  
> My curiosity comes from using this attribute to conduct network traffic modeling.
> As I have seen, some researchers are using the Argus seq number as input to a modeling system, but, given your response, this attribute introduces a bias and should be discarded to model network traffic.
>  
> Thanks again and good work!
> GB
>  
> Da: carter at qosient.com <mailto:carter at qosient.com>
> Inviato: venerdì 6 settembre 2019 15:56
> A: Argus <mailto:argus-info at lists.andrew.cmu.edu>
> Oggetto: Re: [ARGUS] Attribute clarification
>  
> Hey Giampaolo,
> The Argus sequence number is a monotonically increasing record number in the ARGUS_TRANSPORT_DSR, the structure in the argus record that is used to identify the data source of data during data transport between nodes.  The 32-bit int seqnum is generated by the originating Argus data source, and is used to help understand if you’re losing data, how many have you seen, etc….  It is a curious number when you think about how you would want to get and process flow records, especially when you want to merge, aggregate and or filter the records to get to an answer to a question.  But it is there to help when needed (debugging, integrity checks when using udp transport, etc ….).
>  
> When Argus generates a flow record, the output stage puts an Argus source id and a sequence number in the “trans” data sub record.  Historically, the trans dsr has been an interesting topic.  Should it used for hop to hop loss detection ?  If so then each stage of an argus data pipeline, source -> radium -> radium -> radium -> disk, would want to either overwrite the DSR data for its transport, or we would need to add a trans dsr for each stage of the pipe.   Because the 'seq' is tied to the ‘srcid’, we have used it to provide the information needed to recover missing data from the originating source.
>  
> For data recovery, the idea is that the argus data generator would have local storage with a small retention time (days), structured as a standard argus archive, say in 5 minute files, and collectors of the full stream will use the seqnum to realize that data needs to be requested from the originator, say when the pipeline is interrupted, to recover 5 min of data. Or you could be very clever and do a selective recovery of a full set of missing seqnum’s.
>  
> Now these recovery methods are not in the open source project, but the seqnum is there to enable anyone to do this in their production network.
>  
> Having any problems ????
>  
> Carter
>  
> 
> 
> On Sep 6, 2019, at 6:49 AM, Giampaolo Bovenzi <giampaolo.bovenzi at unina.it <mailto:giampaolo.bovenzi at unina.it>> wrote:
>  
> Hello everyone,
>  
> I need some clarification on the “Argus sequence number” attribute, because there are no specific explainations in the documentation.
> In particular, I need to understand how it is constructed.
>  
> Thaks for your help,
> Giampaolo Bovenzi.
>  
> _______________________________________________
> argus mailing list
> argus at qosient.com <mailto:argus at qosient.com>
> https://pairlist1.pair.net/mailman/listinfo/argus <https://pairlist1.pair.net/mailman/listinfo/argus>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20190906/907ee82d/attachment.html>