[ARGUS] Attribute clarification

[carter at qosient.com]

Hey Giampaolo,
The Argus sequence number is a monotonically increasing record number in the ARGUS_TRANSPORT_DSR, the structure in the argus record that is used to identify the data source of data during data transport between nodes.  The 32-bit int seqnum is generated by the originating Argus data source, and is used to help understand if you’re losing data, how many have you seen, etc….  It is a curious number when you think about how you would want to get and process flow records, especially when you want to merge, aggregate and or filter the records to get to an answer to a question.  But it is there to help when needed (debugging, integrity checks when using udp transport, etc ….).

When Argus generates a flow record, the output stage puts an Argus source id and a sequence number in the “trans” data sub record.  Historically, the trans dsr has been an interesting topic.  Should it used for hop to hop loss detection ?  If so then each stage of an argus data pipeline, source -> radium -> radium -> radium -> disk, would want to either overwrite the DSR data for its transport, or we would need to add a trans dsr for each stage of the pipe.   Because the 'seq' is tied to the ‘srcid’, we have used it to provide the information needed to recover missing data from the originating source.

For data recovery, the idea is that the argus data generator would have local storage with a small retention time (days), structured as a standard argus archive, say in 5 minute files, and collectors of the full stream will use the seqnum to realize that data needs to be requested from the originator, say when the pipeline is interrupted, to recover 5 min of data. Or you could be very clever and do a selective recovery of a full set of missing seqnum’s.

Now these recovery methods are not in the open source project, but the seqnum is there to enable anyone to do this in their production network.

Having any problems ????

Carter


> On Sep 6, 2019, at 6:49 AM, Giampaolo Bovenzi <giampaolo.bovenzi at unina.it> wrote:
> 
> Hello everyone,
>  
> I need some clarification on the “Argus sequence number” attribute, because there are no specific explainations in the documentation.
> In particular, I need to understand how it is constructed.
>  
> Thaks for your help,
> Giampaolo Bovenzi.
>  
> _______________________________________________
> argus mailing list
> argus at qosient.com <mailto:argus at qosient.com>
> https://pairlist1.pair.net/mailman/listinfo/argus <https://pairlist1.pair.net/mailman/listinfo/argus>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20190906/bfb556f6/attachment.html>