[ARGUS] Malfunctioning of argus -S option for some pcaps

[Hang Guo]

I see. Argus uses this single queue to simulate having a seperate timer for
each flow.  Thanks for the great explanation Carter!

-Hang

On Wed, Jul 3, 2019 at 7:06 AM <carter at qosient.com> wrote:

> Hey Hang,
> Each flow cache that is being tracked is placed in a single "status”
> queue, in start time order.  The end of the queue is used to manage the
> status interval.   The output processor just looks at the end of this queue
> to decide if it's time to report the flow cache's values.  This saves
> having to manage a timer for each flow cache, which can number in the
> millions.
>
> Of course, each flow cache entry has an independent timestamp, the first
> packet’s timestamp, which is the “start time".  This works, because argus,
> as a real-time flow sensor, reads packets off the wire, where all the
> packets MUST come in in time order (nature of networking a time stamping
> each packet as it comes in, using the same clock).  When argus reads
> packets from a file, or a stream of packets from a distributed packet
> broker, it gets a little weird, as the arrival of packets are not
> real-time, and can be delayed, shaped and / or reordered.
>
> To deal with the weirdness, argus uses a very specific strategy.  When not
> reading realtime packets using a real-time sensor, you have to decide if
> you want to “believe" the timestamp in the packet or do you instead use the
> realtime system clock when argus receives the packet (whenever that might
> be).  Argus uses the timestamp provided by the packet capture facility as
> the realtime clock, and plays a lot of games trying to figure out what to
> do next when there are gaps in time.  The reasons are huge, from a
> practical perspective, as you want deterministic results when processing
> packets.  The only requirement is that the packets come in mostly in time
> order.
>
> Carter
> [image: QoSient]
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__qosient.com&d=DwMFaQ&c=clK7kQUTWtAVEOVIgvi0NU5BOUHhpN0H8p7CSfnc_gI&r=GkcNqHVldHLLhP7B8K__Og&m=uOYAcJz5kWdwAlN2DkpHXdW1nhIwo7QmA_wgvPT6JV0&s=DFLtrPSbsLRZvb0pYx7iRl0e13IlVODQh3zRsEoPb_Q&e=>
>
> Carter Bullard   <carter at qosient.com>•  Chairman / Founder / CTO
> 150 E 57th Street, Suite 12D
> New York, New York 10022-2795
> Phone +1.212.588.9133 • Mobile +1.917.497.9494
>
>
>
> On Jul 2, 2019, at 10:48 PM, Hang Guo <hangguo at usc.edu> wrote:
>
> Thanks Carter, that saves life!!!!
>
> Could you help me understand one more bit about -S option: when implemting
> this report interval (-S), I guess argus uses one timer for all flows
> instead of a seperate timer for each flow right?
>
> Thanks,
> -Hang
>
> On Tue, Jul 2, 2019 at 6:28 PM <carter at qosient.com> wrote:
>
>> Hey Hang,
>> The problem is that your packets are not in time order, so argus is doing
>> the right thing.  Best example is around packet # 3848, where the packet
>> timestamps jumps back 6.5 hours.  Nothing argus can do with that but tally
>> the packets … and all the flows will be messed up for a little while, until
>> the packet timestamps move ahead in front of the largest startime.  If you
>> can get the packets sorted in time, then things should work fine.
>>
>> Carter
>>
>> On Jul 2, 2019, at 4:19 PM, Hang Guo <hangguo at usc.edu> wrote:
>>
>> Hi,
>>
>> I found argus -S option malfunctioning for some pcaps. For example, when
>> running argus -S 10 with the pcap attached (MAC and IP anamoyzed, payload
>> dropped for privacy), instead of reportting every 5-tuple flows every 10
>> seconds, duration of some reported 5-tuple flows (as pasted below) are
>> hundreds of seconds. Just wonder what is the possible cause and is there a
>> fix to this?
>>
>> argus -S 10 -r dur_test_anon.pcapng -w - | ra -c "," -r - -s dur | sort
>>> -nr | head -5
>>> 4865.550781
>>> 296.601562
>>> 296.393066
>>> 294.411255
>>> 292.840790
>>
>>
>> Thanks,
>> -Hang
>> <dur_test_anon.pcapng>
>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20190703/c3206ddc/attachment-0001.html>