Basic question regarding how flows are build.

[el draco]

Thanks a lot John and Carter. I totally forgot about the reporting
status interval!

For the offline analysis of the flows, it is better for me to have a
very large status interval. Now I have what I was expecting.

Racluster is not an option for my research because my model is using
the time differences between flows to compute the periodicity of the
connection. And the times between flows are lost in racluster.
I know there are inter-packet arrival statistics and at some point I
though about having inter-flows arrival statistics in racluster,
however having only the average and mean of these times is not enough
for me. I'm working on a model that uses all the time differences and
then it models the behavior of the connection.

Thanks again for your help! Argus is the best tool for this.
sebas

On Wed, Jan 8, 2014 at 5:13 PM, Carter Bullard <carter at qosient.com> wrote:
> Hey Sebas,
> You are generating multiple 5 second status reports for the same flow.
>
> You can change the status interval, using the “-S <secs>” option, or
> you can aggregate the multiple status reports using racluster().
>
>    argus -r pcap.file -S 60 -w - | ra
>    argus -r pcap.file -w - | racluster
>
> If you want infinite interval, just use a very large number for -S.
>
> The flow status interval is the basis for argus flow generation,
> and it provides a deterministic behavior for flow reporting
> regardless of the flow.  It allows the sensor output to be sorted
> in time, and it removes the requirement that the sensor respond
> to the state of traffic on the wire (very important) !!!!
>
> This strategy also allows you to realize very early that a flow
> is in the network, and it provides periodic indications of its
> presence, when you want to build real-time applications around
> live sensor data.  It sure does beat having to wait until a
> flow is closed to get a report (which could take months).
>
> The advantages of getting multiple records per flow far outweighs
> the shortcomings of trying to cache variable duration flows and
> report only when they are done, in the sensor.
>
> Post processing the sensor output to generate the output you want
> scales in performance and function.  At least that is why it is
> the way that it is.
>
>
> Carter
>
>
> On Jan 8, 2014, at 9:38 AM, el draco <eldraco at gmail.com> wrote:
>
> Hi list, first of all Happy new year to you all!
>
> I have been analyzing some botnet traffic lately and I come across a
> simple question that I was not able to answer myself. Maybe you can
> help me.
>
> I had a simple tcp connection that is encrypted (attached test.pcap).
> If you look at it with wireshark and you search for Connections, you
> will see only one connection.
>
> Even if you use tools like tcpflow, it will find 2 flows. One for each
> direction.
>
> However, using latest argus 3.0.7.5 and latest argus-clients 3.0.7.18,
> it will find 15 bidirectional flows:
>
> StartTime,Dur,Label,Proto,SrcAddr,Sport,Dir,DstAddr,Dport,State,sTos,dTos,TotPkts,TotBytes
> 1970/02/16 02:58:09.809464,3.966546,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,SPA_SPA,0,0,38,24813
> 1970/02/16 02:58:15.153867,4.980870,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,19,19458
> 1970/02/16 02:58:20.469524,4.854012,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,54,45924
> 1970/02/16 02:58:25.518589,3.926445,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,43,33042
> 1970/02/16 02:58:30.823824,4.604630,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,53,44886
> 1970/02/16 02:58:36.259177,4.311461,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,68,49712
> 1970/02/16 02:58:41.263806,4.964994,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,57,44038
> 1970/02/16 02:58:46.597238,4.228112,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,77,59454
> 1970/02/16 02:58:51.967958,3.855925,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,43,38526
> 1970/02/16 02:58:57.525492,4.545178,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,52,45816
> 1970/02/16 02:59:02.704674,4.825829,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,87,68846
> 1970/02/16 02:59:07.732428,4.602490,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,87,68914
> 1970/02/16 02:59:12.884814,4.989438,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,83,67242
> 1970/02/16 02:59:17.902786,4.494702,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,A_PA,0,0,81,62886
> 1970/02/16 02:59:23.103462,0.371401,,tcp,10.0.2.106,59540,
> ->,151.233.138.31,9338,FPA_FA,0,0,6,328
>
> Do you know what is happening? Why argus is seeing so many flows here?
>
> I'm attaching also both configurations so you can try it. I also tried
> with default configurations and is the same.
>
> What is special in this botnet traffic that is causing this?
>
> This was giving me some trouble since I was counting the amount of
> flows on each connection when I realized this.
>
> Thanks in advance!
> sebas
> <test.pcap><test.biargus><argus.conf><ra.conf>
>
>
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E 57th Street Suite 12D
> New York, New York  10022
>
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
>
>
>
>
>