​ Netflow - Direction field in argus?

[Carter Bullard]

Hey Vlad,
The " dir " field is really just a convenience, to indicate what
the direction algorithm thinks is the direction.  The src and dst
assignments are always based on the direction of the first
packet seen, unless the direction algorithm thinks otherwise, i.e.
when there is no ' ? '.

So, why '?', two possible reasons.  You aren't getting all the packets,
and so you aren't seeing the SYN or SYN_ACK.  The algorithm only needs
one of the two packets to figure it out.  When you aren't getting
all the packets, you see " g "s in the flgs field (gaps) of TCP
and ESP traffic.

The second cause happens when argus times out a TCP cache, and
then later, traffic shows up for the flow.  We have to reestablish
the cache and track what argus thinks is new traffic and report on
the activity.  Argus uses a 60 second idle timer for every TCP
connection, by default. This is configurable in the argus.conf file.

So, crank open a TCP, type for a while, get up to get a nice
refreshing beverage or ice tea, and come back, hit a key, and
argus will be tracking a flow where it didn't see the TCP
connection establishment, and the next status record will
get a question mark in the direction field.

We need to time out the TCPs to get the memory back.  60s may be
a bit aggressive.  You can tweak the numbers, but be sure and
check argus's memory use after you make a change.

The key here is if you see a ' ? ' TCP flow, the specific flow
was probably active in the near past.  I have some flows that
are persistent, like imap TCP connections that are months old.
These give me ' ? ' all the time, and to find the initial
flow report, I may have to go back months and months.

So, its not a big deal, unless you do service specific analytics,
and you can't map the service port based on range or content.

Hopefully that is helpful !!!

Carter



On Jun 13, 2013, at 12:16 PM, Vlad Grigorescu <vladg at cmu.edu> wrote:

> On Jun 13, 2013, at 11:10 AM, Carter Bullard <carter at qosient.com> wrote:
> 
>> Because most Internet protocols have packets that are going in
>> both directions, the direction of the flow is not determined
>> by the direction of the packets, as that would always be
>> " both ", which is not a direction.  Argus's flow direction
>> is based on the direction of the first packet observed.  And
>> then for connection-oriented protocols, like TCP, we provide
>> originator status information, so the clients can correct
>> for direction when needed.
>> 
>> Netflow does not provide enough information to make a definitive
>> determination of who initiated TCP establishment.  Netflow or's
>> the TCP flags throughout the life of the TCP flow, and so you
>> cannot know who sent the SYN, nor who sent the SYN_ACK, when
>> the connection transfers data and there are lone ACK packets
>> on the path.  The resulting flags field will both have SYN_ACK
>> indications, minimally.
> 
> This is something that I've been confused about for a while, and I apologize if it's been answered before. We don't collect NetFlow with argus; we give it a copy of all the live traffic. The limitations with NetFlow make sense, but even with the full packet data, we often see records where it couldn't identify directionality. If this is just based on the direction of the first packet, we'd expect to always see a direction (albeit it could very easily be wrong).
> 
> Any idea what would be causing this?
> 
> Thanks,
> 
>  --Vlad Grigorescu
>    Carnegie Mellon University

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130613/cb60de70/attachment.bin>