Argus vs SiLK

[Carter Bullard]

Chris,
I think you missed the point.  George used to use YAF+SiLK, but has switched to Argus.
I don't think he is going to switch back this week, but I don't want to talk for George.  He
is very capable, indeed.

For you to say that YAF is "always bi-flow" doesn't really mean that YAF+SiLK users can
work with or process bi-directional flow data.  And it doesn't mean that YAF, in its "bi-flow"
records, contains the semantics that people need from bi-directional monitoring.

YAF has "bi-flow" counters for IP based Type P flows (flows where the IP packet types
are the same on both sides of the flow), but there isn't any bi-directional state that YAF
is tracking or reporting.  Argus supports 14 bi-directional flow models, most of which are
not supported by YAF.  Bi-directional flows like Layer 2 and Sub-layer 3 only flows,
7+ tuple IP application and control plane flows, OSI flows, Infiniband flows, and
of course there are the Type P1/P2 ones, like:

    1) ICMP packets being paired with the flow that caused the ICMP
       packet to be generated.

    2) Broadcast requests being paired with unicast responses, for
       both Layer 2 and Layer 3 flows.

    3) Layer 3/4 flows that have different Layer 2/3/4 encapsulations
       (one side of the flow is, say, IPv4 over ethernet based TEREDO
       [ethernet-IPv4-udp-IPv6-IPv4], the return side is just 
       IPv4 over a PPP link).   YAF can do GRE tunnel parsing, could
       YAF match up two halves of a flow, where one is in a GRE tunnel
       and the other side is not?  I don't think it can do that.  
       That would seem important though, for network forensics and all.

I'm pretty sure that YAF+SiLK is completely incapable of processing any king of bi-directional
flow data, and so it can't formulate or process forensics queries that are based on network
properties like connectivity, or reachability or availability.  YAF+SiLK can't be used to realize
the state of a network of connected devices, because it does not capture enough information
to reflect the state of a network of connected devices.

I think that is what George was referring to.

It is completely untrue that flow data is so unique that databases can't effectively process it.
I work with the National Center for Data Mining, and they think they can process flow
data without any problems, for a lot less money than what you are touting.

Argus does fine with many SQL systems, and we use MySQL in the distribution, because it
doesn't cost  anything, and its works great for most sites.  Your sponsors spend money
with other groups to do database work with YAF data.   So it is not true that your sponsor
could care less.  They just aren't asking you to do that work.

Carter



On Jul 28, 2010, at 1:49 PM, Chris Inacio wrote:

> George,
> 
> We have stuff we use internally to do that, or similar work.  (A lot of python lying around…)  We've never productized most of it.  NAF can still be downloaded, and you can do that for biflow, but its a little long in the tooth.
> 
> That said if Argus is working for you, why switch?  If there is some other reason for you to switch to YAF, and you really want to go to SQL - we can have an offline discussion.  Really, most SQL systems aren't really made to handle flow information well.  I want to do some testing with Infobright / MySQL.  If it performs well, then we might release something that will populate a SQL system from YAF.  Unfortunately, while you might think that's great (and maybe even Carter too,) my sponsors could care less.  So this is something that we have to do on our own mostly.
> 
> 
> Chris Inacio
> inacio at cert.org
> 
> 
> On Jul 28, 2010, at 7:55 AM, George Jones wrote:
> 
>> On Tue, Jul 27, 2010 at 9:24 PM, Chris Inacio <inacio at cert.org> wrote:
>> 
>> 
>> On Jul 26, 2010, at 2:26 PM, Carter Bullard wrote:
>> 
>> 
>> YAF is always biflow, there is a command line switch for it to emit into 2 uniflow records; internally it is completely biflow - no options.
>> 
>> But it's a moot point unless you have a set of analysis tools behind it that can operate on the biflow.   Is there a biflow/IPFIX-aware version of SiLK?
>> Is there some other tool set that I'm not are of (YAF->database) that consumes IPFIX/biflow ?
>>  
>> ---George Jones

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100728/0714be17/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100728/0714be17/attachment.bin>