argus-3.x request (forwarded)

[Carter Bullard]

Hey Martin,
You should be using argus-3.0!!!  Or at least be playing with it ;o)
Could you resend this response to the argus mailing list, or can I?

So some clarification on your 2 items.

  1.  Duplicate packets
           So this is an interesting problem because there is very little information in
           the packet to help you differentiate between duplicates and real retransmissions.
           They aren't actually always identical packets, as the L2 information or TTLs maybe
           off, as the mirror may not be on the first hop link/interface.

           There are a few ideas.  The best would be to reject packets with different L2/tunnel
           id's but identical IP (ignoring the TTL) and transport identifiers (especially any sequence
           numbers) that arrive less than one RTT for the flow.  The RTT can be determined,
           when possible, and we can come up with reasonable default values (< 100uSec).

           Currently, argus only has cached information from the last packet  seen in either
           direction.  So if the packet train is something like this:

               1, 1, 2, 2, 3, 3, 4, 4

          we can figure it out.  But if its something like this:
               1, 2, 3, 1, 2, 3,  4, 5, 6, 4, 5, 6

          then a simple strategy is not going to do it for us.

          Programs like editcap() attempt to remove duplicates by keeping an MD5 cksum of the
          last 4 packets in a cache and rejecting matches.  This is doable, but it also is simple
          and would have some issues.

          I suspect we can do something like the two above.  Keep a hash with the timestamp, 
          on a per flow basis, look at the L2 info and keep a hashes in a queue for awhile, to
          identify duplicates, and account for them in an additional counter.

          The trick will be to inspect a bunch of packet files that capture this situation and check
          to see how best to identify the duplicates.  I can start working on this problem now, if
          we have the files.

  2.  DNS transaction capture

          You can do this today with argus.  A user data buffer capture of 256 will capture all
           the data needed to do what you want with DNS, and the program radump() will
           printout all the DNS information you need to do the tracking.  The problem is that
           with this strategy you capture 256 bytes of every flow, and that maybe an issue for
           some sites.

          Item #3 of work items for 2010, mentions control plane flow monitoring.
          DNS is THE internet Call Control protocol.  (see slides 35 and 36 of the FloCon
          argus 1/2 day tutorial, Introduction to Argus).

           So, we'll have specific support for DNS tracking in argus-3.0.4.  What this
          really means is that we will capture all the payload data in the control plane flow,
          DNS included (also DHCP, ARP, STP, RIP, OSPF, ISIS, BGP, SIP, RSVP) , so you
          don't have to grab 256 bytes of every transaction to get the control plane flows to grab
          what you want.

With regard to your perfect world, I agree, and the approach is that those jobs (correlation
between flow information elements) are the jobs of argus clients and information systems.
If argus is doing the good job, then the data is captured, but external programs are
needed to track this information.  I do this with DHCP data, but where the IP address user
mappings come from is usually an information system outside the observation domain of
argus.

The thing that we are going to add in argus-3.0.4 (this work is completed by the way) is to have
end system argus event generators provide information like, "this process owns this flow",
and "this user owns this process".  So that if you instrument your end systems, you get explicit
information.

So, what do you think?

Carter

On Jan 25, 2010, at 9:01 AM, Martin xxxxxx  wrote:

> 
>> Subject: [ARGUS] flocon 2010 presentations on the web
>> From: Carter Bullard <carter at qosient.com>
>> To: Argus <argus-info at lists.andrew.cmu.edu>
>> Date: Fri, 22 Jan 2010 14:00:43 -0500
>> 
>> Gentle people,
>> I've updated the argus home page and I've put a list of what I was going
>> to do for version 3.0.4.  If you have any ideas, I'd love to include them!!!
> 
> Hi Carter!
> 
> Two things I've been missing in my argus data:
> 
> 1.
> You already have:
>             s      -  Src TCP packet retransmissions
>             d      -  Dst TCP packet retransmissions
>             *      -  Both Src and Dst TCP retransmissions
> 
> I would like argus to distinguish between retransmissions and duplicate copies of a frame.
> 
> Why, you ask?
> Well, because it is very common that customers setup faulty SPAN mirroring. So the sensor (i.e. argus) receive two identical copies of a frame.
> (In HP procurve switches, it is even "common" to have one copy of packets in one direction but two copies in the other...)
> 
> The problem is how the switches deal with "in", "out", "both" mirroring and VLAN-mirroring (opposite to port mirroring).
> 
> 
> Right now the unwanted extra copies register as "retransmissions" even though no TCP retransmission has occurred.
> 
> I would like Argus to be able to distinguish between the two scenarios so it don't give false retransmission statistics and to help me spot customers with a faulty SPAN setup.
> 
> 
> 
> 2.
> I would like argus to store all DNS requests and/or responses (configurable).
> This way I would have a database of requested hostnames which can be used to:
> * match lookups against a database of known bad hostnames/strings
> * afterwards be able to figure out the actual hostname of a web server without the payload from the GET request header (the "Host:" line).
> 
> 
> (I currently use Argus 2.x, so if any of the above is already invented, I'm sorry to have wasted your time with this email :-)   )
> 
> 
> /Martin
> 
> 
> 
> PS.
> In a perfect world, I would like argus to be able to keep state of the "identity" behind IPs. I.e. argus should know how to decode specific protocols and look for data that might identify this IP (apart from the current IP and Mac).
> Example:
> From Windows NetBIOS packets you can get the hostname and MAC address of an IP (get MAC even if the sensor is not located on the same segment as the client).
> From NetBIOS/SMB packets you can get usernames, this is usually nice information to have when trying to determine "who did the p2p filetransfer" or whatever.
> From DNS responses you can get hostnames for IPs.
> From DHCP/bootp you can get hostnames for IPs.
> 
> Apart from the vast work of developing all the protocol decoding needed, you would also need a smart way to store changes in Identification, and even harder - methods to query this information based on time.
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100125/0fdc412c/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100125/0fdc412c/attachment.bin>