Argus-info Digest, Vol 42, Issue 1
Ken A
ka at pacific.net
Thu Feb 5 17:52:45 EST 2009
Carter Bullard wrote:
> Oh and I forgot to mention what the graph was.
>
> That 3D ArgusGraph is a graph of UDT performance analysis. UDT is a new
> transport protocol developed at the National Data Mining Center, at
> the Univ of Chicago.
>
> The idea is that you need to correlate the window size, the flow control
> state,
> loss indications and the expected (calculated) available rate to
> understand if
> UDT is "filling up the pipe" efficiently.
>
> So we configured argus to generate argus status records every
> millisecond, and
> graphed the pertinent metrics in a 3D waterfall, which made it real easy
> to 'eye ball' that a problem existed. More of a demonstration than an
> analytic, but
> sometimes you have to demonstrate the problem, before anyone will agree
> that
> there is a problem to solve. Worked very well!!!!
>
> The graph shows that there was an inefficiency in how UDT adjusted to
> host flow control
> conditions, thus the big gaps. The session is trying to move a file
> from one machine
> to another at multi-gigabit speeds, but the receiving machine's disk is
> a bit slow, so
> it flow controls the sender occasionally. That is what it is suppose to
> do, but
> the inefficiency lies in how quickly the receiver notifies the sender to
> fire it back up,
> when the disk is ready to receive again.
>
> They thought the problem was network loss.
> Just thought you'd like to know.
>
Absolutely! Very interesting stuff.
Thanks,
Ken
> Carter
>
>
> On Feb 5, 2009, at 3:51 PM, Ken A wrote:
>
>> CS Lee wrote:
>>> hi Carter,
>>> I grab your control plane presentation from here -
>>> https://tools.netsa.cert.org/wiki/display/tt/FloCon+2009
>>
>> Wow. Where can I get the ArgusGraph that generated that nice 3d graph
>> in the presentation? That's a nice visualization of whatever it is. :-)
>> Ken
>>
>>
>>> I'm definitely a user, can't guarantee to be customer unless my boss
>>> says
>>> yes ;]
>>> The mysql support is definitely a long wait ;]
>>> On Thu, Feb 5, 2009 at 11:51 PM, Carter Bullard <carter at qosient.com>
>>> wrote:
>>>> Thanks, I don't think I gave them slides notes, .... didn't know if
>>>> itwas
>>>> intelligible without some form of narrative.
>>>>
>>>> So I have a bunch of mysql database support for argus almost ready
>>>> for public consumption, would you be a potential customer/user?
>>>>
>>>> I use it primarily to hold real-time "ratop" like state so multiple
>>>> people/programs can work off of the data at the same time, but
>>>> some people have had good luck using it to hold flow data and
>>>> to keep track of say all the IP addresses they've seen since the
>>>> beginning, in real-time.
>>>>
>>>> Carter
>>>>
>>>>
>>>> On Feb 4, 2009, at 9:11 PM, CS Lee wrote:
>>>>
>>>> hi Carter,
>>>>
>>>> Thanks. Things are all good, btw, I read your presentation in
>>>> Flocon2009,
>>>> that's really cool stuff.
>>>>
>>>> On Thu, Feb 5, 2009 at 3:21 AM, Carter Bullard <carter at qosient.com>
>>>> wrote:
>>>>
>>>>> Hmmmm,I have them in the development directory, here. Let me see
>>>>> what's
>>>>> up.
>>>>> If we can fix the ongoing v2 -> v3 conversion problem soon, I'll
>>>>> release
>>>>> another round of client programs, and it will definitely have them in
>>>>> there.
>>>>>
>>>>> How are things?
>>>>>
>>>>> Carter
>>>>>
>>>>> On Feb 4, 2009, at 1:22 PM, CS Lee wrote:
>>>>>
>>>>> hi carter,
>>>>>
>>>>> i can't find both rauserdata() and raservice() in argus 3.0
>>>>> distribution
>>>>> which I downloaded from
>>>>>
>>>>> ftp://qosient.com/dev/argus-3.0/
>>>>>
>>>>> Thanks!
>>>>>
>>>>>
>>>>> On Tue, Feb 3, 2009 at 11:46 PM,
>>>>> <argus-info-request at lists.andrew.cmu.edu
>>>>>> wrote:
>>>>>> Send Argus-info mailing list submissions to
>>>>>> argus-info at lists.andrew.cmu.edu
>>>>>>
>>>>>> To subscribe or unsubscribe via the World Wide Web, visit
>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
>>>>>> or, via email, send a message with subject or body 'help' to
>>>>>> argus-info-request at lists.andrew.cmu.edu
>>>>>>
>>>>>> You can reach the person managing the list at
>>>>>> argus-info-owner at lists.andrew.cmu.edu
>>>>>>
>>>>>> When replying, please edit your Subject line so it is more specific
>>>>>> than "Re: Contents of Argus-info digest..."
>>>>>>
>>>>>>
>>>>>> Today's Topics:
>>>>>>
>>>>>> 1. ArgusGenerateRecord: packet size type not defined
>>>>>> (Michael Grinnell)
>>>>>> 2. argus usage (Oguz Yarimtepe)
>>>>>> 3. Re: argus usage (Stewart Gray)
>>>>>> 4. Re: ArgusGenerateRecord: packet size type not defined
>>>>>> (Peter Van Epp)
>>>>>> 5. Re: argus usage (Peter Van Epp)
>>>>>> 6. Re: argus usage (Oguz Yarimtepe)
>>>>>> 7. Re: ArgusGenerateRecord: packet size type not defined
>>>>>> (Carter Bullard)
>>>>>> 8. argus user data buffer analysis (Carter Bullard)
>>>>>>
>>>>>>
>>>>>> ----------------------------------------------------------------------
>>>>>>
>>>>>>
>>>>>> Message: 1
>>>>>> Date: Mon, 02 Feb 2009 13:39:44 -0500
>>>>>> From: Michael Grinnell <grinnell at american.edu>
>>>>>> Subject: [ARGUS] ArgusGenerateRecord: packet size type not defined
>>>>>> To: argus-info <argus-info at lists.andrew.cmu.edu>
>>>>>> Message-ID: <49873DF0.6090305 at american.edu>
>>>>>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> Periodically Argus dies on my test system with the error
>>>>>> "ArgusGenerateRecord: packet size type not defined." The time
>>>>>> between
>>>>>> these errors varies, sometimes it's only a minute or two after argus
>>>>>> starts, other times it can be > 15 minutes. I've tried running a
>>>>>> simultaneous tcpdump, then running the resulting capture file through
>>>>>> argus, but I can't replicate the error. I also don't see any glaring
>>>>>> errors in the capture file around the time it dies. This happens
>>>>>> with
>>>>>> argus 3.0.0 and with argus 3.0.1 beta2. The system is running CentOS
>>>>>> 5.2 and is listening on a dedicated interface (NC7782, bnx2
>>>>>> driver) to a
>>>>>> span port off of a Cisco switch. I have also updated to the
>>>>>> newest bnx2
>>>>>> drivers, but it still recurs. I'm trying to scare up another NIC
>>>>>> to try
>>>>>> as well.
>>>>>>
>>>>>> Thoughts?
>>>>>>
>>>>>> --
>>>>>> Michael Grinnell
>>>>>> Information Security Engineer
>>>>>> The American University
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 2
>>>>>> Date: Mon, 02 Feb 2009 22:50:20 +0200
>>>>>> From: Oguz Yarimtepe <comp.ogz at gmail.com>
>>>>>> Subject: [ARGUS] argus usage
>>>>>> To: argus-info at lists.andrew.cmu.edu
>>>>>> Message-ID: <1233607820.7050.12.camel at gezenti>
>>>>>> Content-Type: text/plain
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I want some information about argus capabilities. I tried argus a bit
>>>>>> and read the wiki. I want to learn whether it is possible to get
>>>>>> application level information from a flow record by using argus, like
>>>>>> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
>>>>>> As i understood from its usage it is possible to get these
>>>>>> information
>>>>>> indirectly by analyzing the argus output but maybe there is a way
>>>>>> that
>>>>>> argus serves that i don't know.
>>>>>>
>>>>>> Thanx.
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 3
>>>>>> Date: Tue, 3 Feb 2009 10:26:06 +1300
>>>>>> From: "Stewart Gray" <Stewart.Gray at safecom.co.nz>
>>>>>> Subject: Re: [ARGUS] argus usage
>>>>>> To: <argus-info at lists.andrew.cmu.edu>
>>>>>> Message-ID:
>>>>>> <
>>>>>> 4C08B31BFF3B5D42BCAC255ADDD6EE5EA8769D at tasbe116.advancedsolutions.co.nz>
>>>>>>
>>>>>>
>>>>>> Content-Type: text/plain; charset="us-ascii"
>>>>>>
>>>>>> Hi Oguz,
>>>>>>
>>>>>> Argus supports the Berkeley packet filter (as other apps like tcpdump
>>>>>> and ngrep do) so you're able to lock queries down to port numbers,
>>>>>> specific hosts etc. Knowing the port number an application runs on
>>>>>> means
>>>>>> it fairly easy to pull out transactions for a particular protocol.
>>>>>> Keep
>>>>>> in mind it doesn't operate at layer 7, flow information for a
>>>>>> protocol
>>>>>> such as HTTP will be very similar to SMTP. If you're after more
>>>>>> in-depth
>>>>>> analysis of a transaction, i.e. what happened during an http
>>>>>> connection
>>>>>> (GET, POST, HEAD etc) you're probably best to be taking full packet
>>>>>> captures alongside argus so you can run them through another tool
>>>>>> such
>>>>>> as wireshark.
>>>>>>
>>>>>> Have a look at this wiki page for some common usages -
>>>>>> http://nsmwiki.org/index.php?title=Argus.
>>>>>>
>>>>>> Cheers,
>>>>>>
>>>>>> Stewart
>>>>>>
>>>>>> -----Original Message-----
>>>>>> From:
>>>>>> argus-info-bounces+stewart.gray=safecom.co.nz at lists.andrew.cmu.edu
>>>>>> [mailto:argus-info-bounces+stewart.gray<argus-info-bounces%2Bstewart.gray>
>>>>>>
>>>>>> =safecom.co.nz at lists.andrew.cmu.e
>>>>>> du] On Behalf Of Oguz Yarimtepe
>>>>>> Sent: Tuesday, 3 February 2009 9:50 a.m.
>>>>>> To: argus-info at lists.andrew.cmu.edu
>>>>>> Subject: [ARGUS] argus usage
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I want some information about argus capabilities. I tried argus a bit
>>>>>> and read the wiki. I want to learn whether it is possible to get
>>>>>> application level information from a flow record by using argus, like
>>>>>> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
>>>>>> As i understood from its usage it is possible to get these
>>>>>> information
>>>>>> indirectly by analyzing the argus output but maybe there is a way
>>>>>> that
>>>>>> argus serves that i don't know.
>>>>>>
>>>>>> Thanx.
>>>>>>
>>>>>>
>>>>>> #####################################################################################
>>>>>>
>>>>>> Important: This electronic message and attachments (if any) are
>>>>>> confidential
>>>>>> and may be legally privileged. If you are not the intended
>>>>>> recipient do
>>>>>> not
>>>>>> copy, disclose or use the contents in any way. Please let us know by
>>>>>> return
>>>>>> e-mail immediately and then destroy this message.
>>>>>>
>>>>>> #####################################################################################
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 4
>>>>>> Date: Mon, 2 Feb 2009 20:51:05 -0800
>>>>>> From: Peter Van Epp <vanepp at sfu.ca>
>>>>>> Subject: Re: [ARGUS] ArgusGenerateRecord: packet size type not
>>>>>> defined
>>>>>> To: argus-info at lists.andrew.cmu.edu
>>>>>> Message-ID: <20090203045105.GA26496 at sfu.ca>
>>>>>> Content-Type: text/plain; charset=us-ascii
>>>>>>
>>>>>> On Mon, Feb 02, 2009 at 01:39:44PM -0500, Michael Grinnell wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> Periodically Argus dies on my test system with the error
>>>>>>> "ArgusGenerateRecord: packet size type not defined." The time
>>>>>>> between
>>>>>>> these errors varies, sometimes it's only a minute or two after argus
>>>>>>> starts, other times it can be > 15 minutes. I've tried running a
>>>>>>> simultaneous tcpdump, then running the resulting capture file
>>>>>>> through
>>>>>>> argus, but I can't replicate the error. I also don't see any
>>>>>>> glaring
>>>>>>> errors in the capture file around the time it dies. This happens
>>>>>>> with
>>>>>>> argus 3.0.0 and with argus 3.0.1 beta2. The system is running
>>>>>>> CentOS
>>>>>>> 5.2 and is listening on a dedicated interface (NC7782, bnx2
>>>>>>> driver) to
>>>>>> a
>>>>>>> span port off of a Cisco switch. I have also updated to the newest
>>>>>> bnx2
>>>>>>> drivers, but it still recurs. I'm trying to scare up another NIC to
>>>>>> try
>>>>>>> as well.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> --
>>>>>>> Michael Grinnell
>>>>>>> Information Security Engineer
>>>>>>> The American University
>>>>>> Setting
>>>>>>
>>>>>> ARGUS_PACKET_CAPTURE_FILE="/var/log/argus/packet.out"
>>>>>>
>>>>>> in an argus.rc file will capture the input packets from pcap in to
>>>>>> the
>>>>>> specified file. If you can get lucky and get a failure before you
>>>>>> run out
>>>>>> of
>>>>>> disk space one of the last packets in the file should tell us what
>>>>>> argus
>>>>>> isn't
>>>>>> liking. On a busy link this file will get large fast but if it
>>>>>> sometimes
>>>>>> fails quickly you may be lucky (you are also likely to see packet
>>>>>> loss
>>>>>> due to
>>>>>> the disk I/O on the sensor but hopefully the fault will still occur).
>>>>>> It looks like the argus record is malformed (it is complaining
>>>>>> that it
>>>>>> doesn't recognize the type in argus/ArgusModeler.c at line 2904 in
>>>>>> the
>>>>>> argus-3.0.1.beta.2 code). A dump of the offending packet should tell
>>>>>> Carter
>>>>>> why (or if the incoming packet is corrupted which is also possible).
>>>>>>
>>>>>> Peter Van Epp
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 5
>>>>>> Date: Mon, 2 Feb 2009 21:06:59 -0800
>>>>>> From: Peter Van Epp <vanepp at sfu.ca>
>>>>>> Subject: Re: [ARGUS] argus usage
>>>>>> To: argus-info at lists.andrew.cmu.edu
>>>>>> Message-ID: <20090203050659.GA8930 at sfu.ca>
>>>>>> Content-Type: text/plain; charset=us-ascii
>>>>>>
>>>>>> On Mon, Feb 02, 2009 at 10:50:20PM +0200, Oguz Yarimtepe wrote:
>>>>>>> Hi,
>>>>>>>
>>>>>>> I want some information about argus capabilities. I tried argus a
>>>>>>> bit
>>>>>>> and read the wiki. I want to learn whether it is possible to get
>>>>>>> application level information from a flow record by using argus,
>>>>>>> like
>>>>>>> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
>>>>>>> As i understood from its usage it is possible to get these
>>>>>>> information
>>>>>>> indirectly by analyzing the argus output but maybe there is a way
>>>>>>> that
>>>>>>> argus serves that i don't know.
>>>>>>>
>>>>>>> Thanx.
>>>>>>>
>>>>>> Depends on what you need. If you enable user data capture
>>>>>> (the -U
>>>>>> option on the argus) it will capture up to the first 512 bytes of the
>>>>>> user
>>>>>> data of the flow. That may or may not give you enough information
>>>>>> about
>>>>>> the
>>>>>> flow to do what you want. Note that on a fast link best results
>>>>>> are going
>>>>>> to
>>>>>> occur using a DAG card as the data capture adds a fairly heavy
>>>>>> load to
>>>>>> the
>>>>>> server. To display the data with ra (for instance) you need to use
>>>>>> the -s
>>>>>> command to add suser and duser to the output (as in
>>>>>>
>>>>>> ra -r argus_file -n -s +suser:512 -s +duser:512
>>>>>>
>>>>>> which will tack the user data on the end of the line. This of course
>>>>>> raises a
>>>>>> number of sticky privacy issues that you need to have considered and
>>>>>> gotten
>>>>>> approved by appropriate management of the link you are tapping
>>>>>> (which may
>>>>>> or
>>>>>> may not be you :-)).
>>>>>>
>>>>>> Peter Van Epp
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 6
>>>>>> Date: Tue, 3 Feb 2009 07:59:33 +0200
>>>>>> From: Oguz Yarimtepe <comp.ogz at gmail.com>
>>>>>> Subject: Re: [ARGUS] argus usage
>>>>>> To: argus-info at lists.andrew.cmu.edu
>>>>>> Message-ID:
>>>>>> <20831c740902022159q176610e5nb954f5d96e211d13 at mail.gmail.com>
>>>>>> Content-Type: text/plain; charset="iso-8859-9"
>>>>>>
>>>>>>> Depends on what you need. If you enable user data capture
>>>>>>> (the
>>>>>> -U
>>>>>>> option on the argus) it will capture up to the first 512 bytes of
>>>>>>> the
>>>>>> user
>>>>>>> data of the flow. That may or may not give you enough information
>>>>>>> about
>>>>>> the
>>>>>>> flow to do what you want. Note that on a fast link best results are
>>>>>> going
>>>>>>> to
>>>>>>> occur using a DAG card as the data capture adds a fairly heavy
>>>>>>> load to
>>>>>> the
>>>>>>> server. To display the data with ra (for instance) you need to
>>>>>>> use the
>>>>>> -s
>>>>>>> command to add suser and duser to the output (as in
>>>>>>>
>>>>>>> ra -r argus_file -n -s +suser:512 -s +duser:512
>>>>>>>
>>>>>>> which will tack the user data on the end of the line. This of course
>>>>>> raises
>>>>>>> a
>>>>>>> number of sticky privacy issues that you need to have considered and
>>>>>> gotten
>>>>>>> approved by appropriate management of the link you are tapping
>>>>>>> (which
>>>>>> may
>>>>>>> or
>>>>>>> may not be you :-)).
>>>>>>>
>>>>>>> Peter Van Epp
>>>>>>
>>>>>> What i am willing to do is to characterize the network traffic by
>>>>>> using
>>>>>> some characteristics derived from flow information. My final decision
>>>>>> about
>>>>>> a flow record will be whether the flow belongs to a chat session,
>>>>>> a mail
>>>>>> transfer, a FTP connection, a web browsing, ...
>>>>>>
>>>>>> I had discovered Bro which has identifiers related with high level
>>>>>> protocols. The protocol family that it supports is not as much as
>>>>>> Argus
>>>>>> does
>>>>>> so i was planning to go on with Argus.
>>>>>>
>>>>>>
>>>>>> --
>>>>>> O?uz Yar?mtepe
>>>>>> www.loopbacking.info
>>>>>> -------------- next part --------------
>>>>>> An HTML attachment was scrubbed...
>>>>>> URL:
>>>>>> https://lists.andrew.cmu.edu/mailman/private/argus-info/attachments/20090203/2e9c8a9c/attachment-0001.html
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 7
>>>>>> Date: Tue, 3 Feb 2009 10:07:03 -0500
>>>>>> From: Carter Bullard <carter at qosient.com>
>>>>>> Subject: Re: [ARGUS] ArgusGenerateRecord: packet size type not
>>>>>> defined
>>>>>> To: Michael Grinnell <grinnell at american.edu>
>>>>>> Cc: argus-info <argus-info at lists.andrew.cmu.edu>
>>>>>> Message-ID: <D2F59B8C-3239-475E-AE5C-4DCE7C819151 at qosient.com>
>>>>>> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>>>>>>
>>>>>> Hey Michael,
>>>>>> Wow, pretty obvious bug in the flow's packet size reporting feature.
>>>>>> Try this patch, and we'll see if it doesn't work for you.
>>>>>>
>>>>>> Carter
>>>>>>
>>>>>> ==== //depot/argus/argus/argus/ArgusModeler.c#62 -
>>>>>> /home/carter/argus/
>>>>>> argus/argus/ArgusModeler.c ====
>>>>>> 2872c2872
>>>>>> < if (psize->src.psizemax > 0)
>>>>>> ---
>>>>>> > if (psize->dst.psizemax > 0)
>>>>>>
>>>>>>
>>>>>> On Feb 2, 2009, at 1:39 PM, Michael Grinnell wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>> Periodically Argus dies on my test system with the error
>>>>>>> "ArgusGenerateRecord: packet size type not defined." The time
>>>>>>> between these errors varies, sometimes it's only a minute or two
>>>>>>> after argus starts, other times it can be > 15 minutes. I've tried
>>>>>>> running a simultaneous tcpdump, then running the resulting capture
>>>>>>> file through argus, but I can't replicate the error. I also don't
>>>>>>> see any glaring errors in the capture file around the time it dies.
>>>>>>> This happens with argus 3.0.0 and with argus 3.0.1 beta2. The
>>>>>>> system is running CentOS 5.2 and is listening on a dedicated
>>>>>>> interface (NC7782, bnx2 driver) to a span port off of a Cisco
>>>>>>> switch. I have also updated to the newest bnx2 drivers, but it
>>>>>>> still recurs. I'm trying to scare up another NIC to try as well.
>>>>>>>
>>>>>>> Thoughts?
>>>>>>>
>>>>>>> --
>>>>>>> Michael Grinnell
>>>>>>> Information Security Engineer
>>>>>>> The American University
>>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> Message: 8
>>>>>> Date: Tue, 3 Feb 2009 10:46:45 -0500
>>>>>> From: Carter Bullard <carter at qosient.com>
>>>>>> Subject: [ARGUS] argus user data buffer analysis
>>>>>> To: Oguz Yarimtepe <comp.ogz at gmail.com>
>>>>>> Cc: argus-info at lists.andrew.cmu.edu
>>>>>> Message-ID: <8FDB29B8-7B9A-401B-8466-B38557690D00 at qosient.com>
>>>>>> Content-Type: text/plain; charset="utf-8"
>>>>>>
>>>>>> Hey Oguz,
>>>>>> We have undocumented programs in the argus-3.0.0 distribution
>>>>>> that do something similar to what you are interested in, I think.
>>>>>> These are undocumented because its a bit complex, and I haven't
>>>>>> had a chance to describe them yet.
>>>>>>
>>>>>> If you are interested in helping to make these types of programs
>>>>>> useful, I'll be more than happy to describe how they work, and
>>>>>> share my experiences and help to make them better.
>>>>>>
>>>>>> The undocumented program rauserdata() will analyze the user
>>>>>> data portion of argus records, and generate a signature pattern
>>>>>> configuration for the protocols it encounters in the data set.
>>>>>> The algorithm is very simple, and pretty powerful, in that it makes
>>>>>> no assumptions about the user data. But, you have to be careful
>>>>>> with the data that you give the engine. Below is a little
>>>>>> background.
>>>>>>
>>>>>> I have found that for the set of protocols you listed, the
>>>>>> first 32 bytes of data is all that is needed to reliably identify the
>>>>>> protocol type. This is because, each of your protocols have
>>>>>> unique greeting identifiers, and for the ones you listed, the
>>>>>> identifiers are all in ascii.
>>>>>>
>>>>>> Because argus provides multiple status reports for long lived
>>>>>> flows, not all argus records for a given flow will contain the
>>>>>> "first N byte" signatures that you are seeking.
>>>>>>
>>>>>> Using racluster() on your 'primitive' argus data will usually provide
>>>>>> you with the "first N bytes" of user data, so that your search for
>>>>>> tokens and patterns can be reliable.
>>>>>>
>>>>>> Try this out for a while to see if you get anything useful:
>>>>>>
>>>>>> racluster -r /a/days/worth/of/data/of/interest/* -w /tmp/day.cache
>>>>>>
>>>>>> rauserdata -r /tmp/day.cache | less
>>>>>>
>>>>>> You should get an output that is arraigned by 'protocol/port' and
>>>>>> you should see a set of source and destination user data buffers
>>>>>> that have the "greatest likelyhood" patterns for that "proto/port"
>>>>>> pair.
>>>>>>
>>>>>> ra() prints the user data buffer with an ASCII encoding by
>>>>>> default, and
>>>>>> so you should see some patterns in the buffers it outputs.
>>>>>> if you see a '.', that is generally a non printable character.
>>>>>>
>>>>>> The ides is to build up configuration files of signatures using
>>>>>> rauserdata(),
>>>>>> and the program raservices(), will take the rauserdata() output as
>>>>>> a configuration file, and label flows with the tags that identify the
>>>>>> protocol.
>>>>>>
>>>>>> Give it a try, and I'd love to see/hear your comments.
>>>>>>
>>>>>> Carter
>>>>>>
>>>>>> On Feb 3, 2009, at 12:59 AM, Oguz Yarimtepe wrote:
>>>>>>
>>>>>>>
>>>>>>> Depends on what you need. If you enable user data capture
>>>>>>> (the -U
>>>>>>> option on the argus) it will capture up to the first 512 bytes of
>>>>>>> the user
>>>>>>> data of the flow. That may or may not give you enough information
>>>>>>> about the
>>>>>>> flow to do what you want. Note that on a fast link best results are
>>>>>>> going to
>>>>>>> occur using a DAG card as the data capture adds a fairly heavy load
>>>>>>> to the
>>>>>>> server. To display the data with ra (for instance) you need to use
>>>>>>> the -s
>>>>>>> command to add suser and duser to the output (as in
>>>>>>>
>>>>>>> ra -r argus_file -n -s +suser:512 -s +duser:512
>>>>>>>
>>>>>>> which will tack the user data on the end of the line. This of course
>>>>>>> raises a
>>>>>>> number of sticky privacy issues that you need to have considered and
>>>>>>> gotten
>>>>>>> approved by appropriate management of the link you are tapping
>>>>>>> (which may or
>>>>>>> may not be you :-)).
>>>>>>>
>>>>>>> Peter Van Epp
>>>>>>>
>>>>>>> What i am willing to do is to characterize the network traffic by
>>>>>>> using some characteristics derived from flow information. My final
>>>>>>> decision about a flow record will be whether the flow belongs to a
>>>>>>> chat session, a mail transfer, a FTP connection, a web browsing, ...
>>>>>>>
>>>>>>> I had discovered Bro which has identifiers related with high level
>>>>>>> protocols. The protocol family that it supports is not as much as
>>>>>>> Argus does so i was planning to go on with Argus.
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> O?uz Yar?mtepe
>>>>>>> www.loopbacking.info
>>>>>> Carter Bullard
>>>>>> CEO/President
>>>>>> QoSient, LLC
>>>>>> 150 E 57th Street Suite 12D
>>>>>> New York, New York 10022
>>>>>>
>>>>>> +1 212 588-9133 Phone
>>>>>> +1 212 588-9134 Fax
>>>>>>
>>>>>>
>>>>>>
>>>>>> -------------- next part --------------
>>>>>> An HTML attachment was scrubbed...
>>>>>> URL:
>>>>>> https://lists.andrew.cmu.edu/mailman/private/argus-info/attachments/20090203/3f7a6300/attachment.html
>>>>>>
>>>>>>
>>>>>> ------------------------------
>>>>>>
>>>>>> _______________________________________________
>>>>>> Argus-info mailing list
>>>>>> Argus-info at lists.andrew.cmu.edu
>>>>>> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
>>>>>>
>>>>>>
>>>>>> End of Argus-info Digest, Vol 42, Issue 1
>>>>>> *****************************************
>>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Best Regards,
>>>>>
>>>>> CS Lee<geek00L[at]gmail.com>
>>>>>
>>>>> http://geek00l.blogspot.com
>>>>> http://defcraft.net
>>>>>
>>>>>
>>>>> Carter Bullard
>>>>> CEO/President
>>>>> QoSient, LLC
>>>>> 150 E 57th Street Suite 12D
>>>>> New York, New York 10022
>>>>>
>>>>> +1 212 588-9133 Phone
>>>>> +1 212 588-9134 Fax
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>> --
>>>> Best Regards,
>>>>
>>>> CS Lee<geek00L[at]gmail.com>
>>>>
>>>> http://geek00l.blogspot.com
>>>> http://defcraft.net
>>>>
>>>>
>>>> Carter Bullard
>>>> CEO/President
>>>> QoSient, LLC
>>>> 150 E 57th Street Suite 12D
>>>> New York, New York 10022
>>>>
>>>> +1 212 588-9133 Phone
>>>> +1 212 588-9134 Fax
>>>>
>>>>
>>>>
>>>>
>>
>>
>
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E 57th Street Suite 12D
> New York, New York 10022
>
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
>
>
>
More information about the argus
mailing list