Argus-info Digest, Vol 42, Issue 1
Carter Bullard
carter at qosient.com
Wed Feb 4 14:21:02 EST 2009
Hmmmm,
I have them in the development directory, here. Let me see what's up.
If we can fix the ongoing v2 -> v3 conversion problem soon, I'll release
another round of client programs, and it will definitely have them in
there.
How are things?
Carter
On Feb 4, 2009, at 1:22 PM, CS Lee wrote:
> hi carter,
>
> i can't find both rauserdata() and raservice() in argus 3.0
> distribution which I downloaded from
>
> ftp://qosient.com/dev/argus-3.0/
>
> Thanks!
>
>
> On Tue, Feb 3, 2009 at 11:46 PM, <argus-info-request at lists.andrew.cmu.edu
> > wrote:
> Send Argus-info mailing list submissions to
> argus-info at lists.andrew.cmu.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
> or, via email, send a message with subject or body 'help' to
> argus-info-request at lists.andrew.cmu.edu
>
> You can reach the person managing the list at
> argus-info-owner at lists.andrew.cmu.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Argus-info digest..."
>
>
> Today's Topics:
>
> 1. ArgusGenerateRecord: packet size type not defined
> (Michael Grinnell)
> 2. argus usage (Oguz Yarimtepe)
> 3. Re: argus usage (Stewart Gray)
> 4. Re: ArgusGenerateRecord: packet size type not defined
> (Peter Van Epp)
> 5. Re: argus usage (Peter Van Epp)
> 6. Re: argus usage (Oguz Yarimtepe)
> 7. Re: ArgusGenerateRecord: packet size type not defined
> (Carter Bullard)
> 8. argus user data buffer analysis (Carter Bullard)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Mon, 02 Feb 2009 13:39:44 -0500
> From: Michael Grinnell <grinnell at american.edu>
> Subject: [ARGUS] ArgusGenerateRecord: packet size type not defined
> To: argus-info <argus-info at lists.andrew.cmu.edu>
> Message-ID: <49873DF0.6090305 at american.edu>
> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>
> Hi,
>
> Periodically Argus dies on my test system with the error
> "ArgusGenerateRecord: packet size type not defined." The time between
> these errors varies, sometimes it's only a minute or two after argus
> starts, other times it can be > 15 minutes. I've tried running a
> simultaneous tcpdump, then running the resulting capture file through
> argus, but I can't replicate the error. I also don't see any glaring
> errors in the capture file around the time it dies. This happens with
> argus 3.0.0 and with argus 3.0.1 beta2. The system is running CentOS
> 5.2 and is listening on a dedicated interface (NC7782, bnx2 driver)
> to a
> span port off of a Cisco switch. I have also updated to the newest
> bnx2
> drivers, but it still recurs. I'm trying to scare up another NIC to
> try
> as well.
>
> Thoughts?
>
> --
> Michael Grinnell
> Information Security Engineer
> The American University
>
>
> ------------------------------
>
> Message: 2
> Date: Mon, 02 Feb 2009 22:50:20 +0200
> From: Oguz Yarimtepe <comp.ogz at gmail.com>
> Subject: [ARGUS] argus usage
> To: argus-info at lists.andrew.cmu.edu
> Message-ID: <1233607820.7050.12.camel at gezenti>
> Content-Type: text/plain
>
> Hi,
>
> I want some information about argus capabilities. I tried argus a bit
> and read the wiki. I want to learn whether it is possible to get
> application level information from a flow record by using argus, like
> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
> As i understood from its usage it is possible to get these information
> indirectly by analyzing the argus output but maybe there is a way that
> argus serves that i don't know.
>
> Thanx.
>
>
>
> ------------------------------
>
> Message: 3
> Date: Tue, 3 Feb 2009 10:26:06 +1300
> From: "Stewart Gray" <Stewart.Gray at safecom.co.nz>
> Subject: Re: [ARGUS] argus usage
> To: <argus-info at lists.andrew.cmu.edu>
> Message-ID:
> <4C08B31BFF3B5D42BCAC255ADDD6EE5EA8769D at tasbe116.advancedsolutions.co.nz
> >
>
> Content-Type: text/plain; charset="us-ascii"
>
> Hi Oguz,
>
> Argus supports the Berkeley packet filter (as other apps like tcpdump
> and ngrep do) so you're able to lock queries down to port numbers,
> specific hosts etc. Knowing the port number an application runs on
> means
> it fairly easy to pull out transactions for a particular protocol.
> Keep
> in mind it doesn't operate at layer 7, flow information for a protocol
> such as HTTP will be very similar to SMTP. If you're after more in-
> depth
> analysis of a transaction, i.e. what happened during an http
> connection
> (GET, POST, HEAD etc) you're probably best to be taking full packet
> captures alongside argus so you can run them through another tool such
> as wireshark.
>
> Have a look at this wiki page for some common usages -
> http://nsmwiki.org/index.php?title=Argus.
>
> Cheers,
>
> Stewart
>
> -----Original Message-----
> From: argus-info-bounces+stewart.gray=safecom.co.nz at lists.andrew.cmu.edu
> [mailto:argus-info-bounces+stewart.gray=safecom.co.nz at lists.andrew.cmu.e
> du] On Behalf Of Oguz Yarimtepe
> Sent: Tuesday, 3 February 2009 9:50 a.m.
> To: argus-info at lists.andrew.cmu.edu
> Subject: [ARGUS] argus usage
>
> Hi,
>
> I want some information about argus capabilities. I tried argus a bit
> and read the wiki. I want to learn whether it is possible to get
> application level information from a flow record by using argus, like
> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
> As i understood from its usage it is possible to get these information
> indirectly by analyzing the argus output but maybe there is a way that
> argus serves that i don't know.
>
> Thanx.
>
> #####################################################################################
> Important: This electronic message and attachments (if any) are
> confidential
> and may be legally privileged. If you are not the intended recipient
> do not
> copy, disclose or use the contents in any way. Please let us know by
> return
> e-mail immediately and then destroy this message.
> #####################################################################################
>
>
> ------------------------------
>
> Message: 4
> Date: Mon, 2 Feb 2009 20:51:05 -0800
> From: Peter Van Epp <vanepp at sfu.ca>
> Subject: Re: [ARGUS] ArgusGenerateRecord: packet size type not defined
> To: argus-info at lists.andrew.cmu.edu
> Message-ID: <20090203045105.GA26496 at sfu.ca>
> Content-Type: text/plain; charset=us-ascii
>
> On Mon, Feb 02, 2009 at 01:39:44PM -0500, Michael Grinnell wrote:
> > Hi,
> >
> > Periodically Argus dies on my test system with the error
> > "ArgusGenerateRecord: packet size type not defined." The time
> between
> > these errors varies, sometimes it's only a minute or two after argus
> > starts, other times it can be > 15 minutes. I've tried running a
> > simultaneous tcpdump, then running the resulting capture file
> through
> > argus, but I can't replicate the error. I also don't see any
> glaring
> > errors in the capture file around the time it dies. This happens
> with
> > argus 3.0.0 and with argus 3.0.1 beta2. The system is running
> CentOS
> > 5.2 and is listening on a dedicated interface (NC7782, bnx2
> driver) to a
> > span port off of a Cisco switch. I have also updated to the
> newest bnx2
> > drivers, but it still recurs. I'm trying to scare up another NIC
> to try
> > as well.
> >
> > Thoughts?
> >
> > --
> > Michael Grinnell
> > Information Security Engineer
> > The American University
>
> Setting
>
> ARGUS_PACKET_CAPTURE_FILE="/var/log/argus/packet.out"
>
> in an argus.rc file will capture the input packets from pcap in to the
> specified file. If you can get lucky and get a failure before you
> run out of
> disk space one of the last packets in the file should tell us what
> argus isn't
> liking. On a busy link this file will get large fast but if it
> sometimes
> fails quickly you may be lucky (you are also likely to see packet
> loss due to
> the disk I/O on the sensor but hopefully the fault will still occur).
> It looks like the argus record is malformed (it is
> complaining that it
> doesn't recognize the type in argus/ArgusModeler.c at line 2904 in the
> argus-3.0.1.beta.2 code). A dump of the offending packet should tell
> Carter
> why (or if the incoming packet is corrupted which is also possible).
>
> Peter Van Epp
>
>
> ------------------------------
>
> Message: 5
> Date: Mon, 2 Feb 2009 21:06:59 -0800
> From: Peter Van Epp <vanepp at sfu.ca>
> Subject: Re: [ARGUS] argus usage
> To: argus-info at lists.andrew.cmu.edu
> Message-ID: <20090203050659.GA8930 at sfu.ca>
> Content-Type: text/plain; charset=us-ascii
>
> On Mon, Feb 02, 2009 at 10:50:20PM +0200, Oguz Yarimtepe wrote:
> > Hi,
> >
> > I want some information about argus capabilities. I tried argus a
> bit
> > and read the wiki. I want to learn whether it is possible to get
> > application level information from a flow record by using argus,
> like
> > HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
> > As i understood from its usage it is possible to get these
> information
> > indirectly by analyzing the argus output but maybe there is a way
> that
> > argus serves that i don't know.
> >
> > Thanx.
> >
>
> Depends on what you need. If you enable user data capture
> (the -U
> option on the argus) it will capture up to the first 512 bytes of
> the user
> data of the flow. That may or may not give you enough information
> about the
> flow to do what you want. Note that on a fast link best results are
> going to
> occur using a DAG card as the data capture adds a fairly heavy load
> to the
> server. To display the data with ra (for instance) you need to use
> the -s
> command to add suser and duser to the output (as in
>
> ra -r argus_file -n -s +suser:512 -s +duser:512
>
> which will tack the user data on the end of the line. This of course
> raises a
> number of sticky privacy issues that you need to have considered and
> gotten
> approved by appropriate management of the link you are tapping
> (which may or
> may not be you :-)).
>
> Peter Van Epp
>
>
> ------------------------------
>
> Message: 6
> Date: Tue, 3 Feb 2009 07:59:33 +0200
> From: Oguz Yarimtepe <comp.ogz at gmail.com>
> Subject: Re: [ARGUS] argus usage
> To: argus-info at lists.andrew.cmu.edu
> Message-ID:
> <20831c740902022159q176610e5nb954f5d96e211d13 at mail.gmail.com>
> Content-Type: text/plain; charset="iso-8859-9"
>
> >
> > Depends on what you need. If you enable user data capture
> (the -U
> > option on the argus) it will capture up to the first 512 bytes of
> the user
> > data of the flow. That may or may not give you enough information
> about the
> > flow to do what you want. Note that on a fast link best results
> are going
> > to
> > occur using a DAG card as the data capture adds a fairly heavy
> load to the
> > server. To display the data with ra (for instance) you need to use
> the -s
> > command to add suser and duser to the output (as in
> >
> > ra -r argus_file -n -s +suser:512 -s +duser:512
> >
> > which will tack the user data on the end of the line. This of
> course raises
> > a
> > number of sticky privacy issues that you need to have considered
> and gotten
> > approved by appropriate management of the link you are tapping
> (which may
> > or
> > may not be you :-)).
> >
> > Peter Van Epp
>
>
> What i am willing to do is to characterize the network traffic by
> using
> some characteristics derived from flow information. My final
> decision about
> a flow record will be whether the flow belongs to a chat session, a
> mail
> transfer, a FTP connection, a web browsing, ...
>
> I had discovered Bro which has identifiers related with high level
> protocols. The protocol family that it supports is not as much as
> Argus does
> so i was planning to go on with Argus.
>
>
> --
> O?uz Yar?mtepe
> www.loopbacking.info
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: https://lists.andrew.cmu.edu/mailman/private/argus-info/attachments/20090203/2e9c8a9c/attachment-0001.html
>
> ------------------------------
>
> Message: 7
> Date: Tue, 3 Feb 2009 10:07:03 -0500
> From: Carter Bullard <carter at qosient.com>
> Subject: Re: [ARGUS] ArgusGenerateRecord: packet size type not defined
> To: Michael Grinnell <grinnell at american.edu>
> Cc: argus-info <argus-info at lists.andrew.cmu.edu>
> Message-ID: <D2F59B8C-3239-475E-AE5C-4DCE7C819151 at qosient.com>
> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>
> Hey Michael,
> Wow, pretty obvious bug in the flow's packet size reporting feature.
> Try this patch, and we'll see if it doesn't work for you.
>
> Carter
>
> ==== //depot/argus/argus/argus/ArgusModeler.c#62 - /home/carter/argus/
> argus/argus/ArgusModeler.c ====
> 2872c2872
> < if (psize->src.psizemax > 0)
> ---
> > if (psize->dst.psizemax > 0)
>
>
> On Feb 2, 2009, at 1:39 PM, Michael Grinnell wrote:
>
> > Hi,
> >
> > Periodically Argus dies on my test system with the error
> > "ArgusGenerateRecord: packet size type not defined." The time
> > between these errors varies, sometimes it's only a minute or two
> > after argus starts, other times it can be > 15 minutes. I've tried
> > running a simultaneous tcpdump, then running the resulting capture
> > file through argus, but I can't replicate the error. I also don't
> > see any glaring errors in the capture file around the time it dies.
> > This happens with argus 3.0.0 and with argus 3.0.1 beta2. The
> > system is running CentOS 5.2 and is listening on a dedicated
> > interface (NC7782, bnx2 driver) to a span port off of a Cisco
> > switch. I have also updated to the newest bnx2 drivers, but it
> > still recurs. I'm trying to scare up another NIC to try as well.
> >
> > Thoughts?
> >
> > --
> > Michael Grinnell
> > Information Security Engineer
> > The American University
> >
>
>
>
>
>
>
>
> ------------------------------
>
> Message: 8
> Date: Tue, 3 Feb 2009 10:46:45 -0500
> From: Carter Bullard <carter at qosient.com>
> Subject: [ARGUS] argus user data buffer analysis
> To: Oguz Yarimtepe <comp.ogz at gmail.com>
> Cc: argus-info at lists.andrew.cmu.edu
> Message-ID: <8FDB29B8-7B9A-401B-8466-B38557690D00 at qosient.com>
> Content-Type: text/plain; charset="utf-8"
>
> Hey Oguz,
> We have undocumented programs in the argus-3.0.0 distribution
> that do something similar to what you are interested in, I think.
> These are undocumented because its a bit complex, and I haven't
> had a chance to describe them yet.
>
> If you are interested in helping to make these types of programs
> useful, I'll be more than happy to describe how they work, and
> share my experiences and help to make them better.
>
> The undocumented program rauserdata() will analyze the user
> data portion of argus records, and generate a signature pattern
> configuration for the protocols it encounters in the data set.
> The algorithm is very simple, and pretty powerful, in that it makes
> no assumptions about the user data. But, you have to be careful
> with the data that you give the engine. Below is a little background.
>
> I have found that for the set of protocols you listed, the
> first 32 bytes of data is all that is needed to reliably identify the
> protocol type. This is because, each of your protocols have
> unique greeting identifiers, and for the ones you listed, the
> identifiers are all in ascii.
>
> Because argus provides multiple status reports for long lived
> flows, not all argus records for a given flow will contain the
> "first N byte" signatures that you are seeking.
>
> Using racluster() on your 'primitive' argus data will usually provide
> you with the "first N bytes" of user data, so that your search for
> tokens and patterns can be reliable.
>
> Try this out for a while to see if you get anything useful:
>
> racluster -r /a/days/worth/of/data/of/interest/* -w /tmp/day.cache
>
> rauserdata -r /tmp/day.cache | less
>
> You should get an output that is arraigned by 'protocol/port' and
> you should see a set of source and destination user data buffers
> that have the "greatest likelyhood" patterns for that "proto/port"
> pair.
>
> ra() prints the user data buffer with an ASCII encoding by default,
> and
> so you should see some patterns in the buffers it outputs.
> if you see a '.', that is generally a non printable character.
>
> The ides is to build up configuration files of signatures using
> rauserdata(),
> and the program raservices(), will take the rauserdata() output as
> a configuration file, and label flows with the tags that identify the
> protocol.
>
> Give it a try, and I'd love to see/hear your comments.
>
> Carter
>
> On Feb 3, 2009, at 12:59 AM, Oguz Yarimtepe wrote:
>
> >
> >
> > Depends on what you need. If you enable user data capture
> > (the -U
> > option on the argus) it will capture up to the first 512 bytes of
> > the user
> > data of the flow. That may or may not give you enough information
> > about the
> > flow to do what you want. Note that on a fast link best results are
> > going to
> > occur using a DAG card as the data capture adds a fairly heavy load
> > to the
> > server. To display the data with ra (for instance) you need to use
> > the -s
> > command to add suser and duser to the output (as in
> >
> > ra -r argus_file -n -s +suser:512 -s +duser:512
> >
> > which will tack the user data on the end of the line. This of course
> > raises a
> > number of sticky privacy issues that you need to have considered and
> > gotten
> > approved by appropriate management of the link you are tapping
> > (which may or
> > may not be you :-)).
> >
> > Peter Van Epp
> >
> > What i am willing to do is to characterize the network traffic by
> > using some characteristics derived from flow information. My final
> > decision about a flow record will be whether the flow belongs to a
> > chat session, a mail transfer, a FTP connection, a web browsing, ...
> >
> > I had discovered Bro which has identifiers related with high level
> > protocols. The protocol family that it supports is not as much as
> > Argus does so i was planning to go on with Argus.
> >
> >
> > --
> > O?uz Yar?mtepe
> > www.loopbacking.info
>
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E 57th Street Suite 12D
> New York, New York 10022
>
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
>
>
>
> -------------- next part --------------
> An HTML attachment was scrubbed...
> URL: https://lists.andrew.cmu.edu/mailman/private/argus-info/attachments/20090203/3f7a6300/attachment.html
>
> ------------------------------
>
> _______________________________________________
> Argus-info mailing list
> Argus-info at lists.andrew.cmu.edu
> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
>
>
> End of Argus-info Digest, Vol 42, Issue 1
> *****************************************
>
>
>
> --
> Best Regards,
>
> CS Lee<geek00L[at]gmail.com>
>
> http://geek00l.blogspot.com
> http://defcraft.net
Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090204/cd282717/attachment.html>
More information about the argus
mailing list