Argus-info Digest, Vol 42, Issue 1
CS Lee
geek00l at gmail.com
Wed Feb 4 21:11:13 EST 2009
hi Carter,
Thanks. Things are all good, btw, I read your presentation in Flocon2009,
that's really cool stuff.
On Thu, Feb 5, 2009 at 3:21 AM, Carter Bullard <carter at qosient.com> wrote:
> Hmmmm,I have them in the development directory, here. Let me see what's
> up.
> If we can fix the ongoing v2 -> v3 conversion problem soon, I'll release
> another round of client programs, and it will definitely have them in
> there.
>
> How are things?
>
> Carter
>
> On Feb 4, 2009, at 1:22 PM, CS Lee wrote:
>
> hi carter,
>
> i can't find both rauserdata() and raservice() in argus 3.0 distribution
> which I downloaded from
>
> ftp://qosient.com/dev/argus-3.0/
>
> Thanks!
>
>
> On Tue, Feb 3, 2009 at 11:46 PM, <argus-info-request at lists.andrew.cmu.edu>wrote:
>
>> Send Argus-info mailing list submissions to
>> argus-info at lists.andrew.cmu.edu
>>
>> To subscribe or unsubscribe via the World Wide Web, visit
>> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
>> or, via email, send a message with subject or body 'help' to
>> argus-info-request at lists.andrew.cmu.edu
>>
>> You can reach the person managing the list at
>> argus-info-owner at lists.andrew.cmu.edu
>>
>> When replying, please edit your Subject line so it is more specific
>> than "Re: Contents of Argus-info digest..."
>>
>>
>> Today's Topics:
>>
>> 1. ArgusGenerateRecord: packet size type not defined
>> (Michael Grinnell)
>> 2. argus usage (Oguz Yarimtepe)
>> 3. Re: argus usage (Stewart Gray)
>> 4. Re: ArgusGenerateRecord: packet size type not defined
>> (Peter Van Epp)
>> 5. Re: argus usage (Peter Van Epp)
>> 6. Re: argus usage (Oguz Yarimtepe)
>> 7. Re: ArgusGenerateRecord: packet size type not defined
>> (Carter Bullard)
>> 8. argus user data buffer analysis (Carter Bullard)
>>
>>
>> ----------------------------------------------------------------------
>>
>> Message: 1
>> Date: Mon, 02 Feb 2009 13:39:44 -0500
>> From: Michael Grinnell <grinnell at american.edu>
>> Subject: [ARGUS] ArgusGenerateRecord: packet size type not defined
>> To: argus-info <argus-info at lists.andrew.cmu.edu>
>> Message-ID: <49873DF0.6090305 at american.edu>
>> Content-Type: text/plain; charset=ISO-8859-1; format=flowed
>>
>> Hi,
>>
>> Periodically Argus dies on my test system with the error
>> "ArgusGenerateRecord: packet size type not defined." The time between
>> these errors varies, sometimes it's only a minute or two after argus
>> starts, other times it can be > 15 minutes. I've tried running a
>> simultaneous tcpdump, then running the resulting capture file through
>> argus, but I can't replicate the error. I also don't see any glaring
>> errors in the capture file around the time it dies. This happens with
>> argus 3.0.0 and with argus 3.0.1 beta2. The system is running CentOS
>> 5.2 and is listening on a dedicated interface (NC7782, bnx2 driver) to a
>> span port off of a Cisco switch. I have also updated to the newest bnx2
>> drivers, but it still recurs. I'm trying to scare up another NIC to try
>> as well.
>>
>> Thoughts?
>>
>> --
>> Michael Grinnell
>> Information Security Engineer
>> The American University
>>
>>
>> ------------------------------
>>
>> Message: 2
>> Date: Mon, 02 Feb 2009 22:50:20 +0200
>> From: Oguz Yarimtepe <comp.ogz at gmail.com>
>> Subject: [ARGUS] argus usage
>> To: argus-info at lists.andrew.cmu.edu
>> Message-ID: <1233607820.7050.12.camel at gezenti>
>> Content-Type: text/plain
>>
>> Hi,
>>
>> I want some information about argus capabilities. I tried argus a bit
>> and read the wiki. I want to learn whether it is possible to get
>> application level information from a flow record by using argus, like
>> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
>> As i understood from its usage it is possible to get these information
>> indirectly by analyzing the argus output but maybe there is a way that
>> argus serves that i don't know.
>>
>> Thanx.
>>
>>
>>
>> ------------------------------
>>
>> Message: 3
>> Date: Tue, 3 Feb 2009 10:26:06 +1300
>> From: "Stewart Gray" <Stewart.Gray at safecom.co.nz>
>> Subject: Re: [ARGUS] argus usage
>> To: <argus-info at lists.andrew.cmu.edu>
>> Message-ID:
>> <
>> 4C08B31BFF3B5D42BCAC255ADDD6EE5EA8769D at tasbe116.advancedsolutions.co.nz>
>>
>> Content-Type: text/plain; charset="us-ascii"
>>
>> Hi Oguz,
>>
>> Argus supports the Berkeley packet filter (as other apps like tcpdump
>> and ngrep do) so you're able to lock queries down to port numbers,
>> specific hosts etc. Knowing the port number an application runs on means
>> it fairly easy to pull out transactions for a particular protocol. Keep
>> in mind it doesn't operate at layer 7, flow information for a protocol
>> such as HTTP will be very similar to SMTP. If you're after more in-depth
>> analysis of a transaction, i.e. what happened during an http connection
>> (GET, POST, HEAD etc) you're probably best to be taking full packet
>> captures alongside argus so you can run them through another tool such
>> as wireshark.
>>
>> Have a look at this wiki page for some common usages -
>> http://nsmwiki.org/index.php?title=Argus.
>>
>> Cheers,
>>
>> Stewart
>>
>> -----Original Message-----
>> From: argus-info-bounces+stewart.gray=safecom.co.nz at lists.andrew.cmu.edu
>> [mailto:argus-info-bounces+stewart.gray<argus-info-bounces%2Bstewart.gray>
>> =safecom.co.nz at lists.andrew.cmu.e
>> du] On Behalf Of Oguz Yarimtepe
>> Sent: Tuesday, 3 February 2009 9:50 a.m.
>> To: argus-info at lists.andrew.cmu.edu
>> Subject: [ARGUS] argus usage
>>
>> Hi,
>>
>> I want some information about argus capabilities. I tried argus a bit
>> and read the wiki. I want to learn whether it is possible to get
>> application level information from a flow record by using argus, like
>> HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
>> As i understood from its usage it is possible to get these information
>> indirectly by analyzing the argus output but maybe there is a way that
>> argus serves that i don't know.
>>
>> Thanx.
>>
>>
>> #####################################################################################
>> Important: This electronic message and attachments (if any) are
>> confidential
>> and may be legally privileged. If you are not the intended recipient do
>> not
>> copy, disclose or use the contents in any way. Please let us know by
>> return
>> e-mail immediately and then destroy this message.
>>
>> #####################################################################################
>>
>>
>> ------------------------------
>>
>> Message: 4
>> Date: Mon, 2 Feb 2009 20:51:05 -0800
>> From: Peter Van Epp <vanepp at sfu.ca>
>> Subject: Re: [ARGUS] ArgusGenerateRecord: packet size type not defined
>> To: argus-info at lists.andrew.cmu.edu
>> Message-ID: <20090203045105.GA26496 at sfu.ca>
>> Content-Type: text/plain; charset=us-ascii
>>
>> On Mon, Feb 02, 2009 at 01:39:44PM -0500, Michael Grinnell wrote:
>> > Hi,
>> >
>> > Periodically Argus dies on my test system with the error
>> > "ArgusGenerateRecord: packet size type not defined." The time between
>> > these errors varies, sometimes it's only a minute or two after argus
>> > starts, other times it can be > 15 minutes. I've tried running a
>> > simultaneous tcpdump, then running the resulting capture file through
>> > argus, but I can't replicate the error. I also don't see any glaring
>> > errors in the capture file around the time it dies. This happens with
>> > argus 3.0.0 and with argus 3.0.1 beta2. The system is running CentOS
>> > 5.2 and is listening on a dedicated interface (NC7782, bnx2 driver) to a
>> > span port off of a Cisco switch. I have also updated to the newest bnx2
>> > drivers, but it still recurs. I'm trying to scare up another NIC to try
>> > as well.
>> >
>> > Thoughts?
>> >
>> > --
>> > Michael Grinnell
>> > Information Security Engineer
>> > The American University
>>
>> Setting
>>
>> ARGUS_PACKET_CAPTURE_FILE="/var/log/argus/packet.out"
>>
>> in an argus.rc file will capture the input packets from pcap in to the
>> specified file. If you can get lucky and get a failure before you run out
>> of
>> disk space one of the last packets in the file should tell us what argus
>> isn't
>> liking. On a busy link this file will get large fast but if it sometimes
>> fails quickly you may be lucky (you are also likely to see packet loss due
>> to
>> the disk I/O on the sensor but hopefully the fault will still occur).
>> It looks like the argus record is malformed (it is complaining that
>> it
>> doesn't recognize the type in argus/ArgusModeler.c at line 2904 in the
>> argus-3.0.1.beta.2 code). A dump of the offending packet should tell
>> Carter
>> why (or if the incoming packet is corrupted which is also possible).
>>
>> Peter Van Epp
>>
>>
>> ------------------------------
>>
>> Message: 5
>> Date: Mon, 2 Feb 2009 21:06:59 -0800
>> From: Peter Van Epp <vanepp at sfu.ca>
>> Subject: Re: [ARGUS] argus usage
>> To: argus-info at lists.andrew.cmu.edu
>> Message-ID: <20090203050659.GA8930 at sfu.ca>
>> Content-Type: text/plain; charset=us-ascii
>>
>> On Mon, Feb 02, 2009 at 10:50:20PM +0200, Oguz Yarimtepe wrote:
>> > Hi,
>> >
>> > I want some information about argus capabilities. I tried argus a bit
>> > and read the wiki. I want to learn whether it is possible to get
>> > application level information from a flow record by using argus, like
>> > HTTP, HTTPS, IMAP, POP, SMTP, FTP ...
>> > As i understood from its usage it is possible to get these information
>> > indirectly by analyzing the argus output but maybe there is a way that
>> > argus serves that i don't know.
>> >
>> > Thanx.
>> >
>>
>> Depends on what you need. If you enable user data capture (the -U
>> option on the argus) it will capture up to the first 512 bytes of the user
>> data of the flow. That may or may not give you enough information about
>> the
>> flow to do what you want. Note that on a fast link best results are going
>> to
>> occur using a DAG card as the data capture adds a fairly heavy load to the
>> server. To display the data with ra (for instance) you need to use the -s
>> command to add suser and duser to the output (as in
>>
>> ra -r argus_file -n -s +suser:512 -s +duser:512
>>
>> which will tack the user data on the end of the line. This of course
>> raises a
>> number of sticky privacy issues that you need to have considered and
>> gotten
>> approved by appropriate management of the link you are tapping (which may
>> or
>> may not be you :-)).
>>
>> Peter Van Epp
>>
>>
>> ------------------------------
>>
>> Message: 6
>> Date: Tue, 3 Feb 2009 07:59:33 +0200
>> From: Oguz Yarimtepe <comp.ogz at gmail.com>
>> Subject: Re: [ARGUS] argus usage
>> To: argus-info at lists.andrew.cmu.edu
>> Message-ID:
>> <20831c740902022159q176610e5nb954f5d96e211d13 at mail.gmail.com>
>> Content-Type: text/plain; charset="iso-8859-9"
>>
>> >
>> > Depends on what you need. If you enable user data capture (the
>> -U
>> > option on the argus) it will capture up to the first 512 bytes of the
>> user
>> > data of the flow. That may or may not give you enough information about
>> the
>> > flow to do what you want. Note that on a fast link best results are
>> going
>> > to
>> > occur using a DAG card as the data capture adds a fairly heavy load to
>> the
>> > server. To display the data with ra (for instance) you need to use the
>> -s
>> > command to add suser and duser to the output (as in
>> >
>> > ra -r argus_file -n -s +suser:512 -s +duser:512
>> >
>> > which will tack the user data on the end of the line. This of course
>> raises
>> > a
>> > number of sticky privacy issues that you need to have considered and
>> gotten
>> > approved by appropriate management of the link you are tapping (which
>> may
>> > or
>> > may not be you :-)).
>> >
>> > Peter Van Epp
>>
>>
>> What i am willing to do is to characterize the network traffic by using
>> some characteristics derived from flow information. My final decision
>> about
>> a flow record will be whether the flow belongs to a chat session, a mail
>> transfer, a FTP connection, a web browsing, ...
>>
>> I had discovered Bro which has identifiers related with high level
>> protocols. The protocol family that it supports is not as much as Argus
>> does
>> so i was planning to go on with Argus.
>>
>>
>> --
>> O?uz Yar?mtepe
>> www.loopbacking.info
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> https://lists.andrew.cmu.edu/mailman/private/argus-info/attachments/20090203/2e9c8a9c/attachment-0001.html
>>
>> ------------------------------
>>
>> Message: 7
>> Date: Tue, 3 Feb 2009 10:07:03 -0500
>> From: Carter Bullard <carter at qosient.com>
>> Subject: Re: [ARGUS] ArgusGenerateRecord: packet size type not defined
>> To: Michael Grinnell <grinnell at american.edu>
>> Cc: argus-info <argus-info at lists.andrew.cmu.edu>
>> Message-ID: <D2F59B8C-3239-475E-AE5C-4DCE7C819151 at qosient.com>
>> Content-Type: text/plain; charset=US-ASCII; format=flowed; delsp=yes
>>
>> Hey Michael,
>> Wow, pretty obvious bug in the flow's packet size reporting feature.
>> Try this patch, and we'll see if it doesn't work for you.
>>
>> Carter
>>
>> ==== //depot/argus/argus/argus/ArgusModeler.c#62 - /home/carter/argus/
>> argus/argus/ArgusModeler.c ====
>> 2872c2872
>> < if (psize->src.psizemax > 0)
>> ---
>> > if (psize->dst.psizemax > 0)
>>
>>
>> On Feb 2, 2009, at 1:39 PM, Michael Grinnell wrote:
>>
>> > Hi,
>> >
>> > Periodically Argus dies on my test system with the error
>> > "ArgusGenerateRecord: packet size type not defined." The time
>> > between these errors varies, sometimes it's only a minute or two
>> > after argus starts, other times it can be > 15 minutes. I've tried
>> > running a simultaneous tcpdump, then running the resulting capture
>> > file through argus, but I can't replicate the error. I also don't
>> > see any glaring errors in the capture file around the time it dies.
>> > This happens with argus 3.0.0 and with argus 3.0.1 beta2. The
>> > system is running CentOS 5.2 and is listening on a dedicated
>> > interface (NC7782, bnx2 driver) to a span port off of a Cisco
>> > switch. I have also updated to the newest bnx2 drivers, but it
>> > still recurs. I'm trying to scare up another NIC to try as well.
>> >
>> > Thoughts?
>> >
>> > --
>> > Michael Grinnell
>> > Information Security Engineer
>> > The American University
>> >
>>
>>
>>
>>
>>
>>
>>
>> ------------------------------
>>
>> Message: 8
>> Date: Tue, 3 Feb 2009 10:46:45 -0500
>> From: Carter Bullard <carter at qosient.com>
>> Subject: [ARGUS] argus user data buffer analysis
>> To: Oguz Yarimtepe <comp.ogz at gmail.com>
>> Cc: argus-info at lists.andrew.cmu.edu
>> Message-ID: <8FDB29B8-7B9A-401B-8466-B38557690D00 at qosient.com>
>> Content-Type: text/plain; charset="utf-8"
>>
>> Hey Oguz,
>> We have undocumented programs in the argus-3.0.0 distribution
>> that do something similar to what you are interested in, I think.
>> These are undocumented because its a bit complex, and I haven't
>> had a chance to describe them yet.
>>
>> If you are interested in helping to make these types of programs
>> useful, I'll be more than happy to describe how they work, and
>> share my experiences and help to make them better.
>>
>> The undocumented program rauserdata() will analyze the user
>> data portion of argus records, and generate a signature pattern
>> configuration for the protocols it encounters in the data set.
>> The algorithm is very simple, and pretty powerful, in that it makes
>> no assumptions about the user data. But, you have to be careful
>> with the data that you give the engine. Below is a little background.
>>
>> I have found that for the set of protocols you listed, the
>> first 32 bytes of data is all that is needed to reliably identify the
>> protocol type. This is because, each of your protocols have
>> unique greeting identifiers, and for the ones you listed, the
>> identifiers are all in ascii.
>>
>> Because argus provides multiple status reports for long lived
>> flows, not all argus records for a given flow will contain the
>> "first N byte" signatures that you are seeking.
>>
>> Using racluster() on your 'primitive' argus data will usually provide
>> you with the "first N bytes" of user data, so that your search for
>> tokens and patterns can be reliable.
>>
>> Try this out for a while to see if you get anything useful:
>>
>> racluster -r /a/days/worth/of/data/of/interest/* -w /tmp/day.cache
>>
>> rauserdata -r /tmp/day.cache | less
>>
>> You should get an output that is arraigned by 'protocol/port' and
>> you should see a set of source and destination user data buffers
>> that have the "greatest likelyhood" patterns for that "proto/port" pair.
>>
>> ra() prints the user data buffer with an ASCII encoding by default, and
>> so you should see some patterns in the buffers it outputs.
>> if you see a '.', that is generally a non printable character.
>>
>> The ides is to build up configuration files of signatures using
>> rauserdata(),
>> and the program raservices(), will take the rauserdata() output as
>> a configuration file, and label flows with the tags that identify the
>> protocol.
>>
>> Give it a try, and I'd love to see/hear your comments.
>>
>> Carter
>>
>> On Feb 3, 2009, at 12:59 AM, Oguz Yarimtepe wrote:
>>
>> >
>> >
>> > Depends on what you need. If you enable user data capture
>> > (the -U
>> > option on the argus) it will capture up to the first 512 bytes of
>> > the user
>> > data of the flow. That may or may not give you enough information
>> > about the
>> > flow to do what you want. Note that on a fast link best results are
>> > going to
>> > occur using a DAG card as the data capture adds a fairly heavy load
>> > to the
>> > server. To display the data with ra (for instance) you need to use
>> > the -s
>> > command to add suser and duser to the output (as in
>> >
>> > ra -r argus_file -n -s +suser:512 -s +duser:512
>> >
>> > which will tack the user data on the end of the line. This of course
>> > raises a
>> > number of sticky privacy issues that you need to have considered and
>> > gotten
>> > approved by appropriate management of the link you are tapping
>> > (which may or
>> > may not be you :-)).
>> >
>> > Peter Van Epp
>> >
>> > What i am willing to do is to characterize the network traffic by
>> > using some characteristics derived from flow information. My final
>> > decision about a flow record will be whether the flow belongs to a
>> > chat session, a mail transfer, a FTP connection, a web browsing, ...
>> >
>> > I had discovered Bro which has identifiers related with high level
>> > protocols. The protocol family that it supports is not as much as
>> > Argus does so i was planning to go on with Argus.
>> >
>> >
>> > --
>> > O?uz Yar?mtepe
>> > www.loopbacking.info
>>
>> Carter Bullard
>> CEO/President
>> QoSient, LLC
>> 150 E 57th Street Suite 12D
>> New York, New York 10022
>>
>> +1 212 588-9133 Phone
>> +1 212 588-9134 Fax
>>
>>
>>
>> -------------- next part --------------
>> An HTML attachment was scrubbed...
>> URL:
>> https://lists.andrew.cmu.edu/mailman/private/argus-info/attachments/20090203/3f7a6300/attachment.html
>>
>> ------------------------------
>>
>> _______________________________________________
>> Argus-info mailing list
>> Argus-info at lists.andrew.cmu.edu
>> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
>>
>>
>> End of Argus-info Digest, Vol 42, Issue 1
>> *****************************************
>>
>
>
>
> --
> Best Regards,
>
> CS Lee<geek00L[at]gmail.com>
>
> http://geek00l.blogspot.com
> http://defcraft.net
>
>
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E 57th Street Suite 12D
> New York, New York 10022
>
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
>
>
>
>
--
Best Regards,
CS Lee<geek00L[at]gmail.com>
http://geek00l.blogspot.com
http://defcraft.net
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090205/809458a7/attachment.html>
More information about the argus
mailing list