Argus-info Digest, Vol 10, Issue 37
CS Lee
geek00l at gmail.com
Wed Jun 21 10:26:08 EDT 2006
Carter,
Regarding the large tcpdump file, I have a 90G purely raw pcap file, but I
can't share it since it contains sensitive info, however I maybe able to
test it with argus 3.0 rc since I have it installed on my fbsd box, will get
back to you once I got the test case, however I would like to know when you
mentioned large file, what size considered to be large for that matter.
Cheers.
On 6/21/06, argus-info-request at lists.andrew.cmu.edu <
argus-info-request at lists.andrew.cmu.edu> wrote:
>
> Send Argus-info mailing list submissions to
> argus-info at lists.andrew.cmu.edu
>
> To subscribe or unsubscribe via the World Wide Web, visit
> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
> or, via email, send a message with subject or body 'help' to
> argus-info-request at lists.andrew.cmu.edu
>
> You can reach the person managing the list at
> argus-info-owner at lists.andrew.cmu.edu
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of Argus-info digest..."
>
>
> Today's Topics:
>
> 1. Re: Re: racount status and its definition (Carter Bullard)
> 2. Re: racount fix for FreeBSD (Carter Bullard)
> 3. Re: Re: racount status and its definition (Robin Gruyters)
> 4. Re: Re: racount status and its definition (Carter Bullard)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Wed, 21 Jun 2006 08:57:56 -0400
> From: Carter Bullard <carter at qosient.com>
> Subject: Re: [ARGUS] Re: racount status and its definition
> To: Robin Gruyters <r.gruyters at yirdis.nl>
> Cc: Argus <argus-info at lists.andrew.cmu.edu>
> Message-ID: <CD3A6AB1-641D-4660-9805-FA771E11C9A4 at qosient.com>
> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>
> Hey Robin,
> You can try this to see any primary discrepancies. If you find
> a file that does generate count problems, we can zoom in on
> a few things quickly and get at the root of the problem. I modified
> the test script to give you direct comparisons, and its now doing it
> a directory at a time:
>
> %for i in /data2/argus/05/*; do echo $i; racount -r $i/*; racluster -
> R $i -m srcid -s trans pkts
>
> The record values in racluster() are not going to be the same
> as racount(), because racount() includes the management records
> in the counting and racluster does not (since they are not merged).
>
> Carter
>
> On Jun 21, 2006, at 8:32 AM, Carter Bullard wrote:
>
> > Hmmm, are you a bash shell user or a csh?
> > Do me a favor and try something like this (assuming bash)
> >
> > % for i in /data2/argus/05/*/* ; do echo $i; racluster -r $i -m
> > proto -s proto trans; done
> >
> > and see if something doesn't look strange.
> >
> > Carter
> >
> > On Jun 21, 2006, at 4:44 AM, Robin Gruyters wrote:
> >
> >> Hello Carter,
> >>
> >> I have just ran racluster() with "-r" option and I get the same
> >> output.
> >> (without any errors)
> >>
> >> [racluster]
> >> # racluster -r /data2/argus/05/*/* -m proto \
> >> -s proto trans pkts spkts dpkts bytes sbytes dbytes - net
> >> 82.148.219.32/28
> >> esp 1 1 1 0 310
> >> 310 0
> >> gre 40440 16142557 6955364 9187193 4000449819 1192848094
> >> 2807601725
> >> udp 283037 554632 296948 257684 86633957
> >> 35435077 51198880
> >> tcp 144666 69282162 28369630 40912532 3994126059 2934170533
> >> 1059955526
> >> icmp 36644 50347 50270 77 4126254
> >> 4121768 4486
> >> [end racluster]
> >>
> >> Regards,
> >>
> >> Robin
> >>
> >> On Wed, Jun 21, 2006 at 03:32:49AM -0400, Carter Bullard wrote:
> >>> Hey Robin,
> >>> Looks like somethings up. Try calling racluster with the -r /
> >>> data2/argus/05/*/*.
> >>> I suspect that its getting some kind of error on one of the files,
> >>> and is stopping,
> >>> especially since your getting 1/10th the number of tcp transactions
> >>> (144666 vs 1443778).
> >>> I remember that you had a bug report with toooo many filenames, if
> >>> you're
> >>> still getting that problem, we'll have to try to figure out what is
> >>> causing racluster()
> >>> issues.
> >>>
> >>> Sorry for the problems,
> >>>
> >>> Carter
> >>>
> >>>
> >>> On Jun 21, 2006, at 3:21 AM, Robin Gruyters wrote:
> >>>
> >>>> Hi Carter,
> >>>>
> >>>> At the moment we use the output of racount() for our monthly
> >>>> report to
> >>>> customers. To show them how much data they have used. (by proto and
> >>>> total)
> >>>>
> >>>> For me it doesn't matter if this is possible with racount() or
> >>>> racluster(),
> >>>> if I just get the output done.
> >>>> If this is possible with racount() and with the "-M addr" option,
> >>>> great!
> >>>>
> >>>> The other options, like counts on ports etc, that would also be
> >>>> nice to
> >>>> have.
> >>>>
> >>>> Altough you say it is also possible with racluster(), but the
> >>>> outcome is
> >>>> totally different what I get back from racount(). (I mean the
> >>>> numbers)
> >>>>
> >>>> [racount]
> >>>> # racount -ar /data2/argus/05/*/* - net 82.148.219.XXX/28
> >>>> racount records total_pkts src_pkts
> >>>> dst_pkts total_bytes src_bytes dst_bytes
> >>>> tcp 1443778 69225031 28344760
> >>>> 40880271 55494468479 7222126408 48272342071
> >>>> udp 280703 549026 293754
> >>>> 255272 86044190 35139486 50904704
> >>>> icmp 35102 47042 46966
> >>>> 76 3503635 3499223 4412
> >>>> ip 40441 16142558 6955365
> >>>> 9187193 4000450129 1192848404 2807601725
> >>>> sum 1800024 85963657 35640845
> >>>> 50322812 59584466433 8453613521 51130852912
> >>>> [end racount]
> >>>>
> >>>> [racluster]
> >>>> # racluster -R /data2/argus/05 -m proto \
> >>>> -s proto trans pkts spkts dpkts bytes sbytes dbytes - net
> >>>> 82.148.219.XXX/28
> >>>> esp 1 1 1 0 310
> >>>> 310 0
> >>>> gre 40440 16142557 6955364 9187193 4000449819 1192848094
> >>>> 2807601725
> >>>> udp 283037 554632 296948 257684 86633957
> >>>> 35435077 51198880
> >>>> tcp 144666 69282162 28369630 40912532 3994126059 2934170533
> >>>> 1059955526
> >>>> icmp 36644 50347 50270 77 4126254
> >>>> 4121768 4486
> >>>> [end racluster]
> >>>>
> >>>> If you only check the "total bytes" on TCP packets. With racount()
> >>>> I get
> >>>> 55494468479 bytes and with racluster() 3994126059 bytes. That is a
> >>>> huge
> >>>> difference.
> >>>>
> >>>> Is there an explanation for this behaviour?
> >>>>
> >>>> Regards,
> >>>>
> >>>> Robin
> >>>>
> >>>> On Tue, Jun 20, 2006 at 11:04:01AM -0400, Carter Bullard wrote:
> >>>>> Hey Robin et al.,
> >>>>> You have become the target of anything racount() related ;o)
> >>>>>
> >>>>> So, in trying to understand if the "-M proto" option is useful,
> >>>>> I realized that all of the old racount() functions are supported
> >>>>> by racluster(), so I don't want to duplicate features, so I may
> >>>>> end up redefining racount(), but keeping its default behavior.
> >>>>> What I will do for now is leave it as it is, no -A support, but
> >>>>> with the "-M addr" option and then figure out what to do after
> >>>>> that based on the lists opinion.
> >>>>>
> >>>>> I use racount as a quick and dirty way of seeing how big is
> >>>>> an argus data file, and to check if programs like racluster()
> >>>>> preserve the counts when it aggregates records, so the
> >>>>> default mode is great, but we can also generate the exact same
> >>>>> output using racluster(), you just have to type more on the
> >>>>> command line to get the output right. Same goes for the old -a
> >>>>> option:
> >>>>>
> >>>>> The older racount() functions can be done in racluster() as:
> >>>>>
> >>>>> racount -r file
> >>>>> racluster -r file -m srcid -s trans pkts spkts dpkts bytes
> >>>>> sbytes
> >>>>> dbytes
> >>>>>
> >>>>> racount -ar file
> >>>>> racluster -r file -m proto -s proto trans pkts spkts dpkts bytes
> >>>>> sbytes dbytes
> >>>>>
> >>>>> Now, with the '-M addr', we have a unique counting situation,
> >>>>> and so that seems appropriate, and I think there should be more
> >>>>> counting things to do, like ports, mac address types (vendor ids),
> >>>>> that kind of thing.
> >>>>>
> >>>>> So, opinions? If we could discuss the counting requirements,
> >>>>> that
> >>>>> might help define racount a bit.
> >>>>>
> >>>>> Carter
> >>>>>
> >>>>
> >>>
> >>> Carter Bullard
> >>> CEO/President
> >>> QoSient, LLC
> >>> 150 E. 57th Street Suite 12D
> >>> New York, New York 10022
> >>>
> >>> +1 212 588-9133 Phone
> >>> +1 212 588-9134 Fax
> >>>
> >>>
> >>
> >
> > Carter Bullard
> > CEO/President
> > QoSient, LLC
> > 150 E. 57th Street Suite 12D
> > New York, New York 10022
> >
> > +1 212 588-9133 Phone
> > +1 212 588-9134 Fax
> >
> >
> >
>
> Carter Bullard
> CEO/President
> QoSient, LLC
> 150 E. 57th Street Suite 12D
> New York, New York 10022
>
> +1 212 588-9133 Phone
> +1 212 588-9134 Fax
>
>
>
>
> ------------------------------
>
> Message: 2
> Date: Wed, 21 Jun 2006 09:18:23 -0400
> From: Carter Bullard <carter at qosient.com>
> Subject: Re: [ARGUS] racount fix for FreeBSD
> To: Richard Bejtlich <taosecurity at gmail.com>
> Cc: Argus <argus-info at lists.andrew.cmu.edu>
> Message-ID: <82583602-1608-4D9D-B0C4-1A4041BD2A70 at qosient.com>
> Content-Type: text/plain; charset=US-ASCII; format=flowed
>
> Hey Richard,
> Documentation is the bane of my existence. It's like you have
> to account for what you've done, and I always see what isn't
> finished, rather that what was completed. But, with the help
> of all, hopefully it will get done.
>
> You had asked about 64-bit machines and argus-3.0 earlier
> in the year, and argus-3.0 is suppose to be 64 all things, so if
> you still have an interest, could you test argus on some of your
> amd hardware?
>
> I suspect that in order to really test this stuff we'll need
> a canonical packet capture file as a data source. Are there
> any big packet capture files out there in the big bad world?
> I found some trace archives in Japan that at least have
> tcpdump() data that we can use.
>
> Does anyone have a favorite source of data that we can
> use as a test set?
>
> Carter
>
>
> On Jun 20, 2006, at 8:51 AM, Richard Bejtlich wrote:
>
> >
> > Hi Carter,
> >
> > Whatever you decide, documentation is much appreciated. I am sure I
> > make life much harder for myself because I overlook many of Argus'
> > cool yet obscure features.
> >
> > I am really excited to see Argus 3.0 on the way! This is sort of like
> > Christmas for those of us who rely on session data.
> >
> > Sincerely,
> >
> > Richard
> >
>
>
>
>
>
>
> ------------------------------
>
> Message: 3
> Date: Wed, 21 Jun 2006 15:38:14 +0200
> From: Robin Gruyters <r.gruyters at yirdis.nl>
> Subject: Re: [ARGUS] Re: racount status and its definition
> To: Carter Bullard <carter at qosient.com>
> Cc: Argus <argus-info at lists.andrew.cmu.edu>
> Message-ID: <20060621153814.mzfs4le00gs40ss8 at server.yirdis.net>
> Content-Type: text/plain; charset=ISO-8859-1; DelSp="Yes";
> format="flowed"
>
> Hi Carter,
>
> Here is the output with argus-3 commands (racount() and racluster()).
> I have just pointed to the file which gives errors with argus-2.0.6
> (racount()).
>
> [...]
> $ racount -r /data2/argus/05/21/*; \
> racluster -R /data2/argus/05/21 -m srcid -s trans pkts
> racount records total_pkts src_pkts dst_pkts
> total_bytes src_bytes dst_bytes
> sum 13951 771479 284510 486969
> 303901576 42836384 261065192
> 13927 771479
> [...]
>
> Here is the argus-2.0.6 output:
>
> [...]
> $ racount -r /data2/argus/archive/2006/05/21/*
> ArgusWarning: racount[48517]: ArgusReadSocketStream: malformed argus
> record len 17793
>
> racount records total_pkts src_pkts dst_pkts
> total_bytes src_bytes dst_bytes
> sum 149250 1143266 566499 576767
> 355491645 72812173 282679472
> [...]
>
> Regards,
>
> Robin
>
>
> Quoting Carter Bullard <carter at qosient.com>:
>
> > Hey Robin,
> > You can try this to see any primary discrepancies. If you find
> > a file that does generate count problems, we can zoom in on
> > a few things quickly and get at the root of the problem. I modified
> > the test script to give you direct comparisons, and its now doing it
> > a directory at a time:
> >
> > %for i in /data2/argus/05/*; do echo $i; racount -r $i/*; racluster -R
> > $i -m srcid -s trans pkts
> >
> > The record values in racluster() are not going to be the same
> > as racount(), because racount() includes the management records
> > in the counting and racluster does not (since they are not merged).
> >
> > Carter
> >
> > On Jun 21, 2006, at 8:32 AM, Carter Bullard wrote:
> >
> >> Hmmm, are you a bash shell user or a csh?
> >> Do me a favor and try something like this (assuming bash)
> >>
> >> % for i in /data2/argus/05/*/* ; do echo $i; racluster -r $i -m
> >> proto -s proto trans; done
> >>
> >> and see if something doesn't look strange.
> >>
> >> Carter
> >>
> >> On Jun 21, 2006, at 4:44 AM, Robin Gruyters wrote:
> >>
> >>> Hello Carter,
> >>>
> >>> I have just ran racluster() with "-r" option and I get the same
> output.
> >>> (without any errors)
> >>>
> >>> [racluster]
> >>> # racluster -r /data2/argus/05/*/* -m proto \
> >>> -s proto trans pkts spkts dpkts bytes sbytes dbytes - net
> 82.148.219.32/28
> >>>
> esp 1 1 1 0 310 310 0
> >>> gre 40440 16142557 6955364 9187193 4000449819 1192848094
> >>> 2807601725
> >>> udp 283037 554632 296948 257684 86633957 35435077
> >>> 51198880
> >>> tcp 144666 69282162 28369630 40912532 3994126059 2934170533
> >>> 1059955526
> >>> icmp 36644 50347 50270
> 77 4126254 4121768 4486
> >>> [end racluster]
> >>>
> >>> Regards,
> >>>
> >>> Robin
> >>>
> >>> On Wed, Jun 21, 2006 at 03:32:49AM -0400, Carter Bullard wrote:
> >>>> Hey Robin,
> >>>> Looks like somethings up. Try calling racluster with the -r /
> >>>> data2/argus/05/*/*.
> >>>> I suspect that its getting some kind of error on one of the files,
> >>>> and is stopping,
> >>>> especially since your getting 1/10th the number of tcp transactions
> >>>> (144666 vs 1443778).
> >>>> I remember that you had a bug report with toooo many filenames, if
> >>>> you're
> >>>> still getting that problem, we'll have to try to figure out what is
> >>>> causing racluster()
> >>>> issues.
> >>>>
> >>>> Sorry for the problems,
> >>>>
> >>>> Carter
> >>>>
> >>>>
> >>>> On Jun 21, 2006, at 3:21 AM, Robin Gruyters wrote:
> >>>>
> >>>>> Hi Carter,
> >>>>>
> >>>>> At the moment we use the output of racount() for our monthly report
> to
> >>>>> customers. To show them how much data they have used. (by proto and
> >>>>> total)
> >>>>>
> >>>>> For me it doesn't matter if this is possible with racount() or
> >>>>> racluster(),
> >>>>> if I just get the output done.
> >>>>> If this is possible with racount() and with the "-M addr" option,
> >>>>> great!
> >>>>>
> >>>>> The other options, like counts on ports etc, that would also be
> >>>>> nice to
> >>>>> have.
> >>>>>
> >>>>> Altough you say it is also possible with racluster(), but the
> >>>>> outcome is
> >>>>> totally different what I get back from racount(). (I mean the
> numbers)
> >>>>>
> >>>>> [racount]
> >>>>> # racount -ar /data2/argus/05/*/* - net 82.148.219.XXX/28
> >>>>> racount records total_pkts src_pkts
> >>>>> dst_pkts total_bytes src_bytes dst_bytes
> >>>>> tcp 1443778 69225031 28344760
> >>>>> 40880271 55494468479 7222126408 48272342071
> >>>>> udp 280703 549026 293754
> >>>>> 255272 86044190 35139486 50904704
> >>>>> icmp 35102 47042 46966
> >>>>> 76 3503635 3499223 4412
> >>>>> ip 40441 16142558 6955365
> >>>>> 9187193 4000450129 1192848404 2807601725
> >>>>> sum 1800024 85963657 35640845
> >>>>> 50322812 59584466433 8453613521 51130852912
> >>>>> [end racount]
> >>>>>
> >>>>> [racluster]
> >>>>> # racluster -R /data2/argus/05 -m proto \
> >>>>> -s proto trans pkts spkts dpkts bytes sbytes dbytes - net
> >>>>> 82.148.219.XXX/28
> >>>>> esp 1 1 1 0 310
> >>>>> 310 0
> >>>>> gre 40440 16142557 6955364 9187193 4000449819 1192848094
> >>>>> 2807601725
> >>>>> udp 283037 554632 296948 257684 86633957
> >>>>> 35435077 51198880
> >>>>> tcp 144666 69282162 28369630 40912532 3994126059 2934170533
> >>>>> 1059955526
> >>>>> icmp 36644 50347 50270 77 4126254
> >>>>> 4121768 4486
> >>>>> [end racluster]
> >>>>>
> >>>>> If you only check the "total bytes" on TCP packets. With racount()
> >>>>> I get
> >>>>> 55494468479 bytes and with racluster() 3994126059 bytes. That is a
> >>>>> huge
> >>>>> difference.
> >>>>>
> >>>>> Is there an explanation for this behaviour?
> >>>>>
> >>>>> Regards,
> >>>>>
> >>>>> Robin
> >>>>>
> >>>>> On Tue, Jun 20, 2006 at 11:04:01AM -0400, Carter Bullard wrote:
> >>>>>> Hey Robin et al.,
> >>>>>> You have become the target of anything racount() related ;o)
> >>>>>>
> >>>>>> So, in trying to understand if the "-M proto" option is useful,
> >>>>>> I realized that all of the old racount() functions are supported
> >>>>>> by racluster(), so I don't want to duplicate features, so I may
> >>>>>> end up redefining racount(), but keeping its default behavior.
> >>>>>> What I will do for now is leave it as it is, no -A support, but
> >>>>>> with the "-M addr" option and then figure out what to do after
> >>>>>> that based on the lists opinion.
> >>>>>>
> >>>>>> I use racount as a quick and dirty way of seeing how big is
> >>>>>> an argus data file, and to check if programs like racluster()
> >>>>>> preserve the counts when it aggregates records, so the
> >>>>>> default mode is great, but we can also generate the exact same
> >>>>>> output using racluster(), you just have to type more on the
> >>>>>> command line to get the output right. Same goes for the old -a
> >>>>>> option:
> >>>>>>
> >>>>>> The older racount() functions can be done in racluster() as:
> >>>>>>
> >>>>>> racount -r file
> >>>>>> racluster -r file -m srcid -s trans pkts spkts dpkts bytes sbytes
> >>>>>> dbytes
> >>>>>>
> >>>>>> racount -ar file
> >>>>>> racluster -r file -m proto -s proto trans pkts spkts dpkts bytes
> >>>>>> sbytes dbytes
> >>>>>>
> >>>>>> Now, with the '-M addr', we have a unique counting situation,
> >>>>>> and so that seems appropriate, and I think there should be more
> >>>>>> counting things to do, like ports, mac address types (vendor ids),
> >>>>>> that kind of thing.
> >>>>>>
> >>>>>> So, opinions? If we could discuss the counting requirements, that
> >>>>>> might help define racount a bit.
> >>>>>>
> >>>>>> Carter
> >>>>>>
> >>>>>
> >>>>
> >>>> Carter Bullard
> >>>> CEO/President
> >>>> QoSient, LLC
> >>>> 150 E. 57th Street Suite 12D
> >>>> New York, New York 10022
> >>>>
> >>>> +1 212 588-9133 Phone
> >>>> +1 212 588-9134 Fax
> >>>>
> >>>>
> >>>
> >>
> >> Carter Bullard
> >> CEO/President
> >> QoSient, LLC
> >> 150 E. 57th Street Suite 12D
> >> New York, New York 10022
> >>
> >> +1 212 588-9133 Phone
> >> +1 212 588-9134 Fax
> >>
> >>
> >>
> >
> > Carter Bullard
> > CEO/President
> > QoSient, LLC
> > 150 E. 57th Street Suite 12D
> > New York, New York 10022
> >
> > +1 212 588-9133 Phone
> > +1 212 588-9134 Fax
>
>
>
> ------------------------------
>
> Message: 4
> Date: Wed, 21 Jun 2006 09:59:30 -0400
> From: Carter Bullard <carter at qosient.com>
> Subject: Re: [ARGUS] Re: racount status and its definition
> To: Robin Gruyters <r.gruyters at yirdis.nl>
> Cc: Argus <argus-info at lists.andrew.cmu.edu>
> Message-ID: <71AC84C0-6766-4B72-84E2-CE8BBE720ABC at qosient.com>
> Content-Type: text/plain; charset=US-ASCII; delsp=yes; format=flowed
>
> At least argus-3.0 racluster and racount agree, as that is the behavior
> that i've seen. Any malformed record is suspect, of course.
>
> Hmmm, can you share the file, so I can attempt to correct the problem?
> If its customer data, I can understand any sensitivities.
>
> I have a program that I was going to release, after all the dust
> settles,
> that fixes/recovers corrupt argus files, but it is argus-3.0 format only
> and is really experimental and so it won't work here, but I can attempt
> to do the same thing with argus-2.0 data if there is demand.
>
> Carter
>
>
> On Jun 21, 2006, at 9:38 AM, Robin Gruyters wrote:
>
> > Hi Carter,
> >
> > Here is the output with argus-3 commands (racount() and racluster
> > ()). I have just pointed to the file which gives errors with
> > argus-2.0.6 (racount()).
> >
> > [...]
> > $ racount -r /data2/argus/05/21/*; \
> > racluster -R /data2/argus/05/21 -m srcid -s trans pkts
> > racount records total_pkts src_pkts dst_pkts
> > total_bytes src_bytes dst_bytes
> > sum 13951 771479 284510 486969
> > 303901576 42836384 261065192
> > 13927 771479
> > [...]
> >
> > Here is the argus-2.0.6 output:
> >
> > [...]
> > $ racount -r /data2/argus/archive/2006/05/21/*
> > ArgusWarning: racount[48517]: ArgusReadSocketStream: malformed
> > argus record len 17793
> >
> > racount records total_pkts src_pkts
> > dst_pkts total_bytes src_bytes dst_bytes
> > sum 149250 1143266 566499
> > 576767 355491645 72812173 282679472
> > [...]
> >
> > Regards,
> >
> > Robin
> >
> >
> > Quoting Carter Bullard <carter at qosient.com>:
> >
> >> Hey Robin,
> >> You can try this to see any primary discrepancies. If you find
> >> a file that does generate count problems, we can zoom in on
> >> a few things quickly and get at the root of the problem. I modified
> >> the test script to give you direct comparisons, and its now doing it
> >> a directory at a time:
> >>
> >> %for i in /data2/argus/05/*; do echo $i; racount -r $i/*;
> >> racluster -R
> >> $i -m srcid -s trans pkts
> >>
> >> The record values in racluster() are not going to be the same
> >> as racount(), because racount() includes the management records
> >> in the counting and racluster does not (since they are not merged).
> >>
> >> Carter
> >>
> >> On Jun 21, 2006, at 8:32 AM, Carter Bullard wrote:
> >>
> >>> Hmmm, are you a bash shell user or a csh?
> >>> Do me a favor and try something like this (assuming bash)
> >>>
> >>> % for i in /data2/argus/05/*/* ; do echo $i; racluster -r $i -
> >>> m proto -s proto trans; done
> >>>
> >>> and see if something doesn't look strange.
> >>>
> >>> Carter
> >>>
> >>> On Jun 21, 2006, at 4:44 AM, Robin Gruyters wrote:
> >>>
> >>>> Hello Carter,
> >>>>
> >>>> I have just ran racluster() with "-r" option and I get the same
> >>>> output.
> >>>> (without any errors)
> >>>>
> >>>> [racluster]
> >>>> # racluster -r /data2/argus/05/*/* -m proto \
> >>>> -s proto trans pkts spkts dpkts bytes sbytes dbytes - net
> >>>> 82.148.219.32/28
> >>>> esp 1 1 1 0 310
> >>>> 310 0
> >>>> gre 40440 16142557 6955364 9187193 4000449819
> >>>> 1192848094 2807601725
> >>>> udp 283037 554632 296948 257684 86633957
> >>>> 35435077 51198880
> >>>> tcp 144666 69282162 28369630 40912532 3994126059
> >>>> 2934170533 1059955526
> >>>> icmp 36644 50347 50270 77 4126254
> >>>> 4121768 4486
> >>>> [end racluster]
> >>>>
> >>>> Regards,
> >>>>
> >>>> Robin
> >>>>
> >>>> On Wed, Jun 21, 2006 at 03:32:49AM -0400, Carter Bullard wrote:
> >>>>> Hey Robin,
> >>>>> Looks like somethings up. Try calling racluster with the -r /
> >>>>> data2/argus/05/*/*.
> >>>>> I suspect that its getting some kind of error on one of the files,
> >>>>> and is stopping,
> >>>>> especially since your getting 1/10th the number of tcp
> >>>>> transactions
> >>>>> (144666 vs 1443778).
> >>>>> I remember that you had a bug report with toooo many filenames, if
> >>>>> you're
> >>>>> still getting that problem, we'll have to try to figure out
> >>>>> what is
> >>>>> causing racluster()
> >>>>> issues.
> >>>>>
> >>>>> Sorry for the problems,
> >>>>>
> >>>>> Carter
> >>>>>
> >>>>>
> >>>>> On Jun 21, 2006, at 3:21 AM, Robin Gruyters wrote:
> >>>>>
> >>>>>> Hi Carter,
> >>>>>>
> >>>>>> At the moment we use the output of racount() for our monthly
> >>>>>> report to
> >>>>>> customers. To show them how much data they have used. (by
> >>>>>> proto and
> >>>>>> total)
> >>>>>>
> >>>>>> For me it doesn't matter if this is possible with racount() or
> >>>>>> racluster(),
> >>>>>> if I just get the output done.
> >>>>>> If this is possible with racount() and with the "-M addr" option,
> >>>>>> great!
> >>>>>>
> >>>>>> The other options, like counts on ports etc, that would also be
> >>>>>> nice to
> >>>>>> have.
> >>>>>>
> >>>>>> Altough you say it is also possible with racluster(), but the
> >>>>>> outcome is
> >>>>>> totally different what I get back from racount(). (I mean the
> >>>>>> numbers)
> >>>>>>
> >>>>>> [racount]
> >>>>>> # racount -ar /data2/argus/05/*/* - net 82.148.219.XXX/28
> >>>>>> racount records total_pkts src_pkts
> >>>>>> dst_pkts total_bytes src_bytes dst_bytes
> >>>>>> tcp 1443778 69225031 28344760
> >>>>>> 40880271 55494468479 7222126408 48272342071
> >>>>>> udp 280703 549026 293754
> >>>>>> 255272 86044190 35139486 50904704
> >>>>>> icmp 35102 47042 46966
> >>>>>> 76 3503635 3499223 4412
> >>>>>> ip 40441 16142558 6955365
> >>>>>> 9187193 4000450129 1192848404 2807601725
> >>>>>> sum 1800024 85963657 35640845
> >>>>>> 50322812 59584466433 8453613521 51130852912
> >>>>>> [end racount]
> >>>>>>
> >>>>>> [racluster]
> >>>>>> # racluster -R /data2/argus/05 -m proto \
> >>>>>> -s proto trans pkts spkts dpkts bytes sbytes dbytes - net
> >>>>>> 82.148.219.XXX/28
> >>>>>> esp 1 1 1 0 310
> >>>>>> 310 0
> >>>>>> gre 40440 16142557 6955364 9187193 4000449819 1192848094
> >>>>>> 2807601725
> >>>>>> udp 283037 554632 296948 257684 86633957
> >>>>>> 35435077 51198880
> >>>>>> tcp 144666 69282162 28369630 40912532 3994126059 2934170533
> >>>>>> 1059955526
> >>>>>> icmp 36644 50347 50270 77 4126254
> >>>>>> 4121768 4486
> >>>>>> [end racluster]
> >>>>>>
> >>>>>> If you only check the "total bytes" on TCP packets. With
> >>>>>> racount()
> >>>>>> I get
> >>>>>> 55494468479 bytes and with racluster() 3994126059 bytes. That
> >>>>>> is a
> >>>>>> huge
> >>>>>> difference.
> >>>>>>
> >>>>>> Is there an explanation for this behaviour?
> >>>>>>
> >>>>>> Regards,
> >>>>>>
> >>>>>> Robin
> >>>>>>
> >>>>>> On Tue, Jun 20, 2006 at 11:04:01AM -0400, Carter Bullard wrote:
> >>>>>>> Hey Robin et al.,
> >>>>>>> You have become the target of anything racount() related ;o)
> >>>>>>>
> >>>>>>> So, in trying to understand if the "-M proto" option is useful,
> >>>>>>> I realized that all of the old racount() functions are supported
> >>>>>>> by racluster(), so I don't want to duplicate features, so I may
> >>>>>>> end up redefining racount(), but keeping its default behavior.
> >>>>>>> What I will do for now is leave it as it is, no -A support, but
> >>>>>>> with the "-M addr" option and then figure out what to do after
> >>>>>>> that based on the lists opinion.
> >>>>>>>
> >>>>>>> I use racount as a quick and dirty way of seeing how big is
> >>>>>>> an argus data file, and to check if programs like racluster()
> >>>>>>> preserve the counts when it aggregates records, so the
> >>>>>>> default mode is great, but we can also generate the exact same
> >>>>>>> output using racluster(), you just have to type more on the
> >>>>>>> command line to get the output right. Same goes for the old -a
> >>>>>>> option:
> >>>>>>>
> >>>>>>> The older racount() functions can be done in racluster() as:
> >>>>>>>
> >>>>>>> racount -r file
> >>>>>>> racluster -r file -m srcid -s trans pkts spkts dpkts bytes
> >>>>>>> sbytes
> >>>>>>> dbytes
> >>>>>>>
> >>>>>>> racount -ar file
> >>>>>>> racluster -r file -m proto -s proto trans pkts spkts dpkts
> >>>>>>> bytes
> >>>>>>> sbytes dbytes
> >>>>>>>
> >>>>>>> Now, with the '-M addr', we have a unique counting situation,
> >>>>>>> and so that seems appropriate, and I think there should be more
> >>>>>>> counting things to do, like ports, mac address types (vendor
> >>>>>>> ids),
> >>>>>>> that kind of thing.
> >>>>>>>
> >>>>>>> So, opinions? If we could discuss the counting
> >>>>>>> requirements, that
> >>>>>>> might help define racount a bit.
> >>>>>>>
> >>>>>>> Carter
> >>>>>>>
> >>>>>>
>
>
>
>
> ------------------------------
>
> _______________________________________________
> Argus-info mailing list
> Argus-info at lists.andrew.cmu.edu
> https://lists.andrew.cmu.edu/mailman/listinfo/argus-info
>
>
> End of Argus-info Digest, Vol 10, Issue 37
> ******************************************
>
--
Best Regards,
CS Lee<geek00L[at]gmail.com>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20060621/34260af8/attachment.html>
More information about the argus
mailing list