problem experienced with ra client: unequal results

[carter at qosient.com]

Hey Réal,
Peter is right, we do have a little instability with the release candidates, and I've been extremely busy on real work, so, ...., my fault at not getting the fixes out quickly!!

The zero length record is a persistent problem, and I still do not have enough data to fix it properly.  If you can share platform, strategy (radium?) and 64-bit vs 32-bit info and of course a sample data file that causes the error, that would go a long way to fixing the problem.

The reason you get no output, is a result of the pipes, where a failure along the set of pipes, causes the down stream processes to terminate early.

I think rather than filter the errors, I need to fix the problem.  If at all possible, if you can share a file that dies with the error, that would help!!!

Carter

Carter Bullard
QoSient LLC
150 E. 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax

-----Original Message-----
From: real.melancon at videotron.ca

Date: Wed, 04 Jul 2007 01:29:34 
To:argus-info at lists.andrew.cmu.edu
Subject: [ARGUS] problem experienced with ra client: unequal results


Hello List,

I use latest argus daemon as well as latest ra* clients.

We collect data using argus daemon using:

/argus -d -S 60 -F /etc/argus/argus.conf -w /var/log/argus/argus.out -i eth1

Then rotate argus.out every hour (using argusarchive) , which generates files in format:

/var/log/argus/archive/YYYY/MM/DD/argus.YYYY.MM.DD.hh.mm.ss.gz

This works well. For example to get Top Talkers & listeners, we use:

/usr/local/bin/racluster -m matrix -r /var/log/argus/argus.out -w - | /usr/local/bin/rasort -m bytes -w - | /usr/local/bin/ra -nu

For specific days, we use (e.g. July 1st, between 15:00 and 7:00):

/usr/local/bin/racluster -t 01.15:00-17:00 -m matrix -r /var/log/argus/archive/2007/07/01/* -w - | /usr/local/bin/rasort -m bytes -w - | /usr/local/bin/ra -nu

But. Here is the problem.... (sorry for the long introduction)

Sometimes, argus ra client just doesn't output any data. e.g.

/usr/local/bin/racluster -t 02.15:00-17:00 -m matrix -r /var/log/argus/archive/2007/07/02/* -w - | /usr/local/bin/rasort -m bytes -w - | /usr/local/bin/ra -nu

same syntax as before but for a different day. data file size is about same size, but ra doesn't output anything but:
racluster[9842]: 1183512263 ArgusReadStreamSocket (0xb7e30ddc) record length is zero
racluster[9842]: 1183512263 ArgusReadStreamSocket (0xb7e30ddc) record length is zero
racluster[9842]: 1183512273 ArgusReadStreamSocket (0xb7e30ddc) record length is zero

or sometimes, only outputs one line:

/usr/local/bin/racluster -t 30.15:00-17:00 -m matrix -r /var/log/argus/archive/2007/06/30/* -w - | /usr/local/bin/rasort -m bytes -w - | /usr/local/bin/ra -nu

racluster[9869]: 01:26:05.607249 ArgusReadStreamSocket (0xb7dc7ddc) record length is zero

My questions is:

(1) Is there any way to filter out these errors ?

My argus.conf daemon options are:
ARGUS_DAEMON=yes
ARGUS_DEBUG_LEVEL=0
ARGUS_MONITOR_DATA=`hostname`
ARGUS_ACCESS_PORT=0
ARGUS_INTERFACE=eth1
ARGUS_SET_PID=no
ARGUS_GO_PROMISCUOUS=yes
ARGUS_FLOW_STATUS_INTERVAL=60
ARGUS_GENERATE_START_RECORDS=yes
ARGUS_GENERATE_RESPONSE_TIME_DATA=yes
ARGUS_GENERATE_JITTER_DATA=yes
ARGUS_GENERATE_MAC_DATA=no
ARGUS_FILTER_OPTIMIZER=yes
ARGUS_CAPTURE_DATA_LEN=0

Thanks in advance.
Real.



____________________________
Réal Melançon