raservices crashes when processing

[Matt Brown]

Good Morning Carter,





As far as collecting user data, looks good to me:



# radump -r * -s suser duser | wc -l

195492

# radump -r * -s suser duser | grep 's\[0\]=""' | wc -l

36307

# radump -r * -s suser duser | grep 's\[[1-9].*' | wc -l

159184



I used the data file produced with rastream:

rastream -d -S 127.0.0.1:561 -B 15s -M time 1h <x-apple-data-detectors://1> -w
/var/opt/argus/%Y-%m-%d/argus_%T -f /usr/local/bin/rastream.sh



argus running as:

argus -d -i eth0 -P 561



argus.conf with ARGUS_CAPTURE_DATA_LEN set:

# cat /etc/argus.conf | egrep -v '^$|^[#]'

ARGUS_FLOW_TYPE="Bidirectional"

ARGUS_FLOW_KEY="CLASSIC_5_TUPLE"

ARGUS_MONITOR_ID="..." #         // String

ARGUS_SET_PID=yes

ARGUS_PID_PATH="/var/run"

ARGUS_FLOW_STATUS_INTERVAL=60

ARGUS_MAR_STATUS_INTERVAL=300

ARGUS_CAPTURE_DATA_LEN=256







Working off the contents of ../support/Config/sig.std and Dave's great
advice, I performed the following:

# racluster -r * -w day.cache

# rauserdata -r day.cache -M printer="encode32" > /tmp/raservices.conf



Even without editing the file (clearly needs to be analyzed and thinned
down to be useful), I tried to run raservices to analyze some data:



# racluster -r * -s saddr daddr dport suser duser -w - | raservices -f
/tmp/raservices.conf

and it segfaults… (as with Dave)



# racluster -r * -N 50 -s saddr daddr dport suser duser -w - | raservices
-f ~/argus-clients-3.0.7.8/support/Config/std.sig -s saddr daddr dport
bytes label:20

.. produces though and is really very cool!





Thanks for drilling into the problem Dave!

Thanks Carter for the solution: adjusting '#define ARGUSMAXSIGFILE` in
../clients/include/argus_client.h then recompiling

I suppose the file should not need to be larger than 2048 anyway, right?





Does anyone have any interest sharing their own raservice conf file?





I also performed what Dave explained, encoding to all other data types, and
'hex' seems to be accepted by raservices, but segfaults (without changing
ARGUSMAXSIGFILE from 2048):

rauserdata -r day.cache -M printer="hex" | head -n 50 > /tmp/raservices.conf

racluster -r * -s saddr daddr dport suser duser -w - | raservices -f
/tmp/raservices.conf -s saddr daddr dport bytes label:20





So, the raservices conf file should be only as large as it needs to be to
define suser and duser contents as a protocol.  It can contain data encoded
in 32 bit chars, or maybe hex, and if it's over 2048 bytes, you must adjust
the constant and recompile the clients.  Sounds right?





Thanks very much guys!



Matt

On May 16, 2013, at 8:30 AM, Carter Bullard <carter at qosient.com> wrote:

Hey Matt,
This is not a crash, which is a programatic unrecoverable fault.  You just
didn't generate a good raservices() configuration file.

Try using the provided ./support/Config/sig.std, as a starting point for
raservices(), to see if you can get good labels?

Are you sucessfully generating user data yet?

Carter

On May 15, 2013, at 5:55 PM, Matt Brown <matthewbrown at gmail.com> wrote:

Hello all,



I took a day's worth of argus data and, as suggested on
http://thread.gmane.org/gmane.network.argus/6228/focus=6234, I analyzed it
with rauserdata as follows:



#racluster -r * -w day.cache

#rauserdata -r day.cache > /tmp/raservices.conf





I then inspected /tmp/raservices.conf and it's messy (lots of single lines
with arbirary ports, likely sport maybe rpc?), but I figured why not give
raservices a shot:



#racluster -r * -w - | raservices -f raservices.conf



I receive the following error:

raservices[21315]: 16:51:00.727719 RaCreateSrvEntry: format error Service:
http





I straced the process, and I see no occurances of "http" in the output
(other than the writev()); the data appears to be read correctly until a
blank line is read [read(3, "", 4096)                       = 0]:



read(3, "\"  \n\nService: 48956             "..., 4096) = 4096

read(3, "...xxxxxx"  dst ="..., 4096) = 4096

read(3, "xxxx"..., 4096) = 689

read(3, "", 4096)                       = 0

close(3)                                = 0

munmap(0xb766e000, 4096)                = 0

gettimeofday({1368651683, 272271}, NULL) = 0

time(NULL)                              = 1368651683

writev(2, [{"raservices[21523]: 17:01:23.2722"..., 79}, {"\n", 1}],
2raservices[21523]: 17:01:23.272271 RaCreateSrvEntry: format error Service:
http

) = 80





Any idea on why this would be?  Is my data processing flow incorrect?





Both clients are 3.0.7.8.





Thanks,



Matt
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130516/ba503096/attachment.html>