raconvert

[Carter Bullard]

Hey CS Lee,
That's great!!!  There are a large number of argus users that are also using Splunk,
and I've talked to the company about directions on the best way to go, but those
have only started.

If there is anything I can do, holler!!!!

Carter

On Jul 29, 2010, at 11:57 AM, CS Lee wrote:

> hi Carter,
> 
> I was having the problem as well until I tried to get argus data into splunk, and in fact I have almost all the fields in argus extracted and send to splunk, I always put suser data and duser data at last field. My argus data is in csv form and this is how I have it done with splunk though -
> 
> In the props.conf(properties config)
> [argus]
> sourcetype = argus
> REPORT-argus = argus-fields, argus-suser-data, argus-duser-data
> 
> In the transforms.conf(data transformation config)
> [argus-fields]
> DELIMS = ","
> FIELDS = "stime","flags","proto","src_ip","src_port","direction","dst_ip","dst_port","state","duration","pkts","bytes","appbytes","pps","bps","src_pkts","dst_pkts","src_bytes","dst_bytes","src_appbytes","dst_appbytes","src_pps","dst_pps","src_bps","dst_bps" 
> 
> [argus-suser-data]
> REGEX = ,s\[\d+\]=(?<suser_data>.{0,64}),?
> 
> [argus-duser-data]
> REGEX = ,d\[\d+\]=(?<duser_data>.{0,64})
> 
> I don't expect everyone to get the idea at first glance however if you are familiar with splunk or regex this won't be too hard. 
> 
> I'm not trying to promote splunk here but since both of them can be glued together so well, I just want to be able to perform analysis on every field i can obtain from argus record, and graphing them, further generating report. On top of them you can still keep argus record in its own format and processed by ra like tools when you need to do some other post processing which is not offered by splunk web.
> 
> I have argus app to splunk done and plan to release it soon. 
> 
> Cheers ;)
> 
> 
> On Thu, Jul 29, 2010 at 11:15 PM, Carter Bullard <carter at qosient.com> wrote:
> Hey CS Lee,
> Yes, the user buffers do need some work.  So how do other systems, like csv,
> deal with delimiters in the output?  Is there a universal escape strategy?
> 
> Good to see you around.
> Carter
> 
> On Jul 28, 2010, at 11:23 AM, CS Lee wrote:
> 
>> hi Carter,
>> 
>> How's life, think I'm back and will blog more about argus and flow stuffs!
>> 
>> Regarding raconvert, the tricky part I see would be converting user data field that is printed because I used to have the problem when using , or other character as delimeter and end up need to do additional parsing to get user data extracted properly in the ascii flow records.
>> 
>> Gentle people,
>> There is a new program in the clients distribution, raconvert(), with manpage.
>> 
>> This program is designed to convert ASCII based argus files to binary argus
>> data records.   The ASCII must have a single character delimiter, such as a ',',
>> but you can specify the delimiter, using the "-c char" option.
>> 
>>   ra -r argus.file -c ,  > /tmp/ra.txt
>>   raconvert -r /tmp/ra.txt -w - | ra
>> 
>> raconvert() is not complete.  Currently, I'm handling maybe 50 out of the 180
>> something fields that we can printout, but its time to put it out there, so if you
>> try to use it, and some fields don't get converted, send me a sample ascii file,
>> and I'll add the support that your field.
>> 
>> The records that we generate may not be complete.  It depends on how much
>> information you provide in the ascii records.  For instance if you only have
>> the "StartTime" field, without the "LastTime" field, the resulting binary argus
>> record will have a duration of 0, so you want to ensure that you have enough
>> information in the ascii output to convey all that you want.
>> 
>> Also, the name suggests that it should be able to do conversion, which may
>> imply that it converts more than just one thing to another, so, ......,
>> if you have any ideas as to what you would like to convert, just holler, and
>> I'll see what I can do.
>> 
>> I will try to add XML conversion before the summer is done.
>> 
>> So why this program?  The primary reason is to support moving argus data
>> around in environments that don't like binary data.  You convert the records
>> to ASCII, printing as many fields as practical, move the file to the next location,
>> and then convert them back to binary records so you can do work with them.
>> Some high security places need this type of support.  But you could also use
>> it as a means to create an argus data editor, if you wanted.
>> 
>> Hope you find this useful,
>> 
>> Carter
>> 
>> -- 
>> Best Regards,
>> 
>> CS Lee<geek00L[at]gmail.com>
>> 
>> http://geek00l.blogspot.com
>> http://defcraft.net
> 
> 




-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100729/5e9aa183/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100729/5e9aa183/attachment.bin>