[ARGUS] Argus importing zeek data

Thu Mar 31 09:11:54 EDT 2022

I had to fetch argus/client manually and install it a while back. It's been
running smoothly.
I remember when installing security onion you have the option to install
argus, but they removed the option some time ago.

On Thu, Mar 31, 2022 at 9:05 AM Carter Bullard <carter at qosient.com> wrote:

> Not sure where security onion gets its argus ... the github.com is pretty
> new !!
> Carter
>
> On Mar 31, 2022, at 8:49 AM, Monah Baki <monahbaki at gmail.com> wrote:
>
> 
> Morning Carter,
>
> Apparently security onion does not work. I was able to install and run
> successfully on a plain Centos 7 running zeek 4.0.5.
>
> Thanks
> Monah
>
> On Wed, Mar 30, 2022 at 5:32 PM Carter Bullard <carter at qosient.com> wrote:
>
>> Hey Monah,
>> I made some more changes and its working for me with your data ...
>>  please refetch and pull ... and all should be good ... if not ...  try a
>> '-X' to clear out any conf that maybe in the way ...
>>
>> if no joy, run with '-D 5' and send the output !!!
>>
>> Carter
>>
>> On Mar 30, 2022, at 5:22 PM, Monah Baki <monahbaki at gmail.com> wrote:
>>
>> 
>> Hi Carter,
>>
>> Downloaded the new clients-master, and still having the same issue.
>>
>> No matter what RACONVERT_TIME_FORMAT I use, the output looks like:
>> 00:00:00.14370365*
>>
>> Monah
>>
>> On Wed, Mar 30, 2022 at 12:10 PM Carter Bullard <carter at qosient.com>
>> wrote:
>>
>>> Hey Monah,
>>> I fixed the issue so that your data and the default conf file will work.
>>> I noticed that your timestamps were not parsed correctly, as you had not
>>> set the correct time parsing method in the conf file.
>>> I fixed that as well.
>>>
>>> Fetch the new master to get the config and the new code …
>>>
>>> Carter
>>>
>>> On Mar 30, 2022, at 10:42 AM, Carter Bullard <carter at qosient.com> wrote:
>>>
>>> Hey Monah,
>>> Its the community_id key … its not in the raconvert.zeek.conf file …
>>> raconvert should ignore fields that it doesn’t know, so I’ll fix that ...
>>> Try this configuration file … I added ‘community_id’ to
>>> the RACONVERT_FIELD_SPECIFIER variable, and mapped the value to the
>>> label …
>>>
>>> Carter
>>>
>>> <raconvert.zeek.conf>
>>>
>>>
>>> On Mar 30, 2022, at 10:27 AM, Monah Baki <monahbaki at gmail.com> wrote:
>>>
>>> Morning Carter
>>>
>>> We are running security onion with zeek. Here is a sample
>>>
>>> cat /nsm/zeek/logs/current/conn.log
>>>
>>> {"ts":1648648794.073199,"uid":"C9v77B3hIQWKghwEc2","id.orig_h":"172.16.100.149","id.orig_p":55909,"id.resp_h":"172.16.86.65","id.resp_p":6027,"proto":"tcp","conn_state":"S0","local_orig":true,"local_resp":tru
>>>
>>> e,"missed_bytes":0,"history":"S","orig_pkts":1,"orig_ip_bytes":52,"resp_pkts":0,"resp_ip_bytes":0,"community_id":"1:xacMILtZUoqN7bOt5I2J4n3s6UU="}
>>>
>>> {"ts":1648648794.108026,"uid":"CVYLk43Gi0NzjmuUj6","id.orig_h":"172.16.245.73","id.orig_p":53536,"id.resp_h":"172.16.86.14","id.resp_p":8530,"proto":"tcp","duration":0.015297889709472657,"orig_bytes":104,"res
>>>
>>> p_bytes":505,"conn_state":"SF","local_orig":true,"local_resp":true,"missed_bytes":0,"history":"ShADfFa","orig_pkts":5,"orig_ip_bytes":316,"resp_pkts":3,"resp_ip_bytes":637,"community_id":"1:waj0R3ypJ68ox0zi78
>>> BBw07KepY="}
>>>
>>> If I use  /usr/local/bin/raconvert -f /etc/raconvert.zeek.conf -r
>>> /nsm/zeek/logs/current/conn.log -w <filename>, the filename does not get
>>> created
>>>
>>> Had to run the command without the -w, as you can see below
>>>
>>> [root at sosensor admin]# /usr/local/bin/raconvert -f
>>> /etc/raconvert.zeek.conf -r /nsm/zeek/logs/current/conn.log
>>>          StartTime      Flgs  Proto            SrcAddr  Sport   Dir
>>>        DstAddr  Dport  TotPkts   TotBytes State
>>>   06:53:52.4313291
>>> 00:00:00.13203538*
>>> 00:00:00.13203538*
>>> 00:00:00.13203538*
>>> 00:00:00.13203538*
>>> 00:00:00.13203538*
>>> 00:00:00.13203538*
>>> 00:00:00.13203538*
>>>
>>> Am I missing anything?
>>>
>>>
>>> Monah
>>>
>>>
>>> On Sun, Mar 27, 2022 at 11:52 AM Carter Bullard <carter at qosient.com>
>>> wrote:
>>>
>>>> Gentle persons,
>>>> I’ve pushed new code to the GitHub openargus clients repo,
>>>> https://github.com/openargus/clients , and bumped the version to
>>>> 3.0.8.4.
>>>> This specific push adds foreign flow data imports using raconvert.1,
>>>> and I’ve added a raconvert.zeek.conf that enables raconvert.1 to read zeek
>>>> conn logs.
>>>>
>>>> To convert a zeek conn log to argus binary:
>>>>    % raconvert -f raconvert.zeek.conf -r zeek.conn.log -w argus.file
>>>>
>>>> The provided raconvert.zeek.conf conversion map, maps all of the zeek
>>>> fields that I had available.  The conversion map is pretty chunky, as you
>>>> need to specify the allowed key, value pairs, specify types, and identify
>>>> where in an argus record the data will go …  If there isn’t a native argus
>>>> attribute for the zeek field, say the “uid”, raconvert.1 can map the key,
>>>> value pair it to the argus label, which you can filter, search, etc ….
>>>>
>>>> Converting zeek json data to argus binary reduces the size of the files
>>>> by 2.5:1 or about 1/3rd.  And gzip’d argus binaries are smaller than gzip’d
>>>> json zeek.conn.logs, by around 1.3:1 … not much but it is not worse :O)
>>>>
>>>> Converting the zeek logs to argus enables processing zeek data with
>>>> argus clients programs like racluster.1, say if you want to generate
>>>> baselines, or to generate different views.  The clients enable you to add
>>>> country codes, and ASNs for addresses simply, and I like using
>>>> rafilteraddr.1 with address lists from 3rd party intelligence like Firehol
>>>> to check to see if the zeek data has any reputation hits ... and you can of
>>>> course view the data with ratop.1 …
>>>>
>>>> Please grab this client code and test it out, if interested …
>>>>
>>>> I would love any opinions about the new GitHub software approach.  If
>>>> you have any suggestions, please email the list or to me ...
>>>> Hope all is most excellent,
>>>>
>>>> Carter
>>>>
>>>>
>>> _______________________________________________
>>> argus mailing list
>>> argus at qosient.com
>>> https://pairlist1.pair.net/mailman/listinfo/argus
>>>
>>>
>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20220331/adca0272/attachment.htm>