ragrep newbie question ?

[Carter Bullard]

Hey John,
The ability to search for arbitrary strings, regardless of whether they are
particular domain names or if they are just embedded tags, is important.
So lets tackle this problem, I just need some discussion on what we
want to do.

The regular expression feature that is now in every client, was not really intended
for heavy duty greping.  ragrep() was a port of the entire regular expression
library.  Since we don't need to do that anymore, I integrated regex support 
to the common library, and retired ragrep(), as it didn't support that many features.

If we are going to do some serious searching, we probably should bring
back ragrep(), so we can do things like pass a file that contains the
regular expression rather than rely on the command line (need this to
get past the buffer limits of ARG_MAX).

If we do bring back ragrep(), what features, options etc should we support?

If we use ragrep() like grep(), an idea would be to printout filenames that contain
matches, rather than print the records, as we do now.  We could support various
options, like grep() does, to print the actual matches, i.e. the argus records that
contain the patterns.   We could support options like grep's  "-u" option to print
out the byte offsets of matching records, etc... ("-s +0offset")

Do we want to take this approach?  With the library support already in, this
shouldn't take long to do.  I could have the 'read the expression from a file'
support in a new ragrep() in a day or so.

What do you think?

Carter



On Mar 15, 2010, at 2:02 PM, John Kennedy wrote:

> How many OR'd names until a seg fault occurs?  A program similar to rafilteraddr.pl would come in handy for domain names
>  
> There are thousands if not millions of malicious domains and auditing for these gets a little interesting at times.  Building and maintaining rules for outbreaks in an IDS can get quite cumbersome. I think using a tool similar to rafilteraddr.pl would help keep a list concise and could be fast enough for integration into other avenues.
> 
> I am just curious if others audit for known malicious domains in their environment?  Is it even worth the trouble?
> 
> John
> 
> On Sun, Mar 14, 2010 at 4:58 PM, Stéphane Peters <stephane.peters at forem.be> wrote:
> Hello Carter,
> 
> could I do case insensitive searches with ra* clients ?
> Maybe the "-i" option of ragrep could be added in the next release ?
> 
> Meanwhile, let me share a command used to look for evidences of some virus / malware infection.
> It works pretty well without parenthesis; are they needed in this case ?
> The RE is split on several lines to permit a nice posting, it seems limited to 4096 bytes, about 200 site names.
>  #!/bin/bash
>  s="PPIHelper.com"
>  s="$s|solfire.aljosaborkovic.com"
>  s="$s|kukutrustnet777.info"
>  s="$s|www.kjwre.*fqwieluoi.info"
>  s="$s|l33t.brand-clothes.net"
>  s="$s|pica.banjalucke-ljepotice.ru"
>  s="$s|maellisromance.com"
>  s="$s|217.32.75.74"
>  s="$s|pingaksh.com"
>  s="$s|radio.irib.ir"
>  s="$s|regal-mont.pondi.hr"
>  s="$s|sandra.prichaonica.com"
>  s="$s|sasgrowth.com"
>  s="$s|snowboard619.w.interia.pl"
>  s="$s|spargeunid.go.ro"
>  s="$s|stakrix.st.funpic.de"
>  s="$s|us516757.bizhostnet.com"
>  s="$s|www.abassiehmunicipality.com"
>  s="$s|www.polaris.ge"
>  s="$s|www.railwayservices.be"
>  s="$s|www.senaauto.ge"
>  s="$s|ziyagokalpilkogretim72.meb.k12.tr"
>  ragrep -s "+suser:50 -bytes" -i -e "$s" $* - udp port 53
> Here an example of how it is used:
> ragrep-sality.sh -nr $file -w /tmp/sality-traces.ra
> 
> More details on the wiki site  :
>  http://nsmwiki.org/Argus#ragrep_example:_Finding_Palevo_.2F_Sality_virus_activity
> 
> julien a écrit :
> 
> Hello,
> 
> I have a small problem with ragrep, that I recently begin to use
> following this paper
> http://www.rawpacket.org/anonymous/papers/Argus-PracticalBotNetDetection.pdf
> 
> When I launch the following request:
> 
> $ ragrep -z -i -e '(http|https|ftp|get|post|head)' -r $src_log - dst
> port 80 or dst port 443 or dst port 8080
> (same without filter)
> 
> I get nothing. $src_log is an argus file converted from a pcap (with argus)
> In Ethereal, there is some http traffic and ragrep doesn't get it. why ?
> bad command line, pcap conversion or something else ?
> 
> Thanks.
> Best regards,
> 
>        Julien
> 
>  
> Regards,
> 
> -- 
> Stephane.Peters at forem.be.
> 
> 



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100317/2fa8bf28/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20100317/2fa8bf28/attachment.bin>