Feature request: grep hex strings with -e

Carter Bullard carter at qosient.com
Wed Oct 10 11:04:34 EDT 2012 

Hmmm, I'll upload a new 3.0.7.3 without.  Well, it will take a little while to figure
out how to provide you with hex base binary regular expression matching.
Especially if its a regular expression library specific issue.  

I'll put in compile time conditionals for when REG_ENHANCED is defined.
That maybe the best I can do.

Here is the patch to fix the argus_grep compile problem.

==== //depot/argus/clients/common/argus_grep.c#11 - /Volumes/Users/carter/argus/clients/common/argus_grep.c ====
57c57
<          int options = REG_EXTENDED | REG_ENHANCED | REG_NOSUB;
---
>          int options = REG_EXTENDED | REG_NOSUB;
59a60,62
> #if defined(REG_ENHANCED)
>          options |= REG_ENHANCED;
> #endif

Carter

On Oct 10, 2012, at 9:42 AM, Markku Parviainen <maketsi at gmail.com> wrote:

> Hi,
> 
> I have been busy a couple of days and wasn't able to test your patch until now.
> 
> Thank you for the quick response on changes. Unfortunately constant
> REG_ENHANCED is not defined in CentOS-version of /usr/include/regex.h,
> and thus your newer client packages fail to compile. The actual error
> is also easily missed on build flood, producing an installation that
> contain just a couple of client binaries (5 to be exact). Logs below.
> 
> I browsed around a bit in regex.h, but couldn't yet find an option
> that would enable \xNN notation. There are a lot of options there, but
> at the first glance, none of them seemed to have anything to do with
> that. I'm not entirely sure if it's still possible with that library
> though.
> 
> I downloaded the source for my local gnu grep, which supports \xNN
> notation via 'perl' grepping (-P option). It seems that it is using
> libpcre library for matching (pcre_compile). Dunno if that's helping
> you.
> 
> 
> gcc -O3 -I. -I../include   -DHAVE_CONFIG_H -DARGUS_SYSLOG -c ./argus_grep.c
> ./argus_grep.c: In function 'ArgusInitializeGrep':
> ./argus_grep.c:57: error: 'REG_ENHANCED' undeclared (first use in this function)
> ./argus_grep.c:57: error: (Each undeclared identifier is reported only once
> ./argus_grep.c:57: error: for each function it appears in.)
> make[1]: *** [argus_grep.o] Error 1
> make[1]: Leaving directory `--/argus-clients-3.0.7.3/common'
> ....
> make[1]: Entering directory `--/argus-clients-3.0.7.3/clients'
> gcc -O3 -I. -I../include -I../common  -DHAVE_CONFIG_H -c ./ra.c
> make[1]: *** No rule to make target `../lib/argus_client.a', needed by
> `../bin/ra'.  Stop.
> make[1]: Leaving directory `--/argus-clients-3.0.7.3/clients'
> making in ./examples
> ...
> 
> # fgrep -r -i REG_ENHANCED /usr/include/
> #
> 
> Package version used is argus-clients-3.0.7.3.
> 
> # gcc -v
> Using built-in specs.
> Target: x86_64-redhat-linux
> Configured with: ../configure --prefix=/usr --mandir=/usr/share/man
> --infodir=/usr/share/info --enable-shared --enable-threads=posix
> --enable-checking=release --with-system-zlib --enable-__cxa_atexit
> --disable-libunwind-exceptions --enable-libgcj-multifile
> --enable-languages=c,c++,objc,obj-c++,java,fortran,ada
> --enable-java-awt=gtk --disable-dssi --disable-plugin
> --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre
> --with-cpu=generic --host=x86_64-redhat-linux
> Thread model: posix
> gcc version 4.1.2 20080704 (Red Hat 4.1.2-51)
> 
> # uname -a
> Linux srv 2.6.18-274.17.1.el5 #1 SMP Tue Jan 10 17:25:58 EST 2012
> x86_64 x86_64 x86_64 GNU/Linux
> 
> # cat /etc/redhat-release
> CentOS release 5.7 (Final)
> 
> Btw. I use my opportunity here to bring this compile warning up too.
> Probably nothing, didn't evaluate:
> 
> ./rarpwatch.c: In function 'RaProcessRecord':
> ./rarpwatch.c:451: warning: passing argument 1 of
> 'RaValidateArpFlowRecord' makes integer from pointer without a cast
> ./rarpwatch.c:451: warning: passing argument 2 of
> 'RaValidateArpFlowRecord' makes integer from pointer without a cast
> 
> 
> 2012/10/5 Carter Bullard <carter at qosient.com>:
>> Hey Markku,
>> OK, try this patch on your clients code, to see if it works.  Not sure what version
>> you're running, so I'm going to assume its argus-clients-3.0.7.1.  The patch's
>> exact line numbers may not be exact, so change the lines that look similar to
>> the new line, and all should be fine.
>> 
>> thoth:common carter$ p4 diff ...
>> ==== //depot/argus/clients/common/argus_grep.c#11 - /Volumes/Users/carter/argus/clients/common/argus_grep.c ====
>> 57c57
>> <          int options = REG_EXTENDED | REG_NOSUB;;
>> ---
>>>         int options = REG_EXTENDED | REG_ENHANCED | REG_NOSUB;
>> 
>> 
>> On my Mac OS X, I needed to add this option to the regular expression compile
>> code to get this to work as you suggested.  Not sure if that is a Mac thing or not.
>> Here is what I get now:
>> 
>> thoth:common carter$ ../bin/ra -r /tmp/*09.35* -e s:\\x41\\x70 -s +suser:32
>>                 StartTime        Dur      Flgs  Proto            SrcAddr  Sport   Dir            DstAddr  Dport  SrcPkts  DstPkts     SrcBytes     DstBytes State                 srcUdata
>> 2012/10/05.09:36:11.842712   0.099477  e           udp       192.168.0.33.mdns      ->        224.0.0.251.mdns          2        0          357            0   INT s[32]=.............Apt._tivo-videostre
>> 2012/10/05.09:37:50.429561   0.095191  e           udp       192.168.0.33.mdns      ->        224.0.0.251.mdns          2        0          357            0   INT s[32]=.............Apt._tivo-videostre
>> 2012/10/05.09:39:29.012663   0.101437  e           udp       192.168.0.33.mdns      ->        224.0.0.251.mdns          2        0          357            0   INT s[32]=.............Apt._tivo-videostre
>> 
>> Holler if that does it for you.
>> 
>> Carter
>> 
>> 
>> On Oct 5, 2012, at 8:55 AM, Carter Bullard <carter at qosient.com> wrote:
>> 
>>> Hey Markku,
>>> I'll look into it today.  Need to run under gdb() to see what actually makes it to recomp().  It maybe as dumb as double quotes vs single quotes, but I'll check it out before lunch.
>>> 
>>> Carter
>>> 
>>> On Oct 5, 2012, at 2:04 AM, Markku Parviainen <maketsi at gmail.com> wrote:
>>> 
>>>> 2012/10/4 Carter Bullard <carter at qosient.com>:
>>>>> ra* clients use the available regular expression library, and should support hexadecimal codes for matching now.
>>>>> So, there is nothing keeping ra* from doing hexidecimal code matching.  Because you have to use '\xNN' to specify the
>>>>> codes, when you provide it on the command line, you may need to escape the ' \ ' to get it past the shell.
>>>> 
>>>> The param was already quoted so that the shell (bash) would not interfere.
>>>> Anyway, for some reason it just doesn't work. I attached a sample (240
>>>> bytes) for your analysis.
>>>> 
>>>> # ra -r regex-anon.ra -M printer=encode32 -s suser:32
>>>>                              srcUdata
>>>> s[32]=333712F228948DABC9C0D199D1C3B00F
>>>> 
>>>> # ra -r regex-anon.ra -e '\x33'
>>>> # ra -r regex-anon.ra -e '\\x33'
>>>> # ra -r regex-anon.ra -e '33'
>>>> 
>>>> None of them produce anything (whereas only the first one should). Ideas?
>>>> 
>>>> I tried enabling debug output, but even -D10 does not produce any
>>>> lines about regex behaviour.
>>>> The system is CentOS 5.7 64bit, gcc v4.1.2, ra v3.0.7.1.
>>>> 
>>>> 
>>>> Btw. To confirm what the shell is delivering to the prog when \x is
>>>> single quoted:
>>>> 
>>>> # echo '\x33'
>>>> \x33
>>>> # echo \x33
>>>> x33
>>>> # perl -e 'print join(", ", @ARGV) ."\n"' -- -e '\x33'
>>>> -e, \x33
>>>> # echo '\\x33'
>>>> \\x33
>>>> <regex-anon.ra>
>>> 
>> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121010/5b1e57e7/attachment.bin>

More information about the argus mailing list