Label issues with last week's release
Carter Bullard
carter at qosient.com
Wed Sep 4 10:03:03 EDT 2013
Hey David,
I noticed that the compiler generated slightly inconsistent code
between filtering for the src and dst country codes. I got
this just now:
thoth:common carter$ ra -b - src co ZZ
(000) ldh dsr[19][4]
(001) jeq #0x0 jt 4 jf 2
(002) jeq #0x5a5a jt 3 jf 4
(003) ret #150
(004) ret #0
thoth:common carter$ ra -b - dst co ZZ
(000) ldh dsr[19][4]
(001) jeq #0x0 jt 5 jf 2
(002) ldh dsr[19][6]
(003) jeq #0x5a5a jt 4 jf 5
(004) ret #150
(005) ret #0
So the dst country code test, for some reason is testing the src
value. Not good. So here is a patch that clears it all up.
==== //depot/argus/clients/common/argus_code.c#81 - /Volumes/Users/carter/argus/clients/common/argus_code.c ====
2602c2602
< struct ablock *b0 = NULL, *b1 = NULL, *tmp;
---
> struct ablock *b0 = NULL, *b1 = NULL;
2611,2613d2610
< b0 = Argusgen_cmp(ARGUS_COCODE_INDEX, soffset, NFF_H, 0, Q_EQUAL);
< Argusgen_not(b0);
<
2626d2622
< tmp = Argusgen_cmp(ARGUS_COCODE_INDEX, soffset, NFF_H, val, Q_EQUAL);
2628c2624,2625
< Argusgen_or(tmp, b1);
---
> b0 = Argusgen_cmp(ARGUS_COCODE_INDEX, soffset, NFF_H, val, Q_EQUAL);
> Argusgen_or(b0, b1);
2632d2628
< tmp = Argusgen_cmp(ARGUS_COCODE_INDEX, soffset, NFF_H, val, Q_EQUAL);
2634c2630,2631
< Argusgen_and(tmp, b1);
---
> b0 = Argusgen_cmp(ARGUS_COCODE_INDEX, soffset, NFF_H, val, Q_EQUAL);
> Argusgen_and(b0, b1);
2638,2639d2634
< Argusgen_and(b0, b1);
<
I'll upload a new argus-clients tomorrow, which will have
all these issues fixed.
Carter
On Sep 3, 2013, at 9:50 PM, "David Edelman" <dedelman at iname.com> wrote:
> Carter,
>
> In yet another demonstration of Murphy being the strongest force in nature, I followed all of the instructions and everything worked just fine. NB: I took great pains to ensure that I did not make install to keep the modified version out of the wild. Everything worked exactly as advertised. I then checked with the original client that caused the problem and now it works.
>
> I’ll keep on watching to see what happens going forward, but at least I know how to run the filter stuff through the debugger. BTW for the cocode stuff ArgusFilterOrig line 391 is a good breakpoint.
>
> --Dave
>
>
> From: Carter Bullard [mailto:carter at qosient.com]
> Sent: Tuesday, September 03, 2013 7:48 PM
> To: David Edelman
> Cc: 'Argus'
> Subject: Re: [ARGUS] Label issues with last week's release
>
> Hey Dave,
> There is a problem getting debug info from the filter code, as the compiler is a forked process.
> In ./common/argus_code.c, you will need to comment out line 102, recompile, and now
> -D9 with the -b option will show most of the compiler operations. The best way to
> debug the actual filtering, is to run the modified clients under gdb(), and break in
> ArgusFilterRecord().
>
> You will need to remember to correct the modification to argus_code.c, when you're
> done, as all clients will dump core when processing filters, with the ARGUS_FORK
> commented out.
>
> Carter
>
> On Sep 3, 2013, at 7:34 PM, "David Edelman" <dedelman at iname.com> wrote:
>
>
> Carter,
>
> For this test, Argus is running on a 64 bit system and the clients are running on a 64 bit system. The dump of the filters match yours. When I run the client on the same system as Argus, there is no difference. If there a debug value that I can set to check the action of the filtering?
>
> --Dave
>
>
>
> ra -S rodnel-new:561 -s sco dco - dst co ZZ
> [No output]
>
> ra -S rodnel-new:561 -s sco dco - src co ZZ
> [No output]
>
> ra -S rodnel-new:561 -s sco dco - co ZZ
> [No output]
>
> ra -S rodnel-new:561 -s sco dco - not dst co ZZ
> sCo dCo
> ZZ US
> ZZ US
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
>
> ra -S rodnel-new:561 -s sco dco - not src co ZZ
> sCo dCo
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
>
> ra -S rodnel-new:561 -s sco dco - not co ZZ
> sCo dCo
> ZZ ZZ
> ZZ ZZ
> ZZ GB
> ZZ ZZ
> ZZ ZZ
> ZZ ZZ
>
>
> From: Carter Bullard [mailto:carter at qosient.com]
> Sent: Tuesday, September 03, 2013 1:09 PM
> To: David Edelman
> Cc: Argus (argus-info at lists.andrew.cmu.edu)
> Subject: Re: [ARGUS] Label issues with last week's release
>
> Hey Dave,
> This is what I get for the two filters…
>
> thoth:common carter$ ra -r * -b - not src co ZZ
> (000) ldh dsr[19][4]
> (001) jeq #0x0 jt 4 jf 2
> (002) jeq #0x5a5a jt 3 jf 4
> (003) ret #0
> (004) ret #150
>
> thoth:common carter$ ra -r * -b - src co ZZ
> (000) ldh dsr[19][4]
> (001) jeq #0x0 jt 4 jf 2
> (002) jeq #0x5a5a jt 3 jf 4
> (003) ret #150
> (004) ret #0
>
> So this looks right, on Mac OS X 10.8.4 with gcc i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00). Load the 16-bit value 4 bytes into the COCODE DSR, and if not zero, compare with "ZZ".
>
> And, the filters works fine on this and all the 64-bit Linux's I have.
> Are you on a 32-bit machine?
>
> Carter
>
>
>
>
> On Sep 3, 2013, at 12:30 PM, David Edelman <dedelman at iname.com> wrote:
>
>
>
> For flow records with valid cocode values including a lot of ZZs I get
>
> ra -r * -b - src co ZZ
>
> (000) ldh dsr[19][4]
> (001) jeq #0x0 jt 4. jf 2
> (002) jeq #0x5a5a jt 3 jf 4
> (003) ret #0
> (004) ret #150
>
>
> If I filter on not src co ZZ I get all the records including ZZ
>
>
> Dave Edelman
>
>
> On Sep 3, 2013, at 12:09, David Edelman <dedelman at iname.com> wrote:
>
> I'm not sure if this is related. If not I'll start a new thread.
>
> The sco and dco values are not making it into the cocode DSR correctly. I tried to kill the DSR and the clients still baulk.
>
> The ralabel config seems to play a role for some of the problems
>
> Filtering on co also seems to be a problem and the generated BPF code looks odd. This may call for the use of offsetof in 64 bit environments.
>
> More when I can send samples.
>
>
> Dave Edelman
>
>
> On Sep 3, 2013, at 8:13, Carter Bullard <carter at qosient.com> wrote:
>
> Hey Craig,
> So I'm back and its time to fix this label problem, as others are starting to report it.
> It may take me a little while to figure out where we are on this bug. If you have any
> new observations, sending to the list would be great...
>
> Radium is stable for you ( > 4 days ?).
>
> Please send a separate email around your rastream() issue. Better to have separate threads for each problem.
>
> Carter
>
> On Aug 26, 2013, at 9:52 PM, Craig Merchant <cmerchant at responsys.com> wrote:
>
>
>
> Carter,
>
> From previous emails sent to the list I was under the impression that you had discovered some differences between how ralabel handles labels vs how radium does. Did those differences get resolved in the latest release? I’m still seeing some issues with duplicate labels…
>
> For example… For IP 10.10.10.10, my label file (in order) looks like:
>
> 10.10.10.0/24 internal,DC2,I5-Apache-UI
> 10.10.10.10 apacheui,ri5
>
> Yet my Argus events look like:
>
> "1377565200.000,e *,tcp,63.166.75.3,0,->,10.10.10.10,9051,1599,685729,SRPA_SPA,56,300.000,478334,207395,2.857,2.467,0.443,US,ZZ,daddr=internal#DC2#I5-Apache-UI#apacheui#ri5#apacheui#ri5#apacheui#ri5#apacheui#ri5"
>
> Not sure why the host-specific label is repeated 4 times, but that’s consistent with other events I’m seeing.
>
> In other (good) news, it would seem like the radium crashing issue has been fixed in the latest release. I know we are a corner case as far as radium stability goes, but it’s been running for four days without stopping and that’s a positive change.
>
> We may be experiencing some issues with rastream though… I sent you our rastream search and the shell scripts it calls at the end of each time interval. It’s been working fine for a few days, but it seems to have lost track of the files it’s processing, so no new events are coming in and old files aren’t being deleted. I’ll dive deeper into that that this week and see if I can narrow down the problem.
>
> Thanksa again for all the fixes!
>
> C
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130904/cbb99c76/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130904/cbb99c76/attachment.bin>
More information about the argus
mailing list