Two questions
Carter Bullard
carter at qosient.com
Mon Feb 4 22:14:24 EST 2013
Hey Craig,
I found the label memory leak in radium. Its a simple patch, which I've included below.
Not sure if you're a patch person, but add lines that start with a " + ", and modify the
lines that begin with a " ! ", to look like the second set of lines that start with a " ! ".
The problem was that we needed to add the label to a copy of the argus record, not make
a copy of the argus record that has the label.
We have enough fixes for me to put up a new argus-clients-3.0.7.x, but I was waiting
for the netflow v9 support for IPv6 to be verified, and to have some more progress on
fixing rapolicy().
I'll try to get a new argus-clients-3.0.7.5, late tomorrow. If not then, it will probably be on Friday.
Carter
==== //depot/argus/clients/clients/radium.c#8 - /Volumes/Users/carter/argus/clients/clients/radium.c ====
***************
*** 401,406 ****
--- 401,408 ----
void
RaProcessRecord (struct ArgusParserStruct *parser, struct ArgusRecordStruct *argus)
{
+ struct ArgusRecordStruct *ns;
+
switch (argus->hdr.type & 0xF0) {
case ARGUS_MAR: {
struct ArgusRecord *rec = (struct ArgusRecord *)argus->dsrs[0];
***************
*** 525,535 ****
}
}
! if (parser->ArgusLabeler != NULL)
! ArgusLabelRecord(parser, argus);
! ArgusPushBackList(parser->ArgusOutput->ArgusOutputList,
! (struct ArgusListRecord *)ArgusCopyRecordStruct(argus), ARGUS_LOCK);
#if defined(ARGUS_THREADS)
pthread_cond_signal(&parser->ArgusOutput->ArgusOutputList->cond);
--- 527,538 ----
}
}
! if ((ns = ArgusCopyRecordStruct(argus)) != NULL) {
! if (parser->ArgusLabeler != NULL)
! ArgusLabelRecord(parser, ns);
! ArgusPushBackList(parser->ArgusOutput->ArgusOutputList, (struct ArgusListRecord *) ns, ARGUS_LOCK);
! }
#if defined(ARGUS_THREADS)
pthread_cond_signal(&parser->ArgusOutput->ArgusOutputList->cond);
On Feb 1, 2013, at 3:58 PM, Craig Merchant <cmerchant at responsys.com> wrote:
> Great explanation, Carter. Much appreciated.
>
> I’ve been using ralabel –w - | ra –r - > outputfile to get my flows into a non-binary format, so I’m guessing that I’m not taking advantage of any record aggregation and flow corrections.
>
> From my (limited) experience with the aggregation tools, racluster seems like it’s meant to aggregate records from a file.
>
> One thing I haven’t figured out is when it’s better to use rasplit vs .rabins vs. rastream. All of them support splitting up the Argus stream by time. If I want to split my stream by 5 minute increments, is one client better than the other for dealing with a flow that stays open for 15 minutes?
>
> Thx.
>
> Craig
>
>
> From: Carter Bullard [mailto:carter at qosient.com]
> Sent: Friday, February 01, 2013 6:19 AM
> To: Craig Merchant
> Cc: Argus (argus-info at lists.andrew.cmu.edu)
> Subject: Re: [ARGUS] Two questions
>
> Hey Craig,
> The address based approach for specifying direction is not a bad strategy, L2 is more reliable than L3 identifiers.
> The problem with using L4 identifiers (port numbers) to make direction decisions, is that it you can get fooled really easy, and tricked even worse.
>
> If we can't make a definitive statement as to the originator, the record we generate reflects the behavior on the wire, which has some truth to it, at least.
>
> All programs like racluster (all the aggregators) know how to deal with status records that are uncertain, and correct the direction when it can.
> Such as a persistent but idle TCP connection. When you aggregate over the complete time of the connection, racluster() will get the direction correctly.
>
> So, when there is ambiguity, the direction reflects the direction of the first packet seen, that is very important information. Your analytic should be aware that the ' ? ' is an important indicator, and adjust for it. If I provide port based direction overrides, I'll also have to indicate in the record that the direction was unknown, and it was changed from wire behavior. We can do that, but it does start to get even more complicated.
>
> To filter the flows with ' ? ', use " tcp and (not syn or not synack)".
>
> Carter
>
>
> On Jan 31, 2013, at 9:25 PM, Craig Merchant <cmerchant at responsys.com> wrote:
>
>
> Thanks, Carter. Looks like your example had an additional \ before the ). This is what worked for me:
>
> [root at ids-manager01-dc1 ~]# ralabel -S 192.168.1.40:561 -s +label:200 -f /usr/local/argus/ralabel.conf -w - | ra -r - -s +label:200 –M -label="(?\!whitelisted)"
>
> We turned Argus on a couple days ago and setup a few security searches, such as looking for unusual outbound SSH traffic. There were several instances of high port to high port connections where Argus was confused about the direction. It would definitely simplify that kind of analysis if Argus could be told that a list of IPs, protocols, and ports should always be the destination of a flow.
>
> Thanks for being responsive to the community!
>
> Craig
>
> From: Carter Bullard [mailto:carter at qosient.com]
> Sent: Thursday, January 31, 2013 12:49 PM
> To: Craig Merchant
> Cc: Argus (argus-info at lists.andrew.cmu.edu)
> Subject: Re: [ARGUS] Two questions
>
> Hey Craig,
> The bash shell is getting in the way, and is trying to convert your " !whitelisted " into a history
> command. " ! " is a special character, so escape the ! using a backslash. This should work:
>
> label="(?\!whitelisted\)"
>
> We have in argus-clients-3.0.7.4, which is on the development server, the ability for you
> to suggest which should be the source and destination when it can't be determined from
> the traffic. You specify local addresses, and then you can say you want the local on the
> left of right, when there is a " ? ". Works pretty good, but its all address based, not service
> based. We can add that if that is something you think would be useful.
>
> Carter
>
> On Jan 31, 2013, at 2:23 PM, Craig Merchant <cmerchant at responsys.com> wrote:
>
>
>
> Is it possible to do a negative regex match on flow labels? I tried doing a negative lookahead –M label=”(?!whitelisted)”, but got an error:
>
> -bash: !whitelisted: event not found
>
> I can do a reverse match using egrep if I have to. But negative label matching would be a nice feature.
>
> Second question… When an ra* client connects to argusd or radium and sees a flow that was established before it connected, what is the logic that the client uses to guess the direction of the flow? And is there any way to influence it or control it? It would be great if Argus could use something like an iana label file or NMAP scan so that known open ports on a server would be given precedence as the destination.
>
> Thanks Carter and the rest of the community for all the help!
>
> Craig
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130204/4eaea732/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130204/4eaea732/attachment.bin>
More information about the argus
mailing list