pcr and
Frank
argus-mailinglist-1524134246 at f-block.org
Fri Apr 20 11:19:59 EDT 2018
Hi Carter,
thx for your quick answer.
1. That did the trick, thx!
2. Yes, i've set ARGUS_FLOW_KEY="CLASSIC_5_TUPLE"
The reasons for my big ARGUS_FLOW_STATUS_INTERVAL value:
- i don't need instant live updates about active flows
- i don't want those updates filling up my log files ( we have many long
connections going on, and when there is a status update entry for each
of them every second, it gets unnecessarily big)
- As far as i understand, flow information are written as soon as the
flow is "done" (timeout reached, connection closed,...), so this value
shouldn't affect those, right?
Ok, what i want in the end is a list of saddr daddr pairs that have long
connections with each other, based on the mean field.
I have for example those records:
rafilteraddr -r argus.out -f /tmp/filter -p0 -s -state -s -flgs -s -dir
-s +mean +seq
StartTime Proto SrcAddr Sport DstAddr Dport
TotPkts TotBytes Mean Seq
21:37:19 tcp 10.10.10.10.42972
10.20.20.20.https 32536 26592499 1790 30487007
22:07:19 tcp 10.10.10.10.42972
10.20.20.20.https 4961 3942584 1795 30487007
22:37:21 tcp 10.10.10.10.42972
10.20.20.20.https 1125 272903 1791 30487007
23:07:21 tcp 10.10.10.10.42972
10.20.20.20.https 494 138085 780 30487007
[...]
06:19:14 tcp 10.10.10.10.23391
10.20.20.20.https 730 72320 1793 31292145
06:49:16 tcp 10.10.10.10.23391
10.20.20.20.https 647 57559 1796 31292145
07:19:22 tcp 10.10.10.10.23391
10.20.20.20.https 701 70449 1793 31292145
07:49:25 tcp 10.10.10.10.23391
10.20.20.20.https 685 60024 1794 31292145
08:19:26 tcp 10.10.10.10.23391
10.20.20.20.https 727 72165 1799 31292145
08:49:35 tcp 10.10.10.10.23391
10.20.20.20.https 672 59209 1793 31292145
There are two connections (the first lines are from the day before )
As i can't aggregate on the sequence number, i use the following command
as my first aggregation step:
rafilteraddr -r argus.out -f /tmp/filter -w - | racluster -m proto saddr
sport daddr dport -M norep -p0 -s -state -s -flgs -s -dir -s +mean +seq
+trans +sum
StartTime Proto SrcAddr Sport DstAddr Dport
TotPkts TotBytes Mean Seq Trans Sum
06:19:14 tcp 10.10.10.10.23391
10.20.20.20.https 8231 776687 0 31292145
0 0
21:37:19 tcp 10.10.10.10.42972
10.20.20.20.https 39116 30946071 0 30487007
0 0
I use the '-M norep' option, as without it, i get this, which gives me
the mean duration time over all records, but not for connections:
StartTime Proto SrcAddr Sport DstAddr Dport
TotPkts TotBytes Mean Seq Trans Sum
06:19:14 tcp 10.10.10.10.23391
10.20.20.20.https 8231 776687 1789 31292145
12 21468
21:37:19 tcp 10.10.10.10.42972
10.20.20.20.https 39116 30946071 1539 30487007
4 6156
And in my last step, i aggregate with saddr daddr:
rafilteraddr -r argus.out -f /tmp/filter -w - | racluster -m proto saddr
sport daddr dport -M norep -w - | racluster -m saddr daddr -p0 -s
-state -s -flgs -s -dir -s +mean +seq +trans +sum
StartTime Proto SrcAddr Sport DstAddr Dport
TotPkts TotBytes Mean Seq Trans Sum
21:37:19 tcp 10.10.10.10
10.20.20.20.https 47347 31722758 13877 31292145
2 27753
For this scenario, it works as expected: there were 2 connections
between those two ip addresses, with a total duration of 27753 and mean
13877.
But as i said, the remaining problem are connections that use the same
5-tuple for multiple connections/flows as with ntp.
Did i understand/do somehting wrong here?
3. Ah, okay, understood.
Thx and Regards,
Frank
On 04/20/2018 03:38 PM, Carter Bullard wrote:
> Hey Frank,
> The PCR requires that you have ARGUS_GENERATE_APPBYTE_METRIC set in argus. The PCR is based on the values of sappbytes and dappbytes, so as a sanity check, print these fields using ra to see if the metric is being generated.
>
> So not sure what you’re wanting to achieve in 2. The 5-tuple aggregation is the default, as that is the “basic” way of forming flows. Its always better if you include the command line that you are using, so we can see whats up.
>
> But, why use such a large ARGUS_FLOW_STATUS_INTERVAL, I use either 5 or 1, and never go much higher. This is the definition of your “quantum" of observation. With a 3 hour interval, man, do you have to wait a long time before anything comes out if your monitoring live traffic. Biggest problem is that your going to see a lot of port reuse in 3 hours, so flows will collide, and you’ll get large flow duration times. If you use default aggregation, then the mean, stddev, max and min are calculated using the ‘due’ field of each record. So it looks like it is working.
>
> Use a small ARGUS_FLOW_STATUS_INTERVAL, and then use aggregation to build up the long lived flows.
>
> So, #3 is a ‘gotcha’ of sorts. “tcp” and “udp” are really shortcut aliases, tcp means “ip proto tcp”, so you're generating filter errors by accident ??? Because tcp and udp and icmp are used so often, they are aliased, seems that all the other protocols (i.e udplite) need the ip proto.
>
> So this works…
>
> ra -b - not udp
> ra -b - not \(ip proto 17\)
>
> So quoting them makes them a string, rather than the keyword, and we then lookup them up, rather than assign the value. Sorry for that inconvenience.
>
> Carter
>
>> On Apr 20, 2018, at 6:39 AM, Frank <argus-mailinglist-1524134246 at f-block.org> wrote:
>>
>> Hi,
>>
>> i'm new to the list and to argus, so i hope i don't repeat already asked
>> questions.
>> First of all: i really like the workflow of argus and its functionality.
>> But i encountered some obstacles that however might also be caused by my
>> misunderstanding.
>>
>> Argus server and client version: 3.0.8.2
>>
>>
>> 1. Is it possible that the PCR functionality is broken? I always get a
>> PCR of -0
>>
>> ra -r argus.out -s sbytes dbytes pcr | head
>> SrcBytes DstBytes PCRatio
>> 83 126 -0.000000
>> 1644 4213 -0.000000
>> 83 126 -0.000000
>> 194 0 -0.000000
>> 194 0 -0.000000
>> 73 89 -0.000000
>> 73 101 -0.000000
>> 74 60 -0.000000
>> 3351 4025 -0.000000
>>
>>
>> 2. Did i see right that it is not possible to aggregate on the argus
>> sequence number, or did i overlook something? The main reason i'd
>> want/need this is for an accurate mean duration time. The best i was
>> able to accomplish was an aggregation (via racluster) over proto src/dst
>> addr/port, but this does e.g. not work with ntp queries.
>> I've set ARGUS_FLOW_STATUS_INTERVAL=1800 , and when i look for long
>> connections and aggregate with '-m proto saddr sport daddr dport' , i
>> get a maximum mean time of 1800, as each status message seems to be
>> treated as a separate flow (at least it looks that way to me). When i
>> also use -M norep, it works for all connections where the selected
>> features are unique for the connection, but, as mentioned before, does
>> not work with ntp connections, or for chatty hosts that use the same
>> source port for a destination host twice or more.
>>
>>
>> 3. Another issue that i encounter is a weird filtering behaviour.
>> For example:
>>
>> ra -r argus.out -- 'not (ip proto udp)'
>> ra[26153]: 1524219408.776398 filter syntax error: 'not (ip proto udp)'
>>
>> ra -r argus.out -- 'not (ip proto tcp)'
>> ra[26219]: 12:19:04.157946 filter syntax error: 'not (ip proto tcp)'
>>
>> ra -r argus.out -- 'not (ip proto udp or tcp)'
>> ra[26233]: 12:19:31.355346 filter syntax error: 'not (ip proto udp or ...'
>>
>> But this works:
>> ra -r argus.out -- 'not (ip proto "udp")'
>> ra -r argus.out -- 'not (ip proto 17 or tcp)'
>> ra -r argus.out -- 'not (ip proto "udp" or tcp)'
>> ra -r argus.out -- 'not udp and not tcp'
>>
>>
>> Thanks in advance and regards,
>> Frank
>>
>>
>>
More information about the argus
mailing list