Problems with racluster
Rafael Barbosa
rrbarbosa at gmail.com
Tue Sep 4 05:29:06 EDT 2012
Hi Carter,
The racluster version you sent to this list seems to cause another bug.
These are the outputs of racluster v3.0.7.1 (correct) and the patched
version you provided:
$> ~/workspace/argus-clients-3.0.7.1/bin/racluster -r port0.argus -f
racluster.conf | grep 13:35:04.950149
13:35:04.950149 e g tcp 172.31.1.102.40767 <?>
172.31.1.123.x11 43 6494 CON
$> ~/workspace/argus-clients-3.0.7.1-patch/bin/racluster -r port0.argus -f
racluster.conf | grep 13:35:04.950149
13:35:04.950149 e g tcp 172.31.1.96.* <?>
172.31.1.96.* 43 6494 CON
I uploaded port0.argus to the ftp.
Best regards,
Rafael Barbosa
http://www.ewi.utwente.nl/~barbosarr/
On Mon, Sep 3, 2012 at 9:38 PM, Carter Bullard <carter at qosient.com> wrote:
> Hy Rafael,
> For the others on the list, your racluster.conf file has an idle timeout
> of 300s
> applied to any flow. The problem that you are seeing is a bug, and is
> caused
> by the timestamp in the management records. The man records are advancing
> the
> ArgusGlobalTime, (it should no do this), which is the master clock for the
> aggregator. When racluster() gets the close MAR record in the file created
> from your f1.pcap file, the timestamp, which is the time the file was
> created, not
> the timestamps of the packets, this causes racluster() to idle out your
> flow cache.
>
> Try this version of your command:
>
> racluster -r f1.argus f2.argus -f ~/config/racluster.conf - not man
>
>
> I will fix this bug later today / tomorrow.
>
> Sorry for the inconvenience,
>
> Carter
>
> On Sep 3, 2012, at 10:13 AM, Rafael Barbosa <rrbarbosa at gmail.com> wrote:
>
> Just uploaded the two pcap files and the racluster.conf to the ftp server
> with the name 'racluster.tar.gz'.
>
> Rafael Barbosa
> http://www.ewi.utwente.nl/~barbosarr/
>
>
>
> On Mon, Sep 3, 2012 at 3:33 PM, Carter Bullard <carter at qosient.com> wrote:
>
>> Hey Rafael,
>> The problem relates to your racluster.conf file. For some reason its
>> timing out
>> the flow in f1.argus, before the f2.argus file shows up ?
>>
>> Send both files so I can debug this. Also send the racluster.conf file,
>> if possible.
>>
>> Carter
>>
>> On Sep 3, 2012, at 4:46 AM, Rafael Barbosa <rrbarbosa at gmail.com> wrote:
>>
>> Hi Carter,
>>
>> The issue with the direction seems to be solved using clients 3.0.7.1:
>> the single duplicated ACK from the previous connection is aggregated in the
>> second connection, and the direction is not reversed.
>>
>> $> ra -r f1.argus f2.argus
>> 21:09:30.970830 e * tcp 172.31.1.102.61722 ->
>> 172.31.1.100.10500 22 6772 FIN
>> 21:09:30.971095 e tcp 172.31.1.100.10500 ?>
>> 172.31.1.102.61722 1 66 CON
>> 21:11:52.493919 e * tcp 172.31.1.102.61722 ->
>> 172.31.1.100.10500 23 6838 FIN
>>
>> $> racluster -r f1.argus f2.argus -f ~/config/racluster.conf
>> StartTime Flgs Proto SrcAddr Sport Dir
>> DstAddr Dport TotPkts TotBytes State
>> 21:09:30.970830 e * tcp 172.31.1.102.61722 ->
>> 172.31.1.100.10500 22 6772 FIN
>> 21:11:52.493919 e * tcp 172.31.1.102.61722 ->
>> 172.31.1.100.10500 24 6904 FIN
>>
>> However, if the duplicated packet is aggregated in the second record, why
>> isn't the 'start time' also set to the time of the duplicated packet? In
>> any case, would it be possible to aggregate the dup packet in the first
>> record?
>>
>> Best regards,
>> Rafael Barbosa
>> http://www.ewi.utwente.nl/~barbosarr/
>>
>>
>>
>> On Sat, Sep 1, 2012 at 6:09 PM, Carter Bullard <carter at qosient.com>wrote:
>>
>>> Hey Rafael,
>>> Give this copy a run to see if it doesn't do the right thing for you.
>>> If so, I'll push the changes to all the aggregators.
>>> Are you using 3.0.6.2 clients or 3.0.7.1 ?
>>> Thanks !!!!!!
>>>
>>> Carter
>>>
>>>
>>>
>>> On Aug 30, 2012, at 8:29 AM, Carter Bullard <carter at qosient.com> wrote:
>>>
>>> Hey Rafael,
>>> Your situation should work fine, if not for the racluster() bug.
>>>
>>> The only problem is that racluster() gives precedence to the first
>>> record, not the record with the definitive data, so the fix should be
>>> simple, I hope. I'll try to fix this today.
>>>
>>> Regarding the model, I was just providing an example racluster.conf.
>>> The status and idle timers are the important part.
>>>
>>> Carter
>>>
>>>
>>> On Aug 30, 2012, at 3:55 AM, Rafael Barbosa <rrbarbosa at gmail.com> wrote:
>>>
>>> Hi Carter,
>>>
>>> I do not care much about the first record, as long as the second record
>>> keeps its correct direction. As you said, the second record contains the
>>> 3-way handshake necessary to decide the flow direction.
>>>
>>> I am not sure if this is important, but my original example is a little
>>> more complicated: I have another pcap file with the rest of the TCP
>>> connection from which the first packet (from the first record in f2.argus)
>>> belongs. The packet that generates the first record f2.argus is a duplicate
>>> for the ACK that completes the teardown of this TCP connection. To make
>>> sure I am clear:
>>>
>>> $> ra -r f1.argus
>>> 21:09:30.970830 e * tcp 172.31.1.102.61722 ->
>>> 172.31.1.100.10500 22 6772 FIN
>>> $> ra -r f2.argus
>>> 21:09:30.971095 e tcp 172.31.1.100.10500 ?>
>>> 172.31.1.102.61722 1 66 CON
>>> 21:11:52.493919 e * tcp 172.31.1.102.61722 ->
>>> 172.31.1.100.10500 23 6838 FIN
>>> $> racluster -r f1.argus f2.argus -f ~/config/racluster.conf
>>> 21:09:30.970830 e * tcp 172.31.1.102.61722 ->
>>> 172.31.1.100.10500 22 6772 FIN
>>> 21:09:30.971095 e * tcp 172.31.1.100.10500 ->
>>> 172.31.1.102.61722 24 6904 FIN
>>>
>>> I just assumed the bug is in the second file. But if you think is
>>> necessary I can provide you with the first argus dump as well.
>>>
>>> Regarding my racluster.conf, I do not really care about real-time, as I
>>> am using argus to post-process pcap files (I do not have access to the
>>> network, just the dumps). But I have a question, why to you suggest to use
>>> the 5-tuple as a "model" in racluster.conf? Isn't this the default behavior?
>>>
>>> And as usual, thanks for the fast reply.
>>>
>>> Best regards,
>>> Rafael Barbosa
>>> http://www.ewi.utwente.nl/~barbosarr/
>>>
>>>
>>>
>>> On Wed, Aug 29, 2012 at 8:19 PM, Carter Bullard <carter at qosient.com>wrote:
>>>
>>>> Hey Rafael,
>>>> Yes, I'll have to fix this bug were the second record is erroneously
>>>> reversed. The fix, however,
>>>> will end up reversing the first record. The second record saw the
>>>> connect setup, so it knows
>>>> the correct direction. The first flow, is just the one packet, without
>>>> any detail, so the direction
>>>> is unknown. When you try to merge the two flows together, the bug is
>>>> that it only looks at the first
>>>> record for the direction, rather than evaluating both, and choosing the
>>>> one that is a better choice.
>>>>
>>>> Is that cool with your line of thinking?
>>>>
>>>> If you're planning on running argus with 300s status intervals, that
>>>> will eliminate any real-time
>>>> possibilities. But if you're thinking about letting argus generate
>>>> shorter lived status records, then
>>>> run racluster() with a racluster.conf file, where you set a status
>>>> timer for all flows at 300s:
>>>>
>>>> filter="" model="saddr daddr proto sport dport" status=300 idle=300
>>>>
>>>> Hopefully this helps !!!!
>>>>
>>>> Carter
>>>>
>>>> On Aug 29, 2012, at 9:22 AM, Rafael Barbosa <rrbarbosa at gmail.com>
>>>> wrote:
>>>>
>>>> Hi all,
>>>>
>>>> I am having some problems with a weird aggregation decision by
>>>> racluster, which inverts the server/client roles (i.e., the direction
>>>> field). This what happens:
>>>>
>>>> $> argus -r f2 -w f2.argus
>>>> $> ra -r f2.argus
>>>> 21:09:30.971095 e tcp 172.31.1.100.10500 ?>
>>>> 172.31.1.102.61722 1 66 CON
>>>> 21:11:52.493919 e * tcp 172.31.1.102.61722 ->
>>>> 172.31.1.100.10500 23 6838 FIN
>>>> $> racluster -r f2.argus -f racluster.conf
>>>> 21:09:30.971095 e * tcp 172.31.1.100.10500 ->
>>>> 172.31.1.102.61722 24 6904 FIN
>>>> $> cat racluster.conf
>>>> filter="" status=0 idle=300
>>>>
>>>> The pcap from which I generated f2.argus is attached to this message.
>>>> The 'f2' pcap file is rather peculiar with duplicated frames and is a
>>>> fragment of a larger capture where TCP ports are re-used in consecutive
>>>> connections, but it is not a "toy" file, it was captured in a real network
>>>> environment.
>>>>
>>>> My goal is to generate flow records with a 300s timeouts, and label
>>>> hosts as servers and clients. Any thoughts on why this inversion happens,
>>>> and how I can work around it?
>>>>
>>>> Thanks for the help,
>>>> Rafael Barbosa
>>>> http://www.ewi.utwente.nl/~barbosarr/
>>>>
>>>> <f2>
>>>>
>>>>
>>>>
>>>
>>>
>>>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120904/625cef3d/attachment.html>
More information about the argus
mailing list