Argus with PF_RING DNA clusters

Fri Oct 5 10:55:26 EDT 2012

Hey Chris,
Sorry for the delay. 

Argus is already structured for multiple threads to eat packets, but there
is work that needs to be done to enable many cores to get into 
the packet processing business for a single interface.  We can do that
kind of effort on the list if that is of interest.   Not hard, but will need some
re-architecting for argus to support multiple cores at this level.

There are two strategies possible here, that I see.  

First involves routing packets to the appropriate cores for processing.  Argus
has been ported to multi-core chips, like Tilera, and they use this technique,
where you run a separate image of argus on each core.  One core reads
packets, and processes the headers to the point where it can decide which 
argus context should process the packets, and then queues the packets for
the independent argi to process.  This is pretty good, as it avoids any form
of locking, but does have the problem of asymmetric work loads.

The other is where you have multiple cores doing the packet header processing,
getting to the point where you have established the flow cache keys, and
calculated the hash values, then a single core does the look ups, and updates
the metrics.  This type of processing has some advantages for hardware
implementations, but does have some issues in software scheduling, as the
amount of header processing can be different for each packet.

If you're interested, and can help me with the PF_RING part, I can do
a lot of the multicore parts.

Carter

On Oct 3, 2012, at 1:44 PM, Chris Wakelin <c.d.wakelin at reading.ac.uk> wrote:

> Hi Carter,
> 
> Just thought I'd better confirm that commenting out the line has fixed
> the problem. I've still got problems with select() which means it maxes
> out a core (doesn't matter too much, I've got 16) which is waiting on
> the PF_RING guys to fix. I the longer term I'd like to run ARGUS
> multithreaded as that should enable each core to handle exactly 1/nth of
> the traffic and reduce memory copies.
> 
> Best Wishes,
> Chris
> 
> On 02/10/12 02:38, Carter Bullard wrote:
>> Hey Chris,
>> I've gone through the code and fixed the packet modifications, so we'll have
>> a fix in the next release.
>> 
>> You should be able to comment out this line in your current code base,
>> without generating any errors, so if you could do that and test your code,
>> that should work and be a good test of the bug.
>> 
>> Give it a try, and if you're still having problems, do send email to the list !!!
>> 
>> Carter
>> 
>> 
>> 
>> On Oct 1, 2012, at 1:03 PM, Chris Wakelin <c.d.wakelin at reading.ac.uk> wrote:
>> 
>>> I've just found in ArgusModeler.c
>>> 
>>>> ArgusCreateIPv6Flow (struct ArgusModelerStruct *model, struct ip6_hdr *ip)
>>>> {
>>> 
>>> ...
>>> 
>>>> 
>>>> #ifdef _LITTLE_ENDIAN
>>>>     ip->ip6_plen = ntohs(ip->ip6_plen);
>>>> #endif 
>>> 
>>> Is that modifying the packet or a copy? It would match my symptoms of
>>> truncated IPv6 packets :-/
>>> 
>>> Best Wishes,
>>> Chris
>>> 
>>> On 27/09/12 22:38, Chris Wakelin wrote:
>>>> Yes, 3.0.6.1 with a fix to ArgusSource.c to allow PF_RING DNA interfaces
>>>> and Intel virtual interfaces (ethX at Y):
>>>> 
>>>>> @@ -4182,7 +4187,7 @@
>>>>>   if (device == NULL)
>>>>>      return;
>>>>> 
>>>>> -   if (strstr(device->name, "dag") || strstr(device->name, "nap")) {
>>>>> +   if (strstr(device->name, "dag") || strstr(device->name, "nap") || strstr(device->name, "dna") || (strstr(device->name, "eth") && strstr(device->name, "@"))) {
>>>>>      for (i = 0; i < src->ArgusInterfaces; i++) {
>>>>>         if (src->ArgusInterface[i].ArgusPd && (pcap_fileno(src->ArgusInterface[i].ArgusPd) > 0))
>>>>>            bzero ((char *)&src->ArgusInterface[i].ifr, sizeof(ifr));
>>>> 
>>>> 
>>>> Best Wishes,
>>>> Chris
>>>> 
>>>> On 27/09/12 22:32, Carter Bullard wrote:
>>>>> Hmmmm, not thinking that we modify packets, but will look.  Your
>>>>> running argus-3.0.6+ ?  Thanks for the bug report !!!!
>>>>> 
>>>>> Carter
>>>>> 
>>>>> On Sep 27, 2012, at 5:00 PM, Chris Wakelin
>>>>> <c.d.wakelin at reading.ac.uk> wrote:
>>>>> 
>>>>>> Hi,
>>>>>> 
>>>>>> I've been having some more problems with ARGUS and PF_RING DNA
>>>>>> clusters. It turns out that with ARGUS running, the other
>>>>>> applications reading the same packets are seeing truncated IPv6
>>>>>> packets. As soon as ARGUS is stopped, things go back to normal.
>>>>>> 
>>>>>> E.g. tcpdump output:
>>>>>> 
>>>>>>> 18:45:36.174466600 IP6 truncated-ip6 - 5355 bytes
>>>>>>> missing!2001:630:53:26:5026:29a2:5863:dbaf.65226 >
>>>>>>> 2a00:1450:400c:c06::5d.443: Flags [.], seq 2651079285:2651084641,
>>>>>>> ack 1774208329, win 259, length 5356 18:45:36.174535600 IP6
>>>>>>> truncated-ip6 - 8160 bytes missing!2a00:1450:400c:c06::5d.443 >
>>>>>>> 2001:630:53:26:5026:29a2:5863:dbaf.65226: Flags [.], seq 1:8161,
>>>>>>> ack 1, win 272, options [nop,nop,sack 1 {0:1}], length 8160
>>>>>> 
>>>>>> The PF_RING clusters use a zero-copy mechanism which means that
>>>>>> each application is seeing the exact same chunk of memory. Is it
>>>>>> possible that ARGUS is modifying this, in particular for the IPv6
>>>>>> handling?
>>>>>> 
>>>>>> The "select() returning immediately" problem is still there, but
>>>>>> the PF_RING authors say they're working on a fix. They don't think
>>>>>> the IPv6 issue is related.
>>>>>> 
>>>>>> Best Wishes, Chris
>>>>>> 
>>>> 
>>> 
>>> 
>>> -- 
>>> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
>>> Christopher Wakelin,                           c.d.wakelin at reading.ac.uk
>>> IT Services Centre, The University of Reading,  Tel: +44 (0)118 378 2908
>>> Whiteknights, Reading, RG6 6AF, UK              Fax: +44 (0)118 975 3094
>>> 
> 
> 
> -- 
> --+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+---+-
> Christopher Wakelin,                           c.d.wakelin at reading.ac.uk
> IT Services Centre, The University of Reading,  Tel: +44 (0)118 378 8439
> Whiteknights, Reading, RG6 2AF, UK              Fax: +44 (0)118 975 3094
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20121005/d399af70/attachment.bin>