hole in the argus archive on theorygroup.org

[Peter Van Epp]

> 
> Peter,
> 
> Here is the configuration of 3 of our probe hosts at CMU.
> Not much tweaking to get 400 Mb/sec out of them with
> the commercial code of Argus.  We really haven't tried to
> push the limit on them. We have some ideas on how to make
> them fly, but remember, we're just interested in not dropping
> anything from the core auditing effort whish is about 400 Mb/sec max.

	At the end of the day thats all I have time for as well :-) The bottom
line is likely to be if I can buy something that will just work without me
doing all the interesting poking, money is going to be spent because we can
get capital money much more easily than staff resources. Thats why I have been
doing an eval on a commercial IDS system, the boss was hoping for a capital 
intensive silver bullet (but I don't think we found one).

> 
> So far we're not dropping anything and we have about 10-15%
> of cpu left without much tweaking done.  One thing to note,
> Argus and really fly when it doesn't have to write to disk.

	Thats good to know. I'm currently going to disk on the sensor machine
(and then copying to an analysis machine) looks like I need to modify that
idea.

> I.e. interrupts become a problem on any system with disk writes.
> What we do is just have the probes run Argus without writing to
> disk and collect data from them via the network (port 561/tcp).
> Therefore, we don't have to load the system with disk writes.
> We're pretty much collecting the kitchen sink when it comes to
> data, so on a 400 Mb/sec network, Argus outputs about
> 1.5 Mb/sec of audit data.  Not much when you consider the
> amount of data you're auditing.  We could reduce this to
> 1 Mb/sec easily, and then some.
> 
> 	...cd
> 
> p.s. On the "Turbo" zero copy packet capture code in Linux,
>      I understand that it only helps when you're doing
>      packet filtering to reduce coping in the kernel.

	Ah! I haven't looked at it at all, I just heard "zero copy" and assumed
they were avoiding the kernel/user space copy by moving page table entries (I
know of such a project for OpenBSD, although I don't think its out yet).

> P.s.s. We'd like to play with a NetBSD box, since I hear the
>      network code is very fast.  But it's not a priority since
>      our probes are keeping up.  Time to concentrate on
>      data analysis.
> 

	Yes, I know that feeling well :-) There are reported "networking"
improvements in FreeBSD 4.5 that I'd like to poke at, but time never seems
to come available for doing it :-). Our current wireless authentication solution
(Vernier) which is FreeBSD based is predicting a 3 fold increase in throughput
with a new release. I'm guessing some (or perhaps a lot) of that throughput
increase will be moving to FreeBSD 4.5 (or that they have lots of smart folks
internally which is also probably true).
	I'm getting ready to buy a pair of SysConnect SK-9843 GigE cards (our 
beowolf cluster has one, but not two) and convince Martin that benchmarking 
them for me would be a good thing :-).  Martin looked over the available GigE 
cards and decided the SysConnect ones were best performing / best supported 
(apparantly the Intel driver is not entirely open source so Intel may need to 
make changes when the kernel changes and the SysConnect is open source, and 
also the most expensive which is why he only has one :-)).

> probe 1:
>   - Dell Precision 530
>   - Pentium Xeon 1.7GHz, 1gb ram, Linux 2.4
>   - 3com 3c985 gige card
>   - Syskonnect ?? single gige card (don't know the model) 
> 
> probe 2:
>   - Dell Optiplex GX240
>   - Pentium 3 1.8GHz, 650mb ram, linux 2.4
>   - 2x Intel gige cards (model ??) - low # of interrupts
> 
> probe 3:
>   - Dell optiplex gx100
>   - Pentium 3 900MHz, 256mb ram, linux 2.4
>   - Intel gige card (model ??) - low # of interrupts
>

	Any sense that one of these works better than the others (I'm guessing
#1 is probably the star :-))?

Peter Van Epp / Operations and Technical Support 
Simon Fraser University, Burnaby, B.C. Canada