Argus 2.0 features

Wed Jul 5 16:51:52 EDT 2000

It all depends on what your doing.  If you've written
clients that pre-process the data or are pulling LOTS of data
from multiple interfaces, you want to consider a quick machine.
Note, that's only in extreme cases when your talking about >1G
day logs, or LOTS of cpu time pre-processing.  I only have
experience with multiple processors on Solaris and it worked well.
Other OS's should work as well.

It also depends on the need to access  historical data.  If your
writing clients that throw data into a db and you need to go
back to the raw Argus files to extract more information frequently,
get some big disks and consider an archiving device that you don't
have to reload too frequently.

I'd consider cheap hosts for collection devices, then worry about
processing requirements later if you need it.  If your not doing
anything huge, don't worry about it.

	...Chas

> -----Original Message-----
> From: owner-argus at lists.andrew.cmu.edu
> [mailto:owner-argus at lists.andrew.cmu.edu]On Behalf Of Peter Van Epp
> Sent: Wednesday, July 05, 2000 8:48 AM
> To: argus at lists.andrew.cmu.edu
> Subject: Re: Argus 2.0 features
>
>
> <snip>
> >
> > Lastly an aside -- I am about to buy a new machine which will be used
> > to store and analyse argus records.  Anybody have any experience on
> > what factors limit the performance of ra and friends running on disk
> > based logs?  Oh, BTW money, is tight ;-)
> >
> > I thought of a fast and wide scsi disk and 128MB memory 500Mhz
> > processor.  Not taking the standard IDE disks increases the price
> > considerably does anyone have a feeling for what difference this will
> > make to performance?
>
> 	Well I can probably help here :-). At the moment I have my
> production
> Argus server (P2 450, 256 Megs, dual 9 gig fast wide SCSI):
>
> ids /kernel: CPU: Pentium II (quarter-micron) (451.02-MHz 686-class CPU)
> ids /kernel: Origin = "GenuineIntel"  Id = 0x652  Stepping=2
> ids /kernel: Features=0x183fbff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,C
> X8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,<b24>>
> ids /kernel: real memory  = 268435456 (262144K bytes)
> ids /kernel: ahc0: <Adaptec aic7890/91 Ultra2 SCSI adapter> rev
> 0x00 int a irq 10 on pci0.14.0
> ids /kernel: ahc0: aic7890/91 Wide Channel A, SCSI Id=7, 16/255 SCBs
> ids /kernel: fxp0: <Intel EtherExpress Pro 10/100B Ethernet> rev
> 0x05 int a irq 9 on pci0.18.0
> ids /kernel: da1: <WDIGTL WDE9100 1.50> Fixed Direct Access SCSI 2 device
> ids /kernel: da1: 40.0MB/s transfers (20.0MHz, offset 15, 16bit)
> ids /kernel: da1: 8683MB (17783204 512 byte sectors: 255H 63S/T1106C)
> ids /kernel: da0 at ahc0 bus 0 target 0 lun 0
> ids /kernel: da0: <WDIGTL WDE9100 1.50> Fixed Direct Access SCSI2 device
> ids /kernel: da0: 40.0MB/s transfers (20.0MHz, offset 15, 16bit)
> ids /kernel: da0: 8683MB (17783204 512 byte sectors: 255H 63S/T1106C)
>
> 	And my about to be production server: P3 600, 256 megs, 10
> gig UDMA66
> IDE drive.
>
> demoa /kernel: CPU: Pentium III/Pentium III Xeon (598.62-MHz
> 686-class CPU)
> demoa /kernel: Origin = "GenuineIntel"  Id = 0x681  Stepping = 1
> demoa /kernel:
> Features=0x387f9ff<FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,SEP,MTRR,PGE
> ,MCA,CMOV,PAT,PSE36,PN,MMX,FXSR,XMM>
> demoa /kernel: real memory  = 268423168 (262132K bytes)
> demoa /kernel: atapci0: <Intel PIIX4 ATA33 controller> port
> 0x1820-0x182f at device 7.1 on pci0
> demoa /kernel: ad0: 9779MB <WDC WD102BA> [19870/16/63] at
> ata0-master using UDMA33
>
> 	Which happens to be currently running in parallel with my production
> machine. That means I can do an A/B comparison of the same data files for
> you on both machines (noting that there is a CPU advantage on the
> new box):
>
> Production machine:
>
> -rw-r--r--  1 root    wheel   82406756 Jul  5 00:00 argus.2000_07_04_06.gz
> -rw-r--r--  1 root    wheel   11395059 Jul  5 06:30 argus.2000_07_05_00.gz
>
> date
> Wed Jul  5 08:14:57 PDT 2000
> ra -r argus.2000_07_04_06.gz -r argus.2000_07_05_00.gz -c -n
> 203.3u 32.2s 4:19.96 90.6% 93+6960k 0+0io 0pf+0w
> date
> Wed Jul  5 08:19:17 PDT 2000
>
>
> Test machine: (noting the CPU advantage is somewhat offset by there being
> 		two argus servers running on here at the moment):
>
> -rw-r--r--  1 root  wheel  80504074 Jul  5 00:00
> dmz.argus.2000_07_04_06_30.gz
> -rw-r--r--  1 root  wheel  11071693 Jul  5 06:30
> dmz.argus.2000_07_05_00_00.gz
>
> date
> Wed Jul  5 08:14:58 PDT 2000
> ra -r dmz.argus.2000_07_04_06_30.gz -r dmz.argus.2000_07_05_00_00.gz -c -n
> 121.1u 18.4s 2:38.24 88.2% 105+6646k 0+0io 0pf+0w
> date
> Wed Jul  5 08:17:36 PDT 2000
>
> 	The UDMA33 IDE looks to be faster (and a lot cheaper) than the SCSI.
> I appear to be able to fill the wire from disk with UDMA 33 IDE
> and tcpreplay
> as well.
>
>
> >
> > It will run either Linux or FreeBSD, again any opinions?
> >
>
> 	Obviously FreeBSD from my point of view :-).  As part of a
> presentation
> I'm doing in August I am preparing a step by step install guide
> of FreeBSD and
> argus which may make this one a little easier (and should motivate me to
> fix up various things I have been meaning to). I need to test
> OpenBSD (much
> harder to install) to see if it is faster than FreeBSD, but haven't gotten
> to it yet ...
>
>
>
> > Is there anyway I can trade memory for disk reading performance on
> > these OSes.
>
> 	Run a filesystem in memory/swap (I do that with /tmp) but it is
> volitile on a boot.
>
> Peter Van Epp / Operations and Technical Support
> Simon Fraser University, Burnaby, B.C. Canada
>