Hardware specs?

[Carter Bullard]

Hey James,
Resource demands for a radium node really depend on what you want to do with it.
The collection and redistribution of flow data does not require a lot of resources.  100K-200K
records per second per core is not a big job, 1M argus records is getting toward limits of
modern Sandy Bridge like Intel processors with some good PCIe based memory and
networking interfaces, using the stock argus-clients code.

2K records / transactions per second would not be a high demand environment for
current argus-client implementations.  With that as the only consideration, a good laptop
will work as a radium collection and redistribution node.

But if you want to use the node for semantic enhancement (labeling), analytics,
storage and archiving, those demands can become non-trivial.

All ra* programs are multi-threaded, so having a few cores is a good thing.  4 is a good
number, only because its greater than 1 and less than 16, but having a few cores does
go a long way to maintaining resources for the basic collection and distribution function,
and offering capacity to do other things.  Many are using 16-32 core systems to do the
full Monty, so to speak, collection, archiving, analytics, database etc....

Sorting data files, aggregation, and many multi-pass analytics will benefit from lots of RAM,
and good bus throughput.  16GB minimum is a decent rule of thumb, with 8GB would be a 
good minimum requirement.

Storage demands can get up there, depending on the granularity of flow reporting and
number of flows, so lots of disk is also a good thing for storage, with long retention times.
Monthly trending may require some persistent storage, unless you know what you want
to trend, so a few TB at least.

There are a few groups at Stanford that are currently doing some pretty demanding argus data
generation, collection and processing.  Maybe we can hook you up with those guys to
talk about their experiences, and what they are using ?

Carter


On Mar 6, 2013, at 8:12 PM, "James A. Robinson" <jimr at highwire.stanford.edu> wrote:

> I was hunting around on the FAQ for "hardware" and "memory", but I
> don't see any recommendations for the hardware to use to run a radium
> server.  I imagine it all depends on how much data one wants to
> collect.
> 
> If we want to monitor two dozen web servers that, collectively, serve
> something like 2000 requests per second with associated database and
> NFS traffic, would anyone have recommendations on how much memory we
> ought to have on the radium server?  Will SAS vs SATA disk be
> important?
> 
> What I'm looking to do initially is collect information on HTTP
> traffic counts (requests per second, bytes per second) for subnets
> connecting to ours.  Later I'd like to be able to put together reports
> on the total amount of NFS traffic between our backend systems.
> 
> Jim
> 
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> James A. Robinson                       jimr at highwire.stanford.edu
> HighWire | Stanford University          http://highwire.stanford.edu/
> +1 650 7237294 (Work)                   +1 650 7259335 (Fax)
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2589 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130307/574b04ad/attachment.bin>