possible radium issue

[Carter Bullard]

No, this doesn't seem right at all.   A couple of suggestions.
Don't use the "-M norep" for this type of aggregation (basically it
just throws away the AGR dsr as the records are being written
out, and in some apps this is great, but not necessary  here).

How are your rasplit()s called, I suspect there may be an issue with  
that.

In most cases, you don't need the hourly and daily rasplit()  
processes, because
you can generate both of these from your 10 min split files.  All  
depends on
whether you want the hourly and daily files updated continuously, or  
if you
can get away with updating them, say every 10 minutes.

It looks like racluster() is faulting reading one of the files.   When  
it does that,
the pipe closes down, and your racount() reports just the records it  
receives.
Just need to find the bad file, and then try to figure out how it got  
corrupted
(at least that is my guess).

what are the totals for each of the individual files in your  
example(s) without
the clustering?

Carter



On Jul 8, 2009, at 4:24 PM, Phillip Deneault wrote:

> I'm running the beta.8 code.  I have a single radium instance  
> collecting data from dozens of locations and 3 rasplit processes  
> connecting to that radium process, one for 10 minute slices, 1 for  
> hourlies, and 1 for dailies.
>
> It *seems* as if the data I'm recording is lower than what I should  
> have.  I say this because I get drastically different counts when I  
> check locally recorded data vs. radium recorded data.
>
> Please yell at me if I am doing this wrong, I performed the  
> racluster in an attempt to normalize the flow counts a little.
>
> Locally recorded data tallies like this.(logs rotated daily, so I  
> picked a convenient hour).
>
> # racluster -t 14 -M norep -r /var/log/argus/argus.out -w - |  
> racount -r -
> racount   records     total_pkts     src_pkts       dst_pkts  
> total_bytes        src_bytes          dst_bytes
>    sum   52385       134978         134813         165  
> 9982211            9970637            11574
>
> However, when I run a tally on the hourlies and the slices collected  
> by radium, I get two different flow counts, neither of which come  
> anywhere close.
>
> (SLICES)
> # racluster -M norep -r argus-07.08.2009-14.50.00.out  
> argus-07.08.2009-14.40.00.out argus-07.08.2009-14.30.00.out  
> argus-07.08.2009-14.20.00.out argus-07.08.2009-14.10.00.out  
> argus-07.08.2009-14.00.00.out -w - | racount -r -
> racount   records     total_pkts     src_pkts       dst_pkts  
> total_bytes        src_bytes          dst_bytes
>    sum   631         1920           1397           523  
> 507980             210286             297694
>
> (HOURLIES)
> # racluster -M norep -r argus-07.08.2009-14.00.00.out -w - | racount  
> -r -
> racount   records     total_pkts     src_pkts       dst_pkts  
> total_bytes        src_bytes          dst_bytes
>    sum   252         447            348            99  
> 95012              57022              37990
>
> Is this a bug, or me doing something wrong?
>
> Thanks,
> Phil
>





-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090708/1f6595d3/attachment.bin>