racluster ArgusRemoveFromQueue errors

[carter at qosient.com]

Hey Cedric!!!
Thanks for the bug fixes!!!!!!  And I'm glad that they are that easy!!!!!!!!

Carter 

Carter Bullard
QoSient LLC
150 E. 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax

-----Original Message-----
From: VIEAU Cédric 172196 <cedric.vieau at cea.fr>

Date: Fri, 29 Jun 2007 11:33:40 
To:"Carter Bullard" <carter at qosient.com>
Cc:<argus-info at lists.andrew.cmu.edu>
Subject: RE: [ARGUS] racluster ArgusRemoveFromQueue errors


Hey Carter,

regarding my issue with racluster and the idle records that are not reported, I think I found where the problem is.

When racluster processes a new record, and when it finds a matching record in its queue, both records are merged. Then, in order to keep the queue sorted, the merged record should be moved to the end of the queue, but it's not the case.
Here is the fix :

--- clients/racluster.c.orig	2007-06-08 16:32:23.000000000 +0200
+++ clients/racluster.c	2007-06-29 11:01:57.000000000 +0200
@@ -406,6 +407,8 @@
                }
 
                ArgusMergeRecords (agg, tns, ns);
+               ArgusRemoveFromQueue (agg->queue, &tns->qhdr, ARGUS_NOLOCK);
+               ArgusAddToQueue (agg->queue, &tns->qhdr, ARGUS_NOLOCK);
 
             } else {
                struct ArgusFlow *flow = (struct ArgusFlow *) &ns->canon.flow;


Regards,
Cedric

> -----Message d'origine-----
> De : Carter Bullard [mailto:carter at qosient.com] 
> Envoyé : mercredi 20 juin 2007 02:22
> À : VIEAU Cédric 172196
> Objet : Re: TR : [ARGUS] racluster ArgusRemoveFromQueue errors
> 
> Hey Cedric,
> Thanks for the patch, I'll incorporate it right now!!!!   
> Yes, I think I know what
> maybe the issue, let me take a look and I'll see if my hunch 
> is correct!!!
> 
> Carter
> 
> 
> VIEAU Cédric 172196 wrote:
> 
> >-------- Message d'origine--------
> >De: argus-info-bounces at lists.andrew.cmu.edu de la part de 
> VIEAU Cédric 172196
> >Date: mer. 13/06/2007 13:53
> >À: Carter Bullard
> >Cc: argus-info at lists.andrew.cmu.edu
> >Objet : [ARGUS] racluster ArgusRemoveFromQueue errors
> > 
> >Hey Carter,
> >
> >I'm using argus-client-3.rc44 on a debian amd64, and I'm 
> having some "ArgusRemoveFromQueue() obj not in queue" errors 
> with racluster.
> >
> >I think the problem is when idle records are reported by 
> racluster and removed from the queue (in ArgusClientTimeout), 
> they are not removed from the hash table. So when 
> RaProcessThisRecord finds a 'tns' in hash, it no longer 
> exists in the queue, and trying to put the record at the end 
> of the queue by removing/adding it fails.
> >
> >Adding a call to ArgusRemoveHashEntry() seems to correct the 
> issue (see patch below). 
> >
> >Regarding the same section of the code, I have another issue 
> with racluster.
> >When reading flows from a radium (with -S flag), idle 
> records are almost never reported. I found that most of the 
> time, 'nslt' is greater than 'glt', 
> >so ((glt - nslt) >= agg->idleint) is always false. 
> >Any idea why ArgusFetchLastTime(ns) would report a timestamp 
> greater than the current ArgusGlobalTime ?
> >
> >My workaround (when Sflag is used) is to use the queue 
> timestamp to compute idle time :
> > if (ArgusParser->Sflag)
> >   nslt  = (double)(ns->qhdr.lasttime.tv_sec * 1.0) + 
> (double)(ns->qhdr.lasttime.tv_usec/1000000.0);
> >It works for me, but it's certainly not a good solution...
> >    
> >thanks,
> >Cedric
> >
> >--- argus-clients-3.0.0.rc.44.orig/clients/racluster.c  
> 2007-06-08 16:32:23.000000000 +0200
> >+++ argus-clients-3.0.0.rc.44/clients/racluster.c       
> 2007-06-13 11:48:52.000000000 +0200
> >@@ -241,10 +241,11 @@
> >       while (!done) {
> >          if (agg->queue->start) {
> >             struct ArgusRecordStruct *ns = (struct 
> ArgusRecordStruct *) agg->queue->start;
> >             double nslt = ArgusFetchLastTime(ns);
> >             double glt  = 
> (double)(ArgusParser->ArgusGlobalTime.tv_sec * 1.0) + 
> (double)(ArgusParser->ArgusGlobalTime.tv_usec/1000000.0);
> > 
> >             if (agg->idleint && ((glt - nslt) >= agg->idleint)) {
> >+               ArgusRemoveHashEntry(ns->htblhdr);
> >                ArgusRemoveFromQueue(agg->queue, &ns->qhdr, 
> ARGUS_NOLOCK);
> >                RaSendArgusRecord(ns);
> >                ArgusDeleteRecordStruct (ArgusParser, ns);
> >
> >
> >  
> >
> 
>