Bug in racluster: "Too many open files"

[Carter Bullard]

Hey Rafael,
We must not be closing the configuration.  When you use the " ind " mode, the code " instantiates "
a new independent modeler for each aggregation,  and the modeler, because of the class definitions,
want to get the configuration using the same method, so they all want to open the file.

Not sure about your line numbers, as this fix was applied to argus-clients-3.0.7.5/common/argus_client.c,
so be sure and add the lines that start with the " + ", in the routine ArgusParseAggregator (), which
is in the file ./common/argus_client.c, toward the end of the routine.

Here is a patch:

-------- Begin diff -c output -----------
*** argus_client.c.orig	Fri Feb 15 10:50:11 2013
--- argus_client.c	Fri Feb 15 10:50:05 2013
***************
*** 11605,11610 ****
--- 11605,11612 ----
        if (pstr != NULL) free(pstr);
     }
  
+    if (fd != NULL) fclose(fd);
+ 
     if ((agg = ArgusNewAggregator(parser, NULL)) == NULL)
        ArgusLog (LOG_ERR, "ArgusClientInit: ArgusNewAggregator error");

------- End diff -c output -------------

Yell if this doesn't work for you !!!

Carter

On Feb 15, 2013, at 9:07 AM, Rafael Barbosa <rrbarbosa at gmail.com> wrote:

> Hi Carter,
> 
> I am trying to run racluster in directory with around 2000 files. Each file is a 5 min "split" generated with rasplit, the way you suggested to work with large amounts of data.
> 
> The following command works:
> $> racluster -R split -M ind > /dev/null
> 
> However, if I add a racluster configuration file, I get the following error:
> $> racluster -f ~/config/racluster.conf -R split -M ind > /dev/null
> racluster[60174]: 14:33:49.335532 open 'split/argus.2010.10.22.00.25.00': Too many open files
> racluster[60174]: 14:33:49.335800 /home/barbosarr/config/racluster.conf: Too many open files
> 
> When looking at the 'lsof' output, it seems that racluster.conf is opened multiple times. I assume it is opened once for each file in the 'split' folder and never closed.
> 
> An excerpt of the 'lsof' output:
> racluster 60174         barbosarr    3r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr    4r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr    5r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr    6r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr    7r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr    8r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr    9r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   10r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   11r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   12r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   13r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   14r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   15r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   16r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   17r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   18r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   19r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   20r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   21r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   22r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   23r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> racluster 60174         barbosarr   24r      REG               8,17       96 704907370 /mnt/data/home/barbosarr/config/racluster.conf
> 
> The content of racluster.conf:
> #Filter:every record, no record status output, record time out 5min
> filter="" status=0 idle=300
> 
> I am running Racluster Version 3.0.7.3.
> 
> Regards,
> Rafael Barbosa
> http://www.ewi.utwente.nl/~barbosarr/

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130215/958cf451/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130215/958cf451/attachment.bin>