possible radium issue

[Carter Bullard]

OK!!!  Progress.
So, if I'm reading this right, rasplit() seems to be ignoring the  
srcid on
some outputs?

Rasplit() does cache the output files, and tries to be smart about open
file descriptors, so its not reading a record, opening the output file,
writing a single record, and closing the file.  We keep a list of open
files per srcid, when the "$srcid" keywork is in the output filename.
It must be that we're confusing which file to use?

Does this seem to describe where we are?

Carter

On Jul 9, 2009, at 11:18 AM, Phillip Deneault wrote:

> Carter Bullard wrote:
> > One thing to check is whether rasplit() is generating a file  
> somewhere else
>> in your file system, say if the "srcid" is screwy, or if time goes  
>> to zero.  Are your dates in the file name looking alright?
>
> More progress.  The time boundaries look ok, but if I compare all  
> file sizes of that time period, one is always WAY bigger than the  
> other ones and it seems to be randomly picked as to which file, in  
> which $srcid directory it will be.
>
>> I would recommend that you add a few more directories in your  
>> target path.
>> Unix has a bad performance issue when the files/directory get above  
>> say
>> 200 or so.  Thats why I add a %Y/%m/%d for the slices, so that the  
>> file
>> count doesn't get too  high.
>
> This isn't so much of a problem since my retention on these files is  
> relatively small, only a day or two.  They are being processed into  
> other things and then removed.
>
> Phil
>




-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090709/7d9fb6d5/attachment.bin>