possible radium issue

[Carter Bullard]

The tell tale symptom would be that you have slices with multiple
srcid's in them, even though your output filename has the
$srcid directive in it.

I'm working this now, if this is your problem, then I hope to have
a fix very soon.

Carter

On Jul 9, 2009, at 11:36 AM, Phillip Deneault wrote:

> Ummm, I think so?  You know your code best. :-)
>
> If I'm reading this right, you have a pool of file handles to use.   
> You believe the pool is working fine except that its ignoring the  
> srcid when choose which entry in the pool to use?  If so, yes, that  
> is very possibly what could be happening.
>
> Phil
>
> Carter Bullard wrote:
>> OK!!!  Progress.
>> So, if I'm reading this right, rasplit() seems to be ignoring the  
>> srcid on
>> some outputs?
>> Rasplit() does cache the output files, and tries to be smart about  
>> open
>> file descriptors, so its not reading a record, opening the output  
>> file,
>> writing a single record, and closing the file.  We keep a list of  
>> open
>> files per srcid, when the "$srcid" keywork is in the output filename.
>> It must be that we're confusing which file to use?
>> Does this seem to describe where we are?
>> Carter
>> On Jul 9, 2009, at 11:18 AM, Phillip Deneault wrote:
>>> Carter Bullard wrote:
>>> > One thing to check is whether rasplit() is generating a file  
>>> somewhere else
>>>> in your file system, say if the "srcid" is screwy, or if time  
>>>> goes to zero.  Are your dates in the file name looking alright?
>>>
>>> More progress.  The time boundaries look ok, but if I compare all  
>>> file sizes of that time period, one is always WAY bigger than the  
>>> other ones and it seems to be randomly picked as to which file, in  
>>> which $srcid directory it will be.
>>>
>>>> I would recommend that you add a few more directories in your  
>>>> target path.
>>>> Unix has a bad performance issue when the files/directory get  
>>>> above say
>>>> 200 or so.  Thats why I add a %Y/%m/%d for the slices, so that  
>>>> the file
>>>> count doesn't get too  high.
>>>
>>> This isn't so much of a problem since my retention on these files  
>>> is relatively small, only a day or two.  They are being processed  
>>> into other things and then removed.
>>>
>>> Phil
>>>
>
> -- 
> --------------------------------------------------------------------
>  WPI Information Technology will never ask for your password and
> you should never give it.  http://www.wpi.edu/+infosec/phishing.html
> --------------------------------------------------------------------
> Phil Deneault                               Network Security Officer
> deneault at wpi.edu                                Information Security
> http://www.wpi.edu/~deneault/        Worcester Polytechnic Institute
> --------------------------------------------------------------------
>

Carter Bullard
CEO/President
QoSient, LLC
150 E 57th Street Suite 12D
New York, New York  10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax



-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3815 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20090709/4ffda155/attachment.bin>