possible radium issue
Phillip Deneault
deneault at WPI.EDU
Thu Jul 9 11:18:27 EDT 2009
Carter Bullard wrote:
> One thing to check is whether rasplit() is generating a file
somewhere else
> in your file system, say if the "srcid" is screwy, or if time goes to
> zero. Are your dates in the file name looking alright?
More progress. The time boundaries look ok, but if I compare all file
sizes of that time period, one is always WAY bigger than the other ones
and it seems to be randomly picked as to which file, in which $srcid
directory it will be.
> I would recommend that you add a few more directories in your target path.
> Unix has a bad performance issue when the files/directory get above say
> 200 or so. Thats why I add a %Y/%m/%d for the slices, so that the file
> count doesn't get too high.
This isn't so much of a problem since my retention on these files is
relatively small, only a day or two. They are being processed into
other things and then removed.
Phil
More information about the argus
mailing list