Looks like a new bug in clients ...

[Carter Bullard]

Argus does a stat on the output filename, periodically, to see if it  
needs
to close the moved file's file descriptor, and create a new one.  I  
think
your internal clock is getting screwed and so 'check filename' timestamp
isn't getting exceeded, so it doesn't check and keeps writing out to the
old file.  The file is growing somewhere in the filesystem, but the  
link to
it is gone, so, ....

I'm working on the leak, but the timestamp issue is still something
that could be  a mystery still.

Carter



On Aug 17, 2007, at 3:42 PM, Peter Van Epp wrote:

> 	The latest clients.rc.48 seems to have broken log file switching
> somehow. The archived log file looks to end early (this should have  
> been the
> noon switch):
>
> 07-08-17 11:40:58  e          tcp      142.58.61.205.1971      - 
> >     209.85.171.166.80            2        3         1034          
> 1677   CON
> 07-08-17 11:43:12  eIs        tcp     151.21.221.110.64696     - 
> >       206.12.16.87.41417         2        0           
> 134            0   REQ
> 07-08-17 11:43:12  e          tcp     192.75.243.138.4645      - 
> >      209.85.167.99.80            2        2         1125           
> 443   CON
>
> and it didn't actually create a new com_argus when it did  the  
> switch (which
> it thinks it did at noon from the archive log):
>
> + gzip com_argus.2007.08.17.11.00.00.0
> + TIMESTAMP=2007.08.17.12.00.54
> 2007.08.17.12.00.54 compression completed
> + mv com_argus.2007.08.17.11.00.00.0.gz /archive/argus3/ 
> com_argus.archive/2007/08/17
> 2007.08.17.12.00.54 argusarchive exiting successfully
>
> 	But an lsof of the ra task indicates no output file present ... Ah
> illumination just struck as I was typing this. It is trying to use the
> file that was moved and compressed instead of creating a new  
> com_argus:
>
> stat(/Volumes/Teradactyl/argus/argus/com_argus. 
> 2007.08.17.11.00.00.0): No such file or directory
>
> 	this is now /Teradactyl/argus/argus/com_argus. 
> 2007.08.17.11.00.00.0.gz
> the archived file, ra hasn't recognized it needs to recreate
> /var/log/argus/com_argus for some reason and is trying to write to  
> the moved
> archive file.
> 	This may be a threads issue, since a test just now caused the file
> to be recreated correctly when I moved it. I'll recompile clients  
> without
> threads and see if that helps.
>
> lsof | grep ra3
> ra3        1388 vanepp  cwd     VDIR      14,10       2856        
> 122 /Volumes/Teradactyl/argus/argus
> ra3        1388 vanepp  txt     VREG       14,2    1289728    
> 1159587 /usr/local/bin/ra3
> ra3        1388 vanepp  txt     VREG       14,2    1797576    
> 1315128 /usr/lib/dyld
> ra3        1388 vanepp  txt     VREG       14,2    4402196    
> 1636735 /usr/lib/libSystem.B.dylib
> ra3        1388 vanepp    0r    VCHR        3,2        0t0  
> 127907460 /dev/null
> ra3        1388 vanepp    1w    VREG      14,10  649543026      
> 50850 /Volumes/Teradactyl/argus/argus.logs/com_ra3.log
> ra3        1388 vanepp    2w    VREG      14,10  649543026      
> 50850 /Volumes/Teradactyl/argus/argus.logs/com_ra3.log
> ra3        1388 vanepp    3r  PSXSHM 0x07c29104        
> 4096           obj=0x07dd17f8
> ra3        1388 vanepp    4u    IPv4 0x0829ea94        0t0        
> TCP test4.ucs.sfu.ca:51828->hcids.ucs.sfu.ca:rmonitor (ESTABLISHED)
> ra3        1388 vanepp    5u     
> VREG                                 stat(/Volumes/Teradactyl/argus/ 
> argus/com_argus.2007.08.17.11.00.00.0): No such file or directory
>
> Peter Van Epp / Operations and Technical Support
> Simon Fraser University, Burnaby, B.C. Canada
>