ra split dies with a floating point divide error when processing large file
Russell Fulton
r.fulton at auckland.ac.nz
Tue Oct 16 00:07:14 EDT 2018
The log rollover process broke and I was left with some large flow files so I tried to break them up with rasplit but if died about a third of the way through the file...
rful011 at secmgrprd02:/usr/local/tools/notify-framework$ sudo ls -lh /data/argus/data/dmzo.big
-rw-r--r-- 1 argus argus 516G Oct 15 08:59 /data/argus/data/dmzo.big
[1]+ Floating point exception(core dumped) rasplit -r /data/argus/data/dmzo.big -M time 1h -w "%Y/%m/%d/dmzo.%H.%M.%S"
Oct 15 18:07:36 secmgrprd02 kernel: [4584251.907460] traps: rasplit[5534] trap divide error ip:556d5ebf7bc4 sp:7ffc39708be0 error:0 in rasplit[556d5eb89000+b3000]
I tried re running the job with a -t parameter to start processing at the beginning of the hour in which it crashed with the same result, this did output some data.
Tried again this time starting -t the hour after the point where is crashed and it still dies and does not produce an output.
I conclude it is something to do with the process of reading the file that has the problem not the splitting.
Has anyone used rasplit on really big files? This one is weeks data and is nearly 500GB.
Russell
More information about the argus
mailing list