rastream and %T in -w

Thu May 9 11:45:10 EDT 2013

Hey Matt,
Yes, the time used to generate the file will be rounded to the
time resolution you specify,  but I would suspect that the value 
should be 00:00:00, not 01:00:00.

rasplit() and rastream() share the same filename generation
logic.  If you take a file of argus data, on this specific machine,
does it also generate the 01:00:00 in the filenames it creates ?
Just one record should cause the situation/bug (if its a bug), so
don't need much data.

Carter

Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

On May 9, 2013, at 10:34 AM, Matt Brown <matthewbrown at gmail.com> wrote:

> Re-looped the list.
> 
> I had grabbed the *-latest.tar.gz off the site the other day.
> 
> Upgrading to 3.0.7.8 does not solve the problem with the file naming scheme.
> 
> I hadn't previously answered, but the times are the local TZ in the records and are not UTC in the filenames; simply always 01:00:00 regardless of when the file is created/process is started.
> 
> 
> strace shows nothing useful at all.  I did verify that the md5 of /etc/localtime and the correct timezone file are the same.
> ...
> 
> Ah hah... `-M 1d`, so resolution of the time format strings is to the day, not the given time format string.  I do find this a little odd.  Carter, what do you think?
> 
> "Problem" resolved.
> 
> 
> Thanks,
> 
> Matt
> 
> 
> 
> 
> On May 8, 2013, at 11:14 PM, Carter Bullard <carter at qosient.com> wrote:
> 
>> Hey Matt,
>> Are the times off by a fixed amount of time each time, like the offset to GMT ?
>> We did have a bug in time processing that should be fixed in argus-clients-3.0.7.8
>> which is on the server at:
>>    http://qosient.com/argus/dev/argus-clients-3.0.7.8.tar.gz
>> 
>> If you grab that and the problem goes away, I would think that it was the tm_gmoffset
>> bug that is biting.
>> 
>> Carter 
>> 
>> On May 8, 2013, at 9:21 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>> 
>>> Sorry Carter.  I did use `-B 15s`
>>> 
>>> I will attempt to dump something with `strace` with harder evidence of the problem.
>>> 
>>> %Y-%m-%d do equate to the strftime() values, but %T, %H, %M, and %S do not.
>>> 
>>> Do you have any suggestions for further troubleshoot the problem?
>>> 
>>> 
>>> 
>>> 
>>> On Wed, May 8, 2013 at 7:37 PM, Carter Bullard <carter at qosient.com> wrote:
>>> Hey Matt,
>>> Don't forget you need a "-B secs" option for rastream() to work.
>>> If you are reading argus data, set it to 2x your flow status interval.
>>> I haven't used a "-", but it shouldn't make a difference.  I use '.'
>>> Carter
>>> 
>>> On May 8, 2013, at 7:25 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>>> 
>>>> Hello Carter,
>>>> 
>>>> Thanks for writing back quickly.
>>>> 
>>>> 
>>>> If I start rastream as follows:
>>>> rastream -S 127.0.0.1:561 -M time 1d -w /var/opt/argus//%Y-%m-%d/argus_%T
>>>> 
>>>> the generated file is named:
>>>> /var/opt/argus//%Y-%m-%d/argus_01:00:00
>>>> 
>>>> 
>>>> As is the case with %H %M and %S == 01 00 and 00
>>>> 
>>>> I pulled these variables from the man page of strftime() http://linux.die.net/man/3/strftime
>>>> 
>>>> 
>>>> I've finally got around to implementing argus in a real way to complement the project flow-inspector, which presents flow data via a web interface using a few d3.js visualizations.  The commit that extends support for the data source of an "rasqlinserted" argus DB can be reviewed: https://github.com/constcast/flow-inspector/commit/e800598c3481c8ec6a44b103d98906668f612546.  It would be great to have an ra* client that would BLPOP() data into a redis queue.  A python script takes in a few IPFIX information elements about the flows and inserts them into a backend DB (mysql, oracle, or mongo).  I've been going back and forth with Lothar Braun who has been quite responsive.
>>>> 
>>>> 
>>>> Thanks again for your help,
>>>> 
>>>> Matt Brown
>>>> 
>>>> 
>>>> 
>>>> On Wed, May 8, 2013 at 3:49 PM, Carter Bullard <carter at qosient.com> wrote:
>>>> Hey Matt,
>>>> Not sure, from your description, what is up.
>>>> So, your calling rastream() against a file or a stream?
>>>> Parameters ?
>>>> 
>>>> Since rastream() gets its time from the records, are those correct?
>>>> 
>>>> Carter
>>>> 
>>>> 
>>>> 
>>>> On May 8, 2013, at 1:52 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>>>> 
>>>> > Hello all,
>>>> >
>>>> > With 3.0.6.2 I am seeing something odd with rastream's -w.
>>>> >
>>>> > It appears to not deal with %T %H %M or %S properly, not returning
>>>> > now(), but starting with 01:00:00 and 01 00 00 respectively.
>>>> >
>>>> > Why is this?
>>>> >
>>>> >
>>>> > Unfortunately gmane's search function seems to not be functioning.
>>>> >
>>>> >
>>>> > Any assistance is appreciated.
>>>> >
>>>> >
>>>> > Thanks,
>>>> >
>>>> > Matt Brown
>>>> >
>>>> 
>>>> 
>>> 
>>> 
>> 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130509/b1488af4/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130509/b1488af4/attachment.bin>