rastream and %T in -w

Thu May 9 12:42:09 EDT 2013

Yes, that would make more sense.  Check this out:

# rasplit -S 127.0.0.1:561 -M time 1d -w /var/opt/argus/%m-%d/argus_%T

# cd 05-09/

# ls -al

total 12

drwxr-xr-x. 2 root root 4096 May  9 12:36 .

drwxr-xr-x. 4 root root 4096 May  9 12:36 ..

-rw-r--r--. 1 root root 3260 May  9 12:36
argus_00:00:00<x-apple-data-detectors://3>

# cd ..

# rm -rf 05-09/

# rastream -B 1s -S 127.0.0.1:561 -M time 1d -w
/var/opt/argus/%m-%d/argus_%T

# cd 05-09/

# ls -al

total 12

drwxr-xr-x. 2 root root 4096 May  9 12:38 .

drwxr-xr-x. 4 root root 4096 May  9 12:38 ..

-rw-r--r--. 1 root root  256 May  9 12:38
argus_01:00:00<x-apple-data-detectors://7>

# cd ..

# rm -rf 05-09/

Is this reproducible with 3.0.7.8?

Thanks!

Matt

On May 9, 2013, at 11:45 AM, Carter Bullard <carter at qosient.com> wrote:

Hey Matt,
Yes, the time used to generate the file will be rounded to the
time resolution you specify,  but I would suspect that the value
should be 00:00:00, not 01:00:00.

rasplit() and rastream() share the same filename generation
logic.  If you take a file of argus data, on this specific machine,
does it also generate the 01:00:00 in the filenames it creates ?
Just one record should cause the situation/bug (if its a bug), so
don't need much data.

Carter

Carter Bullard
CEO/President
QoSient, LLC
150 E. 57th Street Suite 12D
New York, New York 10022

+1 212 588-9133 Phone
+1 212 588-9134 Fax

On May 9, 2013, at 10:34 AM, Matt Brown <matthewbrown at gmail.com> wrote:

Re-looped the list.

I had grabbed the *-latest.tar.gz off the site the other day.

Upgrading to 3.0.7.8 does not solve the problem with the file naming scheme.

I hadn't previously answered, but the times are the local TZ in the records
and are not UTC in the filenames; simply always 01:00:00 regardless of when
the file is created/process is started.

strace shows nothing useful at all.  I did verify that the md5 of
/etc/localtime and the correct timezone file are the same.
...

Ah hah... `-M 1d`, so resolution of the time format strings is to the day,
not the given time format string.  I do find this a little odd.  Carter,
what do you think?

"Problem" resolved.

Thanks,

Matt

On May 8, 2013, at 11:14 PM, Carter Bullard <carter at qosient.com> wrote:

Hey Matt,
Are the times off by a fixed amount of time each time, like the offset to
GMT ?
We did have a bug in time processing that should be fixed in
argus-clients-3.0.7.8
which is on the server at:
   http://qosient.com/argus/dev/argus-clients-3.0.7.8.tar.gz

If you grab that and the problem goes away, I would think that it was the
tm_gmoffset
bug that is biting.

Carter

On May 8, 2013, at 9:21 PM, Matt Brown <matthewbrown at gmail.com> wrote:

Sorry Carter.  I did use `-B 15s`

I will attempt to dump something with `strace` with harder evidence of the
problem.

%Y-%m-%d do equate to the strftime() values, but %T, %H, %M, and %S do not.

Do you have any suggestions for further troubleshoot the problem?

On Wed, May 8, 2013 at 7:37 PM, Carter Bullard <carter at qosient.com> wrote:

> Hey Matt,
> Don't forget you need a "-B secs" option for rastream() to work.
> If you are reading argus data, set it to 2x your flow status interval.
> I haven't used a "-", but it shouldn't make a difference.  I use '.'
> Carter
>
> On May 8, 2013, at 7:25 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>
> Hello Carter,
>
> Thanks for writing back quickly.
>
>
> If I start rastream as follows:
> rastream -S 127.0.0.1:561 -M time 1d -w /var/opt/argus//%Y-%m-%d/argus_%T
>
> the generated file is named:
> /var/opt/argus//%Y-%m-%d/argus_01:00:00
>
>
> As is the case with %H %M and %S == 01 00 and 00
>
> I pulled these variables from the man page of strftime()
> http://linux.die.net/man/3/strftime
>
>
> I've finally got around to implementing argus in a real way to complement
> the project flow-inspector, which presents flow data via a web interface
> using a few d3.js visualizations.  The commit that extends support for the
> data source of an "rasqlinserted" argus DB can be reviewed:
> https://github.com/constcast/flow-inspector/commit/e800598c3481c8ec6a44b103d98906668f612546.
>  It would be great to have an ra* client that would BLPOP() data into a
> redis queue.  A python script takes in a few IPFIX information elements
> about the flows and inserts them into a backend DB (mysql, oracle, or
> mongo).  I've been going back and forth with Lothar Braun who has been
> quite responsive.
>
>
> Thanks again for your help,
>
> Matt Brown
>
>
>
> On Wed, May 8, 2013 at 3:49 PM, Carter Bullard <carter at qosient.com> wrote:
>
>> Hey Matt,
>> Not sure, from your description, what is up.
>> So, your calling rastream() against a file or a stream?
>> Parameters ?
>>
>> Since rastream() gets its time from the records, are those correct?
>>
>> Carter
>>
>>
>>
>> On May 8, 2013, at 1:52 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>>
>> > Hello all,
>> >
>> > With 3.0.6.2 I am seeing something odd with rastream's -w.
>> >
>> > It appears to not deal with %T %H %M or %S properly, not returning
>> > now(), but starting with 01:00:00 and 01 00 00 respectively.
>> >
>> > Why is this?
>> >
>> >
>> > Unfortunately gmane's search function seems to not be functioning.
>> >
>> >
>> > Any assistance is appreciated.
>> >
>> >
>> > Thanks,
>> >
>> > Matt Brown
>> >
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130509/3b104724/attachment.html>