rastream and %T in -w
Carter Bullard
carter at qosient.com
Thu May 9 13:31:09 EDT 2013
Hey Matt,
OK, a bug !!!! I maybe able to reproduce this. Let me check later today.
Carter
On May 9, 2013, at 12:42 PM, Matt Brown <matthewbrown at gmail.com> wrote:
> Yes, that would make more sense. Check this out:
>
> # rasplit -S 127.0.0.1:561 -M time 1d -w /var/opt/argus/%m-%d/argus_%T
> # cd 05-09/
> # ls -al
> total 12
> drwxr-xr-x. 2 root root 4096 May 9 12:36 .
> drwxr-xr-x. 4 root root 4096 May 9 12:36 ..
> -rw-r--r--. 1 root root 3260 May 9 12:36 argus_00:00:00
> # cd ..
> # rm -rf 05-09/
>
> # rastream -B 1s -S 127.0.0.1:561 -M time 1d -w /var/opt/argus/%m-%d/argus_%T
> # cd 05-09/
> # ls -al
> total 12
> drwxr-xr-x. 2 root root 4096 May 9 12:38 .
> drwxr-xr-x. 4 root root 4096 May 9 12:38 ..
> -rw-r--r--. 1 root root 256 May 9 12:38 argus_01:00:00
> # cd ..
> # rm -rf 05-09/
>
> Is this reproducible with 3.0.7.8?
>
>
> Thanks!
>
> Matt
>
>
>
> On May 9, 2013, at 11:45 AM, Carter Bullard <carter at qosient.com> wrote:
>
>> Hey Matt,
>> Yes, the time used to generate the file will be rounded to the
>> time resolution you specify, but I would suspect that the value
>> should be 00:00:00, not 01:00:00.
>>
>> rasplit() and rastream() share the same filename generation
>> logic. If you take a file of argus data, on this specific machine,
>> does it also generate the 01:00:00 in the filenames it creates ?
>> Just one record should cause the situation/bug (if its a bug), so
>> don't need much data.
>>
>> Carter
>>
>> Carter Bullard
>> CEO/President
>> QoSient, LLC
>> 150 E. 57th Street Suite 12D
>> New York, New York 10022
>>
>> +1 212 588-9133 Phone
>> +1 212 588-9134 Fax
>>
>>
>>
>> On May 9, 2013, at 10:34 AM, Matt Brown <matthewbrown at gmail.com> wrote:
>>
>>> Re-looped the list.
>>>
>>> I had grabbed the *-latest.tar.gz off the site the other day.
>>>
>>> Upgrading to 3.0.7.8 does not solve the problem with the file naming scheme.
>>>
>>> I hadn't previously answered, but the times are the local TZ in the records and are not UTC in the filenames; simply always 01:00:00 regardless of when the file is created/process is started.
>>>
>>>
>>> strace shows nothing useful at all. I did verify that the md5 of /etc/localtime and the correct timezone file are the same.
>>> ...
>>>
>>> Ah hah... `-M 1d`, so resolution of the time format strings is to the day, not the given time format string. I do find this a little odd. Carter, what do you think?
>>>
>>> "Problem" resolved.
>>>
>>>
>>> Thanks,
>>>
>>> Matt
>>>
>>>
>>>
>>>
>>> On May 8, 2013, at 11:14 PM, Carter Bullard <carter at qosient.com> wrote:
>>>
>>>> Hey Matt,
>>>> Are the times off by a fixed amount of time each time, like the offset to GMT ?
>>>> We did have a bug in time processing that should be fixed in argus-clients-3.0.7.8
>>>> which is on the server at:
>>>> http://qosient.com/argus/dev/argus-clients-3.0.7.8.tar.gz
>>>>
>>>> If you grab that and the problem goes away, I would think that it was the tm_gmoffset
>>>> bug that is biting.
>>>>
>>>> Carter
>>>>
>>>> On May 8, 2013, at 9:21 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>>>>
>>>>> Sorry Carter. I did use `-B 15s`
>>>>>
>>>>> I will attempt to dump something with `strace` with harder evidence of the problem.
>>>>>
>>>>> %Y-%m-%d do equate to the strftime() values, but %T, %H, %M, and %S do not.
>>>>>
>>>>> Do you have any suggestions for further troubleshoot the problem?
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> On Wed, May 8, 2013 at 7:37 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>> Hey Matt,
>>>>> Don't forget you need a "-B secs" option for rastream() to work.
>>>>> If you are reading argus data, set it to 2x your flow status interval.
>>>>> I haven't used a "-", but it shouldn't make a difference. I use '.'
>>>>> Carter
>>>>>
>>>>> On May 8, 2013, at 7:25 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>>>>>
>>>>>> Hello Carter,
>>>>>>
>>>>>> Thanks for writing back quickly.
>>>>>>
>>>>>>
>>>>>> If I start rastream as follows:
>>>>>> rastream -S 127.0.0.1:561 -M time 1d -w /var/opt/argus//%Y-%m-%d/argus_%T
>>>>>>
>>>>>> the generated file is named:
>>>>>> /var/opt/argus//%Y-%m-%d/argus_01:00:00
>>>>>>
>>>>>>
>>>>>> As is the case with %H %M and %S == 01 00 and 00
>>>>>>
>>>>>> I pulled these variables from the man page of strftime() http://linux.die.net/man/3/strftime
>>>>>>
>>>>>>
>>>>>> I've finally got around to implementing argus in a real way to complement the project flow-inspector, which presents flow data via a web interface using a few d3.js visualizations. The commit that extends support for the data source of an "rasqlinserted" argus DB can be reviewed: https://github.com/constcast/flow-inspector/commit/e800598c3481c8ec6a44b103d98906668f612546. It would be great to have an ra* client that would BLPOP() data into a redis queue. A python script takes in a few IPFIX information elements about the flows and inserts them into a backend DB (mysql, oracle, or mongo). I've been going back and forth with Lothar Braun who has been quite responsive.
>>>>>>
>>>>>>
>>>>>> Thanks again for your help,
>>>>>>
>>>>>> Matt Brown
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, May 8, 2013 at 3:49 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>>> Hey Matt,
>>>>>> Not sure, from your description, what is up.
>>>>>> So, your calling rastream() against a file or a stream?
>>>>>> Parameters ?
>>>>>>
>>>>>> Since rastream() gets its time from the records, are those correct?
>>>>>>
>>>>>> Carter
>>>>>>
>>>>>>
>>>>>>
>>>>>> On May 8, 2013, at 1:52 PM, Matt Brown <matthewbrown at gmail.com> wrote:
>>>>>>
>>>>>> > Hello all,
>>>>>> >
>>>>>> > With 3.0.6.2 I am seeing something odd with rastream's -w.
>>>>>> >
>>>>>> > It appears to not deal with %T %H %M or %S properly, not returning
>>>>>> > now(), but starting with 01:00:00 and 01 00 00 respectively.
>>>>>> >
>>>>>> > Why is this?
>>>>>> >
>>>>>> >
>>>>>> > Unfortunately gmane's search function seems to not be functioning.
>>>>>> >
>>>>>> >
>>>>>> > Any assistance is appreciated.
>>>>>> >
>>>>>> >
>>>>>> > Thanks,
>>>>>> >
>>>>>> > Matt Brown
>>>>>> >
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130509/64821f51/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20130509/64821f51/attachment.bin>
More information about the argus
mailing list