Radium/ra client tools flow timestamps oddities with direct Netflow

[Drew Dixon]

I suppose to boil it down, I can't really seem to understand why the
timestamps are off by 8 hours in the future when the netflow data is
certainly not delayed in being processed by radium/racluster more than an
hour or so at the very most, for some flows- but probably more like
ballpark ~10 minutes or so on average.  Right now the only thing that might
make sense is that radium is not calculating the timestamps properly but
I'm not certain.

I'm aware of the -T option in radium but I'm concerned that if this ~49 day
calculation is applicable here that the threshold will be a moving target
and if I were to set -T to 8 hours (haven't tested that yet), then again I
might be worried about something that isn't relevant.   This ~49 day
calculation is something that I was informed about by a network engineer
and that person was who brought up the concern with it being a moving
target that could create problems in the future with the time sync
threshold setting (-T) in radium is a static configuration setting (which
it is AFAIK).

Any help/input etc. is greatly appreciated.

-Drew

On Fri, Dec 1, 2017 at 4:41 PM, Drew Dixon <dwdixon at umich.edu> wrote:

> Hi all,
>
> Currently as an interim solution we are reading an older Netflow v5
> conversion we had already available to us due to radium not supporting
> IPFIX/v10 flow data.
>
> Netflow v5 --> radium --> racluster --> xz archives <-- ra client tool <--
> user
>
> The flow data we are getting is great and useful but I'm seeing some very
> strange issues with timestamps using just the default options...the
> timestamps appear to be about ~8 hours in the future (from current EST) for
> some reason...ntp is fine everything is setup properly with the time on
> everything upstream of the box running radium/racluster and on that box as
> well.
>
> Is there not support for properly calculating timestamps when using radium
> for reading Netflow data directly from network devices (without generating
> our own off the wire using an argus server)?  I know radium/ra prints the
> start time of the flow by default from what I can tell reading the docs.
>
> Still not fully certain on all this but I took some pcaps before the data
> we are reading with radium gets processed by radium and from what I
> understand it's something like this in order to get the actual timestamp of
> the flow:
>
> In the pcap, the timestamp (listed above the individual flows, once per
> netflow packet) is set to ~49 days ago (49 days, 17 hours, 2 minutes and 47
> seconds, exactly) to allow for the maximum possible values (up to overflow
> per flow).  So I believe you need to add the StartTime seconds (per flow)
> to the TimeStamp (per packet) to get the actual real/current start time
> timestamp of the flow.
>
> I'm not sure if I'm on the right track with figuring this all out, I dug
> into all the time stamp and time sync options available in radium/ra and
> nothing really seems to fit/be an appropriate solution or really do
> anything for me.
>
> Does any of this sound familiar to anyone?
>
> For example flows coming in off the wire right now (~16:30) after being
> read/parsed by radium are showing a timestamp of (~00:30).
>
> Am I missing something obvious here or maybe not so obvious?  How can I
> get radium to calculate the timestamps properly or adjust properly and
> display them accurately in the radium parsed flow data?
>
> Thank you much in advance,
>
> -Drew
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20171204/9c405d97/attachment.html>