Using argus-clients for netflow collection and display
Carter Bullard
carter at qosient.com
Fri Apr 4 15:33:44 EDT 2014
Hey Jesse,
The first run of racount() indicates that all the records are uni-directional.
No dst pkts or dst bytes gt 0.
If you look at the output of the second run, with the “ -M rmon”, the source and
destination metrics are identical. So the “ -M rmon “ option is not doing what
you think, so don’t use it until you want an RMON style of metric from your data.
No problems at all with collection. Try running racluster() on the file, and see
what you get ???
racluster -r argus.2014.04.04.14.30.00 -w - | racount
How did you call ra() to get the single DNS request ???
Carter
On Apr 4, 2014, at 2:44 PM, Jesse Bowling <jessebowling at gmail.com> wrote:
> OK, one step closer to understanding! :)
>
> I'm collecting like this:
>
> /usr/local/bin/rasplit -M time 5m -C 10.0.0.1:9996 -w /argus/netflow/%Y/%m/%d/argus.%Y.%m.%d.%H.%M.%S -d
>
> and see this in a recent file:
>
> # ra -r argus.2014.04.04.14.30.00 -w - |racount
> racount records total_pkts src_pkts dst_pkts total_bytes src_bytes dst_bytes
> sum 118680 3366456 3366456 0 3609376863 3609376863 0
>
> # ra -r argus.2014.04.04.14.30.00 -M rmon -w - |racount
> racount records total_pkts src_pkts dst_pkts total_bytes src_bytes dst_bytes
> sum 240375 7505584 3752792 3752792 8077575906 4038787953 4038787953
>
> So, could this be a collection issue? The netflow source only sending half the conversation? Filtering in on a single DNS transaction as before shows:
>
> StartTime Flgs Proto SrcAddr Sport Dir DstAddr Dport TotPkts TotBytes State SrcPkts DstPkts SrcBytes DstBytes
> 04/04/14 14:31:09.573000 Ne 17 100.0.1.8.53 -> 100.0.1.135.53504 1 163 REQ 1 0 163 0
>
>
> Collection issue? Understanding issue?
>
> Cheers,
>
> Jesse
>
>
>
>
> On Fri, Apr 4, 2014 at 2:24 PM, Carter Bullard <carter at qosient.com> wrote:
> Hey Jesse,
> No, not right !!!! The non-aggregating ra* programs will read netflow records
> and convert them into argus records that will have unidirectional metrics.
> To merge unidirectional netflow data into bi-directional data, you need to use
> an argus aggregator, such as racluster() or rabins(), they will merge the uni-
> directional netflow records and create bi-directional argus records.
>
> If you take a file of netflow records, and count them up, and compare the totals
> to the output of racluster() on that same file of netflow records, you should get
> the same totals for bytes and packets, with a reduction in the total number
> of records.
>
> The RMON option simply creates a reversed clone of every record on input,
> doubling the traffic records but presenting with both directions. This allows you
> to pick specific singular objects, like the Source Address, and get all the
> statistics straight.
>
> Compare these two calls:
>
> ra -r netflow.file -w - | racount
> ra -r netflow.file -M rmon -w - | racount
>
> All the metrics will be 2X, so its doing something completely different
> than what you think.
>
> Printout spkts and dpkts and sbytes and dbytes, rather that pkts and bytes.
>
> Carter
>
>
>
>
> On Apr 4, 2014, at 1:57 PM, Jesse Bowling <jessebowling at gmail.com> wrote:
>
> > I've been experimenting with using argus-clients to collect netflow records, and ran into some confusion. I believe I've worked out some of the issue, but wanted to confirm some items.
> >
> > I believe that argus is writing out netflow records as bi-directional records, rather than as the unidirectional records that netflow is generated in. For example, I see something like this:
> >
> > ra -r - - \(host 100.0.1.103 and port 53\) and \(host 100.0.1.30 and port 10239\)
> > StartTime Flgs Proto SrcAddr Sport Dir DstAddr Dport TotPkts TotBytes State
> > 03/31/14 22:57:47.865000 Ne 17 100.0.1.103.53 -> 100.0.1.30.10239 1 78 REQ
> >
> > In this case 100.0.1.103 is the DNS server, and 100.0.1.30 is the client. I found it odd that only a single packet was sent. However when I look at it this way:
> >
> > ra -r - -M rmon - \(host 100.0.1.103 and port 53\) and \(host 100.0.1.30 and port 10239\)
> > StartTime Flgs Proto Host Sport Dir DstAddr Dport TotPkts TotBytes State
> > 03/31/14 22:57:47.865000 Ne 17 100.0.1.103.53 -> 100.0.1.30.10239 1 78 REQ
> > 03/31/14 22:57:47.865000 Ne 17 100.0.1.30.10239 <- 100.0.1.103.53 1 78 REQ
> >
> > It would seem to make more sense as I can see both sides. My confusion is in the count for total bytes/packets...I find it unlikely that the exact same size packets were sent in query and answer (given how often this happens in the data), and if this is an aggregated record, why is the packet count not two? I don't believe the timestamps are granular enough to reveal timing differences, since we're relying on the router to generate the records, so I can understand the confusion for which server originated the traffic.
> >
> > So...Are argus-clients automatically creating bi-directional records from the netflow collected? How accurate can I expect those records to be? Is there an option to write netflow collected records as unidirectional? Would that even be a good idea?
> >
> > I fully appreciate some of the reasons that argus was created (to deal with just this sort of reliance on router implementations), but what pitfalls do I need to look out for if I'm using argus to collect those records?
> >
> > Thanks,
> >
> > Jesse
> >
> > --
> > Jesse Bowling
> >
>
>
>
>
> --
> Jesse Bowling
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140404/2dc6924d/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20140404/2dc6924d/attachment.bin>
More information about the argus
mailing list