[flow-tools] FW: Has anyone tried this before / know how possible it is?

[Mark Fullmer]

NetFlow v5, 6, 7, and 8 do have per flow sequence numbers.  Flow-tools
does not currently store this information.

NetFlow v9 has per packet sequence numbers so trying to de-duplicate
streams based on sequence numbers will not work with the next gen protocol
as it currently stands.

As others have suggested, there are probably easier ways to do this.  One
simple solution would be to run a total bytes report on each collector, the
one with the largest byte count is used for billing.  If they differ
greatly something went wrong.  One other sanity check would be to run
a daily per interface input and/or output report and then check those
values with the interface counters available via SNMP.

Encouraging Cisco and other vendors to move forward with a NetFlow v9
like implementation over SCTP, failover, and a reporting mechanism to
instrument lost flow data would also help in building a resilient
collection system...

mark

On Wed, Jan 29, 2003 at 10:15:11AM +1100, Will Lotto wrote:
> 
> > G'day all,
> > 
> > I don't know if I'm trying to do the impossible or not, but I'm
> > gathering that if I've thought of it someone else has before me.
> > 
> > Since the netflow flows are UDP, and a program on a linux box is a
> > little less reliable than the IOS on a 7200, I'd like to setup two
> > collectors for the same stream (either via multicast or telling the
> > 7200 it has multiple collectors), which dump the streams into separate
> > directories; then a program that gets the two directories and creates
> > one file with no duplicate streams.
> > 
> > Ie.
> > 
> >                         ----[collector] --> [Dir #1]----
> >                       /                                         \
> > [Cisco 7200] <                                             > [ Merged
> > flows (duplicates removed) ] ---> [Stats / Billing / etc.]
> >                      \                                          /
> >                        ----[collector] --> [Dir #2] ----
> > 
> > 
> > 
> > This way either one of the collectors can fail, and the second will
> > still collect streams, then the merge process can happen at any time.
> > 
> > Will the sequence number of the UDP packets help me, or is this not
> > possible with flow-tools?
> > 
> > 
> > Thanks,
> > 
> > Will Lotto
> > Systems Administrator
> > Bendigo Community Telco