Anonymization of argus flow data

[Kaustubh Gadkari]

On Tue, Sep 3, 2013 at 6:00 AM, Carter Bullard <carter at qosient.com> wrote:
> Hmmmm,
> There shouldn't be any performance issues with anonymizing a file, if your
> just
> anonymizing the IP addresses.  How many addresses are in the file?
> What does your ranonymize.conf file look like?   How much memory is it
> using?
>

I am not quite sure how many IP addresses there are in the file. My
ranonymize.conf looks like this:

RANON_PRESERVE_ETHERNET_VENDOR=yes
RANON_PRESERVE_BROADCAST_ADDRESS=yes
RANON_NET_ANONYMIZATION=sequential
RANON_HOST_ANONYMIZATION=sequential
RANON_PRESERVE_NET_ADDRESS_HIERARCHY=class

I took a look at how much memory ranonymize is using .. the usage is
about 42% on a machine with 32GB RAM.

> ranonymize() can be a little complex O(nLogN + C), but it should be
> in the same time frame as racount().  How long does it take for racount()
> to read the file?
>

I am running racount right now .. I will post results once it finishes.

> Just a rule of thumb. If a ra* program doesn't complete in a few minutes,
> you
> should stop it and try to figure out if there is a memory problem or not.
>

Thanks, I'll keep this in mind :)

Thanks,
Kaustubh

> Carter
>
> On Sep 2, 2013, at 2:20 PM, Kaustubh Gadkari <kaustubh.gadkari at gmail.com>
> wrote:
>
> Hi,
>
> I have a set of argus flow data captured at our data capture vantage point,
> and I want to anonymize the IP addresses (both source and destination) fully
> i.e. I want to replace both the addresses, using a prefix preserving
> technique. I have tried using ranonymize, but it is taking an extremely long
> time to anonymize the file (I started the process a couple of months ago, on
> a ~125GB file, and the output file size today is only ~30GB).
>
> Can anyone suggest the right way to go about anonymizing the data set I
> have? Is ranonymize the right tool for the job?
>
> Thanks,
> Kaustubh
>
> --
> Kaustubh Gadkari
>
>



-- 
Kaustubh Gadkari