Name it V3.1.0 ? [Re: Re: argus client obfuscation

Mon Jan 23 12:06:24 EST 2012

Hey Stephane,
Thanks for the response !!!!  Changes done, I'm going to put up clients-3.0.5.30 today.
If you see something that needs attention, don't hesitate to send email !!!

Version numbering is an interesting beast. I have been planning on this one being
3.0.6.  I was saving 3.1.0, which would be the next release, for the full sflow, netflowv9,
version, but its pretty arbitrary.   In support of your suggestion, we have introduced
some new concepts into the clients, such as full flow-tools support etc…, so 
no problem rethinking the strategy.  The more minds on it, I think the better it gets.

Is this an important idea?  Good idea, or idea :o)
If important, I'll definitely consider it.  We do have lots of numbers available.

Carter

On Jan 23, 2012, at 11:53 AM, Stéphane Peters wrote:

> Hi Carter, 
> 
> the change you are about to make is a good one for me.
> And it is sufficiently important to put a kind of warning sign in front of it.
> I would suggest using the version number 3.1.x rather than 3.0.6.x or something else to emphasize the change.
> As the file format is identical, no need to increment the major release number;
> or have you some other ideas about the minor version numbering ?
> 
> 
> 
> Carter Bullard a écrit :
>> 
>> Gentle people,
>> Sorry for the delayed response.  I've thought about this a bit, so here is my opinion.
>> Argus is really the only flow based system that captures user data and makes use of it, so we do need to do the right, whatever that may be, regarding flow data and attributes like user data.
>> 
>> With regard to processing of user data, we provide a lot of searching, scanning, merging, and dumping support.   We can provide more, but for the most part, we're pretty good.
>> With regard to printing, we provide multi-format printing, we have hex dumping, ascii, 32-bit and 64-bit encoding support.  We do protocol decode, with radump().   With all these, we provide truncation … so we're doing pretty good.
>> 
>> With regard to archiving, we provide options to strip and truncate user data, and we provide user supported protection/conversion through reconvert()'s  binary -> ascii -> binary support.  This allows you to do your own thing, but we could do a lot better (like describe how to use this).
>> 
>> With regard to user data protection, we do a great job with anonymization by throwing the user data away.  Probably need to describe this better.
>> 
>> The obfuscation was originally put in to protect clear text passwords from inadvertent disclosure by novice users.  Whether this is helpful or not is really debatable, but its a good idea on paper, and it was inserted to demonstrate that flow technology can address the security concerns of storing user data for long periods of time.  The concept is that there is stuff in the user data that needs some protection.  Simple password obfuscation does distinguish argus technology from packet technology, like wireshark and tcpdump. 
>> 
>> Now, all of this sounds great, but the reality is that regardless of how we print the data, we are still capturing the passwords and storing them.  By not printing them in standard printing, is an opportunity to mislead the novice and the seasoned user (as we've seen in this discussion).  So the obfuscation support is somewhat misleading, which is a bad thing with such an important concept.
>> 
>> So, as a result of this potential confusion, I am going to change the default behavior, because I think the false conclusion that the data is not in the flow records is worse than the protection provided to the naive.  I have sent email to the groups that originally asked for the feature, and they are cool with changes, as long as its well documented.
>> 
>> Obfuscation was an undocumented feature of the user data ASCII printer.  The user data printer formats are well documented in the rarc.5 manpage, but not anywhere else.  So, I'm going to add another mode of printing, ASCII_OBFUSCATE, and add that as the second option in the rarc.5 manpage, with the description that this provides some password obfuscation protection.  If we choose to obfuscate anything else, we'll add options for the option.
>> 
>> I am going to remove obfuscation from the ASCII printer. The ASCII printer will remain the default.
>> 
>> ra* programs currently support the "-M ascii", "-M hex", "-M encode32", and "-M encode64" command-line options, which are undocumented.
>> I will change this support to " -M printer=ascii", "-M printer=hex" ….. , add descriptions in the manpage and ra() usage output.
>> 
>> With the documented support to defined the printer on the command line,  I will remove the "-x" option.  The -x was not documented, so I believe that it is safe to recover the letter for future options.
>> 
>> So what does this all mean:
>> 
>>    Remove obfuscation from the default ASCII user data printer.
>>    Create the new user data printer option, "obfuscate" or "ascii-obfuscate", and document it in the .rarc and command-line manpages.
>>    Modify the "-M <printer>" option, to be "-M printer=<printer>" options, and document these features.
>>    Remove the "-x" option.
>> 
>> Now I do understand many of the issues here, and I really like the notion of formalizing the behavior, and having a reason for it.
>> Argus provides the ability to have a system wide rarc file, with specific user rarc files, so that the system can specify a policy and
>> users can override that policy.  In discussion how to use that we can talk about obfuscation and how to make it the system default,
>> if that is desired.
>> 
>> I'm going to implement this today, and put it out.  If people feel really strongly about it, keep those comments coming.
>> 
>> I am very happy that the list worked as well as it did on this topic.  While I personally think that some form of protection
>> is necessary, many on the list pointed out a problem that I hadn't considered; how the obfuscation may mislead users,
>> which seems in the end to be more damaging.  I do hope everyone sees this as a good group solution.
>> 
>> Thanks !!!!!!
>> 
>> Carter
>> 
>> 
>> 
>> On Jan 22, 2012, at 1:48 PM, Rafael Barbosa wrote:
>> 
>>> I understand that not obfuscating data is the default behavior of most (if not all) network monitoring tools. But I also think Clauss raised a valid point: argus already obfuscate the data and changing the default behavior could cause problems to some people.
>>> 
>>> In addition, I think adding some .rarc option would be enough to make everyone happy.
>>> 
>>> Best regards,
>>> Rafael Barbosa
>>> http://www.ewi.utwente.nl/~barbosarr/
>>> 
>>> 
>>> 
>>> On Fri, Jan 20, 2012 at 6:25 PM, CS Lee <geek00l at gmail.com> wrote:
>>> hi Rafael,
>>> 
>>> I think people may confuse about my thought, it is not about "I want the raw data" mindset, the internet community who uses libpcap based tools such as tcpdump, snort, argus do expect the default behavior of the tools are to capture and show the raw data(I think most people do use tcpdump, ngrep and the rest before coming into argus), people may be caught in surprise when they see it is obfuscated, I for one did see the obfuscation and quite confusing before but I was focusing on other side of issues so not really look into them. 
>>> 
>>> However with Carter mentioned it will be documented, I don't see it as problem, I just voice my opinion over here since we are living in the world of open source ;)
>>> 
>>> Cheers ;]
>>> 
>>> -- 
>>> Best Regards,
>>> 
>>> CS Lee<geek00L[at]gmail.com>
>>> 
>>> http://geek00l.blogspot.com
>>> http://defcraft.net
>>> 
>> 
> Regards,
> -- 
> Stephane.Peters at forem.be.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120123/a90287dd/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20120123/a90287dd/attachment.bin>