color support, pcre and pf_ring issues

Tue Oct 23 05:43:58 EDT 2012

Hey Elof,
There are many reasons to color a record or a part of a record, and I'm thinking of having a list of methods that result in a painters algorithm, to generate the final result.  

Somewhere in that list of color methods is address coloring.  So if you want the whole record RED, because of some fault condition, but you want the addresses colored because of locality, you would specify the fault method first, the address method second.

In the implementation so far, we know what each column is, and the coloring can be targeted at any field, group of fields or the whole record.  So if you have a minimum byte count condition, as an example, you should have a way of specifying that conditional against the field and then indicate what fileds get what color.  I'm working with our standard filters, like we use for aggregation, with a field=color directive.  Does that sound useful ?

Carter

On Oct 23, 2012, at 4:11 AM, elof2 at sentor.se wrote:

> 
> On Mon, 22 Oct 2012, Carter Bullard wrote:
>> So, can we agree to support 8 accent colors:
>>  YELLOW, ORANGE, RED, MAGENTA, VIOLET, BLUE, CYAN and GREEN
>> and 8 monotone colors, to be used as background, foreground, and content tones:
>>  BLACK, CHARCHOL, DARK_GREY, BLUE_GREY, GREY, LIGHT_GREY,  BEIGE, WHITE
>> these are my suggestions, any other suggestions are welcome.
> 
> Sounds good.
> 
> Two other areas that need some thought are:
> * setting the background color
>  - to highlight something you've searched for?
>  - to highlight flows that trigger some alert?
> * using bold fonts
>  - to highlight things
>  (instead of using different fonts, perhaps all you need are two shades of each accent color, where the "bold" one would be slightly lighter than the normal color)
> 
> 
>> Using these standard color names, we can support an address configuration file that looks like:
>> 0.0.0.0/8-192.167.255.255/32    Internet  ORANGE
>> 192.168.3.0/24-223.0.0.0/8      Internet  ORANGE
>> 224.0.0.0-232.255.255.255       Multicast  BLUE_GREY
>> 255.255.255.255                 Broadcast  DARK_GREY
>> 192.168.0.0/16    QoSient             GREEN
>> 192.168.0.0/24    Wired               BLUE
>> 192.168.0.67      SMTP                BLUE
>> 192.168.1.0/24    Switzerland         BLUE
>> 192.168.2.0/24    Wireless            BLUE
>> 207.237.36.98     Public QoSient.com  CYAN
> 
> Sounds good.
> 
> 
>> So, with this, we aren't supporting a standard semantic for remote, local, me, they,
>> but you can set the color to the ranges, and you can say what the semantics really
>> are??
> 
> The above is enough for me.
> 
> 
>> Got to think about  the " auto " mode, but its on the feature list.
> 
> Perhaps you can look at commands like 'ls' and steal some ideas from there. :-)
> 
> /Elof
> 
> 
>> On Oct 22, 2012, at 11:45 AM, elof2 at sentor.se wrote:
>> 
>>> 
>>>> To turn color on, there is a new .rarc variable, and it takes yes and no for an answer.
>>>> RA_COLOR_SUPPORT="yes"
>>> 
>>> I would request an "auto" mode for RA_COLOR_SUPPORT, so ra* commands run via cron etc will yield plain text while ra* commands run in an interactive tty yield color output.
>>> 
>>>> Some sites, based on observation domain, would probably want to use L2 ethernet addresses,
>>>> or source MPLS or VLAN tags to say what is local.  And I suspect some would like to have
>>>> an extensive list of CIDR addresses, and want to put a " color to use " in the configuration.
>>> 
>>> Yes, extensive list of CIDR addresses + a " color to use " in the configuration would be great.
>>> 
>>>> I suspect that coloring the network part, and the host part of an address different colors would
>>>> be interesting… but not sure if that is useful.
>>> 
>>> Nah, I don't see a scenario where this kind of coloring would be needed.
>>> 
>>>> So, if there are opinions on color, and how to use it, I'd love to hear about it before
>>>> I release the color version of ratop().
>>> 
>>> Personally I've been using color like this:
>>> * light green = IP belong to the customer and is site-local to the argus sensor (or in some times local to the person creating the report).
>>> 
>>> * green = IP belong to the customer but is remote, like a server at a remote office
>>> 
>>> * red = IP does not belong to the customer, i.e. it is external.
>>> 
>>> * purple = IP should be treated as if it were external, like the inside of a NAT:ing loadbalancer.
>>> 
>>> * cyan = excluded IP
>>> 
>>> 
>>> 
>>> Examples:
>>> My customer has two offices and one DC.
>>> 10.10.0.0/16 = Office A
>>> 10.20.0.0/16 = Office B
>>> 172.16.0.0/24 = Private DMZ at DC
>>> 111.111.111.0/24 = Public DMZ at DC
>>> 
>>> A GET-request to the internet from a client at Office A:
>>> The argus sensor is listening to traffic at Office A.
>>> 10.10.0.50 -> 123.123.123.123 80
>>> light green       red
>>> 
>>> 
>>> A GET-request to a server at DC, from a client at Office A:
>>> The argus sensor is listening to traffic at Office A.
>>> I haven't bothered with coloring public and private customer IPs differently.
>>> 10.10.0.50 -> 111.111.111.10 80   or
>>> light green      green
>>> 10.10.0.50 -> 172.16.0.10 80
>>> light green      green
>>> 
>>> 
>>> A GET-request to a server at DC, from a client on the internet:
>>> The argus sensor is listening to traffic at DC.
>>> 123.123.123.123 -> 111.111.111.10 80
>>>    red               green
>>> 
>>> 
>>> An incoming GET-request from the internet to a web-server on the private DMZ has its src IP source-NATed by the lb like this:
>>> Original packet: 123.123.123.123 -> 111.111.111.10 80
>>> After the lb:    172.16.0.1 -> 172.16.0.10 80
>>> 
>>> The argus sensor is listening to traffic at DC (inside the lb).
>>> 172.16.0.1 -> 172.16.0.10 80
>>> purple      light green
>>> Even though this traffic physically come from 172.16 and go to 172.16, i.e. from and to the customer, external configuration stating that 172.16.0.1 should be treated as if it was an external host colors this flow as purple -> light green instead of light green -> light green.
>>> 
>>> 
>>> A customer network technician login to my customer portal. In his account profile it says he belongs to Office B. When he looks at the reports, IPs local to him (i.e. Office B) is shown in light green while Office A and DC IPs are colored in normal green.
>>> 
>>> 
>>> Cyan should normally not be seen, but in some cases the customer SPAN is forced to include some garbage that you really don't want to see (and that you won't filter out via bpf). I simply color this in cyan to distinguish this traffic from the green and red sides.
>>> 
>>> 
>>> So everything that has to do with the customer traffic use different shades of green and everything that has to do with the evil Internet use red shades throughout all reporting systems.
>>> 
>>> 
>>> 
>>> That's how I roll... :-)
>>> 
>>> /Elof
>> 