Argus 3.0 / 32-bit overflow / backward compatibility

[Carter Bullard]

Hey Ilija,
Argus itself doesn't support 64-bit counters, but the clients do.  It is the difference between flow
data generation and flow data processing.

We do not recommend running argus such that it holds flow information for very large periods of time.
I run with ARGUS_FLOW_STATUS_INTERVAL set to 1s or 5s in every installation I do, and as
a result the 32-bit counter strategy has worked quite well.  For 10-100 Gbps interface monitoring,
I think the 32-bit value is still the way to go, with the clients aggregating the status reports to generate the
final flow record, where we do support 64-bit reporting.

If you think this is a flaw, then lets discuss.  Adding 64-bit counters will increase the flow cache
for every flow tracked by 48 bytes (src and dst pkt, byte, and appbyte counters) and for TCP we'll need to
add another 64 bytes or so, as we have a lot of other counters.  Can be a huge deal if we're tracking
1M flows, but our current cache size is pretty big, when you add jitter and other features.

Not sure what is up with your argus-2.0 data, as we should be able to read it with out issue.
argus-clients-3.0.5.19, in the developers section, may be able to read it fine, if there was a but.
If you can share the 2.x file, I'll check out what might be the issue.

Carter

On Sep 22, 2011, at 1:52 PM, Ilija Baniski wrote:

> Hello all,
> 
> I am doing some testing prior to upgrading to 3.0. 64-bit byte counters are an important reason for the upgrade, while the ability of the 3.0 clients to read argus 2.0 data make the switch that much sweeter. However, I haven't been able to get either of these two features to work. I have the latest argus-3.0.4 and argus-clients-3.0.4.1 built. I know these two are not very closely related, but I'll ask both questions here.
> 
> I read that these two features should be supported here:
> http://permalink.gmane.org/gmane.network.argus/3601
> http://article.gmane.org/gmane.network.argus/3782
> 
> 1.
> To test the 4GB byte overflow I performed a ~5GB file transfer twice: once with STATUS_INTERVAL=60 and then with STATUS_INTERVAL=600. These were the results:
> 
> # ./ra -Xnr /opt/data/argus/var/log/argus.out - host 192.168.111.39
>   11:45:51.241371  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       674666  691670398   CON
>   11:46:51.241496  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       728665  746638654   CON
>   11:47:51.241563  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       729780  748390504   CON
>   11:48:51.241597  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       733293  752190798   CON
>   11:49:51.241631  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       733537  752519850   CON
>   11:50:51.241659  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       734255  752514878   CON
>   11:51:51.241729  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       732836  751412980   CON
>   11:52:51.241770  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22       374739  384106534   FIN
> # ./racluster -Xnr /opt/data/argus/var/log/argus.out - port 49460
>   11:45:51.241371  e d       tcp    192.168.111.137.49460     ->     192.168.111.39.22      5441771 5579444596   FIN
> 
> Which shows the transfer of the ~5GB file. Now with only a single interval for the transfer:
> 
> # ./ra -Xnr /opt/data/argus/var/log/argus.out - host 192.168.111.39
>   11:33:23.249123  e d       tcp    192.168.111.137.47119     ->     192.168.111.39.22      5443943 1284774452   FIN
> # ./racluster -Xnr /opt/data/argus/var/log/argus.out - port 47119
>   11:33:23.249123  e d       tcp    192.168.111.137.47119     ->     192.168.111.39.22      5443943 1284774452   FIN
> 
> Which shows that the byte count has wrapped after 4GB (1284774452 + 4x1024x1024x1024 = 5579741748).
> 
> So I guess the question is, what am I doing wrong? Do I need to compile argus in a certain way or configure it differently? Or perhaps this is related to the clients (unlikely since racluster seems to have no problem showing the 5GB if the data is there).
> 
> 
> 2.
> To check the backward compatibility I used the ra tool from the argus-clients-3.0.4.1 to try and read a file written by argus 2.0. Here are some output snippets of me trying to read the file with both, an old client and new one:
> 
> The old:
> # ra -h
> Ra Version 2.0.6.fixes.1
> ...
> # ra -r argus.2011-09-13-07-00-01.gz | head -2
> 09-07-11 11:18:07.535387           man                    10.2.1.37  v2.0                                     1 0          0        0         0            0           STA
> 09-13-11 02:49:01.914562           tcp                    10.2.2.86.8280        ->                69.184.252.19.8292       3        6         384          435         CON
> 
> and the new:
> # ./ra -Xh
> Ra Version 3.0.4.1
> ...
> # ./ra -Xr argus.2011-09-13-07-00-01.gz
> <no output at all>
> 
> So back to the same questions: what am I doing wrong/how can I use the new clients to read the old data? In my random attempts to make this happen I did try to unzip the file first, but that made no difference.
> 
> 
> Thanks for any help,
> Ilija
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20110922/a8c2906b/attachment.bin>