Argus 3.0 / 32-bit overflow / backward compatibility

Carter Bullard carter at
Thu Sep 22 22:52:36 EDT 2011

Hey Ilija,
I've checked every version of argus-2.x that I have, and non of the client programs
I have can read the argus-2 file that you sent.  The starting MAR record in your
file is too big.  Its 144 bytes long when its suppose to be 128 bytes.

Did you compile this on a 64-bit machine?  Reason I ask, argus-2.x was not
64-bit ready, and compiling on 64-bit would cause struct size problems.
Clients compiled on the same architecture would be able to read flow data
generated by a 'broken' argus, as they would have the same size problems,
but the resulting files would not be portable, even to 32-bit argus-2.x programs.

That is my best guess as to why your argus-2.x files aren't working with argus-3.x,
but its just a guess.

One thing you could do is write out the argus-2.x data in ascii, using your existing
argus-2.x clients, and then read convert that ascii back to argus-3.0 flow data
using raconvert()?


On Sep 22, 2011, at 3:36 PM, Ilija Baniski wrote:

> Hey Carter,
> Thanks for the quick response and provided insight. You are right, with the correct configuration 32-bit counters should be sufficient for the good majority of the cases. However, setting the ARGUS_FLOW_STATUS_INTERVAL to a very low value would be an unnecessary overhead for most flows, I think. So, how does an alternative configuration feature (or perhaps this is something that already exists and I have missed?) where argus will produce "flow status" on its own when it has a byte counter that is ready to overflow? Such an option should allow for less aggressively outputting to disk while making it harder to loose data by having a poorly configured argus.
> As for the backwards compatibility, unfortunately I did not have much luck even with the newest development release. I also ran it across a number of files in the argus archive I have available. I am attaching one of the smallest argus 2.x files that I have (unsuccessfully) tested against.
> Thanks,
> Ilija
> On 11-09-22 02:29 PM, Carter Bullard wrote:
>> Hey Ilija,
>> Argus itself doesn't support 64-bit counters, but the clients do.  It is the difference between flow
>> data generation and flow data processing.
>> We do not recommend running argus such that it holds flow information for very large periods of time.
>> I run with ARGUS_FLOW_STATUS_INTERVAL set to 1s or 5s in every installation I do, and as
>> a result the 32-bit counter strategy has worked quite well.  For 10-100 Gbps interface monitoring,
>> I think the 32-bit value is still the way to go, with the clients aggregating the status reports to generate the
>> final flow record, where we do support 64-bit reporting.
>> If you think this is a flaw, then lets discuss.  Adding 64-bit counters will increase the flow cache
>> for every flow tracked by 48 bytes (src and dst pkt, byte, and appbyte counters) and for TCP we'll need to
>> add another 64 bytes or so, as we have a lot of other counters.  Can be a huge deal if we're tracking
>> 1M flows, but our current cache size is pretty big, when you add jitter and other features.
>> Not sure what is up with your argus-2.0 data, as we should be able to read it with out issue.
>> argus-clients-, in the developers section, may be able to read it fine, if there was a but.
>> If you can share the 2.x file, I'll check out what might be the issue.
>> Carter
>> On Sep 22, 2011, at 1:52 PM, Ilija Baniski wrote:
>>> Hello all,
>>> I am doing some testing prior to upgrading to 3.0. 64-bit byte counters are an important reason for the upgrade, while the ability of the 3.0 clients to read argus 2.0 data make the switch that much sweeter. However, I haven't been able to get either of these two features to work. I have the latest argus-3.0.4 and argus-clients- built. I know these two are not very closely related, but I'll ask both questions here.
>>> I read that these two features should be supported here:
>>> 1.
>>> To test the 4GB byte overflow I performed a ~5GB file transfer twice: once with STATUS_INTERVAL=60 and then with STATUS_INTERVAL=600. These were the results:
>>> # ./ra -Xnr /opt/data/argus/var/log/argus.out - host
>>>   11:45:51.241371  e d       tcp     ->       674666  691670398   CON
>>>   11:46:51.241496  e d       tcp     ->       728665  746638654   CON
>>>   11:47:51.241563  e d       tcp     ->       729780  748390504   CON
>>>   11:48:51.241597  e d       tcp     ->       733293  752190798   CON
>>>   11:49:51.241631  e d       tcp     ->       733537  752519850   CON
>>>   11:50:51.241659  e d       tcp     ->       734255  752514878   CON
>>>   11:51:51.241729  e d       tcp     ->       732836  751412980   CON
>>>   11:52:51.241770  e d       tcp     ->       374739  384106534   FIN
>>> # ./racluster -Xnr /opt/data/argus/var/log/argus.out - port 49460
>>>   11:45:51.241371  e d       tcp     ->      5441771 5579444596   FIN
>>> Which shows the transfer of the ~5GB file. Now with only a single interval for the transfer:
>>> # ./ra -Xnr /opt/data/argus/var/log/argus.out - host
>>>   11:33:23.249123  e d       tcp     ->      5443943 1284774452   FIN
>>> # ./racluster -Xnr /opt/data/argus/var/log/argus.out - port 47119
>>>   11:33:23.249123  e d       tcp     ->      5443943 1284774452   FIN
>>> Which shows that the byte count has wrapped after 4GB (1284774452 + 4x1024x1024x1024 = 5579741748).
>>> So I guess the question is, what am I doing wrong? Do I need to compile argus in a certain way or configure it differently? Or perhaps this is related to the clients (unlikely since racluster seems to have no problem showing the 5GB if the data is there).
>>> 2.
>>> To check the backward compatibility I used the ra tool from the argus-clients- to try and read a file written by argus 2.0. Here are some output snippets of me trying to read the file with both, an old client and new one:
>>> The old:
>>> # ra -h
>>> Ra Version 2.0.6.fixes.1
>>> ...
>>> # ra -r argus.2011-09-13-07-00-01.gz | head -2
>>> 09-07-11 11:18:07.535387           man            v2.0                                     1 0          0        0         0            0           STA
>>> 09-13-11 02:49:01.914562           tcp                  ->              3        6         384          435         CON
>>> and the new:
>>> # ./ra -Xh
>>> Ra Version
>>> ...
>>> # ./ra -Xr argus.2011-09-13-07-00-01.gz
>>> <no output at all>
>>> So back to the same questions: what am I doing wrong/how can I use the new clients to read the old data? In my random attempts to make this happen I did try to unzip the file first, but that made no difference.
>>> Thanks for any help,
>>> Ilija
> <argus-sample.out>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 4367 bytes
Desc: not available
URL: <>

More information about the argus mailing list