Argus memory issues
Peter Van Epp
vanepp at sfu.ca
Sun Aug 19 23:34:47 EDT 2007
On Sun, Aug 19, 2007 at 11:00:07PM -0400, Carter Bullard wrote:
> So it looks like we could be just processing a massive amount of
> flows. I do have a fixed memory model for argus, so that we don't
> grow over a certain memory foot print, we'll aggressively reclaim
> records and hold low packet flows for only the status interval,
> (usually 5 secs), that may work very well for you.
>
> So what are we thinking then, are we not leaking memory and you've got
> a lot of flows? Should I switch to thinking about how to keep your
> argus better behaved?
>
> Carter
>
Yes sort of. The puzzling part is that my 2.0.6 system is still chugging
along fine on the same data (but without user data) that is currently choking
3.0 (at this point without threads, user data or even output to a client).
argus-3.0 (chosen from an hour that didn't seem to run out of memory :-)):
test4:/var/log/argus vanepp$ ra3 -r /archive/argus3/com_argus.archive/2007/08/15/com_argus.2007.08.15.14.00.00.0.gz -n man
07-08-15 14:00:24 man 1172637 0 2786116 2 980417 27043 2786116 2980874864 CON
07-08-15 14:01:24 man 1195209 0 2840347 2 966201 27590 2840347 3040374744 CON
07-08-15 14:02:24 man 1217026 0 2891733 2 1000500 27616 2891733 3098033784 CON
07-08-15 14:03:24 man 1239216 0 2944188 2 1000624 27052 2944188 3155959864 CON
07-08-15 14:04:24 man 1259885 0 2993361 2 1045410 26387 2993361 3210878984 CON
07-08-15 14:05:24 man 1286106 0 3054195 2 1035640 27954 3054195 3275422984 CON
07-08-15 14:06:24 man 1306951 0 3102666 2 941859 28763 3102666 3331541144 CON
07-08-15 14:07:24 man 1327541 0 3150287 2 976290 29432 3150287 3387613104 CON
07-08-15 14:08:24 man 1354025 0 3205331 2 969769 30268 3205331 3450989784 CON
07-08-15 14:09:24 man 1374984 0 3253937 2 970571 28929 3253937 3507411144 CON
07-08-15 14:10:24 man 1393511 0 3297526 2 922896 27879 3297526 3559330344 CON
07-08-15 14:11:24 man 1412644 0 3341769 2 911490 29793 3341769 3612703624 CON
07-08-15 14:12:24 man 1432303 0 3388174 2 950633 28603 3388174 3667129144 CON
07-08-15 14:13:24 man 1452892 0 3436351 2 952619 29755 3436351 3723621824 CON
07-08-15 14:14:24 man 1472367 0 3481550 2 968650 29561 3481550 3777819144 CON
07-08-15 14:15:24 man 1492529 0 3529187 2 1011118 28944 3529187 3833547544 CON
07-08-15 14:16:24 man 1513232 0 3577628 2 992951 27440 3577628 3889140944 CON
07-08-15 14:17:24 man 1534016 0 3626696 2 1014643 27236 3626696 3945081104 CON
07-08-15 14:18:24 man 1555291 0 3676958 2 1037774 26918 3676958 4001647144 CON
07-08-15 14:19:24 man 1576296 0 3726748 2 1032067 26140 3726748 4057477624 CON
07-08-15 14:20:24 man 1595717 0 3773567 2 982867 26605 3773567 4111139264 CON
07-08-15 14:21:24 man 1614388 0 3817237 2 1001749 25952 3817237 4162150304 CON
07-08-15 14:22:24 man 1633226 0 3862001 2 1043290 25980 3862001 4213899504 CON
07-08-15 14:23:24 man 1653651 0 3910911 2 981144 29099 3910911 4270437224 CON
07-08-15 14:24:24 man 1674914 0 3961760 2 951822 29301 3961760 33357568 CON
07-08-15 14:25:24 man 1694708 0 4007603 2 959291 28480 4007603 87145088 CON
07-08-15 14:26:24 man 1715179 0 4055899 2 918180 28432 4055899 142542528 CON
07-08-15 14:27:24 man 1735529 0 4103903 2 930887 28628 4103903 197986648 CON
07-08-15 14:28:24 man 1756438 0 4153460 2 964800 29293 4153460 254923688 CON
07-08-15 14:29:24 man 1775701 0 4198996 2 861857 28488 4198996 308770728 CON
07-08-15 14:30:24 man 1795511 0 4246466 2 905501 28822 4246466 364036888 CON
07-08-15 14:31:24 man 1815059 0 4293349 2 842655 25969 4293349 417321168 CON
07-08-15 14:32:24 man 1834548 0 4340388 2 805952 25792 4340388 470724008 CON
07-08-15 14:33:24 man 1854168 0 4387327 2 814452 25715 4387327 524127728 CON
07-08-15 14:34:24 man 1873533 0 4433935 2 830901 25440 4433935 576737568 CON
07-08-15 14:35:24 man 1892766 0 4480385 2 841707 25568 4480385 629654888 CON
07-08-15 14:36:24 man 1911651 0 4525593 2 922177 26136 4525593 681994888 CON
07-08-15 14:37:24 man 1930756 0 4571308 2 966213 26494 4571308 734984768 CON
07-08-15 14:38:24 man 1955339 0 4627545 2 989488 25548 4627545 795340128 CON
07-08-15 14:39:24 man 1974897 0 4673706 2 896919 25180 4673706 847976928 CON
07-08-15 14:40:24 man 1994202 0 4718619 2 792368 26060 4718619 900271048 CON
07-08-15 14:41:24 man 2014358 0 4766426 2 836465 28547 4766426 955643808 CON
07-08-15 14:42:24 man 2032739 0 4809490 2 876541 26979 4809490 1006861768 CON
07-08-15 14:43:24 man 2052275 0 4855722 2 853278 28120 4855722 1060911008 CON
07-08-15 14:44:24 man 2071728 0 4901761 2 797199 28042 4901761 1114472568 CON
07-08-15 14:45:24 man 2092752 0 4951286 2 889380 29089 4951286 1171663688 CON
07-08-15 14:46:24 man 2113794 0 4999579 2 899038 29712 4999579 1228471528 CON
07-08-15 14:47:24 man 2133195 0 5045435 2 817684 29277 5045435 1283055208 CON
07-08-15 14:48:24 man 2153557 0 5094075 2 867185 29250 5094075 1339515288 CON
07-08-15 14:49:24 man 2173843 0 5143886 2 886131 28150 5143886 1395881088 CON
07-08-15 14:50:24 man 2193392 0 5190511 2 898308 27491 5190511 1450181448 CON
07-08-15 14:51:24 man 2210999 0 5232842 2 854348 25653 5232842 1500051688 CON
07-08-15 14:52:24 man 2228308 0 5274645 2 824314 24738 5274645 1548967728 CON
07-08-15 14:53:24 man 2246959 0 5320603 2 877264 25841 5320603 1601377368 CON
07-08-15 14:54:24 man 2266623 0 5368579 2 878698 25483 5368579 1654951448 CON
07-08-15 14:55:24 man 2285971 0 5415552 2 926918 25879 5415552 1707955448 CON
07-08-15 14:56:24 man 2301960 0 5445780 2 389347 15687 5445780 1742634408 CON
07-08-15 14:57:24 man 2314613 0 5466755 2 147994 10038 5466755 1765850208 CON
07-08-15 14:58:24 man 2325099 0 5484015 2 97277 7996 5484015 1784658408 CON
07-08-15 14:59:24 man 2333757 0 5498041 2 68135 6704 5498041 1799888208 CON
test4:/var/log/argus vanepp$
The same hour from the 2.0.6 sensor (which is listening on the same regen
tap as 3.0) the flows should be the same but 2.0.6 is handling it and 3.0 isn't
(and the 2.0.6 sensor has only 1 gig of memory not 4)
nepp at sniffer:/var/log/argus> ra -r /usr/local/argus/com_argus.archive/2007/08/15/com_argus.2007.08.15.14.00.00.0.gz -nn man
22 Jun 07 07:36:23 man 229.97.122.203 v2.0 1 0 0 0 0 0 STA
15 Aug 07 13:56:27 man 229.97.122.203 v2.0 2299836494 1308855001157 0 3426164216 130783 CON
15 Aug 07 14:01:27 man 229.97.122.203 v2.0 2300022324 1242885022304 0 3454202706 139418 CON
15 Aug 07 14:06:27 man 229.97.122.203 v2.0 2300205390 1201094752165 0 3171199840 135152 CON
15 Aug 07 14:11:27 man 229.97.122.203 v2.0 2300376928 1220234876630 0 3360631524 122151 CON
15 Aug 07 14:16:27 man 229.97.122.203 v2.0 2300552396 1232065068302 0 3576425899 122621 CON
15 Aug 07 14:21:27 man 229.97.122.203 v2.0 2300728449 1211074847986 0 3346915526 129684 CON
15 Aug 07 14:26:27 man 229.97.122.203 v2.0 2300904485 1200494506089 0 2933591340 127401 CON
15 Aug 07 14:31:27 man 229.97.122.203 v2.0 2301070396 1197084221477 0 2643569732 119551 CON
15 Aug 07 14:36:27 man 229.97.122.203 v2.0 2301243118 1203024476135 0 2957584557 125656 CON
15 Aug 07 14:41:27 man 229.97.122.203 v2.0 2301419626 1213234313786 0 2692091848 126701 CON
15 Aug 07 14:46:27 man 229.97.122.203 v2.0 2301595126 1227394322412 0 2660038990 122037 CON
15 Aug 07 14:51:27 man 229.97.122.203 v2.0 2301758903 1228524444856 0 2784876374 118980 CON
vanepp at sniffer:/var/log/argus>
the 2.0.6 sensor has a much smaller footprint on the same traffic:
%!ps
ps auxwwww | grep argus
root 944 2.0 20.5 215068 214168 ?? Ss 22Jun07 3162:09.10 /usr/local/bin/argus_bpf -dJR -P 561 -i em2 -i em3
I'm wondering if 3.0 is keeping more flows open for some reason and
thus eating a lot more memory (or is failing to close flows it should be and
running out of memory because of it). At this point it looks like the memory
allocation is pretty clean just something is using too much of it.
Peter Van Epp / Operations and Technical Support
Simon Fraser University, Burnaby, B.C. Canada
More information about the argus
mailing list