[Fwd: Broken Reset Filter, possible bug]

Wed Mar 12 08:15:56 EDT 2008

Hey Nick,
Can you send me a tcpdump file of the packet that you printed?  That's all I need to fix this problem.

Which one has the correct time, tcpdump or argus?

Microsoft tends to send lone RESETs long after the flow has closed and gone.  Argus has forgotten the original flow,  and so it can only report that there was a lone reaet.
Racluster() will merge the RESETs back into the original flow.

Carter

Carter Bullard
QoSient LLC
150 E. 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax

-----Original Message-----
From: Nick Diel <ndiel at engr.colostate.edu>

Date: Tue, 11 Mar 2008 16:22:56 
To:Carter Bullard <carter at qosient.com>
Subject: Re: [ARGUS] [Fwd: Broken Reset Filter, possible bug]

Carter,

 It looks like most cases are fixed, but there still seems to be a problem when a reset packet is the first packet of a status flow.  The "not reset" filter does not work for these flows.

 In my case most of these flows are unidirectional traffic and I am seeing the second half of a packet scan (so single packet flows).  Of course these flows may also be backscatter or just happens to be where argus started another status record.

 Here is an example of where argus started a new status record.  What is also interesting is argus did not pickup on them synack.

 [diel at lander-nic ~]$ ra -nn -r /capture/dump-110707/pcap/*.argus.out -  host 164.109.105.136 and host 63.63.36.131 and port 56959
    09:13:51.068894  e           6       63.63.36.131.56959     ->    164.109.105.136.25            2        120   ACC
    09:13:51.462053  e           6    164.109.105.136.25        ->       63.63.36.131.56959        28       3604   RST
 [diel at lander-nic ~]$ ra -z -nn -r /capture/dump-110707/pcap/*.argus.out -  host 164.109.105.136 and host 63.63.36.131 and port 56959
    09:13:51.068894  e           6       63.63.36.131.56959     ->    164.109.105.136.25            2        120    sS
    09:13:51.462053  e           6    164.109.105.136.25        ->       63.63.36.131.56959        28       3604    sR

 tcpdump output for the flow:
 10:13:51.068894 IP 63.63.36.131.56959 > 164.109.105.136.25: S 51058137:51058137(0) win 24000 <mss 536>
 10:13:51.082038 IP 164.109.105.136.25 > 63.63.36.131.56959: S 3373374872:3373374872(0) ack 51058138 win 1380 <mss 1380>
 10:13:51.462053 IP 63.63.36.131.56959 > 164.109.105.136.25: R 51058138:51058138(0) win 0
 10:13:53.670082 IP 63.63.36.131.56959 > 164.109.105.136.25: S 51058137:51058137(0) win 24000 <mss 536>
 10:13:53.681723 IP 164.109.105.136.25 > 63.63.36.131.56959: S 3059316086:3059316086(0) ack 51058138 win 1380 <mss 1380>
 10:13:53.907031 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 1 win 24000
 10:13:53.923542 IP 164.109.105.136.25 > 63.63.36.131.56959: P 1:58(57) ack 1 win 65535
 10:13:54.109642 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 58 win 24000
 10:13:54.217980 IP 63.63.36.131.56959 > 164.109.105.136.25: P 1:19(18) ack 58 win 24000
 10:13:54.236573 IP 164.109.105.136.25 > 63.63.36.131.56959: P 58:86(28) ack 19 win 65517
 10:13:54.425915 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 86 win 24000
 10:13:54.436759 IP 164.109.105.136.25 > 63.63.36.131.56959: P 86:161(75) ack 19 win 65517
 10:13:54.623756 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 161 win 24000
 10:13:54.699171 IP 63.63.36.131.56959 > 164.109.105.136.25: P 19:103(84) ack 161 win 24000
 10:13:54.714101 IP 164.109.105.136.25 > 63.63.36.131.56959: P 161:213(52) ack 103 win 65433
 10:13:55.052449 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 213 win 24000
 10:13:55.064664 IP 164.109.105.136.25 > 63.63.36.131.56959: P 213:270(57) ack 103 win 65433
 10:13:55.264147 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 270 win 24000
 10:13:55.280955 IP 63.63.36.131.56959 > 164.109.105.136.25: . 103:639(536) ack 270 win 24000
 10:13:55.319876 IP 63.63.36.131.56959 > 164.109.105.136.25: . 639:1175(536) ack 270 win 24000
 10:13:55.331134 IP 164.109.105.136.25 > 63.63.36.131.56959: . ack 1175 win 65535
 10:13:55.520197 IP 63.63.36.131.56959 > 164.109.105.136.25: P 1175:1685(510) ack 270 win 24000
 10:13:55.548274 IP 164.109.105.136.25 > 63.63.36.131.56959: P 270:298(28) ack 1685 win 65025
 10:13:55.743295 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 298 win 24000
 10:13:55.743823 IP 63.63.36.131.56959 > 164.109.105.136.25: P 1685:1691(6) ack 298 win 24000
 10:13:55.759119 IP 164.109.105.136.25 > 63.63.36.131.56959: P 298:313(15) ack 1691 win 65019
 10:13:55.762272 IP 164.109.105.136.25 > 63.63.36.131.56959: F 313:313(0) ack 1691 win 65019
 10:13:55.947930 IP 63.63.36.131.56959 > 164.109.105.136.25: . ack 313 win 23985
 10:13:55.949641 IP 63.63.36.131.56959 > 164.109.105.136.25: R 1691:1691(0) ack 313 win 24000
 10:13:55.958569 IP 63.63.36.131.56959 > 164.109.105.136.25: R 1691:1691(0) ack 314 win 24000

 Nick
 PS any pointers on getting tcpdump and argus to resolve to same time zone?

 Carter Bullard wrote: Hey Nick, 
So I have a new argus and clients up on the server. 
This should fix problems from argus()'s perspective. 
I'll add code later this week to fix ra* programs so they 
can deal with the bug. 

Carter

On Mar 10, 2008, at 7:00 PM, Nick Diel wrote: 

 Carter,

 Those packets were extracted from a larger pcap file.  Maybe they need to be part of a larger pcap file to generate the problem.  If you want to evaluate it more, I can do more testing for you.  I can also look into getting you the original large pcap file (500mb).  Though if the clients handle the issue fine, maybe you don't mind it occurring.

 Thanks for state keyword.  It looks like it is listed as status in the man page in the detailed/spelled out list and state right above in the condensed list.

 Thanks for your help,
 Nick

 Carter Bullard wrote: 

Hey Nick, 
These packets actually don't demonstrate the bug, but the fixed code 
at least  does the right thing, so at least I didn't break anything. 

The keyword is "state" not "status". 

Carter 

On Mar 10, 2008, at 4:00 PM, Nick Diel wrote: 

 Carter,

 I will test it tonight.  Also I have included a pcap that has a flow in it that generated an argus record in question.  I didn't want to put the pcap file on the mailing list so I am sending it only to you.  It is anonymized and has it data stripped, but still want to limit where the flow goes.  Flows that just have a single reset packet in them also seemed likely to cause an issue in Argus (reset filter doesn't work on them).

 Also after first looks it looks like -s status does not work (was trying to make it work for finding flows that are causing the above problem).  Though I haven't tested it completely.

 Nick

 Carter Bullard wrote: So I have a fix, or at least something that should fix the problem. 
I would rather fix argus, than put some bandaid in the client programs, 
but because I don't have any packets to test, we'll have to just put 
it in place and see what happens.  I will clean up the filters as well, 
but they actually look ok at this point. 

I'll put up a new argus and clients tonight, if you could pick it 
up and give it a run, that would be great.  I will later put in a fix 
to correct the clients so that the historical data will work as well. 

Carter 

On Mar 10, 2008, at 3:16 PM, Nick Diel wrote: 

 The ones below were aggregated, but I checked my original argus files (non aggregated) and found they contain the same problem.  I generated my original argus files by reading pcap files, perhaps that is where the problem lies?

 Do you need a sample pcap file that generated one of these flows?

 Nick

 Carter Bullard wrote: Hey Nick, 
Hmmmm, looks like its an argus bug, in that we have a flow 
status indication of reset, but we don't have the TH_RST bit 
flipped in the flags field, which is what the filter is looking at. 

So the question is, did these records come straight from argus() 
or did they get aggregated along the way? 

Carter 

On Mar 10, 2008, at 1:03 PM, Nick Diel wrote: 

 I am not sure if this lists block emails with attachments (my original didn't go through).  So I am sending this without the attachment.  Let me know if I need to send attachments through other channels.

 Nick

 -------- Original Message -------- 
 Subject: Broken Reset Filter, possible bug 
 Date: Mon, 10 Mar 2008 10:37:22 -0600 
 From: Nick Diel <ndiel at engr.colostate.edu> <mailto:ndiel at engr.colostate.edu> 
 To: argus-info at lists.andrew.cmu.edu <mailto:argus-info at lists.andrew.cmu.edu> 

 It appears that the filter "not reset" only works some of the time. I noticed this trying to use the filter: "not ((syn or synack) and (fin or finack or reset)." Then doing the following command gave me lots of entries with resets. diel at morte:~$ ra -r /lander/dump-110707/argus/port80/argus.out -z - "not ((syn or synack) and (fin or finack or reset))" | egrep "(s|S).*R" 09:01:29.947638 e tcp 184.3.136.54.3000 -> 87.123.62.161.www 9 3331 sSER 09:01:29.949058 e tcp 184.3.136.54.3001 -> 87.123.62.149.www 9 3545 sSER 09:01:29.950426 e tcp 184.3.136.54.3002 -> 87.123.62.169.www 17 7560 sSER 09:01:29.952302 e tcp 165.72.129.19.1170 -> 34.208.115.191.www 27 7275 sER 09:01:29.958305 e d tcp 184.3.136.54.3003 -> 87.123.62.145.www 12 4352 sSER 09:01:29.959940 e tcp 184.3.136.54.3004 -> 87.123.62.134.www 12 6051 sSER 09:01:29.960089 e tcp 184.3.136.54.3005 -> 87.123.62.148.www 12 6247 sSER 09:01:29.968459 e tcp 164.34.224.106.3188 -> 201.79.50.56.www 22 13350 sSER 09:01:29.997532 e tcp 165.1.181.212.3415 -> 50.17.254.186.www 5 1076 sER 09:01:30.032242 e tcp 164.109.105.182.www -> 35.69.108.245.11889 2 120 SR 09:01:30.044153 e tcp 165.1.181.212.3416 -> 50.17.254.187.www 5 1076 sER 09:01:30.067214 e tcp 111.237.214.170.www -> 157.216.170.55.2676 3 180 SR 09:01:30.089935 e s tcp 184.3.186.45.2907 <?> 192.48.235.79.www 188 107626 EfR 09:01:30.100910 e tcp 166.240.172.90.59566 -> 87.123.62.128.www 13 6634 sSER 09:01:30.103517 e tcp 164.61.150.68.3211 -> 110.11.67.104.www 12 4974 sSER 09:01:30.115002 e tcp 166.240.172.90.59567 -> 87.123.62.187.www 20 10100 sSER 09:01:30.136030 e tcp 154.188.62.27.44895 -> 192.48.236.65.www 45 25645 sER 09:01:30.196479 e tcp 164.34.106.254.4943 -> 111.101.7.193.www 94 89890 sSER 09:01:30.203517 e tcp 37.41.109.12.www -> 161.164.184.133.49588 26 8562 SER 09:01:30.219369 e tcp 138.123.105.215.3823 -> 164.34.222.121.www 474 39800 sER 09:01:30.227540 e tcp 110.87.125.122.1297 -> 164.61.200.76.www 7 2018 sSER 09:01:30.262842 e tcp 36.177.74.81.www -> 184.3.145.197.1789 13 13719 SEfR 09:01:30.309841 e tcp 36.7.245.63.www -> 161.164.204.185.4015 4 1043 SER 09:01:30.346086 e tcp 164.109.84.242.www -> 93.188.153.157.1149 35 24250 SER 09:01:30.398819 e tcp 154.188.84.193.35920 -> 36.132.97.243.www 7 1326 sSER 09:01:30.422082 eI S tcp 62.122.56.21.1760 ?> 165.72.184.212.www 4 240 ER 09:01:30.476588 e tcp 110.87.125.122.14990 -> 164.61.200.229.www 66 59404 sSER 09:01:30.477389 e tcp 186.218.90.219.8462 -> 37.28.161.174.www 10 3217 sSER 09:01:30.493936 e tcp 186.218.123.100.57084 -> 192.48.236.127.www 11 3698 sER 09:01:30.528469 e tcp 157.216.249.22.3938 -> 201.79.50.57.www 16 9274 sSER 09:01:30.574783 e tcp 165.72.159.144.49402 -> 192.48.236.205.www 7 3187 sER 09:01:30.608215 e d tcp 161.164.237.92.2270 -> 192.73.184.102.www 42 33320 sSER .......... I have attached a file with a few flow entries in them that have this behavior. Notice this unusual output with this sample file: diel at morte:~$ ra -r resetExample.argus -z 09:01:44.589353 e tcp 164.109.174.238.42969 -> 36.174.95.100.www 34 20482 sSER 09:01:44.925242 e tcp 164.109.174.238.42970 -> 36.174.95.100.www 74 38620 sSER 09:01:45.112644 e tcp 164.109.174.238.42971 -> 36.174.95.100.www 69 36967 sSER 09:02:12.675158 e tcp 164.109.174.238.42973 -> 36.174.95.100.www 108 68948 sSER 09:03:41.503642 e tcp 164.109.174.238.42975 -> 36.174.95.100.www 51 33970 sSER 09:06:27.431113 e tcp 164.109.174.238.42998 -> 36.174.95.100.www 7 2681 sSER versus diel at morte:~$ ra -r resetExample.argus -z - not reset 09:01:44.589353 e tcp 164.109.174.238.42969 -> 36.174.95.100.www 34 20482 sSER 09:02:12.675158 e tcp 164.109.174.238.42973 -> 36.174.95.100.www 108 68948 sSER 09:03:41.503642 e tcp 164.109.174.238.42975 -> 36.174.95.100.www 51 33970 sSER Nick 

 <samplePackets.pcap> 