[ARGUS] BSD argus/Linux ra problems?

[Michael Sanderson]

Peter Van Epp wrote:
> On Mon, Nov 22, 2004 at 02:34:36AM -0800, Michael Sanderson wrote:
> 
> 	Since Michael is just across town from me I gave him access to a couple
> of my test sensors. Core2 seems to like the SUSE 9 sensor but not the FreeBSD
> one, so it looks to be a Core2 to BSD issue OK, although the stuff below is
> fairly bizzare since I don't see why the debug level would sometimes make 
> things work correctly (unless this is a timing issue somehow).

This does appear to be a timing issue, but really it looks like a TCP window 
size issue and how ArgusReadConnection() does its reads.

Both SuSE and BSD servers do the expected SYN -> SYN/ACK -> ACK three way 
handshake for TCP connection setup.  What happens afterwards differs:

Here is a SuSE server responding (sorry for the line wrap).

23:31:44.424464 IP SuSE.561 > FC2-client.33521: P 1:129(128)
ack 1 win 5792 <nop,nop,timestamp 549562400 644656284>
23:31:44.424618 IP FC2-client.33521 > SuSE.561: . ack 129 win 46 
<nop,nop,timestamp 644656286 549562400>
23:31:45.544900 IP SuSE.561 > FC2-client.33521: P 129:261(132) ack 1 win 5792 
<nop,nop,timestamp 549563520 644656286>
23:31:45.544922 IP FC2-client.33521 > SuSE.561: . ack 261 win 46 
<nop,nop,timestamp 644657406 549563520>
23:31:45.823623 IP FC2-client.33521 > SuSE.561: F 1:1(0) ack
261 win 46 <nop,nop,timestamp 644657685 549563520>

The SuSE server sends us the full 128 bytes of data that ra needs for 
determining the kind of data flow and getting the initial MAR record. 
Everything is cool and off goes the client (killed pretty fast here, hence the 
FIN after only a single packet from the server).  Seems like SuSE is ignoring 
the window size.


Here is a BSD server:

23:33:34.949845 IP BSD.561 > FC2-client.33526: . 1:47(46) ack 1 win 57920 
<nop,nop,timestamp 420203931 644766815>
23:33:34.950016 IP FC2-client.33526 > BSD.561: . ack 47 win 46 
<nop,nop,timestamp 644766821 420203931>
23:33:34.950629 IP FC2-client.33526 > BSD.561: F 1:1(0) ack 47 win 46 
<nop,nop,timestamp 644766822 420203931>
23:33:34.951075 IP BSD.561 > FC2-client.33526: P 47:129(82) ack 1 win 57920 
<nop,nop,timestamp 420203931 644766821>
23:33:34.951117 IP FC2-client.33526 > BSD.561: R 1486636851:1486636851(0) win 0

Here we see that the BSD server/OS only sends 46 bytes of data initially.  16 
bytes of the first packet gets read at the beginning of ArgusReadConnection() 
leaving only 30 bytes to get read as the MAR data.  The current code says that 
this is too small (ie. not a whole MAR record), closes the connection (hence the 
FIN) and exits since there are no servers to read from.

I can work around it by introducing a sleep(1) in argus_parse.c'main() between 
the ArgusGetServerSocket() and ArgusReadConnection().  If I do that, I can 
connect to the servers on my own OpenBSD 3.6 box and Peter's FreeBSD 4.10 box.

This seems to be related to the FC2 box saying that it has a window size of 46 
in its acknowledgement of the TCP SYN/SYNACK.

23:31:44.422646 IP FC2-client.33521 > BSD.561: . ack 1 win 46 <nop,nop,timestamp 
644656284 549562398>

BSD seems to be honouring that window size (atleast for the first packet), while 
SuSE does not.  (Why FC2 is using a window size of 46 bytes is beyond me, though 
it does this for ssh connections coming out of that box as well.  Why BSD 
subsequently ignores the window size of 46 bytes is also beyond me.)

Carter, if I'm reading all of this right, then ArgusReadConnection(), 
particularly where it is dealing with the MAR record, should be made more 
robust, along the lines of ArgusReadStreamSocket.  There shouldn't be an 
assumption made that, if a positive number of bytes is read, the whole MAR 
record is available.

The flip side of this is that it is a weirdo case, very dependent upon Linux 
kernel release and what OS the argus sensor runs on, possibly even the amount of 
memory in the client machine (since window size is related to available receive 
buffer space, which is related to total memory in the box).

-- 
Michael Sanderson                   sanders at cs.ubc.ca
UBC Computer Science		http://www.cs.ubc.ca/spider/sanders/
					604 822 6194