rc.28 code on the server
carter at qosient.com
carter at qosient.com
Wed Aug 30 06:42:38 EDT 2006
Hey peter,
Can you grab packets for some of the reversed flows, so we can see which argus is correct?
Carter
Carter Bullard
QoSient LLC
150 E. 57th Street Suite 12D
New York, New York 10022
+1 212 588-9133 Phone
+1 212 588-9134 Fax
-----Original Message-----
From: Peter Van Epp <vanepp at sfu.ca>
Date: Tue, 29 Aug 2006 21:22:24
To:argus-info at lists.andrew.cmu.edu
Subject: [ARGUS] Re: rc.28 code on the server
rc.28 seems to be in reasonably good shape. It inverts some flows so
I needed to invert them back in the perl script to do the comparison, at which
point app bytes looks to be just plain wrong, and the jitters and vlans didn't
get reversed with the rest of the fields:
./ra_test.pl vlan.argus
sabytes 127 672256967
dabytes 206 671891464
sjit 1745403.155905
djit 1745403.16
svlan 0x0000 0x00d3
dvlan 0x00d3 0x0000
line: 1 fields in error: djit,sabytes,dabytes,dvlan,sjit,svlan,
1151432428.958661,1151432675.930102,1,246.971441,246.971441,142.78.66.138,142.58.211.41,tcp,4206,210,0,0,255,255,554,541,206,127,6,7,17.95,17.52,0.02,0.03,0.0000,0.0000,3848370891,q,0:0:20:ab:e6:73,8:11:88:5:5d:1d,->,,1745403.155905,FIN,,d[16]=".l..........>...",64733,0,9305,,,0x00d3,0x0000,0xcfbc
1151432428.958661,1151432675.930102,1,246.971441,246.971436,142.58.211.41,142.78.66.138,tcp,210,4206,0,0,255,255,541,554,672256967,671891464,7,6,17.524,17.945,0.028,0.024,0,0,229.97.122.203, v ,8:11:88:5:5d:1d,0:0:20:ab:e6:73,->,,1745403.16,FIN,s[16]=".l..........>...",,0,64733,9305,,,0x00d3,0x0000,0xcfbc,0xcfbc
but other than that it looks happy. Below is a copy of the new perl
script (with the note the the test 2.0.6 ra is now called ra2):
#!/usr/bin/perl
if ($#ARGV < 0) {
print "usage: ra_test argus2_input_file\n";
}
open (RA2, "/usr/local/bin/ra2 -d 16 -F ra2.conf.full -r $ARGV[0] |")
|| die "can't open $ARGV[0] $!";
open (RA32, "/usr/local/bin/ra3 -F ra3.conf.full -r $ARGV[0] |")
|| die "can't open $ARGV[0] $!";
# save the initial labels and eat the first ra2 man line (as not in ra32)
$label2 = <RA2>;
$labels32 = <RA32>;
$_ = <RA2>;
$line = 0;
while (<RA2>) {
# Read the ra2 file with ra2
$line++;
chop;
# Read the argus2 file with ra3 first so we can supress blanks in the user
# data fields and set the counts for the ra2 data ...
$ra32 = <RA32>;
chop $ra32;
($start32, $end32, $trans32, $dur32, $avgdur32, $saddr32, $daddr32, $proto32, $sport32, $dport32, $stos32, $dtos32, $sttl32, $dttl32, $sbytes32, $dbytes32, $sabytes32, $dabytes32, $spkts32, $dpkts32, $srate32, $drate32, $sload32, $dload32, $sloss32, $dloss32, $srcid32, $flgs32, $smac32, $dmac32, $dir32, $sjit32, $djit32, $state32, $rest) = split(/,/,$ra32,35);
if ($rest =~ /^s\[(\d+)\]=\"(.+)$/) {
$count = $1;
$rest = $2;
$data = substr ($rest,0,$count);
$rest = substr ($rest, $count+2);
$suser32 = "s\[$count\]=\"$data\"";
} else {
# eat the comma even if no data!
$data = "";
$rest = substr ($rest,1)
}
# now strip out any blanks (because ra2 is going to) in the data field.
$data =~ s/ //g;
# and save that as the length of the actual ra2 source data field.
$scount = length($data);
if ($rest =~ /^d\[(\d+)\]=\"(.+)$/) {
$count = $1;
$rest = $2;
$data = substr ($rest,0,$count);
$rest = substr ($rest, $count+2);
$duser32 = "d\[$count\]=\"$data\"";
} else {
# eat the comma even if no data!
$data = "";
$rest = substr ($rest,1)
}
# now strip out any blanks (because ra2 is going to) in the data field.
$data =~ s/ //g;
# and save that as the length of the actual ra2 source data field.
$dcount = length($data);
($swin32, $dwin32, $seq32, $smpls32, $dmpls32, $svlan32, $dvlan32, $sipid32, $dipid32) = split(/,/,$rest);
$flgs32 =~ s/ //g;
$ra2 = $_;
($start2, $end2, $trans2, $dur2, $avgdur2, $saddr2, $daddr2, $proto2, $sport2, $dport2, $stos2, $dtos2, $sttl2, $dttl2, $sbytes2, $dbytes2, $sabytes2, $dabytes2, $spkts2, $dpkts2, $srate2, $drate2, $sload2, $dload2, $sloss2, $dloss2, $srcid2, $flgs2, $smac2, $dmac2, $dir2, $sjit2, $djit2, $state2, $rest) = split(/,/,$_,35);
# V3 is inverting some flows so detect that and invert them back ...
$rev = "n";
if (($saddr2 ne $daddr2) && ($saddr2 eq $daddr32) && ($daddr2 eq $saddr32)) {
$rev = "y";
$tsaddr2 = $saddr2;
$saddr2 = $daddr2;
$daddr2 = $tsaddr2;
$tsport2 = $sport2;
$sport2 = $dport2;
$dport2 = $tsport2;
$tstos2 = $stos2;
$stos2 = $dtos2;
$dtos2 = $tstos2;
$tsttl2 = $sttl2;
$sttl2 = $dttl2;
$dttl2 = $tsttl2;
$tsbytes2 = $sbytes2;
$sbytes2 = $dbytes2;
$dbytes2 = $tsbytes2;
$tsabytes2 = $sabytes2;
$sabytes2 = $dabytes2;
$dabytes2 = $tsabytes2;
$tspkts2 = $spkts2;
$spkts2 = $dpkts2;
$dpkts2 = $tspkts2;
$tsrate2 = $srate2;
$srate2 = $drate2;
$drate2 = $tsrate2;
$tsload2 = $sload2;
$sload2 = $dload2;
$dload2 = $tsload2;
$tsloss2 = $sloss2;
$sloss2 = $dloss2;
$dloss2 = $tsloss2;
$tsmac2 = $smac2;
$smac2 = $dmac2;
$dmac2 = $tsmac2;
$tsjit2 = $sjit2;
$sjit2 = $djit2;
$djit2 = $tsjit2;
}
# if the reverse flag is set invert the ra3 data first ...
if ($rev eq "y") {
$tscount = $scount;
$scount = $dcount;
$dcount = $tscount;
}
if ($rest =~ /^s\[(\d+)\]=\"(.+)$/) {
$count = $1;
$rest = $2;
# while keeping the real count use the count calculated from
# the ra3 data to actually do the copy ...
$data = substr ($rest,0,$scount);
$rest = substr ($rest, $scount+2);
if ($rev eq "y") {
$duser2 = "d\[$count\]=\"$data\"";
} else {
$suser2 = "s\[$count\]=\"$data\"";
}
} else {
# eat the comma even if no data!
$rest = substr ($rest,1)
}
if ($rest =~ /^d\[(\d+)\]=\"(.+)$/) {
$count = $1;
$rest = $2;
# while keeping the real count use the count calculated from
# the ra3 data to actually do the copy ...
$data = substr ($rest,0,$dcount);
$rest = substr ($rest, $dcount+2);
if ($rev eq "y") {
$suser2 = "s\[$count\]=\"$data\"";
} else {
$duser2 = "d\[$count\]=\"$data\"";
}
} else {
# eat the comma even if no data!
$rest = substr ($rest,1)
}
($swin2, $dwin2, $seq2, $smpls2, $dmpls2, $svlan2, $dvlan2, $ipid2) = split(/,/,$rest);
# Now continue inverting if the reverse flag is set
if ($rev eq "y") {
$tswin2 = $swin2;
$swin2 = $dwin2;
$dwin2 = $tswin2;
$tsmpls2 = $smpls2;
$smpls2 = $dmpls2;
$dmpls2 = $tsmpls2;
$tsvlan2 = $svlan2;
$svlan2 = $dvlan2;
$dvlan2 = $tsvlan2;
}
$flgs2 =~ s/ //g;
if ($proto2 eq "llc") {
if ($sport2 eq "170") {
$sport2 = "sna";
}
if ($dport2 eq "170") {
$dport2 = "sna";
}
if ($sport2 eq "224") {
$sport2 = "net";
}
if ($dport2 eq "224") {
$dport2 = "net";
}
if ($sport2 eq "255") {
$sport2 = "gbl";
}
if ($dport2 eq "255") {
$dport2 = "gbl";
}
}
if ($proto2 eq "nvl") {
$proto2 = "ipx/";
}
if ($proto2 eq "3482") {
$proto2 = "slow";
}
if ($proto2 eq "3492") {
$proto2 = "nlb";
}
if ($proto2 eq "3327") {
$proto2 = "ismp";
}
if (($proto32 eq "rtp") && ($proto2 eq "udp")) {
$proto2 = "rtp";
}
if (($proto2 =~ /\d+/) && ($proto2 == int($proto32 / 10))) {
$proto32 = $proto2;
}
# Make 2.0.6 output match the 3.0 output (no port numbers).
if (($proto2 eq "igmp") || ($proto2 eq "ipni") || ($proto2 eq "arp") ||
($proto2 eq "srp") || ($proto2 eq "vine") || ($proto2 eq "ax.2") ||
(Sproto2 eq "pri-")) {
$sport2 = "";
$dport2 = "";
}
# For esp kill the ports because they are different 2.0.6 to 3.0 ...
if ($proto2 eq "esp") {
$sport2 = "";
$dport2 = "";
$sport32 = "";
$dport32 = "";
}
%comp = "";
$comp{start} = ($start2 eq $start32);
$comp{end} = ($end2 eq $end32);
$comp{trans} = ($trans2 eq $trans32);
$comp{dur} = ($dur2 eq $dur32);
if ((($avdur2 - $avdur32) < 1) && (($avdur2 - $avdur32) > -1 )) {
$comp{avdur} = 1;
}else {
$comp{avdur} = 0;
}
$comp{saddr} = ($saddr2 eq $saddr32);
$comp{daddr} = ($daddr2 eq $daddr32);
if ($proto32 =~ /[a-z]+/) {
$proto32 = substr $proto32, 0 , 4;
}
$comp{proto} = ($proto2 eq $proto32);
if (($comp{proto} != 1) && ($proto2 eq "rdp") && ($proto32 eq "udp")) {
$comp{proto} = 1;
}
# replace 0 with * in port
if ($sport32 eq '*') {
$sport2 = '*';
}
if ($sport32 =~ /^0x/) {
$sport32 = oct ($sport32);
}
if ($sport32 =~ /[a-z]+/) {
$sport32 = substr $sport32, 0, 3;
}
$comp{sport} = ($sport2 eq $sport32);
if ($comp{sport} != 1) {
print "sport $sport2 $sport32\n";
}
# replace 0 with * in port
if ($dport32 eq '*') {
$dport2 = '*';
}
if ($dport32 =~ /^0x/) {
$dport32 = oct ($dport32);
}
if ($dport32 =~ /[a-z]+/) {
$dport32 = substr $dport32, 0, 3;
}
$comp{dport} = ($dport2 eq $dport32);
if ($comp{dport} != 1) {
print "dport $dport2 $dport32\n";
}
# no src pkts no tos or ttl either (bug in 2.0.6)
if ($spkts2 == 0) {
$stos2 = "";
}
$comp{stos} = ($stos2 eq $stos32);
# no dst pkts no tos or ttl either (bug in 2.0.6)
if ($dpkts2 == 0) {
$dtos2 = "";
}
$comp{dtos} = ($dtos2 eq $dtos32);
# no src pkts no tos or ttl either (bug in 2.0.6)
if ($spkts2 == 0) {
$sttl2 = "";
}
$comp{sttl} = ($sttl2 eq $sttl32);
# no src pkts no tos or ttl either (bug in 2.0.6)
if ($dpkts2 == 0) {
$dttl2 = "";
}
$comp{dttl} = ($dttl2 eq $dttl32);
$comp{sbytes} = ($sbytes2 eq $sbytes32);
$comp{dbytes} = ($dbytes2 eq $dbytes32);
$comp{sabytes} = ($sabytes2 eq $sabytes32);
if ($comp{sabytes} != 1) {
print "sabytes $sabytes2 $sabytes32\n";
}
$comp{dabytes} = ($dabytes2 eq $dabytes32);
if ($comp{dabytes} != 1) {
print "dabytes $dabytes2 $dabytes32\n";
}
$comp{spkts} = ($spkts2 eq $spkts32);
$comp{dpkts} = ($dpkts2 eq $dpkts32);
if ((($srate2 - $srate32) < 100) && (($srate2 - $srate32) > -100 )) {
$comp{srate} = 1;
}else {
$comp{srate} = 0;
print "srate $srate2 $srate32\n";
}
if ((($drate2 - $drate32) < 100) && (($drate2 - $drate32) > -100 )) {
$comp{drate} = 1;
}else {
$comp{drate} = 0;
print "drate $drate2 $drate32 $srate33\n";
}
if ($sload2 eq "inf") {
$sload2 = 0.0;
}
if ((($sload2 - $sload32) < 1) && (($sload2 - $sload32) > -1 )) {
$comp{sload} = 1;
}else {
$comp{sload} = 0;
print "sload $sload2 $sload32\n";
}
if ($dload2 eq "inf") {
$dload2 = 0.0;
}
if ((($dload2 - $dload32) < 1) && (($dload2 - $dload32) > -1 )) {
$comp{dload} = 1;
}else {
$comp{dload} = 0;
print "dload $dload2 $dload32 $dload33\n";
}
# Loss is different v2 to v3 and thus breaks ...
# if ((($sloss2 - $sloss32) < 1) && (($sloss2 - $sloss32) > -1 )) {
# $comp{sloss} = 1;
# }else {
# $comp{sloss} = 0;
# print "sloss $sloss2 $sloss32 $sloss33\n";
# }
# if ((($dloss2 - $dloss32) < 1) && (($dloss2 - $dloss32) > -1 )) {
# $comp{dloss} = 1;
# }else {
# $comp{dloss} = 0;
# print "dloss $dloss2 $dloss32 $dloss33\n";
# }
# $comp{srcid} = ($srcid2 eq $srcid32);
while ($flgs2) {
$char = chop $flgs2;
if ($char eq "q") {
$char = "v";
}
$flgs2{$char} = "y";
}
while ($flgs32) {
$char = chop $flgs32;
$flgs32{$char} = "y";
}
$flgs = 1;
foreach $flg (sort keys (%flgs2)) {
if (($flg ne "") && ($flgs32{$flg} ne "y")) {
$flgs = 0;
} else {
$flgs2{$flg} = "";
$flgs32{$flg} = "";
}
}
if ($flgs == 0) {
print "flgs2 = ";
foreach $char (keys %flgs2) {
if ($flgs2{$char} eq "y") {
print "$char";
}
}
print "\nflgs32 = ";
foreach $char (keys %flgs32) {
if ($flgs32{$char} eq "y") {
print "$char";
}
}
print "\n";
}
$comp{flgs} = $flgs;
%flgs2 = "";
%flgs32 = "";
$comp{smac} = ($smac2 eq $smac32);
if ($comp{smac} != 1) {
print "smac $smac2 $smac32\n";
}
$comp{dmac} = ($dmac2 eq $dmac32);
if ($comp{dmac} != 1) {
print "dmac $dmac2 $dmac32\n";
}
if ($proto2 eq "arp") {
$dir2 =~ s/who-has/who/;
}
# comment/uncomment the next line to ignore / test direction flags
# $comp{dir} = ($dir2 eq $dir32);
# Int to float round off leaves quite a bit of slop!
if ((($sjit2 - $sjit32) < 100) && (($sjit2 - $sjit32) > -100 )) {
$comp{sjit} = 1;
} else {
$comp{sjit} = 0;
print "sjit $sjit2 $sjit32\n";
}
if ((($djit2 - $djit32) < 100) && (($djit2 - $djit32) > -100 )) {
$comp{djit} = 1;
}else {
$comp{djit} = 0;
print "djit $djit2 $djit32\n";
}
# comment / uncomment out the next 4 lines to ignore / test state
# $comp{state} = ($state2 eq $state32);
# if ($comp{state} != 1) {
# print "state $state2 $state32\n";
# }
$suser32 =~ s/ //g;
$comp{suser} = ($suser2 eq $suser32);
$duser32 =~ s/ //g;
$comp{duser} = ($duser2 eq $duser32);
if (($swin2 == 0) && ($spkts2 == 0)) {
$swin2 = ""; # 2.0.6 bug
}
$comp{swin} = ($swin2 eq $swin32);
if ($comp{swin} != 1) {
print "swin $swin2 $swin32\n";
}
if (($dwin2 == 0) && ($dpkts2 == 0)) {
$dwin2 = ""; # 2.0.6 bug
}
$comp{dwin} = ($dwin2 eq $dwin32);
if ($comp{dwin} != 1) {
print "dwin $dwin2 $dwin32\n";
}
$comp{seq} = ($seq2 eq $seq32);
$comp{smpls} = ($smpls2 eq $smpls32);
$comp{dmpls} = ($dmpls2 eq $dmpls32);
$comp{svlan} = ($svlan2 eq $svlan32);
if ($comp{svlan} != 1) {
print "svlan $svlan2 $svlan32\n";
}
$comp{dvlan} = ($dvlan2 eq $dvlan32);
if ($comp{dvlan} != 1) {
print "dvlan $dvlan2 $dvlan32\n";
}
if ($sipid32 ne "") {
$comp{sipid} = ($ipid2 eq $sipid32);
if ($proto2 eq "igmp") {
$comp{sipid} = 1;
}
} else {
$comp{sipid} = 1;
}
if ($comp{sipid} != 1) {
print "sipid $ipid2 $sipid32\n";
}
if ($dipid32 ne "") {
$comp{dipid} = ($ipid2 eq $dipid32);
} else {
$comp{dipid} = 1;
}
if ($comp{dipid} != 1) {
print "dipid $ipid2 $dipid32\n";
}
$bad = "no";
$error = "";
foreach $field (keys %comp){
if ($field ne "") {
if ($comp{$field} ne 1) {
$bad = "yes";
$error .= "$field,";
}
}
}
if ($bad eq "yes") {
print "\nline: $line fields in error: $error\n$ra2\n$ra32\n\n";
$a = $a;
}
}
More information about the argus
mailing list