Do you know how to read a pcap file continuously?
Carter Bullard
carter at qosient.com
Mon Oct 7 17:13:00 EDT 2013
Hey Sebas,
Try this version of ArgusSource.c and ArgusSource.h. These should fix trying to read a truncated file.
Replace the ones in the ./argus directory and remake.
Carter
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ArgusSource.c
Type: application/octet-stream
Size: 147564 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131007/1c76572d/attachment.obj>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ArgusSource.h
Type: application/octet-stream
Size: 34802 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131007/1c76572d/attachment-0001.obj>
-------------- next part --------------
On Oct 7, 2013, at 4:17 PM, el draco <eldraco at gmail.com> wrote:
> Hi Carter
>
> The basic stuff we need is to read a common pcap file (no -U in
> tcpdump) continuously with argus.
>
> We are not expecting argus to survive a file deletion.
> If the pcap file is not there or it is deleted, we start over, no
> problem for us.
> If the file is not there when argus starts, argus can exit.
>
> Does this make it easier?
> thanks!
> sebas
>
>
> On Mon, Oct 7, 2013 at 7:10 PM, Carter Bullard <carter at qosient.com> wrote:
>> Hey Sebas,
>> I can recover from the truncated file error condition, but
>> its a bit expensive, pcap_close(), look for a change in size
>> then pcap_open() lseek() …, but its really expensive.
>>
>> If you can describe the conditions that you expect, like
>> should survive file deletion, etc… Then we can wrap this
>> thing up pretty quick.
>>
>> Carter
>>
>>
>>
>> On Oct 7, 2013, at 11:18 AM, el draco <eldraco at gmail.com> wrote:
>>
>>> Hi list, we have been playing a little bit with reading a pcap file
>>> continuously. I'm resending our posts so they remain in the list.
>>>
>>> Sorry that you have to read the mails backwards in time to make sanse of this.
>>> In summary, you can read a pcap file continuously. Better if the
>>> tcpdump is using -U, but there is a work-around if it is not. Thanks
>>> Carter for all your work.
>>>
>>> sebas
>>>
>>> On Oct 7, 2013, at 10:21 AM, el draco <eldraco at gmail.com> wrote:
>>>> Hi Carter!
>>>>
>>>> Ufff that's an issue. While I can run it with -U if I can choose to,
>>>> right now we are using virtualbox
>>>> -tracefile option to crate the pcap file. It means that when we start
>>>> a vm, virtualbox automatically creates the pcap file for us. And
>>>> unfortunately, virtualbox is not running the capture with -U.
>>>>
>>>> However I managed to find a workaround!! It is ugly... again.. but it works.
>>>>
>>>> So, first you have a non-buffered pcap file running (without -U), like
>>>> the one from virtualbox -tracefile
>>>> tcpdump -n -s0 -i eth0 -w /tmp/test.pcap -v
>>>>
>>>> Then you have a second tcpdump capture file, translating the packets
>>>> and adding the buffer...
>>>> tail -n 1000 -f /tmp/test.pcap | tcpdump -n -s0 -r - -U -w /tmp/test.pcap2
>>>>
>>>> This second pcap file is buffered, so now argus can read it without problems:
>>>> ./bin/argus -f -r /tmp/test.pcap2 -w /tmp/test.argus -P 2040
>>>>
>>>> And you can see the argus data with ra continuously:
>>>> ra -n -S localhost:2040
>>>>
>>>>
>>>> With these commands I'm now able to read the argus data continuously
>>>> from a pcap file.
>>>>
>>>> The only issue I had is this:
>>>> If I stop the first pcap, then the second pcap exits (the tail command
>>>> in fact), and the argus and ra commands keep running. However, if I
>>>> start the first pcap and second pcap again... the argus is NOT getting
>>>> the data, and the ra command shows nothing. If I restart the argus,
>>>> everything works again.
>>>> Not an issue for me, but I report it in case it helps.
>>>>
>>>> Thanks a lot for your work on this topic!
>>>> cheers
>>>> sebas
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> On Mon, Oct 7, 2013 at 1:07 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>> Hey sebas,
>>>>> You MUST run tcpdump with the -U option to guarantee that the file
>>>>> contains data on packet boundaries.
>>>>>
>>>>> Carter
>>>>>
>>>>>
>>>>>
>>>>> On Oct 7, 2013, at 5:25 AM, el draco <eldraco at gmail.com> wrote:
>>>>>
>>>>>> Verification of argus-3.0.7.5
>>>>>>
>>>>>> 1st experiment
>>>>>> ----------------------
>>>>>> In one root console: tcpdump -n -s0 -i eth0 -v -w /tmp/test.pcap
>>>>>>
>>>>>> In another root console:
>>>>>> ./bin/argus -h
>>>>>> Argus Version 3.0.7.5
>>>>>> ./bin/argus -f -r /tmp/test.pcap -w /tmp/test.argus
>>>>>>
>>>>>> After some reading, It exits with this error:
>>>>>> ArgusError: 07 Oct 13 08:02:18.126142 ArgusGetPackets: pcap_dispatch()
>>>>>> returned bogus savefile header
>>>>>>
>>>>>> The /tmp/test.argus file is created with the flows. And if I run it
>>>>>> again, the flows are updated.
>>>>>> However the argus process dies when it reaches the end of the file.
>>>>>>
>>>>>> 2nd experiment
>>>>>> ----------------------
>>>>>> I deleted the previous files.
>>>>>> In one root console: tcpdump -n -s0 -i eth0 -v -w /tmp/test.pcap
>>>>>> Then I stopped the tcpdump.
>>>>>>
>>>>>> In another root console: ./bin/argus -f -r /tmp/test.pcap -w /tmp/test.argus
>>>>>> The argus keeps running without dying.
>>>>>> The ra -r /tmp/test.argus reads the netfows and exits.
>>>>>>
>>>>>> If I start the tcpdump again, the argus runs for 10 seconds, and then
>>>>>> exits with this error:
>>>>>> ArgusError: 07 Oct 13 09:08:07.562844 ArgusGetPackets:
>>>>>> pcap_dispatch() returned bogus savefile header
>>>>>> The /tmp/test.argus file was updated with the new information and I
>>>>>> could read it with ra.
>>>>>>
>>>>>> 3rd experiment
>>>>>> ----------------------
>>>>>> I deleted the previous files.
>>>>>> In a root console: ./bin/argus -f -i eth0 -w /tmp/test.argus
>>>>>> I can read the file with ra.
>>>>>> This works fine.
>>>>>>
>>>>>> 4th experiment
>>>>>> ----------------------
>>>>>> I deleted the previous files.
>>>>>> In one NON-ROOT console: tcpdump -n -s0 -i eth0 -v -w /tmp/test.pcap
>>>>>> In another NON-ROOT console: ./bin/argus -r /home/sebas/test.pcap -f -P 2040
>>>>>>
>>>>>> After some reading, It exits with the same error
>>>>>> ArgusError: 07 Oct 13 09:14:45.628054 ArgusGetPackets: pcap_dispatch()
>>>>>> returned bogus savefile header
>>>>>>
>>>>>>
>>>>>>
>>>>>> Info about my setup:
>>>>>> ii libpcap-dev 1.4.0-2 all
>>>>>> development library for libpcap (transitional package)
>>>>>> ii libpcap0.8:i386 1.4.0-2 i386
>>>>>> system interface for user-level packet capture
>>>>>>
>>>>>>
>>>>>> Tell me if you need me to test something else.
>>>>>> cheers
>>>>>> sebas
>>>>>>
>>>>>> On Sun, Oct 6, 2013 at 5:50 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>>>> So, this works, but still needs some of the exceptions worked out,
>>>>>>> such as if you rename or delete the packet file, but it should work....
>>>>>>> Please give this version of argus a trial, and send any feedback,
>>>>>>> bugs, additional features needed, etc soon !!!
>>>>>>>
>>>>>>> The man page for argus has been updated for this feature yet,
>>>>>>> so mostly done.
>>>>>>>
>>>>>>> This version has some other changes, so this will serve as general
>>>>>>> testing as well. The other changes are basically idle interface
>>>>>>> timer issues, and shouldn't be an issue.
>>>>>>>
>>>>>>> Use the -f option, along with the -r packetfile option, just like tail.
>>>>>>> This may work with pipes as well, but I didn't test.
>>>>>>>
>>>>>>> Carter
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Oct 6, 2013, at 1:29 PM, el draco <eldraco at gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Carter, We are waiting for it! Yes!!!
>>>>>>>>
>>>>>>>> We are now using tshark and offline argus..., so if you manage to make it work,
>>>>>>>> we will be more than happy to test it.
>>>>>>>>
>>>>>>>> Just tell me what you need from us.
>>>>>>>>
>>>>>>>> Thanks a lot!
>>>>>>>>
>>>
>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 6837 bytes
Desc: not available
URL: <https://pairlist1.pair.net/pipermail/argus/attachments/20131007/1c76572d/attachment.bin>
More information about the argus
mailing list