Do you know how to read a pcap file continuously?

[el draco]

I have successfully implemented a python script that does exactly that:
- open the file
- read it and store the las offset
- flush it
- close the file
- open it
- seek to the stored position
- continue reading

Sorry, only in python.
You can see my implementation in this public project:
http://sourceforge.net/projects/dnmap/
dnmap_server.py file , function read_file_and_fill_nmap_variable()

What I do not know is how these data can be sent to the lipcap library...

Does it help?
sebas

On Wed, Sep 18, 2013 at 11:18 AM, Carter Bullard <carter at qosient.com> wrote:
> I'll attempt to do a few tests, such as keep the file descriptor open, when we get EOF, and start testing to see if the size has changed, and just keep reading when it does grow.  Or we may have to close the file descriptor once we get EOF, holding the last offset, and reopen and seek when we realize the file has grown.
>
> It would be nice to know if either has been done before and people say that it does or doesn't work.   Maybe a bit of serious google searching ???
>
> Carter
>
> On Sep 18, 2013, at 3:45 AM, el draco <eldraco at gmail.com> wrote:
>
>> Thanks a lot Carter!
>>
>> Just to be clear, do you want to test if it is possible to seek() once
>> the file is opened with lipcap right? That can be tricky without
>> modifying the library but we can test it.
>> The nice thing about tail is that it does not use libpcap so it can seek().
>>
>> Or I'm not understanding you and you will use -f to open the pcap
>> file, then seek() and then give the data to libpcap somehow?
>>
>> sebas
>>
>> On Tue, Sep 17, 2013 at 5:23 PM, Carter Bullard <carter at qosient.com> wrote:
>>> Hmmmmm,
>>> That is pretty bizarre that tail works, not really sure how it could.
>>>
>>> OK, so if this is important to you guys, if you can do the homework
>>> on the net to see if we can successfully fseek() once we open the
>>> file, then I'll put it in, but I'll need you guys to test it.
>>>
>>> I did do a little google search and found a few wireshark posts
>>> that suggested that it was doable, but I haven't seen if there were
>>> any reports of success or failure.
>>>
>>> I'll add the "-f" option to argus, and we'll do the same thing
>>> that tail does, ignore the -f on the socket based reads, and
>>> keep at it for the FIFO based reads.
>>>
>>> If you delete the packet file, we'll realize it and start anew,
>>> a least that will be the goal.  I'll add this for the 3.0.8
>>> release, if we find that it works.
>>>
>>> Carter
>>>
>>>
>>>
>>> On Sep 17, 2013, at 11:02 AM, el draco <eldraco at gmail.com> wrote:
>>>
>>>> Thanks Carter. That is exactly what we realized. I would love a seek()
>>>> on libpcap.
>>>>
>>>> I know it is offtoppic but, right now we solve it with something like this:
>>>>
>>>> First start the capture in file.pcap
>>>>
>>>> tail -f test.pcap -n 1000 | tcpdump -n -s0 -r -
>>>>
>>>> That can read a pcap file continuously. You can use tshark instead.
>>>>
>>>> With argus you can do:
>>>> tail -f test.pcap -n 1000 | argus -r - -w test.argus
>>>>
>>>> (the -n is useful if you want to read big files. You must go to the
>>>> start of the file with the -n value)
>>>>
>>>> And after that you can run several times:
>>>> ra -r test.argus
>>>>
>>>> Not the best I know...
>>>> sebas
>>>>
>>>> On Tue, Sep 17, 2013 at 3:58 PM, Carter Bullard <carter at qosient.com> wrote:
>>>>> Hey El Draco,
>>>>> This type of feature would have to be supported by libpcap(),
>>>>> as it is the logic that reads the packets from files.
>>>>>
>>>>> If there is a decent way to seek() into a libpcap file
>>>>> once we've opened the file with pcap_fopen_offline(),
>>>>> then I can put it into argus.
>>>>>
>>>>> Carter
>>>>>
>>>>>
>>>>> On Sep 17, 2013, at 8:12 AM, el draco <eldraco at gmail.com> wrote:
>>>>>
>>>>>> Hi list.
>>>>>>
>>>>>> We need your help.
>>>>>> We have a lot of running pcap captures here that are storing the
>>>>>> packets in pcap files on the disk. These files are continuously
>>>>>> growing and we are using argus to analyze them.
>>>>>> (We can not change and use only argus, we need the pcaps)
>>>>>>
>>>>>> We want to have argus read these pcap files and generate some output
>>>>>> (or a server port waiting for a client or write the data to files).
>>>>>> But argus should run continuously without stopping, like when it is
>>>>>> reading packets from the network. If new packets are added to the pcap
>>>>>> file, we want argus to find them.
>>>>>>
>>>>>> I know that you can not use -r in argus and at the same time open a
>>>>>> port for listening requests.
>>>>>>
>>>>>> Did any of you solve this before? How to continuously analyze a pcap file?
>>>>>>
>>>>>> Also we can't run argus each time we need data, because in large files
>>>>>> it can take up to 5 minutes to read one pcap.
>>>>>>
>>>>>> thanks a lot
>>>>>> sebas
>>