This repository contains the source code for example Analysis Modules for the Apache Strap-Down Pilotage program. It implements the approach described in the ASDP Analysis API Implementation document.
Initializing: This module has a dependency on the ASDP_Render_Module package because it uses
some classes defined in that source directory. It does not build the submodule, just uses it to
read a header and class source file. To initialize the submodule, either use the --recursive flag
on the clone or run the following command in the root directory of this repository:
git submodule update --init --recursive
Required: The ASDP_Analysis_Module_Examples repository requires the ASDP_Core_API library to have been
installed before it is built. This installs by default in a known system location that can
be found automatically. The library source can be obtained using:
git clone https://github.com/arizonaCameraLab/ASDP_Core_API
Build: ASDP_Analysis_Module uses CMake to configure the builds (though other build systems could be used). On Ubuntu Linux, this can be done as follows:
sudo apt install cmake
cd; mkdir src; cd src; git clone https://github.com/arizonaCameraLab/ASDP_Analysis_Module
cd; mkdir -p build/ASDP_Analysis_Module; cd build/ASDP_Analysis_Module
cmake ../../src/ASDP_Analysis_Module
make
The CUDA code will only be built if a CUDA toolkit is available on the computer.
Run: There are both C++ and a CUDA sample programs. Each is run with a single command-line
argument that specifies the name of the network interface card to listen on for servers. For
example, to use the C++ code to listen on a local (test) server, run
./ASDP_Analysis_Module_CXX localhost
There is a Percentile_CXX C++ program that computes and prints the specified percentile pixel intensity
over the image from a given camera. It is run as follows:
./Percentile_CXX localhost <percentile>
It can be run with the --camera option to specify which camera on the ball to analyze, the
default is camera 1 (the first one).
There is an ASDP_Analysis_Module_Median_Threshold CUDA program that uses CUDA-only processing
to compute a median filter and then detects drones where a pixel exceeds a specific threshold
above its 8-direction neighbors with a specified stride. It is run as follows:
./ASDP_Analysis_Module_Median_Threshold localhost
It can be run with the --camera option to specify which camera on the ball to analyze, the
default is to run on all cameras; this can be given multiple times to specify several cameras.
When running, commands can by typed at the console to adjust the verbosity (v#), threshold (t#),
and stride (s#) parameters while the program is running. A verbosity of 2 or higher will print
out the detected drone locations for each frame.
Documentation: The primary documentation is available in DOxygen, and is generated by as part of the build process when DOxygen is available. On Ubuntu Linux, this can be generated as follows:
sudo apt install doxygen
When you build using CMake, it will build DOxygen by default. Open doc_doxygen/html/index.html with a web browser to view the documentation.
Although the C++ workflow is relatively straightforward, the CUDA workflow is more complex as seen in the diagram below. The complex, multi-threaded and multi-streamed behavior and the use of pre-allocated image buffers is required to maintain sufficiently low latency on ingest to avoid dropping frames and to provide sufficient throughput on analysis to keep up with the ingest rate.
The example program generates phantom Drone sightings at the center of each image for each
frame. To modify this, change the return data type structure and the CUDA
kernel code in a copy of the ASDP_Analysis_Module_CUDA.cu file. There are comments
in the code wherever changes are needed that are marked with the keyword "todo" preceded by
an @ sign.
There are command-line arguments to restrict the analysis to a single camera and to specify how many frames to skip between analysis so that the analysis can keep up. If it does not keep up, the program will drop frames and this will cause it to fill up memory and crash. A more robust response to dropped frames (skipping over frames until a new begin-frame message and discarding partial data) may be needed in a production system.
If Boost.Python is available on the system, then there must also be the same version of Python that it was built against. On Windows, set the CMAKE_PREFIX_PATH to point at each of them (you can edit the entry in the CMakeCache.txt file to do this).
On Linux, the following packages are required:
- sudo apt install libboost-python-dev python3-dev libboost-numpy-dev
To run the Video_Stream_Receiver.py also requires sudo apt install python3-opencv.
In this case, the build process will also build the Python bindings for a subset of the ASDP_Core_API library (JSONStringSender and JSONStringReceiver). The resulting .pyd file can be imported and used in Python programs. There is an example Python program, Analysis_Server.py that shows how to use the Python bindings to create an analysis server that sends reports to connected clients or to file depending on the URL specified on the command line.
Note that the C++ methods are modified to return Python tuples when there is a pass-by-reference return value for the C++ method. This is because Python methods can only return a single value. The first entry in the tuple (or the only entry) is the Status code.
Note that the VideoStreamReceiver class will only be wrapped when Boost.Numpy is also available on the system. This is a threaded class implemented in C++ that receives video frames from a server and makes them available as Numpy arrays in Python. There is an example Python program, Video_Stream_Receiver.py that shows how to use the Python bindings to create a video stream receiver and process the received frames. This class exposes the following methods:
VideoStreamReceiver(std::string streamFileName)- Constructor that takes the stream file name as an argument.VideoStreamReceiver(std::string IP, uint32_t serial, uint32_t cameraID, uint32_t frameStride, uint32_t replayID)- Constructor that takes the server IP address, camera serial number, camera ID, frame stride, and replay ID as arguments.GetStatus()- Returns the current status of the video stream receiver.- bp::tuple GetNextFrame(int timeoutMicroseconds) - Retrieves the next frame from the video stream receiver with a specified timeout in microseconds. Returns a tuple containing the time as a Time and the frame as a Numpy array. Empty return on timeout.
DiscardBufferedFrames()- Discards any buffered frames in the video stream receiver. Used to clear out any backlog of frames.