History

libavfi/dnn: add LibTorch as one of DNN backend

PyTorch is an open source machine learning framework that accelerates the path from research prototyping to production deployment. Official website: https://pytorch.org/. We call the C++ library of PyTorch as LibTorch, the same below. To build FFmpeg with LibTorch, please take following steps as reference: 1. download LibTorch C++ library in https://pytorch.org/get-started/locally/, please select C++/Java for language, and other options as your need. Please download cxx11 ABI version: (libtorch-cxx11-abi-shared-with-deps-*.zip). 2. unzip the file to your own dir, with command unzip libtorch-shared-with-deps-latest.zip -d your_dir 3. export libtorch_root/libtorch/include and libtorch_root/libtorch/include/torch/csrc/api/include to $PATH export libtorch_root/libtorch/lib/ to $LD_LIBRARY_PATH 4. config FFmpeg with ../configure --enable-libtorch \ --extra-cflag=-I/libtorch_root/libtorch/include \ --extra-cflag=-I/libtorch_root/libtorch/include/torch/csrc/api/include \ --extra-ldflags=-L/libtorch_root/libtorch/lib/ 5. make To run FFmpeg DNN inference with LibTorch backend: ./ffmpeg -i input.jpg -vf \ dnn_processing=dnn_backend=torch:model=LibTorch_model.pt -y output.jpg The LibTorch_model.pt can be generated by Python with torch.jit.script() api. https://pytorch.org/tutorials/advanced/cpp_export.html. This is pytorch official guide about how to convert and load torchscript model. Please note, torch.jit.trace() is not recommanded, since it does not support ambiguous input size.

Wenbin Chen committed 2 years ago

Commit f4e06641 parent d24b1367 files · +624 −4Browse files

Showing the changes to libavfilter/dnn_interface.h (1 of 7 files). Show all

Unified · Split

libavfilter/dnn_interface.h+1−1
@@ -32,7 +32,7 @@ · 31 unmodified lines
3232
3333 #define DNN_GENERIC_ERROR FFERRTAG('D','N','N','!')
3434
35−typedef enum {DNN_TF = 1, DNN_OV} DNNBackendType;
35+typedef enum {DNN_TF = 1, DNN_OV, DNN_TH} DNNBackendType;
3636
3737 typedef enum {DNN_FLOAT = 1, DNN_UINT8 = 4} DNNDataType;
3838

Keyboard shortcuts

General

Show keyboard shortcuts
?
Focus the current search
/

Issues and pull requests

Open filters
f

Issues, pull requests and Inbox

Select next item
j↓
Select previous item
k↑
Open selected item
Enter
Dismiss filters or clear selection
Esc

Repository code

Go to file
t