1
0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2024-12-23 12:43:46 +02:00
Commit Graph

22 Commits

Author SHA1 Message Date
Guo, Yejun
f4b3c0e55c avfilter/dnn: add a new interface to query dnn model's input info
to support dnn networks more general, we need to know the input info
of the dnn model.

background:
The data type of dnn model's input could be float32, uint8 or fp16, etc.
And the w/h of input image could be fixed or variable.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-10-30 11:07:06 -03:00
Guo, Yejun
e1b45b8596 avfilter/dnn: get the data type of network output from dnn execution result
so,  we can make a filter more general to accept different network
models, by adding a data type convertion after getting data from network.

After we add dt field into struct DNNData, it becomes the same as
DNNInputData, so merge them with one struct: DNNData.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-10-30 11:00:41 -03:00
Guo, Yejun
dff39ea9f0 dnn: add tf.nn.conv2d support for native model
Unlike other tf.*.conv2d layers, tf.nn.conv2d does not create many
nodes (within a scope) in the graph, it just acts like other layers.
tf.nn.conv2d only creates one node in the graph, and no internal
nodes such as 'kernel' are created.

The format of native model file is also changed, a flag named
has_bias is added, so change the version number.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-10-30 10:31:55 -03:00
Guo, Yejun
2558e62713 avfilter/dnn: unify the layer load function in native mode
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-10-15 18:56:54 -03:00
Guo, Yejun
3fd5ac7e92 avfilter/dnn: unify the layer execution function in native mode
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-10-15 18:56:25 -03:00
Guo, Yejun
b78dc27bba avfilter/dnn: add DLT prefix for enum DNNLayerType to avoid potential conflicts
and also change CONV to DLT_CONV2D for better description

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-10-15 16:35:39 -03:00
Guo, Yejun
8f13a557ca libavfilter/dnn: support multiple outputs for native mode
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-09-20 14:51:57 -03:00
Guo, Yejun
75ca94f3cf libavfilter/dnn/dnn_backend_native: find the input operand according to input name
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-09-20 14:51:50 -03:00
Guo, Yejun
b2683c66b2 libavfilter/dnn: add layer maximum for native mode.
The reason to add this layer is that it is used by srcnn in vf_sr.
This layer is currently ignored in native mode. After this patch,
we can add multiple outputs support for native mode.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-09-20 10:57:18 -03:00
Marton Balint
862e020f93 avfilter/dnn: fix inclusion guard in dnn/dnn_backend_native_layer_depth2space.h
Fixes fate-source failure.

Signed-off-by: Marton Balint <cus@passwd.hu>
2019-09-19 21:30:54 +02:00
Guo, Yejun
48133fad05 libavfilter/dnn: separate depth_to_space layer from dnn_backend_native.c to a new file
the logic is that one layer in one separated source file to make
the source files simple for maintaining.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-09-19 11:25:15 -03:00
Guo, Yejun
5f058dd693 libavfilter/dnn: separate conv2d layer from dnn_backend_native.c to a new file
the logic is that one layer in one separated source file to make
the source files simple for maintaining.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-09-19 11:09:25 -03:00
Guo, Yejun
022f50d3fe libavfilter/dnn: add header into native model file
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-09-04 11:13:21 -03:00
Guo, Yejun
83e0b71f66 dnn: export operand info in python script and load in c code
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-08-30 11:41:30 -03:00
Guo, Yejun
2d5e39c13e dnn: change .model file format to put layer number at the end of file
currently, the layer number is at the beginning of the .model file,
so we have to scan twice in python script, the first scan to get the
layer number. Only one scan needed after put the layer number at the
end of .model file.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-08-30 11:41:30 -03:00
Guo, Yejun
09a455a246 dnn: introduce dnn operand (in c code) to hold operand infos within network
the info can be saved in dnn operand object without regenerating again and again,
and it is also needed for layer split/merge, and for memory reuse.

to make things step by step, this patch just focuses on c code,
the change within python script will be added later.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-08-30 11:41:30 -03:00
Jun Zhao
1b0a8e48f1 lavfi/dnn/dnn_backend_native: fix memory leak in error path
fix memory leak in error path

Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
2019-08-20 10:07:38 +08:00
Guo, Yejun
67889d4715 libavfilter/dnn/dnn_backend_tf: add tf.pad support for tensorflow backend with native model.
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-08-19 11:37:16 -03:00
Guo, Yejun
29aeeb3e3e libavfilter/dnn/dnn_backend_tf: fix typo that variable uninitialized.
if it is initialized randomly, the tensorflow lib will report
error message such as:
Attempt to add output -7920 of depth_to_space4 not in range [0, 1) to node with type Identity

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-08-19 11:37:16 -03:00
Guo, Yejun
ccbab41039 dnn: convert tf.pad to native model in python script, and load/execute it in the c code.
since tf.pad is enabled, the conv2d(valid) changes back to its original behavior.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-07-29 12:34:19 -03:00
Guo, Yejun
df8db34552 dnn: add layer pad which is equivalent to tf.pad
the reason to add this layer first is that vf_sr uses it in its
tensorflow model, and the next plan is to update the python script
to convert tf.pad into native model.

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-07-29 12:34:19 -03:00
Guo, Yejun
1b9064e3f4 libavfilter/dnn: move dnn files from libavfilter to libavfilter/dnn
it is expected that there will be more files to support native mode,
so put all the dnn codes under libavfilter/dnn

The main change of this patch is to move the file location, see below:
modified:   libavfilter/Makefile
new file:   libavfilter/dnn/Makefile
renamed:    libavfilter/dnn_backend_native.c -> libavfilter/dnn/dnn_backend_native.c
renamed:    libavfilter/dnn_backend_native.h -> libavfilter/dnn/dnn_backend_native.h
renamed:    libavfilter/dnn_backend_tf.c -> libavfilter/dnn/dnn_backend_tf.c
renamed:    libavfilter/dnn_backend_tf.h -> libavfilter/dnn/dnn_backend_tf.h
renamed:    libavfilter/dnn_interface.c -> libavfilter/dnn/dnn_interface.c

Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
2019-07-26 13:07:43 -03:00