Or it'll cause null pointer dereference if size < sizeof(uint32_t), also
in case tc[0] > 3, the code will report error directly.
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Important part of this algorithm is the double threshold step: pixels
above "high" threshold being kept, pixels below "low" threshold dropped,
pixels in between (weak edges) are kept if they are neighboring "high"
pixels.
The weak edge check uses a neighboring context and should not be applied
on the plane's border. The condition was incorrect and has been fixed in
the commit.
Signed-off-by: Andriy Gelman <andriy.gelman@gmail.com>
Reviewed-by: Andriy Gelman <andriy.gelman@gmail.com>
We should not silently allocate an incorrect sized buffer.
Fixes trac issue #8718.
Signed-off-by: Reimar Döffinger <Reimar.Doeffinger@gmx.de>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Reviewed-by: Guo, Yejun <yejun.guo@intel.com>
floating point precision will cause rgb*max generate different value on
x86_32 and x86_64. have pass fate test on x86_32 and x86_64 by using
lrintf to get the nearest integral value for rgb * max before av_clip.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
We can try with the srcnn model from sr filter.
1) get srcnn.pb model file, see filter sr
2) convert srcnn.pb into openvino model with command:
python mo_tf.py --input_model srcnn.pb --data_type=FP32 --input_shape [1,960,1440,1] --keep_shape_ops
See the script at https://github.com/openvinotoolkit/openvino/tree/master/model-optimizer
We'll see srcnn.xml and srcnn.bin at current path, copy them to the
directory where ffmpeg is.
I have also uploaded the model files at https://github.com/guoyejun/dnn_processing/tree/master/models
3) run with openvino backend:
ffmpeg -i input.jpg -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=openvino:model=srcnn.xml:input=x:output=srcnn/Maximum -y srcnn.ov.jpg
(The input.jpg resolution is 720*480)
Also copy the logs on my skylake machine (4 cpus) locally with openvino backend
and tensorflow backend. just for your information.
$ time ./ffmpeg -i 480p.mp4 -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=tensorflow:model=srcnn.pb:input=x:output=y -y srcnn.tf.mp4
…
frame= 343 fps=2.1 q=31.0 Lsize= 2172kB time=00:00:11.76 bitrate=1511.9kbits/s speed=0.0706x
video:1973kB audio:187kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.517637%
[aac @ 0x2f5db80] Qavg: 454.353
real 2m46.781s
user 9m48.590s
sys 0m55.290s
$ time ./ffmpeg -i 480p.mp4 -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=openvino:model=srcnn.xml:input=x:output=srcnn/Maximum -y srcnn.ov.mp4
…
frame= 343 fps=4.0 q=31.0 Lsize= 2172kB time=00:00:11.76 bitrate=1511.9kbits/s speed=0.137x
video:1973kB audio:187kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.517640%
[aac @ 0x31a9040] Qavg: 454.353
real 1m25.882s
user 5m27.004s
sys 0m0.640s
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
OpenVINO is a Deep Learning Deployment Toolkit at
https://github.com/openvinotoolkit/openvino, it supports CPU, GPU
and heterogeneous plugins to accelerate deep learning inferencing.
Please refer to https://github.com/openvinotoolkit/openvino/blob/master/build-instruction.md
to build openvino (c library is built at the same time). Please add
option -DENABLE_MKL_DNN=ON for cmake to enable CPU path. The header
files and libraries are installed to /usr/local/deployment_tools/inference_engine/
with default options on my system.
To build FFmpeg with openvion, take my system as an example, run with:
$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/deployment_tools/inference_engine/lib/intel64/:/usr/local/deployment_tools/inference_engine/external/tbb/lib/
$ ../ffmpeg/configure --enable-libopenvino --extra-cflags=-I/usr/local/deployment_tools/inference_engine/include/ --extra-ldflags=-L/usr/local/deployment_tools/inference_engine/lib/intel64
$ make
Here are the features provided by OpenVINO inference engine:
- support more DNN model formats
It supports TensorFlow, Caffe, ONNX, MXNet and Kaldi by converting them
into OpenVINO format with a python script. And torth model
can be first converted into ONNX and then to OpenVINO format.
see the script at https://github.com/openvinotoolkit/openvino/tree/master/model-optimizer/mo.py
which also does some optimization at model level.
- optimize at inference stage
It optimizes for X86 CPUs with SSE, AVX etc.
It also optimizes based on OpenCL for Intel GPUs.
(only Intel GPU supported becuase Intel OpenCL extension is used for optimization)
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
Note for info level, one extra \n will be print after the log.
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Currently, the zoompan filter exposes a 'time' variable (missing from docs) for use in
the 'zoom', 'x', and 'y' expressions. This variable is perhaps better named
'out_time' as it represents the timestamp in seconds of each output frame
produced by zoompan. This patch adds aliases 'out_time' and 'ot' for 'time'.
This patch also adds an 'in_time' (alias 'it') variable that provides access
to the timestamp in seconds of each input frame to the zoompan filter.
This helps to design zoompan filters that depend on the input video timestamps.
For example, it makes it easy to zoom in instantly for only some portion of a video.
Both the 'out_time' and 'in_time' variables have been added in the documentation
for zoompan.
Example usage of 'in_time' in the zoompan filter to zoom in 2x for the
first second of the input video and 1x for the rest:
zoompan=z='if(between(in_time,0,1),2,1):d=1'
V2: Fix zoompan filter documentation stating that the time variable
would be NAN if the input timestamp is unknown.
V3: Add 'it' alias for 'in_time. Add 'out_time' and 'ot' aliases for 'time'.
Minor corrections to zoompan docs.
Signed-off-by: exwm <thighsman@protonmail.com>
It can't fail, yet it returns an int and other code checks whether it
failed; yet if it did fail, an AVFrame would leak. One could of course
add an av_frame_free for this (that compilers could optimize away), yet
it is easier to simply stop pretending that disp_palette could fail.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Fix vpad.name leak in error path, move the vpad related operation
only if enabled show IR frequency response.
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
We need at least a few bits of entropy to determine the start index of each
queue, in order to let filters run in parallel as much as possible, and
rand() is not thread safe and disrupts any external API's usage of rand,
so instead replace it with av_get_random_seed.
While it has more overhead than rand, we only run it once per filter upon init.
more math unary operations will be added here
It can be tested with the model file generated with below python scripy:
import tensorflow as tf
import numpy as np
import imageio
in_img = imageio.imread('input.jpeg')
in_img = in_img.astype(np.float32)/255.0
in_data = in_img[np.newaxis, :]
x = tf.placeholder(tf.float32, shape=[1, None, None, 3], name='dnn_in')
x1 = tf.subtract(x, 0.5)
x2 = tf.abs(x1)
y = tf.identity(x2, name='dnn_out')
sess=tf.Session()
sess.run(tf.global_variables_initializer())
graph_def = tf.graph_util.convert_variables_to_constants(sess, sess.graph_def, ['dnn_out'])
tf.train.write_graph(graph_def, '.', 'image_process.pb', as_text=False)
print("image_process.pb generated, please use \
path_to_ffmpeg/tools/python/convert.py to generate image_process.model\n")
output = sess.run(y, feed_dict={x: in_data})
imageio.imsave("out.jpg", np.squeeze(output))
Signed-off-by: Ting Fu <ting.fu@intel.com>
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Use this in vf_spp.c, where the get_pixels operation is done on
unaligned source addresses.
Hook up the x86 (mmx and sse) versions of get_pixels to this
function pointer, as those implementations seem to support unaligned
use.
This fixes fate-filter-spp on armv7.
Signed-off-by: Martin Storsjö <martin@martin.st>
