Motivated by a desire to use vf_libplacebo as a GPU-accelerated
cropping/padding/zooming filter. This commit adds support for setting
the `input/target.crop` fields as dynamic expressions.
Re-use the same generic variables available to other scale and crop type
filters, and also add some more that we can afford as a result of being
able to set these properties dynamically.
It's worth pointing out that `out_t/ot` is currently redundant with
`in_t/t` since it will always contain the same PTS values, but I plan on
changing this in the near future.
I decided to also expose `crop_w/crop_h` and `pos_w/pos_h` as variables
in the expression parser itself, since this enables the fairly common
use case of determining dimensions first and then placing the image
appropriately, such as is done in the default behavior (which centers
the cropped/placed region by default).
In some circumstances, libplacebo will clear the background as a result
of cropping/padding. Currently, this uses the hard-coded default fill
color of black. This option makes this behavior configurable.
Will remove native backend, so change the default backend in filters,
and also remove the python scripts which generate native model file.
Signed-off-by: Ting Fu <ting.fu@intel.com>
Native backend will be removed in following commits, so change the
dnn interface and modify the error message in it first.
Signed-off-by: Ting Fu <ting.fu@intel.com>
Not doing so is an obvious oversight - the ICC profile is tied to the
original colorspace, so if we change it, we should definitely strip this
information.
We should probably also have an extra option to control whether the ICC
profile should be stripped, ignored, or applied, but for now this fixes
an existing bug.
Otherwise main and overlay videos share the same input region. Note NULL
pointer imples the whole overlay video will be processed.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Clarify failure in case of x/y building a too big matrix.
Example:
$ ffmpeg -hide_banner -f lavfi -i color=c=white:size=640x360,unsharp=lx=5:ly=23 -f null -t 1 -
[Parsed_unsharp_1 @ 0x5650e1c30240] luma matrix size (lx/2+ly/2)*2=26 greater than maximum value 25
color=c=white:size=640x360,unsharp=lx=5:ly=23: Invalid argument
Fix trac issue:
http://trac.ffmpeg.org/ticket/6033
Otherwise the output format is not changed when output is in system
memory. For example, the output format is still p010le in the following
case:
$ ffmpeg -qsv_device /dev/dri/renderD128 -f lavfi -i testsrc -vf
"format=p010le,vpp_qsv=extra_hw_frames=8:format=nv12" -f null -
...
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf60.4.100
Stream #0:0: Video: wrapped_avframe, p010le(tv, progressive), 320x240
[SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Add stack_internal.h to the list of skipped headers to fix
make checkheaders fail, it's introduced by commit 742dfa281
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
These fields are supposed to store information about the packet the
frame was decoded from, specifically the byte offset it was stored at
and its size.
However,
- the fields are highly ad-hoc - there is no strong reason why
specifically those (and not any other) packet properties should have a
dedicated field in AVFrame; unlike e.g. the timestamps, there is no
fundamental link between coded packet offset/size and decoded frames
- they only make sense for frames produced by decoding demuxed packets,
and even then it is not always the case that the encoded data was
stored in the file as a contiguous sequence of bytes (in order for pos
to be well-defined)
- pkt_pos was added without much explanation, apparently to allow
passthrough of this information through lavfi in order to handle byte
seeking in ffplay. That is now implemented using arbitrary user data
passthrough in AVFrame.opaque_ref.
- several filters use pkt_pos as a variable available to user-supplied
expressions, but there seems to be no established motivation for using them.
- pkt_size was added for use in ffprobe, but that too is now handled
without using this field. Additonally, the values of this field
produced by libavcodec are flawed, as described in the previous
ffprobe conversion commit.
In summary - these fields are ill-defined and insufficiently motivated,
so deprecate them.
Rescale the timestamp for AVERROR_EOF. This can fix tickets 10261 and
10262.
Tested-by: Dmitry Rogozhkin <dmitry.v.rogozhkin@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
SSIM360Context.ssim360_hist is an array of four pointers to double;
so sizeof(*ssim360_hist[0]) (=sizeof(double)) is the correct size
to use to calculate the amount of memory to allocate, not
sizeof(*ssim360_hist) (which is sizeof(double*)).
Use FF_ALLOCZ_TYPED_ARRAY to avoid this issue altogether.
Fixes Coverity issue #1520671.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Reviewed-by: Jan Ekström <jeebjp@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This has not been functional since a year ago, including in our current
minimum dependency of libplacebo (v4.192.0). It also causes build errors
against libplacebo v6, so it needs to be removed from the code. We can
keep the option around for now, but it should also be removed soon.
Signed-off-by: Niklas Haas <git@haasn.dev>
Signed-off-by: James Almer <jamrial@gmail.com>
This reverts commit 205117d87f.
This didn't fix make checkheaders after all, and also broke compilation in some
scenarios.
Signed-off-by: James Almer <jamrial@gmail.com>
Restores the behavior of naming the instance filter@id, which was accidentally changed
to simpy id in commit f17051eaae.
Fixes ticket #10226.
Signed-off-by: James Almer <jamrial@gmail.com>
Callers currently have two ways of adding filters to a graph - they can
either
- create, initialize, and link them manually
- use one of the avfilter_graph_parse*() functions, which take a
(typically end-user-written) string, split it into individual filter
definitions+options, then create filters, apply options, initialize
filters, and finally link them - all based on information from this
string.
A major problem with the second approach is that it performs many
actions as a single atomic unit, leaving the caller no space to
intervene in between. Such intervention would be useful e.g. to
- modify filter options;
- supply hardware device contexts;
both of which typically must be done before the filter is initialized.
Callers who need such intervention are then forced to invent their own
filtergraph parsing, which is clearly suboptimal.
This commit aims to address this problem by adding a new modular
filtergraph parsing API. It adds a new avfilter_graph_segment_parse()
function to parse a string filtergraph description into an intermediate
tree-like representation (AVFilterGraphSegment and its children).
This intermediate form may then be applied step by step using further
new avfilter_graph_segment*() functions, with user intervention possible
between each step.
Also, replace an AVFilterContext argument with a logging context+private
class, as those are the only things needed in this function.
Will be useful in future commits.
We need to construct the output format list separatedly from the input
format list, because we need to adhere to two extra requirements:
1. Big-endian output formats are always unsupported (runtime error)
2. Combining 'vulkan' with an explicit out_format that is not supported
by the vulkan frame allocation code is illegal and will crash (abort)
As a free side benefit, this rewrite fixes a possible memory leak in the
`fail` path that was present in the old code.
Signed-off-by: Niklas Haas <git@haasn.dev>
It only works on Linux
$ ffmpeg -loglevel verbose -init_hw_device qsv=intel -f lavfi -i \
yuvtestsrc -vf "format=uyvy422,vpp_qsv=format=nv12" -f null -
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
QSVDeintContext and VPPContext have the same base context, and all
features in deinterlace_qsv are implemented in vpp_qsv filter, so
deinterlace_qsv can be taken as a special case of vpp_qsv filter, and we
may use VPPContext with a different option array, preinit callback and
support pixel formats to implement deinterlace_qsv, then remove
QSVDeintContext.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Like what we did for scale_qsv filter, we use QSVVPPContext as a base
context to manage MFX session for deinterlace_qsv filter.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This is used to control the output at frame rate or field rate when
deinterlace is expected and framerate is not specified.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
From x86inc:
> On AMD cpus <=K10, an ordinary ret is slow if it immediately follows either
> a branch or a branch target. So switch to a 2-byte form of ret in that case.
> We can automatically detect "follows a branch", but not a branch target.
> (SSSE3 is a sufficient condition to know that your cpu doesn't have this problem.)
x86inc can automatically determine whether to use REP_RET rather than
REP in most of these cases, so impact is minimal. Additionally, a few
REP_RETs were used unnecessary, despite the return being nowhere near a
branch.
The only CPUs affected were AMD K10s, made between 2007 and 2011, 16
years ago and 12 years ago, respectively.
In the future, everyone involved with x86inc should consider dropping
REP_RETs altogether.
QSVScaleContext and VPPContext have the same base context, and all
features in scale_qsv are implemented in vpp_qsv filter, so scale_qsv
can be taken as a special case of vpp_qsv filter, and we may use
VPPContext with a different option array, preinit callback and supported
pixel formats to implement scale_qsv then remove QSVScaleContext
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Use QSVVPPContext as a base context of QSVScaleContext, hence we may
re-use functions defined for QSVVPPContext to manage MFX session for
scale_qsv filter.
In addition, system memory has been taken into account in
QSVVVPPContext, we may add support for non-QSV pixel formats in the
future.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This brings ff_vf_ssim360 in line with its declaration in allfilters.c;
this discrepancy is actually undefined behaviour.
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The FILTER_INPUTS and FILTER_OUTPUTS macros already set
AVFilter.(inputs|outputs); Clang therefore emits a warning for
this: "initializer overrides prior initialization of this subobject
[-Winitializer-overrides]"
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Don't use "static const" for compile time float constants, but use
defines. This fixes the following error:
src/libavfilter/vf_ssim360.c(549): error C2099: initializer is not a constant
Signed-off-by: Martin Storsjö <martin@martin.st>
Customized SSIM for various projections (and stereo formats) of 360 images and videos.
Further contributions by:
Ashok Mathew Kuruvilla
Matthieu Patou
Yu-Hui Wu
Anton Khirnov
Suggested-By: ffmpeg@fb.com
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The same members between QSVVPPContext and VPPContext are removed from
VPPContext, and async_depth is moved from QSVVPPParam to QSVVPPContext
so that all QSV filters using QSVVPPContext may support async depth.
In addition, we may use QSVVPPContext as base context in other QSV
filters in the future so that we may re-use functions defined in
qsvvpp.c for other QSV filters.
This commit shouldn't change the functionality of vpp_qsv / overlay_qsv.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This is in preparation for reusing the code for other QSV filters. E.g.
deinterlacing_qsv may have an option array without format option
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
QSV filters may set this flag in preinit callback to turn on / off pass
through mode
This is in preparation for reusing the code for other QSV filters. E.g.
scale_qsv filter doesn't support pass through mode.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Set the expected default value for options in this callback, hence we
have the right values even if these options are not included in the
option arrray.
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Special values are:
0 = original width/height
-1 = keep original aspect
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This patch provides default value if the expression is NULL.
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Also fix the naming style in enum var_name.
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This ensures all defined types are supported, even if only to report
their presence and print their size if a custom implementation is
not added.
Signed-off-by: James Almer <jamrial@gmail.com>
The current code will apply them if the options string does not contain
a 'flags' substring, and will do so by appending the graph-level option
string to the filter option string (with the standard ':' separator).
This is flawed in at least the following ways:
- naive substring matching without actually parsing the options string
may lead to false positives (e.g. flags are specified by shorthand)
and false negatives (e.g. the 'flags' substring is not actually the
option name)
- graph-level sws options are not limited to flags, but may set
arbitrary sws options
This commit simply applies the graph-level options with
av_set_options_string() and lets them be overridden as desired by the
user-specified filter options (if any). This is also shorter and avoids
extra string handling.
This function currently treats AVFilterContext options and
filter-private options differently: the former are immediately applied,
while the latter are stored in a dictionary to be applied later.
There is no good reason for having two branches - storing all options in
the dictionary is simpler and achieves the same effect (since it is
later applied with av_opt_set_dict()).
This will also be useful in future commits.
Current code first sets AVFilterContext-level options, then aplies the
leftover on the filter's private data. This is unnecessary, applying the
options to AVFilterContext with the AV_OPT_SEARCH_CHILDREN flag
accomplishes the same effect.
It appears faster than the iterative method on my machine (1.06x
faster), so I'm guessing compilers improved over time (the iterative
version was slightly faster in the past).
Currently, in case of equality on the first color channel, the order of
the ref colors is defined by the hashing function. This commit makes the
sorting deterministic and improve the hierarchical ordering.
This variable is used only for the running weight (used to reach the
target median). The places where we actually need the box weight are
changed to use box->weight.
The variance computation is simple enough now (since we can use the axis
squared errors) that it doesn't need to have a complex lazy computation
logic.
The equalities in the w{r,g,b} range checks make sure longest is never
0. Even if the alpha ended up being selected in get_next_color() it
would cause underread memory accesses in its caller (colormap_insert).
This reverts commit dea673d0d5.
This change cannot work for several reasons, the most obvious ones are:
- the alpha is being part of the scoring of the color difference, even
though we can not interpret the alpha as part of the perception of the
color (we don't even know if it's premultiplied or postmultiplied)
- the colors are averaged with their alpha value which simply cannot
work
The command proposed in the original thread of the patch actually
produces a completely broken file:
ffmpeg -y -loglevel verbose -i fate-suite/apng/o_sample.png -filter_complex "split[split1][split2];[split1]palettegen=max_colors=254:use_alpha=1[pal1];[split2][pal1]paletteuse=use_alpha=1" -frames:v 1 out.png
We can see that many color pixels are off, but more importantly some
colors have a random alpha value: https://imgur.com/eFQ2UK7
I don't see any easy fix for this unfortunately, the approach appears to
be flawed by design.
This filter, when used in the "pad" mode, currently makes the
distinction between limited and full range solely by testing for YUVJ
pixel formats at link setup time. This is deprecated and should be
improved to perform the detection based on the per-frame metadata.
In order to make this distinction based on color range metadata, which
is only known at the time of filtering frames, for simplicity, we simply
allocate two copies of the "black" frame - one for limited range and the
other for full range metadata. This could be done more dynamically (e.g.
as-needed or simply by blitting the appropriate pixel value directly),
but this change is relatively simple and preserves the structure of the
existing code.
This commit actually fixes a bug in FATE - the new output is correct for
the first time. The previous md5 ref was of a frame that incorrectly
combined full-range pixel data with limited-range black fields. The
corresponding result has been updated.
Signed-off-by: Niklas Haas <git@haasn.dev>
This filter currently makes the distinction between limited and full
range by testing for the deprecated YUVJ pixel formats at link setup
time. This is deprecated and should be improved to perform the detection
based on the per-frame metadata.
Rewrite it to calculate the black pixel threshold at the time of
filtering a frame, when metadata about the frame's color range is known.
Doing it this way has the small side benefit of being able to handle
streams with variable metadata, and is not a meaningful performance
cost.
Signed-off-by: Niklas Haas <git@haasn.dev>
Bugfix: The OpenVino DNN backend in the 'async' mode sets
'task->inference_done' to 'complete' prior to data copy from
OpenVino output buffer to task's output frame.
This order causes task destroy in ff_dnn_get_result_common()
prior to model output processing.
Signed-off-by: Rafik Saliev <rafik.f.saliev@intel.com>
PI, PHI and E are defined in libavutil/eval.c, and user may use these
constants for scale_qsv filter, so we needn't re-define these variables
in vf_scale_qsv.c
Reviewed-by: Gyan Doshi <ffmpeg@gyani.pro>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
When process yuv420 frames, FFmpeg uses same alignment on Y/U/V
planes. VPL and MSDK use Y plane's pitch / 2 as U/V planes's
pitch, which makes U/V planes 16-bytes aligned. We need to set
a separate alignment to meet runtime's behaviour.
Now alignment is changed to 16 so that the linesizes of U/V planes
meet the requirment of VPL/MSDK. Add get_buffer.video callback to
qsv filters to change the default get_buffer behaviour.
Now the commandline works fine:
ffmpeg -f rawvideo -pix_fmt yuv420p -s:v 3082x1884 \
-i ./3082x1884.yuv -vf 'vpp_qsv=w=2466:h=1508' -f rawvideo \
-pix_fmt yuv420p 2466_1508.yuv
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
