Callers currently have two ways of adding filters to a graph - they can
either
- create, initialize, and link them manually
- use one of the avfilter_graph_parse*() functions, which take a
(typically end-user-written) string, split it into individual filter
definitions+options, then create filters, apply options, initialize
filters, and finally link them - all based on information from this
string.
A major problem with the second approach is that it performs many
actions as a single atomic unit, leaving the caller no space to
intervene in between. Such intervention would be useful e.g. to
- modify filter options;
- supply hardware device contexts;
both of which typically must be done before the filter is initialized.
Callers who need such intervention are then forced to invent their own
filtergraph parsing, which is clearly suboptimal.
This commit aims to address this problem by adding a new modular
filtergraph parsing API. It adds a new avfilter_graph_segment_parse()
function to parse a string filtergraph description into an intermediate
tree-like representation (AVFilterGraphSegment and its children).
This intermediate form may then be applied step by step using further
new avfilter_graph_segment*() functions, with user intervention possible
between each step.
This script generates the current general assembly voters according to
the criteria of '20 commits in the last 36 months'.
Signed-off-by: J. Dekker <jdek@itanimul.li>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
libavutil/color_utils contains some avpriv_ symbols that map
enum AVTransferCharacteristic values to gamma-curve approximations and
to the actual transfer functions to invert them (i.e. -> linear).
There's two issues with this:
(1) avpriv is evil and should be avoided whenever possible
(2) libavutil/csp.h exposes a public API for handling color that
already handles primaries and matricies
I don't see any reason this API has to be private, so this commit takes
the functionality from avutil/color_utils and merges it into avutil/csp
with an exposed av_ API rather than the previous avpriv_ API.
Every reference to the previous API has been updated to point to the
new one. color_utils.h has been deleted as well. This should not break
any applications as it only contained avpriv_ symbols in the first
place, so nothing in that header could be referenced by other
applications.
Signed-off-by: Leo Izen <leo.izen@gmail.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Analogous to -enc_stats*, but happens right before muxing. Useful
because bitstream filters and the sync queue can modify packets after
encoding and before muxing. Also has access to the muxing timebase.
Current HLS implementation simply skip a failed segment to catch up
the stream, but this is not optimal for some use cases like livestream
recording.
Add an option to retry a failed segment to ensure the output file is
a complete stream.
Signed-off-by: gnattu <gnattuoc@me.com>
Reviewed-by: Steven Liu <liuqi05@kuaishou.com>
Splits the currently handled subtitle at random access point
packets that can be configured to follow a specific output stream.
Currently only subtitle streams which are directly mapped into the
same output in which the heartbeat stream resides are affected.
This way the subtitle - which is known to be shown at this time
can be split and passed to muxer before its full duration is
yet known. This is also a drawback, as this essentially outputs
multiple subtitles from a single input subtitle that continues
over multiple random access points. Thus this feature should not
be utilized in cases where subtitle output latency does not matter.
Co-authored-by: Andrzej Nadachowski <andrzej.nadachowski@24i.com>
Co-authored-by: Bernard Boulay <bernard.boulay@24i.com>
Signed-off-by: Jan Ekström <jan.ekstrom@24i.com>
This is intended to be a more convenient replacement for
reordered_opaque.
Add support for it in the two encoders that offer
AV_CODEC_CAP_ENCODER_REORDERED_OPAQUE: libx264 and libx265. Other
encoders will be supported in future commits.
This reverts commit dea673d0d5.
This change cannot work for several reasons, the most obvious ones are:
- the alpha is being part of the scoring of the color difference, even
though we can not interpret the alpha as part of the perception of the
color (we don't even know if it's premultiplied or postmultiplied)
- the colors are averaged with their alpha value which simply cannot
work
The command proposed in the original thread of the patch actually
produces a completely broken file:
ffmpeg -y -loglevel verbose -i fate-suite/apng/o_sample.png -filter_complex "split[split1][split2];[split1]palettegen=max_colors=254:use_alpha=1[pal1];[split2][pal1]paletteuse=use_alpha=1" -frames:v 1 out.png
We can see that many color pixels are off, but more importantly some
colors have a random alpha value: https://imgur.com/eFQ2UK7
I don't see any easy fix for this unfortunately, the approach appears to
be flawed by design.
New option can be used to avoid creating very short segments with inputs
whose GOP size is variable or unharmonic with segment_time.
Only effective with segment_time.
We shouldn't be providing links to unverified and non-FFmpeg-controlled
content in our documentation.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
We do not endorse or recommend specific third party servers or companies
that users' data will be funneled through.
It is also incorrectly describing how FFmpeg currently works.
Should have been part of 412922cc6f but was
missed.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Bring it up to date with current practice, as the current text does not
cover everything. Drop the reference to unprefixed exported symbols,
which do not exist anymore.
It describes a general development policy, not code formatting. It is
also not true, as these days we tend to prioritize correctness, safety,
and completeness over code size.
It is currently very out of touch with reality.
* declare we are using C99 fully, rather than C90 plus extensions
* mention our use of stdatomic.h
* mention forbidden C99 features, like VLAs and complex numbers
VP6 alpha in EA format is a second VP6 encoded video stream where only the Y
component is used and is interpreted as the alpha channel of the first VP6
stream. The alpha VP6 stream is muxed separately from the main VP6 stream, has
its own stream headers and packet headers. In theory the two streams might not
even have the same resolution (although most likely that is not something that
is seen or supported in the wild), but the format is capable of doing it.
Merged VP6 alpha (also known as the VP6A codec) means that a packet of the
video stream contains the corresponding packet of both VP6 substreams like
this:
{OffsetOfAlpha, DataPacket, AlphaDataPacket}
So data and alpha data of a frame is merged to a single packet, this is how VP6
video with alpha is muxed in FLV and SWF.
The first approach is more like how the demuxer sees data in the EA format,
unfortunately it is different to what the FLV or SWF format expects, so -
having no better place for it in the framework - I decided to do an optional
format conversion in the EA demuxer.
Signed-off-by: Marton Balint <cus@passwd.hu>
Add qsv_transcode example which shows how to use qsv to do hardware
accelerated transcoding, also show how to dynamically set encoding
parameters.
examples:
Normal usage:
qsv_transcode input.mp4 h264_qsv output.mp4 "g 60"
Dynamic setting usage:
qsv_transcode input.mp4 hevc_qsv output.mp4 "g 60 asyne_depth 1"
100 "g 120"
This command initializes codec with gop_size 60 and change it to
120 after 100 frames
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
As a result of a typo in the source code, this option was completely
non-functional. In order to fix it, without breaking the current default
behavior, explicitly change this default to 0.
This behavior is also consistent with how other scale filters behave by
default, so it's probably best to enshrine it anyways.
Drop the reference to directly committing code, because
- it is highly discouraged and very rarely done these days
- there is no good reason NOT to submit patches for review
Add a link to the development policy chapter.
Add skip_frame support to qsvenc. Use per-frame metadata
"qsv_skip_frame" to control it. skip_frame option defines the behavior
of qsv_skip_frame.
no_skip: Frame skipping is disabled.
insert_dummy: Encoder inserts into bitstream frame where all macroblocks
are encoded as skipped.
insert_nothing: Similar to insert_dummy, but encoder inserts nothing.
The skipped frames are still used in brc. For example, gop still include
skipped frames, and the frames after skipped frames will be larger in
size.
brc_only: skip_frame metadata indicates the number of missed frames
before the current frame.
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
There are sill many users of these APIs within libav*, so this commit
introduced too many deprecation warnings, making compilation too noisy and
potentially hiding legit warnings.
Once the remaining users are ported, this can be reapplied.
This reverts commit 76d0038579.
This is a more explicit iteration API rather than using the "magic"
av_dict_get(d, "", t, AV_DICT_IGNORE_SUFFIX) which is not really
trivial to grasp what it does when casually reading through code.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
summary: This patch modifies the `curves` filter with new `interp` option
to let user pick the existing natural cubic spline interpolation
and the new PCHIP interapolation.
reason: The natural cubic spline does not impose monotonicity between
the keypoints. As such, the fitted curve may vary wildly against
user's intension. The PCHIP interpolation is not as smooth as
the natural spline but guarantees the monotonicity. Providing
both options enhances users experience (e.g., reduces the number
of keypoints to realize the desired curve). See the related bug
report for the example of an ill-interpolated curve.
alternate solution:
Both Photoshop and GIMP appear to use monotonic interpolation in
their curve tools, which were the models for this filter. As
such, an alternate solution is to drop the natural spline and
go without the `interp` option.
related bug report: https://trac.ffmpeg.org/ticket/9947 (filed by myself)
Signed-off-by: Takeshi (Kesh) Ikuma <tikuma@hotmail.com>
This enables overriding the rotation as well as horizontal/vertical
flip state of a specific video stream on the input side.
Additionally, switch the singular test that was utilizing the rotation
metadata to instead override the input display rotation, thus leading
to the same result.
The FF_PAD_STRUCTURE is too complex for doxygen to be able to
properly handle, resulting in completely broken AVBPrint documentation.
To fix that, tell Doxygen what to expand that macro to.
Add an AV_PIX_FMT_NE macro for RGB32FBE/RGB32FLE and also one for
RGBA32FBE/RGBA32FLE for packed 32-bit float RGB samples, and also
packed 32-bit float RGBA samples, respectively.
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Leo Izen <leo.izen@gmail.com>
Add PCR at keyframe can be undesirable when -pcr_period is
specified. Add an flag to disable this behavior.
Signed-off-by: Zhao Zhili <zhilizhao@tencent.com>
Apparently this option was intended (the context contains a
currently-unused frame_rate field), but was never added. This results in
the output timebase being unset after config_output(), so the input
audio timebase ends up being used for video output, which is clearly
wrong.
Add an option for setting output video framerate. Also set output frame
durations.
It has been deprecated in favor of the aresample filter for almost 10
years.
Another thing this option can do is drop audio timestamps and have them
generated by the encoding code or the muxer, but
- for encoding, this can already be done with the setpts filter
- for muxing this should almost never be done as timestamp generation by
the muxer is deprecated, but people who really want to do this can use
the setts bitstream filter
Users can't make anything with its content.
Making it opaque might allow us to avoid one level of indirection.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
avcodec_enum_to_chroma_pos() and avcodec_chroma_pos_to_enum()
deal with enum AVChromaLocation which is defined in lavu.
These functions are therefore replaced by
av_chroma_location_enum_to_pos() and av_chroma_location_pos_to_enum().
This commit provides the necessary deprecations. Also already make
these functions wrappers around the corresponding lavu functions
as not doing so would force one to disable deprecation warnings.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
They are intended as replacements for avcodec_enum_to_chroma_pos()
and avcodec_chroma_pos_to_enum().
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
They are also frequently used in libavformat.
This change does not cause any breakage as avcodec.h
includes defs.h.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
There is no check for whether these supposedly redundant PPS
are actually redundant. One could check via memcmp which would
work in practice* (because all content buffers are initially
zero-allocated), but this is not portable as compilers may
trash padding inside structures as they wish.
In case the PPS is not really redundant the output is garbage.
This happens with several files from the FATE-suite. E.g.
h264-conformance/CVCANLMA2_Sony_C.jsv doesn't decode correctly
any more, whereas h264-conformance/CABA3_TOSHIBA_E.264 even
fails in ff_cbs_write_packet(), because the inferred value
of num_ref_idx_l0_active_minus1 mismatches with the value set
in the slice (this happens when num_ref_idx_l0_default_active_minus1
changes in the PPS; the value in the slice header is inferred from
the original PPS's num_ref_idx_l0_default_active_minus1).
*: Unless slice_group_id is used, i.e. unless slice_group_map_type
is six.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
~4x faster than the C version.
The shuffles in the 15pt dim1 are seriously expensive. Not happy with it,
but I'm contempt.
Can be easily converted to pure AVX by removing all vpermpd/vpermps
instructions.
this maps to the vpxenc argument with the same name and the
VP9E_SET_MIN_GF_INTERVAL codec control
Signed-off-by: James Zern <jzern@google.com>
Reviewed-by: Vignesh Venkatasubramanian <vigneshv@google.com>
Add "slice" intra refresh type to h264_qsv and hevc_qsv. This type means
horizontal refresh by slices without overlapping. Also update the doc.
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Currently AVBR is disabled and VBR is the default method if maxrate is
not specified on Linux, but AVBR is the default one if maxrate is not
specified on Windows. In order to make user experience better accross
Linux and Windows, use VBR by default on Windows if maxrate is not
specified. User need to set both avbr_accuracy and avbr_convergence to
non-zero explicitly and not to specify maxrate if AVBR is expected.
In addition, AVBR works for H264 and HEVC only in the SDK.
$ ffmpeg.exe -v verbose -f lavfi -i yuvtestsrc -vf "format=nv12" -c:v
vp9_qsv -f null -
These are the formats we want/need to use when dealing with the Intel
VAAPI decoder for 12bit 4:2:0, 12bit 4:2:2, 10bit 4:4:4 and 12bit 4:4:4
respectively.
As with the already supported Y210 and YUVX (XVUY) formats, they are
based on formats Microsoft picked as their preferred 4:2:2 and 4:4:4
video formats, and Intel ran with it.
P12 and Y212 are simply an extension of 10 bit formats to say 12 bits
will be used, with 4 unused bits instead of 6.
XV30, and XV36, as exotic as they sound, are variants of Y410 and Y412
where the alpha channel is left formally undefined. We prefer these
over the alpha versions because the hardware cannot actually do
anything with the alpha channel and respecting it is just overhead.
Y412/XV46 is a normal looking packed 4 channel format where each
channel is 16bits wide but only the 12msb are used (like P012).
Y410/XV30 packs three 10bit channels in 32bits with 2bits of alpha,
like A/X2RGB10 style formats. This annoying layout forced me to define
the BE version as a bitstream format. It seems like our pixdesc
infrastructure can handle the LE version being byte-defined, but not
when it's reversed. If there's a better way to handle this, please
let me know. Our existing X2 formats all have the 2 bits at the MSB
end, but this format places them at the LSB end and that seems to be
the root of the problem.
It has been deprecated in b4f59beeb4,
but the attribute_deprecated was not set and there was no entry
in APIchanges. This commit adds these and schedules it for removal.
Given that the reason behind the deprecation is exactly the same
as in av_fopen_utf8(), reuse its FF_API_AV_FOPEN_UTF8.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
This is the alphaless version of VUYA that I introduced recently. After
further discussion and noting that the Intel vaapi driver explicitly
lists XYUV as a support format for encoding and decoding 8bit 444
content, we decided to switch our usage and avoid the overhead of
having a declared alpha channel around.
Note that I am not removing VUYA, as this turned out to have another
use, which was to replace the need for v408enc/dec when dealing with
the format.
The vaapi switching will happen in the next change
Add adaptive_i/b feature to hevc_qsv. Adaptive_i allows changing of
frame type from P and B to I. Adaptive_b allows changing of frame type
frome B to P.
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
According to its documentation it returns "pts of the last muxed packet
+ its duration", but the value it actually returns right now is
(possibly guessed) dts after muxer-internal bitstream filtering (if
any).
This function was added for ffmpeg.c, but it is not used there anymore.
Since the value it returns is ill-defined and so inappropriate for any
serious use, deprecate it.
The present default value of 0 will render the overlay video invisible.
A default of 1.0 is consistent with most common use cases.
Signed-off-by: Fei Wang <fei.w.wang@intel.com>
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The "AYUV" format is defined by Microsoft as their preferred format for
4:4:4 content, and so it is the format used by Intel VAAPI and QSV.
As Microsoft like to define their byte ordering in little-endian
fashion, the memory order is reversed, and so our pix_fmt, which
follows memory order, has a reversed name (VUYA).
This functionally already exists, but as pointed out in #9672 and #9673,
requiring users to manually include filters is clumsy, error-prone and
hard to use together with tools like ffplay.
To streamline ICC profile support, add a new AVCodecContext flag to
globally enable reading and writing ICC profiles, automatically, for all
appropriate media types.
Note that this commit only includes the new API. The implementation is
split off to separate commits for readability.
Signed-off-by: Niklas Haas <git@haasn.dev>
The -shortest option (which finishes the output file at the time the
shortest stream ends) is currently implemented by faking the -t option
when an output stream ends. This approach is fragile, since it depends
on the frames/packets being processed in a specific order. E.g. there
are currently some situations in which the output file length will
depend unpredictably on unrelated factors like encoder delay. More
importantly, the present work aiming at splitting various ffmpeg
components into different threads will make this approach completely
unworkable, since the frames/packets will arrive in effectively random
order.
This commit introduces a "sync queue", which is essentially a collection
of FIFOs, one per stream. Frames/packets are submitted to these FIFOs
and are then released for further processing (encoding or muxing) when
it is ensured that the frame in question will not cause its stream to
get ahead of the other streams (the logic is similar to libavformat's
interleaving queue).
These sync queues are then used for encoding and/or muxing when the
-shortest option is specified.
A new option – -shortest_buf_duration – controls the maximum number of
queued packets, to avoid runaway memory usage.
This commit changes the results of the following tests:
- copy-shortest[12]: the last audio frame is now gone. This is
correct, since it actually outlasts the last video frame.
- shortest-sub: the video packets following the last subtitle packet are
now gone. This is also correct.
Using parameter from AVCodecContext to reset qsv codec is more suitable
for MFXVideoENCODE_Reset()'s usage. Per-frame metadata is more suitable
for the usage of mfxEncodeCtrl being passed to
MFXVideoENCODE_EncodeFrameAsync(). Now change it to use the value
from AVCodecContext.
Because q->param is passed to both "in" and "out" parameters when call
MFXVideoENCODE_Query(), the value in q->param may be changed. New
variables are added to store old configuration, so that we can detect
real parameter change.
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The only duration field currently present in AVFrame is pkt_duration,
which is semantically restricted to those frames that are output by
decoders.
Add a new field that stores the frame's duration without regard for how
that frame was produced. Deprecate pkt_duration.
This is a per-file input option that adjusts an input's timestamps
with reference to another input, so that emitted packet timestamps
account for the difference between the start times of the two inputs.
Typical use case is to sync two or more live inputs such as from capture
devices. Both the target and reference input source timestamps should be
based on the same clock source.
If either input lacks starting timestamps, then no sync adjustment is made.
GSoC'22
libavfilter/vf_chromakey_cuda.cu:the CUDA kernel for the filter
libavfilter/vf_chromakey_cuda.c: the C side that calls the kernel and gets user input
libavfilter/allfilters.c: added the filter to it
libavfilter/Makefile: added the filter to it
cuda/cuda_runtime.h: added two math CUDA functions that are used in the filter
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
Add a short hand parameter for making a fixed size grid. The existing
xstack layout parameter syntax gets tedious if all one wants is a
matrix like grid of the input streams. Add a grid option to the xstack
filter that simplifies this use case by simply specifying the number of
rows and columns instead of specific x/y co-ordinate for each stream.
Also updating the filter documentation to explain the new option.
Signed-off-by: Vignesh Venkatasubramanian <vigneshv@google.com>
-shortest stops 'recording' when the shortest output stream ends. The
native or even seek-adjusted duration of the source input stream isn't
considered.
This allows for wider compatibility with older devices, such as those
running iOS 3. The only difference between HLS version 2 and version 3 is
that version 3 supports non-integer EXTINF values, and as such, we can
default to version 2 if we're using whole-integer EXTINFs anyways, when
`-hls_flags round_durations` is set.
As this code seems to otherwise consistently use the lowest compatible
version, this seems to fit in properly with existing behavior.
Testing confirms with that this patch, HLS output can work all the way back
to iOS 3.
Reviewed-by: Steven Liu <liuqi05@kuaishou.com>
Signed-off-by: Lucy <lucy@absolucy.moe>
Enable dynamic QP configuration in runtime on qsv encoder. Through
AVFrame->metadata, we can set key "qsv_config_qp" to change QP
configuration when we encode video in CQP mode.
Signed-off-by: Yue Heng <yue.heng@intel.com>
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Reduces default fragment size from the pulse audio default of 2 sec to 50 ms.
This also has an effect on the size of the returned frames, which will be
around 50 ms as well, making timestamps more accurate.
This should fix the regression in ticket #9776.
Pulseaudio timestamps for monitor sources are still pretty inaccurate for me,
but I don't see how else should we query latencies from the library.
Signed-off-by: Marton Balint <cus@passwd.hu>
