In order to fine-control referencing schemes in VP9 encoding, there
is a need to use VP9E_SET_SVC_REF_FRAME_CONFIG method. This commit
provides a way to use the API through frame metadata.
The AC3 encoder used to be a separate library called "Aften", which
got merged into libavcodec (literally, SVN commits and all).
The merge preserved as much features from the library as possible.
The code had two versions - a fixed point version and a floating
point version. FFmpeg had floating point DSP code used by other
codecs, the AC3 decoder including, so the floating-point DSP was
simply replaced with FFmpeg's own functions.
However, FFmpeg had no fixed-point audio code at that point. So
the encoder brought along its own fixed-point DSP functions,
including a fixed-point MDCT.
The fixed-point MDCT itself is trivially just a float MDCT with a
different type and each multiply being a fixed-point multiply.
So over time, it got refactored, and the FFT used for all other codecs
was templated.
Due to design decisions at the time, the fixed-point version of the
encoder operates at 16-bits of precision. Although convenient, this,
even at the time, was inadequate and inefficient. The encoder is noisy,
does not produce output comparable to the float encoder, and even
rings at higher frequencies due to the badly approximated winow function.
Enter MIPS (owned by Imagination Technologies at the time). They wanted
quick fixed-point decoding on their FPUless cores. So they contributed
patches to template the AC3 decoder so it had both a fixed-point
and a floating-point version. They also did the same for the AAC decoder.
They however, used 32-bit samples. Not 16-bits. And we did not have
32-bit fixed-point DSP functions, including an MDCT. But instead of
templating our MDCT to output 3 versions (float, 32-bit fixed and 16-bit fixed),
they simply copy-pasted their own MDCT into ours, and completely
ifdeffed our own MDCT code out if a 32-bit fixed point MDCT was selected.
This is also the status quo nowadays - 2 separate MDCTs, one which
produces floating point and 16-bit fixed point versions, and one
sort-of integrated which produces 32-bit MDCT.
MIPS weren't all that interested in encoding, so they left the encoder
as-is, and they didn't care much about the ifdeffery, mess or quality - it's
not their problem.
So the MDCT/FFT code has always been a thorn in anyone looking to clean up
code's eye.
Backstory over. Internally AC3 operates on 25-bit fixed-point coefficients.
So for the floating point version, the encoder simply runs the float MDCT,
and converts the resulting coefficients to 25-bit fixed-point, as AC3 is inherently
a fixed-point codec. For the fixed-point version, the input is 16-bit samples,
so to maximize precision the frame samples are analyzed and the highest set
bit is detected via ac3_max_msb_abs_int16(), and the coefficients are then
scaled up via ac3_lshift_int16(), so the input for the FFT is always at least 14 bits,
computed in normalize_samples(). After FFT, the coefficients are scaled up to 25 bits.
This patch simply changes the encoder to accept 32-bit samples, reusing
the already well-optimized 32-bit MDCT code, allowing us to clean up and drop
a large part of a very messy code of ours, as well as prepare for the future lavu/tx
conversion. The coefficients are simply scaled down to 25 bits during windowing,
skipping 2 separate scalings, as the hacks to extend precision are simply no longer
necessary. There's no point in running the MDCT always at 32 bits when you're
going to drop 6 bits off anyway, the headroom is plenty, and the MDCT rounds
properly.
This also makes the encoder even slightly more accurate over the float version,
as there's no coefficient conversion step necessary.
SIZE SAVINGS:
ARM32:
HARDCODED TABLES:
BASE - 10709590
DROP DSP - 10702872 - diff: -6.56KiB
DROP MDCT - 10667932 - diff: -34.12KiB - both: -40.68KiB
DROP FFT - 10336652 - diff: -323.52KiB - all: -364.20KiB
SOFTCODED TABLES:
BASE - 9685096
DROP DSP - 9678378 - diff: -6.56KiB
DROP MDCT - 9643466 - diff: -34.09KiB - both: -40.65KiB
DROP FFT - 9573918 - diff: -67.92KiB - all: -108.57KiB
ARM64:
HARDCODED TABLES:
BASE - 14641112
DROP DSP - 14633806 - diff: -7.13KiB
DROP MDCT - 14604812 - diff: -28.31KiB - both: -35.45KiB
DROP FFT - 14286826 - diff: -310.53KiB - all: -345.98KiB
SOFTCODED TABLES:
BASE - 13636238
DROP DSP - 13628932 - diff: -7.13KiB
DROP MDCT - 13599866 - diff: -28.38KiB - both: -35.52KiB
DROP FFT - 13542080 - diff: -56.43KiB - all: -91.95KiB
x86:
HARDCODED TABLES:
BASE - 12367336
DROP DSP - 12354698 - diff: -12.34KiB
DROP MDCT - 12331024 - diff: -23.12KiB - both: -35.46KiB
DROP FFT - 12029788 - diff: -294.18KiB - all: -329.64KiB
SOFTCODED TABLES:
BASE - 11358094
DROP DSP - 11345456 - diff: -12.34KiB
DROP MDCT - 11321742 - diff: -23.16KiB - both: -35.50KiB
DROP FFT - 11276946 - diff: -43.75KiB - all: -79.25KiB
PERFORMANCE (10min random s32le):
ARM32 - before - 39.9x - 0m15.046s
ARM32 - after - 28.2x - 0m21.525s
Speed: -30%
ARM64 - before - 36.1x - 0m16.637s
ARM64 - after - 36.0x - 0m16.727s
Speed: -0.5%
x86 - before - 184x - 0m3.277s
x86 - after - 190x - 0m3.187s
Speed: +3%
Add 2 new options:
- reconnect_on_http_error - a list of http status codes that should be
retried. the list can contain explicit status codes / the strings
4xx/5xx.
- reconnect_on_network_error - reconnects on arbitrary errors during
connect, e.g. ECONNRESET/ETIMEDOUT
the retry employs the same exponential backoff logic as the existing
reconnect/reconnect_at_eof flags.
related tickets:
https://trac.ffmpeg.org/ticket/6066https://trac.ffmpeg.org/ticket/7768
Signed-off-by: Marton Balint <cus@passwd.hu>
Do it only when requested with the AV_CODEC_EXPORT_DATA_VIDEO_ENC_PARAMS
flag.
Drop previous code using the long-deprecated AV_FRAME_DATA_QP_TABLE*
API. Temporarily disable fate-filter-pp, fate-filter-pp7,
fate-filter-spp. They will be reenabled once these filters are converted
in following commits.
because hls_enc_key and hls_enc_iv get 16byte char
for example:
-hls_enc_key 0123456789abcdef -hls_enc_iv abcdefghijklmnop
Reviewed-by: Gyan Doshi <ffmpeg@gyani.pro>
Signed-off-by: Steven Liu <liuqi05@kuaishou.com>
At present, progress stats are updated at a hardcoded interval of
half a second. For long processes, this can lead to bloated
logs and progress reports.
Users can now set a custom period using option -stats_period
Default is kept at 0.5 seconds.
The cropdetect filter, at present, skips the first two frames. This
behaviour is hardcoded.
New option 'skip' allows users to change this. Convenient for when
input is a single image or a trimmed video stream.
Default is kept at 2 to preserve current behaviour.
the init.mp4 can be expanded with strftime the same way as
hls_segment_filename.
Signed-off-by: Nikola Pajkovsky <nikola@pajkovsky.cz>
Signed-off-by: liuqi05 <liuqi05@kuaishou.com>
Also changes some default values for options after this change.
This makes distinction between feedback and wet option.
Before they would produce same output if values were swapped.
They add considerable complexity to frame-threading implementation,
which includes an unavoidably leaking error path, while the advantages
of this option to the users are highly dubious.
It should be always possible and desirable for the callers to make their
get_buffer2() implementation thread-safe, so deprecate this option.
This patch introduces a new frame side data type AVFilmGrainParams for use
with video codecs which support it.
It can save a lot of memory used for duplicate processed reference frames and
reduce copies when applying film grain during presentation.
This patch adds a select_region option to the xcbgrab input device.
If set to 1, the user will be prompted to select the grabbing area
graphically by clicking and dragging. A rectangle will be drawn to
mark the grabbing area. A single click with no dragging will select
the whole screen. The option overwrites the video_size, grab_x, and
grab_y options if set by the user.
For testing, just set the select_region option as follows:
ffmpeg -f x11grab -select_region 1 -i :0.0 output.mp4
The drawing happens directly on the root window using standard rubber
banding techniques, so it is very efficient and doesn't depend on any
X extensions or compositors.
Reviewed-by: Andriy Gelman <andriy.gelman@gmail.com>
Signed-off-by: Omar Emara <mail@OmarEmara.dev>
This way the old max queue size limit based behavior for streams
where each individual packet is large is kept, while for smaller
streams more packets can be buffered (current default is at 50
megabytes per stream).
For some explanation, by default ffmpeg copies packets from before
the appointed seek point/start time and puts them into the local
muxing queue. Before, it getting utilized was much less likely
since as soon as the filter chain was initialized, the encoder
(and thus output stream) was also initialized.
Now, since we will be pushing the encoder initialization to when the
first AVFrame is decoded and filtered - which only happens after
the exact seek point is hit as packets are ignored until then -
this queue will be seeing much more usage.
In more layman's terms, this attempts to fix cases such as where:
- seek point ends up being 5 seconds before requested time.
- audio is set to copy, and thus immediately begins filling the
muxing queue.
- video is being encoded, and thus all received packets are skipped
until the requested time is hit.
then we can set the rtp read timeout instead of infinite timeout.
How to test(5s timeout):
./ffprobe -i rtp://192.168.1.67:1234?timeout=5000000
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
The manual states "there is virtually no reason to use that encoder.".
It supports less sample formats than the native encoder, is less efficient
than the native encoder and is also slower and pretty much remains untested.
libwavpack also isn't being fuzzed, which given that we plug the parameters
without any sanitizing them looks concerning.
A common pattern e.g. in libavcodec is replacing/updating buffer
references: unref old one, ref new one. This function allows simplifying
such code and avoiding unnecessary refs+unrefs if the references are
already equivalent.
Allow to set the EOF timestamp.
Also: doc/filters/testsrc*: specify the rounding of the duration option.
The changes in the ref files are right.
For filter-fps-down, the graph is testsrc2=r=7:d=3.5,fps=3.
3.5=24.5/7, so the EOF of testsrc2 will have PTS 25/7.
25/7=(10+5/7)/3, so the EOF PTS for fps should be 11/7,
and the output should contain a frame at PTS 10.
For filter-fps-up, the graph is testsrc2=r=3:d=2,fps=7,
for filter-fps-up-round-down and filter-fps-up-round-up
it is the same with explicit rounding options.
But there is no rounding: testsrc2 produces exactly 6 frames
and 2 seconds, fps converts it into exactly 14 frames.
The tests should probably be adjusted to restore them to
a useful coverage.
Expressions for option fontsize of video filter drawtext have been
supported since commit 6442e4ab3c.
Signed-off-by: Andrei Rybak <rybak.a.v@gmail.com>
Revised-by: Gyan Doshi <ffmpeg@gyani.pro>
This patch allows setting a compression ratio and to
set multiple layers. The user has to input a compression
ratio for each layer.
The per layer compression ration can be set as follows:
-layer_rates "r1,r2,...rn"
for to create 'n' layers.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Requires some extraneous top side and bottom front channels to be
defined.
According to STD-B59v2, the defined channel layout is:
- FL
- FR
- FC
- LFE1
- BL
- BR
- FLc
- FRc
- BC
- LFE2
- SiL
- SiR
- TpFL
- TpFR
- TpFC
- TpC
- TpBL
- TpBR
- TpSiL
- TpSiR
- TpBC
- BtFC
- BtFL
- BtFR
Dimensions are normally specified as width x height, and this will match
the same option to libaom-av1.
Remove the indirection through the private context at the same time.
The tile_rows/cols options currently do a confusingly different thing to
the options of the same name on other encoders like libvpx and libaom.
There is no backward-compatibility reason to implement the log2 behaviour
as there was for libaom, so just get rid of them entirely.
Threaded input can increase smoothness of e.g. x11grab significantly. Before
this patch, in order to activate threaded input the user had to specify a
"dummy" additional input, with this change it is no longer required.
Signed-off-by: Marton Balint <cus@passwd.hu>
This utility helps avoid undefined behavior when doing things like
checking how much memory we need to allocate for an image before we have
allocated a buffer.
Signed-off-by: Brian Kim <bkkim@google.com>
Signed-off-by: James Almer <jamrial@gmail.com>
remove the timeout option docs part for HTTP protocol and add
auth_type option part.
Reviewed-by: Gyan Doshi <ffmpeg@gyani.pro>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Some legacy applications such as AVI2MVE expect raw RGB bitmaps
to be stored bottom-up, whereas our RIFF BITMAPINFOHEADER assumes
they are always stored top-down and thus write a negative value
for height. This can prevent reading of these files.
Option flipped_raw_rgb added to AVI and Matroska muxers
which will write positive value for height when enabled.
Note that the user has to flip the bitmaps beforehand using other
means such as the vflip filter.
broken since:
aa5c6f382b avcodec/libaomenc: Add command-line options to control the use of partition tools
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: James Zern <jzern@google.com>
This is the only use of 'FontName' with that capitalization, as both
source-code and tests use 'Fontname'. Having consistent capitalization
makes it easier to find the relevant source from the docs.
See these examples for other uses:
libavcodec/ass_split.c:68
tests/ref/fate/sub-cc:9
We can try with the srcnn model from sr filter.
1) get srcnn.pb model file, see filter sr
2) convert srcnn.pb into openvino model with command:
python mo_tf.py --input_model srcnn.pb --data_type=FP32 --input_shape [1,960,1440,1] --keep_shape_ops
See the script at https://github.com/openvinotoolkit/openvino/tree/master/model-optimizer
We'll see srcnn.xml and srcnn.bin at current path, copy them to the
directory where ffmpeg is.
I have also uploaded the model files at https://github.com/guoyejun/dnn_processing/tree/master/models
3) run with openvino backend:
ffmpeg -i input.jpg -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=openvino:model=srcnn.xml:input=x:output=srcnn/Maximum -y srcnn.ov.jpg
(The input.jpg resolution is 720*480)
Also copy the logs on my skylake machine (4 cpus) locally with openvino backend
and tensorflow backend. just for your information.
$ time ./ffmpeg -i 480p.mp4 -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=tensorflow:model=srcnn.pb:input=x:output=y -y srcnn.tf.mp4
…
frame= 343 fps=2.1 q=31.0 Lsize= 2172kB time=00:00:11.76 bitrate=1511.9kbits/s speed=0.0706x
video:1973kB audio:187kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.517637%
[aac @ 0x2f5db80] Qavg: 454.353
real 2m46.781s
user 9m48.590s
sys 0m55.290s
$ time ./ffmpeg -i 480p.mp4 -vf format=yuv420p,scale=w=iw*2:h=ih*2,dnn_processing=dnn_backend=openvino:model=srcnn.xml:input=x:output=srcnn/Maximum -y srcnn.ov.mp4
…
frame= 343 fps=4.0 q=31.0 Lsize= 2172kB time=00:00:11.76 bitrate=1511.9kbits/s speed=0.137x
video:1973kB audio:187kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.517640%
[aac @ 0x31a9040] Qavg: 454.353
real 1m25.882s
user 5m27.004s
sys 0m0.640s
Signed-off-by: Guo, Yejun <yejun.guo@intel.com>
Signed-off-by: Pedro Arthur <bygrandao@gmail.com>
This patch adds the control for enabling rectangular partitions, 1:4/4:1
partitions and AB shape partitions.
Signed-off-by: Wang Cao <wangcao@google.com>
Signed-off-by: James Zern <jzern@google.com>
Currently, the zoompan filter exposes a 'time' variable (missing from docs) for use in
the 'zoom', 'x', and 'y' expressions. This variable is perhaps better named
'out_time' as it represents the timestamp in seconds of each output frame
produced by zoompan. This patch adds aliases 'out_time' and 'ot' for 'time'.
This patch also adds an 'in_time' (alias 'it') variable that provides access
to the timestamp in seconds of each input frame to the zoompan filter.
This helps to design zoompan filters that depend on the input video timestamps.
For example, it makes it easy to zoom in instantly for only some portion of a video.
Both the 'out_time' and 'in_time' variables have been added in the documentation
for zoompan.
Example usage of 'in_time' in the zoompan filter to zoom in 2x for the
first second of the input video and 1x for the rest:
zoompan=z='if(between(in_time,0,1),2,1):d=1'
V2: Fix zoompan filter documentation stating that the time variable
would be NAN if the input timestamp is unknown.
V3: Add 'it' alias for 'in_time. Add 'out_time' and 'ot' aliases for 'time'.
Minor corrections to zoompan docs.
Signed-off-by: exwm <thighsman@protonmail.com>
users are getting mislead by the integer, although profile
can support both const string and integer.
http://ffmpeg.org/pipermail/ffmpeg-user/2020-June/049025.html
Also fix the order of high and main, it's not my intention.
Signed-off-by: Limin Wang <lance.lmwang@gmail.com>
Currently, ffmpeg inserts scale filter by default in the filter graph
to force the whole decoded stream to scale into the same size with the
first frame. It's not quite make sense in resolution changing cases if
user wants the rawvideo without any scale.
Using autoscale/noautoscale as an output option to indicate whether auto
inserting the scale filter in the filter graph:
-noautoscale or -autoscale 0:
disable the default auto scale filter inserting.
ffmpeg -y -i input.mp4 out1.yuv -noautoscale out2.yuv -autoscale 0 out3.yuv
Update docs.
Suggested-by: Mark Thompson <sw@jkqxz.net>
Reviewed-by: Nicolas George <george@nsup.org>
Signed-off-by: U. Artie Eoff <ullysses.a.eoff@intel.com>
Signed-off-by: Linjie Fu <linjie.fu@intel.com>
Max region ID is 87. Also the region affects not only the G0 charset but G2 and
the national subset as well.
Signed-off-by: Marton Balint <cus@passwd.hu>
Use opaque iteration state instead of the previous child class. This
mirrors similar changes done in lavf/lavc.
Deprecate the av_opt_child_class_next() API.
The "-deinterlace" was deprecated since d7edd35, over eight years
ago.
Refer to deinterlacing filters instead.
Signed-off-by: Moritz Barsnick <barsnick@gmx.net>
This allows for users who derive devices to set options for the
new device context they derive.
The main use case of this is to allow users to enable extensions
(such as surface drawing extensions) in Vulkan while deriving from
the device their frames are on. That way, users don't need to write
any initialization code themselves, since the Vulkan spec invalidates
mixing instances, physical devices and active devices.
Apart from Vulkan, other hwcontexts ignore the opts argument since they
don't support options at all (or in VAAPI and OpenCL's case, options are
currently only used for device selection, which device_derive overrides).
This change makes it possible for child encoders to define custom profile
option names which can be used for setting the AVCodecContext->profile.
Also rename unit name to something rather unique, so it won't be used elsewhere.
Signed-off-by: Marton Balint <cus@passwd.hu>
This will be used for AVCodecContext->profile. By specifying constants in the
encoders we won't have to use the common AVCodecContext options table and
different encoders can use the same profile name even with different values.
Signed-off-by: Marton Balint <cus@passwd.hu>
Both are codec properties and not encoder capabilities. The relevant
AVCodecDescriptor.props flags exist for this purpose.
Signed-off-by: James Almer <jamrial@gmail.com>
This contains encoder wrappers for H264, HEVC, AAC, AC3 and MP3.
This is based on top of an original patch by wm4
<nfxjfg@googlemail.com>. The original patch supported both encoding
and decoding, but this patch only includes encoding.
The patch contains further changes by Paweł Wegner
<pawel.wegner95@gmail.com> (primarily for splitting out the encoding
parts of the original patch) and further cleanup, build compatibility
fixes and tweaks for use with Qualcomm encoders by Martin Storsjö.
Signed-off-by: Martin Storsjö <martin@martin.st>
The Matroska muxer has always mapped the title tag to the FileDescription
element for attachments streams since support for writing attachments
was added in commit c7a63a521b. This
commit merely documents this fact.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
