Writing the duration SimpleTag is special: It's size is
reserved in advance via an EBML Void element (if seekable)
and this reserved space is overwritten when writing the trailer;
it does not use put_ebml_string().
The string to write is created via snprintf on a buffer
of size 20; this buffer is then written via put_ebml_binary()
with a size of 20.
EBML strings need not be zero-terminated; if not, they
are implicitly terminated by the element's length field.
snprintf() always zero-terminates the buffer, i.e.
the last byte can be discarded when using an EBML string.
This patch does this.
The FATE changes are as expected: One byte saved for every
track; the only exception is the matroska-qt-mode test:
An additional byte is saved because an additional byte
could be saved from the enclosing Tags length field.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Do it only for video (the only thing for type for which HDR10+
makes sense).
This effectively reverts changes to several FATE ref-files
made in bda44f0f39.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
One that is fine enough to represent all DV audio sample rates. Audio
packet durations are now sample-accurate.
This largely undoes commit 76fbb0052d. To
avoid breaking the issue fixed by that commit, resync audio timestamps
against video if they get more than one frame apart. The sample from
issue #8762 still works correctly after this commit.
Slightly changes the results of the lavf-dv seektest, due to the audio
timebase being more granular.
Current code will call avpriv_set_pts_info() for each video frame,
possibly setting a different timebase if the stream framerate changes.
This violates API conventions, as the timebase is supposed to stay
constant after stream creation.
Change the demuxer to set a single timebase that is fine enough to
handle all supported DV framerates.
The seek tests change slightly because the new timebase is more
granular.
The field is not specific to Opus.
The mp2fixed encoder signals initial_padding and is used
by both the matroska-encoding-delay test as well as
the lavf-mkv tests which necessitated several FATE ref changes.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
These tables are only used by encoders and only for the current picture;
ergo they need not be put into the picture at all, but rather into
the encoder's context. They also don't need to be refcounted,
because there is only one owner.
In contrast to this, the earlier code refcounts them which
incurs unnecessary overhead. These references are not unreferenced
in ff_mpeg_unref_picture() (they are kept in order to have something
like a buffer pool), so that several buffers are kept at the same
time, although only one is needed, thereby wasting memory.
The code also propagates references to other pictures not part of
the pictures array (namely the copy of the current/next/last picture
in the MpegEncContext which get references of their own). These
references are not unreferenced in ff_mpeg_unref_picture() (the
buffers are probably kept in order to have something like a pool),
yet if the current picture is a B-frame, it gets unreferenced
at the end of ff_mpv_encode_picture() and its slot in the picture
array will therefore be reused the next time; but the copy of the
current picture also still has its references and therefore
these buffers will be made duplicated in order to make them writable
in the next call to ff_mpv_encode_picture(). This is of course
unnecessary.
Finally, ff_find_unused_picture() is supposed to just return
any unused picture and the code is supposed to work with it;
yet for the vsynth*-mpeg4-adap tests the result depends upon
the content of these buffers; given that this patchset
changes the content of these buffers (the initial content is now
the state of these buffers after encoding the last frame;
before this patch the buffers used came from the last picture
that occupied the same slot in the picture array) their ref-files
needed to be changed. This points to a bug somewhere (if one removes
the initialization, one gets uninitialized reads in
adaptive_quantization in ratecontrol.c).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
For 422 frames we should not use hard coded 8 to calculate mb size for
uv plane. Chroma shift should be taken into consideration to be
compatiple with different sampling format.
The error is reported by fate test when av_cpu_max_align() return 64
on the platform supporting AVX512. This is a hidden error and it is
exposed after commit 17a59a634c.
mpeg2enc has a mechanism to reuse frames. When it computes SSE (sum of
squared error) on current mb, reconstructed mb will be wrote to the
previous mb space, so that the memory can be saved. However if the align
is 64, the frame is shared in somewhere else, so the frame cannot be
reused and a new frame to store reconstrued data is created. Because the
height of mb is wrong when compute sse on 422 frame, starting from the
second line of macro block, changed data is read when frame is reused
(we need to read row 16 rather than row 8 if frame is 422), and unchanged
data is read when frame is not reused (a new frame is created so the
original frame will not be changed).
That is why commit 17a59a634c exposes this
issue, because it add av_cpu_max_align() and this function return 64 on
platform supporting AVX512 which lead to creating a frame in mpeg2enc,
and this lead to the different outputs.
Signed-off-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
This is possible by using a dynamic buffer to write them;
said dynamic buffer is (re)used and reset as appropriate.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Also check the (user-provided) tags for being overlong; the earlier
code had an implicit unchecked size_t->int conversion.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Up until now, the Matroska muxer did not use the dispositions it is
given as-is; instead it by default overrode the disposition of the first
track of a kind (audio, video, subtitles) if no track of this kind has
the default disposition set. And up until recently, it also enforced
by default that no more than one track of each kind be marked as
default.
The rationale for the former is that there are lots of containers which
lack the concept of default streams, so that it is not uncommon for no
stream to be marked as default at all; the rationale for the latter was
that up until recently, it was dubious whether the Matroska specification
allowed more than one default stream for track type (e.g. mkvmerge
disallowed it). It was this point which led to the implementation of
the above mentioned behaviour inspired by mkvmerge.
Yet the Matroska specifications have changed and now explicitly allow
to set more than one track of each type as default, so that the main
reason of not using the dispositions as-is was rendered moot. Therefore
this commit changes the default to pass the disposition through.
The matroska-mpegts-remux FATE-test has been updated to still use the
old "infer" mode so that it is still covered by FATE; the
matroska-zero-length-block test has also been updated to cover
the infer_no_subs mode. The references for lots of other FATE tests
needed to be updated because of a newly added FlagDefault element with
value zero (whereas a FlagDefault with value 1 needn't be coded at all,
as it coincided with the default value of said element).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
By using the frame counter (and the video time base) for audio pts we lose some
timestamp precision but we ensure that video and audio coming from the same DV
frame are always in sync.
This patch also makes timestamps after seek consistent and it should also fix
the timestamps when the audio clock is unlocked and have a completely
indpendent clock source. (E.g. runs on fixed 48009 Hz which should have been
exact 48000 Hz)
Fixes out of sync timestamps in ticket #8762.
Signed-off-by: Marton Balint <cus@passwd.hu>
The previous threshold, 4 KB, maybe was reasonable when it was set
(in 2010), but in today's settings and with typical network speeds
and data sizes, it's pretty small. 32 KB probably is a more reasonable
default now, regardless of input.
This changes the test references for two seek tests.
When using the normal seek function, which boils down to the lseek(2)
function, a seek to an out of bounds position doesn't return an error,
but that condition is only reported when doing the subsequent read
(which returns EOF). When doing more seeks by fast forwarding, the
fact that the seeked to destination is out of bounds is noticed and
reported sooner in these cases.
Signed-off-by: Martin Storsjö <martin@martin.st>
The Matroska muxer writes the Chapters early when chapters were already
available when writing the header; in this case any tags pertaining to
these chapters get written, too.
Yet if no chapters had been supplied before writing the header, Chapters
can also be written when writing the trailer if any are supplied. Tags
belonging to these chapters were up until now completely ignored.
This commit changes this: Writing the tags belonging to chapters has
been moved to mkv_write_chapters(). If mkv_write_tags() has not been
called yet (i.e. when chapters are written when writing the header),
the AVIOContext for writing the ordinary Tags element is used, but not
output, as this is left to mkv_write_tags() in order to only write one
Tags element. Yet if mkv_write_tags() has already been called,
mkv_write_chapters() will output a Tags element of its own which only
contains the tags for chapters.
When chapters are available initially, the corresponding tags will now
be the first tags in the Tags element; but the ordering of tags in Tags
is irrelevant anyway.
This commit also makes chapter_id_offset local to mkv_write_chapters()
as it is used only there and not reused at all.
Potentially writing a second Tags element means that the maximum number
of SeekHead entries had to be incremented. All the changes to FATE
result from the ensuing increase in the amount of space reserved for the
SeekHead (21 bytes more).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Several EBML Master elements for which a good upper bound of the final
length was available were nevertheless written without giving an
upper bound of the final length to start_ebml_master(), so that their
length fields were eight bytes long. This has been changed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Using random values for TrackUID and FileUID (as happens when the
AVFMT_FLAG_BITEXACT flag is not set) has the obvious downside of making
the output indeterministic. This commit mitigates this by writing the
potentially random values with a fixed size of eight byte, even if their
actual values would fit into less than eight bytes. This ensures that
even in non-bitexact mode, the differences between two files generated
with the same settings are restricted to a few bytes in the header.
(Namely the SegmentUID, the TrackUIDs (in Tracks as well as when
referencing them via TagTrackUID), the FileUIDs (in Attachments as
well as in TagAttachmentUID) as well as the CRC-32 checksums of the
Info, Tracks, Attachments and Tags level-1-elements.) Without this
patch, there might be an offset/a size difference between two such
files.
The FATE-tests had to be updated because the fixed-sized UIDs are also
used in bitexact mode.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Tags in the Matroska file format can be summarized as follows: There is
a level 1-element called Tags containing one or many Tag elements each
of which in turn contain a Targets element and one or many SimpleTags.
Each SimpleTag roughly corresponds to a single key-value pair similar to
an AVDictionaryEntry. The Targets meanwhile contains information to what
the metadata contained in the SimpleTags contained in the containing Tag
applies (i.e. to the file as a whole or to an individual track).
The Matroska muxer writes such metadata. It puts the metadata of every
stream into a Tag whose Targets makes it point to the corresponding
track. And if the output is seekable, then it also adds another Tag for
each track whose Targets corresponds to the track and where it reserves
space in a SimpleTag to write the duration at the end of the muxing
process into.
Yet there is no reason to write two Tag elements for a track and a few
bytes (typically 24 bytes per track) can be saved by adding the duration
SimpleTag to the other Tag of the same track (if it exists).
FATE has been updated because the output files changed. (Tests that
write to unseekable output (pipes) needn't be updated (no duration tag
has ever been written for them) and the same applies to tests without
further metadata.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until e7ddafd5, the Matroska muxer wrote two SeekHeads: One at the
beginning referencing the main level 1 elements (i.e. not the Clusters)
and one at the end, referencing the Clusters. This second SeekHead was
useless and has therefore been removed. Yet the SeekHead-related
functions and structures are still geared towards this usecase: They
are built around an allocated array of variable size that gets
reallocated every time an element is added to it although the maximum
number of Seek entries is a small compile-time constant, so that one should
rather include the array in the SeekHead structure itself; and said
structure should be contained in the MatroskaMuxContext instead of being
allocated separately.
The earlier code reserved space for a SeekHead with 10 entries, although
we currently write at most 6. Reducing said number implied that every
Matroska/Webm file will be 84 bytes smaller and required to adapt
several FATE tests; furthermore, the reserved amount overestimated the
amount needed for for the SeekHead's length field and how many bytes
need to be reserved to write a EBML Void element, bringing the total
reduction to 89 bytes.
This also fixes a potential segfault: If !mkv->is_live and if the
AVIOContext is initially unseekable when writing the header, the
SeekHead is already written when writing the header and this used to
free the SeekHead-related structures that have been allocated. But if
the AVIOContext happens to be seekable when writing the trailer, it will
be attempted to write the SeekHead again which will lead to segfaults
because the corresponding structures have already been freed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The packet counting based approach caused excessive sdt/pat/pmt for VBR, so
let's use a timestamp based approach instead similar to how we emit PCRs.
SDT/PAT/PMT period should be consistent for both VBR and CBR from now on.
Also change the type of sdt_period and pat_period to AV_OPT_TYPE_DURATION so no
floating point math is necessary.
Fixes ticket #3714.
Signed-off-by: Marton Balint <cus@passwd.hu>
Up until now, the length field of most level 1 elements has been written
using eight bytes, although it is known in advance how much space the
content of said elements will take up so that it would be possible to
determine the minimal amount of bytes for the length field. This
commit changes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Up until now the EBML Header length field has been written with eight
bytes, although the EBML Header is always so small that only one byte
is needed for it. This patch saves seven bytes for every Matroska/Webm
file.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
* commit 'f8df5e2f31a5ba7b30a0e1caaaf5a03c753b3f9b':
tests: Add a convenience function for video-only lavf tests
Merged-by: James Almer <jamrial@gmail.com>
Thanks for the discussion. Here's the next version, now with /25 and removed
ff_log2().
The blocksize of the PCM decoder is hard-coded. This creates
unnecessary delay when reading low-rate (<100Hz) streams. This creates
issues when multiplexing multiple streams, since other inputs are only
opened/read after a low-rate input block was completely read.
This patch decreases the blocksize for low-rate inputs, so
approximately a block is read every 40ms. This decreases the startup
delay when multiplexing inputs with different rates.
Signed-off-by: Philipp M. Scholl <pscholl@bawue.de>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes seek for files with empty edits and files with negative ctts
(dts_shift > 0). Added fate samples and tests.
Signed-off-by: Sasi Inguva <isasi@isasi.mtv.corp.google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The current edit unit cannot be reliably determined for the last packet of a
video stream, because we can't query the start offset of the next edit unit
from the index. This caused missing timestamps for the last video packet.
Therefore from now on, we allow setting the PTS even if we are not sure of the
current edit unit if mxf_set_current_edit_unit returned a specific failure, and
the assumed current edit unit is the last.
Fixes last packet timestamp of:
ffprobe -fflags nofillin -show_packets tests/data/lavf/lavf.mxf -select_streams v
Signed-off-by: Marton Balint <cus@passwd.hu>
Subtract the calculated dts offset from the requested timestamp before
seeking. This fixes an error "Error while filtering: Operation not
permitted" observed with a short file which contains only one key frame
and starts with negative timestamps.
Then, av_index_search_timestamp() returns a valid negative timestamp,
but mov_seek_stream bails out with AVERROR_INVALIDDATA.
Fixes ticket #6139.
Signed-off-by: Jonas Licht <jonas.licht@fem.tu-ilmenau.de>
Signed-off-by: Peter Große <pegro@friiks.de>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
When sidx box support is enabled, the code will skip reading all
trun boxes (each containing ctts entries for samples inthat box).
If seeks are attempted before all ctts values are known, the old
code would dump ctts entries into the wrong location. These are
then used to compute pts values which leads to out of order and
incorrectly timestamped packets.
This patch fixes ctts processing by always using the index returned
by av_add_index_entry() as the ctts_data index. When the index gains
new entries old values are reshuffled as appropriate.
This approach makes sense since the mov demuxer is already relying
on the mapping of AVIndex entries to samples for correct demuxing.
As a result of this all ctts entries are now 1-count. A followup
change will be submitted to remove support for > 1 count entries
which will simplify seeking.
Notes for future improvement:
Probably there are other boxes (stts, stsc, etc) that are impacted
by this issue... this patch only attempts to fix ctts since it
completely breaks packet timestamping.
This patch continues using an array for the ctts data, which is not
the most ideal given the rearrangement that needs to happen (via
memmove as new entries are read in). Ideally AVIndex and the ctts
data would be set-type structures so addition is always worst case
O(lg(n)) instead of the O(n^2) that exists now; this slowdown is
noticeable during seeks.
Signed-off-by: Dale Curtis <dalecurtis@chromium.org>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
As it gives excellent encoding gains at an insignificant speed increase
and passes fate without problems, it should now be safe to enable by
default.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Preparation for potentially disabling merged side data by default in the
libs. Do this in particular because it affects fate tests.
The changed tests either reflect added packet side data, or the changed
packet size due to merged side data removal reducing the packet size.
