Using random values for TrackUID and FileUID (as happens when the
AVFMT_FLAG_BITEXACT flag is not set) has the obvious downside of making
the output indeterministic. This commit mitigates this by writing the
potentially random values with a fixed size of eight byte, even if their
actual values would fit into less than eight bytes. This ensures that
even in non-bitexact mode, the differences between two files generated
with the same settings are restricted to a few bytes in the header.
(Namely the SegmentUID, the TrackUIDs (in Tracks as well as when
referencing them via TagTrackUID), the FileUIDs (in Attachments as
well as in TagAttachmentUID) as well as the CRC-32 checksums of the
Info, Tracks, Attachments and Tags level-1-elements.) Without this
patch, there might be an offset/a size difference between two such
files.
The FATE-tests had to be updated because the fixed-sized UIDs are also
used in bitexact mode.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If there are Attachments to write, the Matroska muxer currently
allocates two objects: An array that contains an entry for each
AttachedFile containing just the stream index of the corresponding
stream and the FileUID used for this AttachedFile; and a structure with
a pointer to said array and a counter for said array. These uids are
generated via code special to Attachments: It uses an AVLFG in the
normal and a sha of the attachment data in the bitexact case. (Said sha
requires an allocation, too.)
But now that an uid is generated for each stream in mkv_init(), there is
no need any more to use special code for generating the FileUIDs of
AttachedFiles: One can simply use the uid already generated for the
corresponding stream. And this makes the whole allocations of the
structures for AttachedFiles as well as the structures itself superfluous.
They have been removed.
In case AVFMT_FLAG_BITEXACT is set, the uids will be different from the
old ones which is the reason why the FATE-test lavf-mkv_attachment
needed to be updated. The old method had the drawback that two
AttachedFiles with the same data would have the same FileUID.
The new one doesn't.
Also notice that the dynamic buffer used to write the Attachments leaks
if an error happens when writing the buffer. By removing the
allocations potential sources of errors have been removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Tags in the Matroska file format can be summarized as follows: There is
a level 1-element called Tags containing one or many Tag elements each
of which in turn contain a Targets element and one or many SimpleTags.
Each SimpleTag roughly corresponds to a single key-value pair similar to
an AVDictionaryEntry. The Targets meanwhile contains information to what
the metadata contained in the SimpleTags contained in the containing Tag
applies (i.e. to the file as a whole or to an individual track).
The Matroska muxer writes such metadata. It puts the metadata of every
stream into a Tag whose Targets makes it point to the corresponding
track. And if the output is seekable, then it also adds another Tag for
each track whose Targets corresponds to the track and where it reserves
space in a SimpleTag to write the duration at the end of the muxing
process into.
Yet there is no reason to write two Tag elements for a track and a few
bytes (typically 24 bytes per track) can be saved by adding the duration
SimpleTag to the other Tag of the same track (if it exists).
FATE has been updated because the output files changed. (Tests that
write to unseekable output (pipes) needn't be updated (no duration tag
has ever been written for them) and the same applies to tests without
further metadata.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
It represents the relationship between them more naturally and will be
useful in the following commits.
Allows significantly more frames in fate-h264-attachment-631 to be
decoded.
containing updated extradata, in this case a new FLAC streaminfo.
Furthermore, it also tests that the Matroska muxer is able to preserve
uncommon channel layouts by adding Vorbis comments to the CodecPrivate.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
It might be used by the Matroska muxer. This is also the reason why the
FATE-tests for muxing WavPack into Matroska needed to be updated: They
now write the correct version 4.07 and not 4.03 as before.
Reviewed-by: David Bryant <david@wavpack.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
mkvmerge versions 6.2 to 40.0 had a bug that made it not propagate the
WavPack extradata (containing the WavPack version) during remuxing from
a Matroska file; currently our demuxer would treat every WavPack block
encountered as invalid data (unless the WavPack stream is to be
discarded (i.e. the streams discard is >= AVDISCARD_ALL)) and try to
resync to the next level 1 element.
Luckily, the WavPack version is currently not really important; so we
fix this problem by assuming a version. David Bryant, the creator of
WavPack, recommended using version 0x410 (the most recent version) for
this. And this is what this commit does.
A FATE-test for this has been added.
Reviewed-by: David Bryant <david@wavpack.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Add overflow test for hevc_add_res when int16_t coeff = -32768.
The result of C is good, while ASM is not.
To verify:
make fate-checkasm-hevc_add_res
ffmpeg/tests/checkasm/checkasm --test=hevc_add_res
./checkasm --test=hevc_add_res
checkasm: using random seed 679391863
MMXEXT:
hevc_add_res_4x4_8_mmxext (hevc_add_res.c:69)
- hevc_add_res.add_residual [FAILED]
SSE2:
hevc_add_res_8x8_8_sse2 (hevc_add_res.c:69)
hevc_add_res_16x16_8_sse2 (hevc_add_res.c:69)
hevc_add_res_32x32_8_sse2 (hevc_add_res.c:69)
- hevc_add_res.add_residual [FAILED]
AVX:
hevc_add_res_8x8_8_avx (hevc_add_res.c:69)
hevc_add_res_16x16_8_avx (hevc_add_res.c:69)
hevc_add_res_32x32_8_avx (hevc_add_res.c:69)
- hevc_add_res.add_residual [FAILED]
AVX2:
hevc_add_res_32x32_8_avx2 (hevc_add_res.c:69)
- hevc_add_res.add_residual [FAILED]
checkasm: 8 of 14 tests have failed
Signed-off-by: Xu Guangxin <guangxin.xu@intel.com>
Signed-off-by: Linjie Fu <linjie.fu@intel.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
check_func will return NULL for functions that have already been tested. If
the func is tested and skipped (which happens several times), there is no
need to prepare data(randomize_buffers and memcpy).
Move relative code in compare_add_res(), prepare data and do check only if
the function is not tested.
Signed-off-by: Linjie Fu <linjie.fu@intel.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Up until e7ddafd5, the Matroska muxer wrote two SeekHeads: One at the
beginning referencing the main level 1 elements (i.e. not the Clusters)
and one at the end, referencing the Clusters. This second SeekHead was
useless and has therefore been removed. Yet the SeekHead-related
functions and structures are still geared towards this usecase: They
are built around an allocated array of variable size that gets
reallocated every time an element is added to it although the maximum
number of Seek entries is a small compile-time constant, so that one should
rather include the array in the SeekHead structure itself; and said
structure should be contained in the MatroskaMuxContext instead of being
allocated separately.
The earlier code reserved space for a SeekHead with 10 entries, although
we currently write at most 6. Reducing said number implied that every
Matroska/Webm file will be 84 bytes smaller and required to adapt
several FATE tests; furthermore, the reserved amount overestimated the
amount needed for for the SeekHead's length field and how many bytes
need to be reserved to write a EBML Void element, bringing the total
reduction to 89 bytes.
This also fixes a potential segfault: If !mkv->is_live and if the
AVIOContext is initially unseekable when writing the header, the
SeekHead is already written when writing the header and this used to
free the SeekHead-related structures that have been allocated. But if
the AVIOContext happens to be seekable when writing the trailer, it will
be attempted to write the SeekHead again which will lead to segfaults
because the corresponding structures have already been freed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The WebM DASH Manifest muxer can write manifests for live streams and
these contain an entry that depends on the time the manifest is written;
an AVOption to make the output reproducible has been added for tests.
But this is unnecessary, as there already is a method for reproducible
output: The AVFMT_FLAG_BITEXACT-flag of the AVFormatContext. Therefore
this commit removes the custom option.
Given that the description of said option contained "private option -
users should never set this" and that it was not documented in
muxers.texi, no deprecation period for this option seemed necessary.
The commands of the FATE-tests for this muxer have been changed to no
longer use this option.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This fixes mpeg2video stream copies to mpeg muxer like this:
ffmpeg -i xdcamhd.mxf -c:v copy output.mpg
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Utilizes a subpicture sample with one decodable subpicture for the
test.
Based on a failing test case in reported by Michael in
https://ffmpeg.org/pipermail/ffmpeg-devel/2019-February/240398.html
which at the time had no test case for it.
Additionally, this is the first test case for the presentation
graphics format.
According to the H.264 specifications, the only NAL units that need to
have four byte startcodes in H.264 Annex B format are SPS/PPS units and
units that start a new access unit. Before af7e953a, the first of these
conditions wasn't upheld as already existing in-band parameter sets
would not automatically be written with a four byte startcode, but only
when they already were at the beginning of their input packets. But it
made four byte startcodes be used too often as every unit that is written
together with a parameter set that is inserted from extradata received a
four byte startcode although a three byte start code would suffice
unless the unit itself were a parameter set.
FATE has been updated to reflect the changes. Although the patch leaves
the extradata unchanged, the size of the extradata according to the FATE
reports changes. This is due to a quirk in ff_h2645_packet_split which
is used by extract_extradata: If the input is Annex B, the first zero of
a four byte startcode is considered a part of the last unit (if any).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The standard does not seem to require the counter to be zero based, but some
checker tools (MyriadBits MXFInspect, Interra Baton) have validations against 0
start...
Fixes ticket #6781.
Signed-off-by: Marton Balint <cus@passwd.hu>
RFC 3986 states that the generic syntax uses the slash ("/"), question mark
("?"), and number sign ("#") characters to delimit components that are
significant to the generic parser's hierarchical interpretation of an
identifier.
Signed-off-by: Marton Balint <cus@passwd.hu>
The tests for concat use this option which is scheduled for removal and
does nothing any more. So remove it; otherwise, these tests would fail
at the next major version bump.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
When a Matroska Block is only stored in compressed form, the size of
the uncompressed block is not explicitly coded and therefore not known
before decompressing it. Therefore the demuxer uses a guess for the
uncompressed size: The first guess is three times the compressed size
and if this is not enough, it is repeatedly incremented by a factor of
three. But when this happens with lzo, the decompression is neither
resumed nor started again. Instead when av_lzo1x_decode indicates that x
bytes of input data could not be decoded, because the output buffer is
already full, the first (not the last) x bytes of the input buffer are
resent for decoding in the next try; they overwrite already decoded
data.
This commit fixes this by instead restarting the decompression anew,
just with a bigger buffer.
This seems to be a regression since 935ec5a1.
A FATE-test for this has been added.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
This test tests that demuxing ProRes that is muxed like it should be in
Matroska (i.e. with the first header ("icpf") atom stripped away) works;
it also tests bz2 decompression as well as the handling of
unknown-length clusters.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Up until now, the microdvd demuxer uses av_strdup() to allocate the
extradata from a string; its length is set to strlen() + 1, i.e.
including the \0 at the end. Upon remuxing, the muxer would simply copy
the extradata at the beginning, including the \0.
This commit changes this by not adding the \0 to the size of the
extradata; the muxer now delimits extradata by inserting a \n. This
required to change the subtitles-microdvd-remux FATE-test.
Furthermore, the extradata is now allocated with zeroed padding.
The microdvd decoder is not affected by this, as it didn't use the size
of the extradata at all, but treated it as a C-string.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
