The Matroska muxer writes the Chapters early when chapters were already
available when writing the header; in this case any tags pertaining to
these chapters get written, too.
Yet if no chapters had been supplied before writing the header, Chapters
can also be written when writing the trailer if any are supplied. Tags
belonging to these chapters were up until now completely ignored.
This commit changes this: Writing the tags belonging to chapters has
been moved to mkv_write_chapters(). If mkv_write_tags() has not been
called yet (i.e. when chapters are written when writing the header),
the AVIOContext for writing the ordinary Tags element is used, but not
output, as this is left to mkv_write_tags() in order to only write one
Tags element. Yet if mkv_write_tags() has already been called,
mkv_write_chapters() will output a Tags element of its own which only
contains the tags for chapters.
When chapters are available initially, the corresponding tags will now
be the first tags in the Tags element; but the ordering of tags in Tags
is irrelevant anyway.
This commit also makes chapter_id_offset local to mkv_write_chapters()
as it is used only there and not reused at all.
Potentially writing a second Tags element means that the maximum number
of SeekHead entries had to be incremented. All the changes to FATE
result from the ensuing increase in the amount of space reserved for the
SeekHead (21 bytes more).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Several EBML Master elements for which a good upper bound of the final
length was available were nevertheless written without giving an
upper bound of the final length to start_ebml_master(), so that their
length fields were eight bytes long. This has been changed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The Matroska muxer currently only adds CuePoints in three cases:
a) For video keyframes. b) For the first audio frame in a new Cluster if
in DASH-mode. c) For subtitles. This means that ordinary Matroska audio
files won't have any Cues which impedes seeking.
This commit changes this. For every track in a file without video track
it is checked and tracked whether a Cue entry has already been added
for said track for the current Cluster. This is used to add a Cue entry
for each first packet of each track in each Cluster.
Implements #3149.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Using random values for TrackUID and FileUID (as happens when the
AVFMT_FLAG_BITEXACT flag is not set) has the obvious downside of making
the output indeterministic. This commit mitigates this by writing the
potentially random values with a fixed size of eight byte, even if their
actual values would fit into less than eight bytes. This ensures that
even in non-bitexact mode, the differences between two files generated
with the same settings are restricted to a few bytes in the header.
(Namely the SegmentUID, the TrackUIDs (in Tracks as well as when
referencing them via TagTrackUID), the FileUIDs (in Attachments as
well as in TagAttachmentUID) as well as the CRC-32 checksums of the
Info, Tracks, Attachments and Tags level-1-elements.) Without this
patch, there might be an offset/a size difference between two such
files.
The FATE-tests had to be updated because the fixed-sized UIDs are also
used in bitexact mode.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
If there are Attachments to write, the Matroska muxer currently
allocates two objects: An array that contains an entry for each
AttachedFile containing just the stream index of the corresponding
stream and the FileUID used for this AttachedFile; and a structure with
a pointer to said array and a counter for said array. These uids are
generated via code special to Attachments: It uses an AVLFG in the
normal and a sha of the attachment data in the bitexact case. (Said sha
requires an allocation, too.)
But now that an uid is generated for each stream in mkv_init(), there is
no need any more to use special code for generating the FileUIDs of
AttachedFiles: One can simply use the uid already generated for the
corresponding stream. And this makes the whole allocations of the
structures for AttachedFiles as well as the structures itself superfluous.
They have been removed.
In case AVFMT_FLAG_BITEXACT is set, the uids will be different from the
old ones which is the reason why the FATE-test lavf-mkv_attachment
needed to be updated. The old method had the drawback that two
AttachedFiles with the same data would have the same FileUID.
The new one doesn't.
Also notice that the dynamic buffer used to write the Attachments leaks
if an error happens when writing the buffer. By removing the
allocations potential sources of errors have been removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Tags in the Matroska file format can be summarized as follows: There is
a level 1-element called Tags containing one or many Tag elements each
of which in turn contain a Targets element and one or many SimpleTags.
Each SimpleTag roughly corresponds to a single key-value pair similar to
an AVDictionaryEntry. The Targets meanwhile contains information to what
the metadata contained in the SimpleTags contained in the containing Tag
applies (i.e. to the file as a whole or to an individual track).
The Matroska muxer writes such metadata. It puts the metadata of every
stream into a Tag whose Targets makes it point to the corresponding
track. And if the output is seekable, then it also adds another Tag for
each track whose Targets corresponds to the track and where it reserves
space in a SimpleTag to write the duration at the end of the muxing
process into.
Yet there is no reason to write two Tag elements for a track and a few
bytes (typically 24 bytes per track) can be saved by adding the duration
SimpleTag to the other Tag of the same track (if it exists).
FATE has been updated because the output files changed. (Tests that
write to unseekable output (pipes) needn't be updated (no duration tag
has ever been written for them) and the same applies to tests without
further metadata.)
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Up until e7ddafd5, the Matroska muxer wrote two SeekHeads: One at the
beginning referencing the main level 1 elements (i.e. not the Clusters)
and one at the end, referencing the Clusters. This second SeekHead was
useless and has therefore been removed. Yet the SeekHead-related
functions and structures are still geared towards this usecase: They
are built around an allocated array of variable size that gets
reallocated every time an element is added to it although the maximum
number of Seek entries is a small compile-time constant, so that one should
rather include the array in the SeekHead structure itself; and said
structure should be contained in the MatroskaMuxContext instead of being
allocated separately.
The earlier code reserved space for a SeekHead with 10 entries, although
we currently write at most 6. Reducing said number implied that every
Matroska/Webm file will be 84 bytes smaller and required to adapt
several FATE tests; furthermore, the reserved amount overestimated the
amount needed for for the SeekHead's length field and how many bytes
need to be reserved to write a EBML Void element, bringing the total
reduction to 89 bytes.
This also fixes a potential segfault: If !mkv->is_live and if the
AVIOContext is initially unseekable when writing the header, the
SeekHead is already written when writing the header and this used to
free the SeekHead-related structures that have been allocated. But if
the AVIOContext happens to be seekable when writing the trailer, it will
be attempted to write the SeekHead again which will lead to segfaults
because the corresponding structures have already been freed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The standard does not seem to require the counter to be zero based, but some
checker tools (MyriadBits MXFInspect, Interra Baton) have validations against 0
start...
Fixes ticket #6781.
Signed-off-by: Marton Balint <cus@passwd.hu>
Fixes: fate-fitsdec-bitpix-64
Possibly Fixes: -nan is outside the range of representable values of type 'unsigned short'
Possibly Fixes: 17769/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_FITS_fuzzer-5678314672357376
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
The packet counting based approach caused excessive sdt/pat/pmt for VBR, so
let's use a timestamp based approach instead similar to how we emit PCRs.
SDT/PAT/PMT period should be consistent for both VBR and CBR from now on.
Also change the type of sdt_period and pat_period to AV_OPT_TYPE_DURATION so no
floating point math is necessary.
Fixes ticket #3714.
Signed-off-by: Marton Balint <cus@passwd.hu>
Up until now, the length field of most level 1 elements has been written
using eight bytes, although it is known in advance how much space the
content of said elements will take up so that it would be possible to
determine the minimal amount of bytes for the length field. This
commit changes this.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Up until now the EBML Header length field has been written with eight
bytes, although the EBML Header is always so small that only one byte
is needed for it. This patch saves seven bytes for every Matroska/Webm
file.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
write_tmcd allows tmcd track to be created with any mode but in
mov_write_header, index for first tmcd track is only set for modes
MP4 or MOV, causing a crash if tmcd creation is attempted with other
modes.
* commit 'f8df5e2f31a5ba7b30a0e1caaaf5a03c753b3f9b':
tests: Add a convenience function for video-only lavf tests
Merged-by: James Almer <jamrial@gmail.com>
* commit 'a70eac7a9b193e8434b5bed90bd72aa4cb688363':
tests: Convert image2pipe tests to non-legacy test scripts
Merged-by: James Almer <jamrial@gmail.com>
ISMV lacks any sort of edit list support, as well as tfxd is
effectively the PTS of the fragment for most intents and purposes.
Thus, if b-frames are requested without negative CTS offsets you
end up with N frames' worth of delay (tfxd PTS plus the CTS offset
of the first sample). Negative CTS offsets enable the first sample
to have CTS=DTS, and thus a/v desync due to b-frame reorder delay
is avoided.
According to EBU tech 3285 supplement 3 the dwPosPeakOfPeaks field
should contain the absolute position to the maximum audio sample value,
but the current implementation writes the relative peak frame index
instead.
Fix the issue by writing the "unknown" value (-1) for now until the
feature is implemented correctly.
Previous version reviewed-by: Peter Bubestinger <p.bubestinger@av-rd.com>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Tobias Rapp <t.rapp@noa-archive.com>
Since there is no information about the source format, "unspecified"
is the correct value to write here.
All tests using the MPEG-2 encoder are updated, as this changes the
header on all outputs.
This reverts commit 04aa09c4bc
and reintroduces 0ff5567a30 that
was temporarily reverted due to minor regressions.
It also reverts e5bce8b4ce that fixed FATE refs.
The fate-ffm change is caused by field_order now being set
on the output format because the first frame arrives earlier.
The fate-mxf change is assumed to be the same.
<@jamrial> durandal_1707: 04aa09c4bc broke fate-lavf-ffm and fate-lavf-mxf
<@durandal_1707> how so?
<@jamrial> one byte changes
<@durandal_1707> jamrial: just update checksums
<@jamrial> durandal_1707: but why did they change at all? the commit you reverted didn't affect them
<@jamrial> why does reverting it affect these tests?
<@jamrial> i don't think updating the checksum without knowing what changed is a good idea
<@durandal_1707> jamrial: the lavfi core is in weird state after removal of recursive code
<@durandal_1707> jamrial: the change is that older ones would get progressive flag set and new one doesnt
<@jamrial> alright
As it gives excellent encoding gains at an insignificant speed increase
and passes fate without problems, it should now be safe to enable by
default.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
This makes sure the actual stream parameters are used, which is
important mainly for hardware decoding+filtering cases, which would
previously require various weird workarounds to handle the fact that a
fake software graph has to be constructed, but never used.
This should also improve behaviour in rare cases where
avformat_find_stream_info() does not provide accurate information.
This merges Libav commit a3a0230. It was previously skipped.
The code in flush_encoders() which sets up a "fake" format wasn't in
Libav. I'm not sure if it's a good idea, but it tends to give
behavior closer to the old one in certain corner cases.
The vp8-size-change gives different result, because now the size of
the first frame is used. libavformat reported the size of the largest
frame for some reason.
The exr tests now use the sample aspect ratio of the first frame. For
some reason libavformat determines 0/1 as aspect ratio, while the
decoder returns the correct one.
The ffm and mxf tests change the field_order values. I'm assuming
another libavformat/decoding mismatch.
Signed-off-by: wm4 <nfxjfg@googlemail.com>
According to the spec[1], a value of 0 means the footer is present and a value
of 1 means it's absent, the exact opposite of header presence flag where 1
means present and 0 absent.
The reason for this is compatibility with APEv1 tags, where there's no header,
footer presence was mandatory for all files, and the flags field was a zeroed
reserved field.
[1] http://wiki.hydrogenaud.io/index.php?title=Ape_Tags_Flags
Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
This would be simpler if codecpar supported AVOptions
modern ffserver should be unaffected by this, older ffserver which required the
muxer to directly access the encoder could have issues with this, but this
direct access is just wrong and unsafe
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This accesses the private encoder context, it should not be used by
the current ffserver it may affect old ffserver versions but i believe
there is consens that accessing the private encoder context from the muxer
is completely wrong.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This reverts commit e0c6b32046.
Said commit changed the behavior of the demuxer and decoder in a non
backwards compatible way.
Demuxers should make extradata available at init if possible, and send
new extradata as side data within a packet if needed.
A better fix for the remuxing crash will follow.
Signed-off-by: James Almer <jamrial@gmail.com>
Also test the fallback to png creation for a single frame.
Reviewed-by: James Almer <jamrial@gmail.com>
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
The dynamic buffer does not contain the CRC32 element so calls to avio_tell()
don't take it into account. This resulted in CueRelativePosition values being
six bytes short.
This is a regression since 6724525a15
Instead of adding yet another custom check for CRC32 to fix a size or an offset,
remove the existing ones and reserve the six bytes in the dynamic buffer.
Signed-off-by: James Almer <jamrial@gmail.com>
We don't currently support values 1 (centimeters), 2 (inches) or 3 (DAR),
only the default value 0 (pixels) which doesn't need to be written.
The fate refs are updated as unknown SAR is now signaled in the output
files with the addition of the new element.
Reviewed-by: Carl Eugen Hoyos <ceffmpeg@gmail.com>
Signed-off-by: James Almer <jamrial@gmail.com>
Attachment tags were being written targeting non-existent streams in the
output file.
Also filter filename and mimetype entries, as they are standard elements
in the Attachment master.
Signed-off-by: James Almer <jamrial@gmail.com>
Implements part of ticket #4347
Tested-by: Dave Rice <dave@dericed.com>
Tested-by: Jerome Martinez <jerome@mediaarea.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Implements part of ticket #4347
Tested-by: Dave Rice <dave@dericed.com>
Tested-by: Jerome Martinez <jerome@mediaarea.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Implements part of ticket #4347
Tested-by: Dave Rice <dave@dericed.com>
Tested-by: Jerome Martinez <jerome@mediaarea.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Implements part of ticket #4347
Tested-by: Dave Rice <dave@dericed.com>
Tested-by: Jerome Martinez <jerome@mediaarea.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Implements part of ticket #4347
Tested-by: Dave Rice <dave@dericed.com>
Tested-by: Jerome Martinez <jerome@mediaarea.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
Implements part of ticket #4347
Tested-by: Dave Rice <dave@dericed.com>
Tested-by: Jerome Martinez <jerome@mediaarea.net>
Reviewed-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: James Almer <jamrial@gmail.com>
This commit is largely based on commit 15e84ed3 from Anton Khirnov
<anton@khirnov.net> which was previously skipped in bbf5ef9d.
There are still a bunch of things raising codecpar related warnings that
need fixing, such as:
- the use of codec->debug in the interactive debug mode
- read_ffserver_streams(): it's probably broken now but there is no test
- lowres stuff
- codec copy apparently required by bitstream filters
The matroska references are updated because they now properly forward
the field_order (previously unknown, now progressive).
Thanks to James Almer for fixing a bunch of FATE issues in this commit.
Signed-off-by: Clément Bœsch <clement@stupeflix.com>
Signed-off-by: James Almer <jamrial@gmail.com>
* commit '393596f9d51134d6e45d81ae129223f4faea1232':
mpegtsenc: stop impersonating ses in sdt
This commit also includes the needed FATE updates later spotted by
Martin Storsjö and fixed in 34effe816f on
Libav side.
Merged-by: Clément Bœsch <u@pkh.me>
* commit 'e3453fd44480d903338c663238bf280215dd9a07':
matroska: Write the field order information
Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
The bits_per_raw_sample represents the number of bits of precision per sample.
The field is added at the logical place, not at the end as the code was just
recently added
This fixes the regression about losing the audio sample precision information
The change in the fate test checksum un-does the change from the merge
Previous version reviewed by: wm4 <nfxjfg@googlemail.com>
Previous version reviewed by: Dominik 'Rathann' Mierzejewski <dominik@greysector.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Currently, AVStream contains an embedded AVCodecContext instance, which
is used by demuxers to export stream parameters to the caller and by
muxers to receive stream parameters from the caller. It is also used
internally as the codec context that is passed to parsers.
In addition, it is also widely used by the callers as the decoding (when
demuxer) or encoding (when muxing) context, though this has been
officially discouraged since Libav 11.
There are multiple important problems with this approach:
- the fields in AVCodecContext are in general one of
* stream parameters
* codec options
* codec state
However, it's not clear which ones are which. It is consequently
unclear which fields are a demuxer allowed to set or a muxer allowed to
read. This leads to erratic behaviour depending on whether decoding or
encoding is being performed or not (and whether it uses the AVStream
embedded codec context).
- various synchronization issues arising from the fact that the same
context is used by several different APIs (muxers/demuxers,
parsers, bitstream filters and encoders/decoders) simultaneously, with
there being no clear rules for who can modify what and the different
processes being typically delayed with respect to each other.
- avformat_find_stream_info() making it necessary to support opening
and closing a single codec context multiple times, thus
complicating the semantics of freeing various allocated objects in the
codec context.
Those problems are resolved by replacing the AVStream embedded codec
context with a newly added AVCodecParameters instance, which stores only
the stream parameters exported by the demuxers or read by the muxers.
This reverts commit 8ed82d8174.
SMPTE S377-1-2009c defines in F.4.1 that the Video Line Map should
always be an array with two 32 bit integers as elements. This is
repeated in G.2.12 with actual examples for progressive content,
where the second value would always be 0.
Additionally, the IRT MXF analyser also lists this as the only
error in the MXF output from ffmpeg: https://mxf-analyser-cloud.irt.de
Reviewed-by: Tomas Härdin <tomas.hardin@codemill.se>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
