Make it clear that there is no timing-dependent behavior. In particular,
there is no state in which both input and output are denied, and where
you have to wait for a while yourself to make progress (apparently some
hardware decoders like to do this).
Avoid wording that makes references to time. It shouldn't be mistaken
for some kind of asynchronous API (like POSIX read() can return EAGAIN
if there is no new input yet). It's a state machine, so try to use
appropriate terms.
Signed-off-by: Diego Biurrun <diego@biurrun.de>
Merges Libav commit 8a60bba0ae.
Apparently the demuxer outputs the wrong padding for HE-AAC (based on
the raw sample rate, or so). aacdec contains a hack to adjust the muxer
padding accordingly before it's used to trim the decoder output. This
modified the packet side data, which in combination with the old
decoding API would change the packet the user passed to the decoder.
This is clearly not allowed, and it breaks running some gapless fate
tests with "-fflags +keepside" applied (without keepside, the packet
metadata is typically newly allocated, essentially making a copy and not
modifying the user's input packet).
This should probably be fixed in the demuxer (and consequently also the
muxer), but for now only fix the immediate problem.
Regression since 946ed78f5f (2012).
Ever since the codecpar changes, this has been always printed when
opening a flv file. This is because the codecpar changes made all
streams to be added lazily as read_packet is called.
There is no reason that draining couldn't return an error or two. But
some decoders don't handle this very well, and might always return an
error. This can lead to API users getting into an infinite loop and
burning CPU, because no progress is made and EOF is never returned.
In fact, ffmpeg.c contains a hack against such a case. It is made
unnecessary with this commit, and removed with the next one. (This
particular error case seems to have been fixed since the hack was
added, though.)
This might lose frames if decoding returns errors during draining.
The filter field is often used to check whether a filter is
configured. If configuring the filter actually fails somewhere in
the middle of it, these fields could still be set to non-NULL, which
lead to other code accessing the half-configured filter graph, which
in turn could lead to crashes within libavfilter.
Solve this by properly resetting all fields.
This was triggered by a fuzzed sample after the recent changes. It's
unknown whether this behavior could be triggered before that.
If a subtitle packet came before the first video frame could be fully
decoded, the subtitle packet would get discarded. This puts the subtitle
into a queue instead, and processes it once the attached filter graph is
initialized.
Some callers (like do_subtitle_out()) call this with an AVPacket that is
not refcounted. This can cause undefined behavior.
Calling av_packet_move_ref() does not make a packet refcounted if it
isn't yet. (And it can't be made to, because it always succeeds,
and can't return ENOMEM.)
Call av_packet_ref() instead to make sure it's refcounted.
I couldn't find a case that is fixed by this with the current code. But
it will fix the fate-pva-demux test with the later patches applied.
Signed-off-by: wm4 <nfxjfg@googlemail.com>
If AVVideotoolboxContext.cv_pix_fmt_type is set to 0, don't set the
kCVPixelBufferPixelFormatTypeKey value on the VT decoder.
This makes VT output its native format, which can be much faster on
some hardware iterations (if the native format does not match with
the requested format, it will be converted, which is slow).
The default is still forcing nv12.
Public fields were added after the private fields (negating the entire
point of this). New private fields go into AVStreamInternal anyway.
The new marker was set by guessing which fields are supposed to be
private and wshich not. recommended_encoder_configuration is accessed by
ffserver_config.c directly, and is supposed to use the public API.
ffmpeg.c accesses AVStream.cur_dts, even though it's a private field,
but that seems to be an older error.
Allow all struct fields to be accessed directly, as long as they're
public.
Before this change, many fields were "public", but could be accessed via
AVOption only. This meant they were effectively not public, but were
present for documentation purposes, which was incredibly confusing at
best.
D3D9Ex uses different driver paths. This helps with "headless"
configurations when no user logs in. Plain D3D9 device creation will
fail if no user is logged in, while it works with D3D9Ex.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Merges Libav commit c2f97f0508.
This is an extended version of the AVFrame.opaque field, which can be
used to attach arbitrary user information to an AVFrame.
The usefulness of the opaque field is rather limited, because it can
store only up to 32 bits of information (or 64 bit on 64 bit systems).
It's not possible to set this field to a memory allocation, because
there is no way to deallocate it correctly.
The opaque_ref field circumvents this by letting the user set an
AVBuffer, which makes the user data refcounted.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Merges Libav commit 04f3bd3496.
This is a bit messy, mainly due to timestamp handling.
decode_video() relied on the fact that it could set dts on a flush/drain
packet. This is not possible with the old API, and won't be. (I think
doing this was very questionable with the old API. Flush packets should
not contain any information; they just cause a FIFO to be emptied.) This
is replaced with checking the best_effort_timestamp for AV_NOPTS_VALUE,
and using the suggested DTS in the drain case.
The modified tests (fate-cavs and others) still fails due to dropping
the last frame. This happens because the timestamp of the last frame
goes backwards (ffprobe -show_frames shows the same thing). I suspect
that this "worked" due to the best effort timestamp logic picking the
DTS over the decreasing PTS. Since this logic is in libavcodec (where
it probably shouldn't be), this can't be easily fixed. The timestamps
of the cavs samples are weird anyway, so I chose not to fix it.
Another strange thing is the timestamp handling in the video path of
process_input_packet (after the decode_video() call). It looks like
the code to increase next_dts and next_pts should be run every time
a frame is decoded - but it's needed even if output is skipped.
With the new decode API, doing this in ffmpeg.c is impractical. There
was resistance against removing the warning, so put it into libavcodec.
Not bothering with reducing the warning to verbose log level for
subsequent wanrings. The warning should be rare, and only happen when
developing new codecs for the old API.
Includes a change suggested by Michael Niedermayer.
This uses a new MMAL feature, which limits the number of extra frames
that can be buffered within the decoder. VIDEO_MAX_NUM_CALLBACKS can
be defined as positive or negative number. Positive numbers are
absolute, and can lead to deadlocks if the user underestimates the
number of required buffers. Negative numbers specify the number of extra
buffers, e.g. -1 means no extra buffer, (-1-N) means N extra buffers.
Set a gratuitous default of -11 (N=10). This is much lower than the
firmware default, which appears to be 96.
This is backwards compatible, but needs a symbol only present in newer
firmware headers. (It's an enum item, so it requires a check in
configure.)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Slight simplification. The result is the same. Also, change the
wording of the message as requested in patch review.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Fixes apparent mmal_port_disable() freezes in ffmmal_stop_decoder() when
calling ffmmal_decode() with flush semantics a large number of times in
a row.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The assert in ffmmal_stop_decoder() could trigger sometimes. The
packets_buffered counter was indeed not correctly maintained, and
packets were not subtracted from it if they were still in the waiting
queue.
For some reason, this happened especially with VC-1.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The deprecated avcodec_decode_video2() and avcodec_decode_audio4()
functions called av_packet_split_side_data() on the input packets. This
is required for packets produced by libavformat with the
AVFMT_FLAG_KEEP_SIDE_DATA flag unset (which is unfortunately the
default).
The new API didn't do this yet, although it didn't matter as no decoder
supports the new API yet. The emulation layer for the old API calls the
old API functions, which took care of the splitting. Add this code to
the new API codec entrypoints too, because we shouldn't send essentially
corrupted data to decoders.
Until now, the decoding API was restricted to outputting 0 or 1 frames
per input packet. It also enforces a somewhat rigid dataflow in general.
This new API seeks to relax these restrictions by decoupling input and
output. Instead of doing a single call on each decode step, which may
consume the packet and may produce output, the new API requires the user
to send input first, and then ask for output.
For now, there are no codecs supporting this API. The API can work with
codecs using the old API, and most code added here is to make them
interoperate. The reverse is not possible, although for audio it might.
From Libav commit 05f66706d1.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Until now, the decoding API was restricted to outputting 0 or 1 frames
per input packet. It also enforces a somewhat rigid dataflow in general.
This new API seeks to relax these restrictions by decoupling input and
output. Instead of doing a single call on each decode step, which may
consume the packet and may produce output, the new API requires the user
to send input first, and then ask for output.
For now, there are no codecs supporting this API. The API can work with
codecs using the old API, and most code added here is to make them
interoperate. The reverse is not possible, although for audio it might.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This uses a new MMAL feature, which limits the number of extra frames
that can be buffered within the decoder. VIDEO_MAX_NUM_CALLBACKS can
be defined as positive or negative number. Positive numbers are
absolute, and can lead to deadlocks if the user underestimates the
number of required buffers. Negative numbers specify the number of extra
buffers, e.g. -1 means no extra buffer, (-1-N) means N extra buffers.
Set a gratuitous default of -11 (N=10). This is much lower than the
firmware default, which appears to be 96.
This is backwards compatible, but needs a symbol only present in newer
firmware headers. (It's an enum item, so it requires a check in
configure.)
Fixes a specific srt sample, which has an event with negative duration.
libavcodec will convert an event with negative duration to an ASS event
which will be displayed forever, which is not wanted here.
Treat negative duration always as unknown duration instead, and show it
until the next subtitle event.
If videotoolbox_common_end_frame failed, then the AVFrame was returned
to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
the decode call indicating success.
These "half-set" AVFrames with dummy buffer are a videotoolbox specific
hack, because the decoder requires an allocated AVFrame for its internal
logic. Videotoolbox on the other hand allocates its frame itself
internally, and outputs it only on end_frame. At this point, the dummy
buffer is replaced with the real frame (unless decoding fails).
Fixes apparent mmal_port_disable() freezes in ffmmal_stop_decoder() when
calling ffmmal_decode() with flush semantics a large number of times in
a row.
The assert in ffmmal_stop_decoder() could trigger sometimes. The
packets_buffered counter was indeed not correctly maintained, and
packets were not subtracted from it if they were still in the waiting
queue.
For some reason, this happened especially with VC-1.
I've got some m4a samples that had jpeg cover art marked as png. Since
these files were supposedly written by iTunes, and other software can
read it (e.g. clementine does), this should be worked around.
Since png has a very simple to detect header, while it's apparently a
real pain to detect jpeg in the general case, try to detect png and
assume jpeg otherwise. Not bothering with bmp, as I have no test case.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Commit 2b3e9bbfb5 caused problems for a
certain API user:
https://code.google.com/p/chromium/issues/detail?id=537725https://code.google.com/p/chromium/issues/detail?id=542032
The problem seems rather arbitrary, because if there's junk, anything
can happen. In this case, the imperfect junk skipping just caused it to
read different junk, from what I can see.
We can improve the accuracy of junk detection by a lot by checking if 2
consecutive frames use the same configuration. While in theory it might
be completely fine for the 1st frame to have a different format than the
2nd frame, it's exceedingly unlikely, and I can't think of a legitimate
use-case.
This is approximately the same mpg123 does for junk skipping. The
set of compared header bits is the same as the libavcodec mp3 parser
uses for similar purposes.
The AVCodecContext.get_format callback is not only used for pixel format
negotiation with the API user, but also for hwaccel init. For the
latter, it's required that some codec parameters, in particular the
codec profile, are set when the callback is invoked.
This patch removes a get_format invocation where this is not guaranteed.
The codec parameters, including the profile, are really set further
below. (The same code path that sets the profile also calls get_format
properly too.)
This just happened to work by coincidence in most cases. For example, if
the API user just copied or reused the AVStream's AVCodecContext when
decoding, the profile would be set properly. But in some cases it
fails., such as with the sample WolfensteinTwitch.mp4 on the samples
server.
Remove the redundant get_format call. Apparently it serves no purpose
anymore, although it is possible that this was different at the time it
was added in commit ffd77f94a2.
This fixes hwaccel usage for API users which do not set the profile
when setting up the AVCodecContext (which is allowed).
Work on the AVFrame references directly.
Instead of setting up a flipped/swapped "view" on the pictures,
flip/swap them when returning decoded frames to the API user.
In some situations, MMAL won't return a decoded frame for certain input
frames. This can happen if a frame fails to decode, or if a packet does
not actually contain a complete frame. In these situations, we would
deadlock (or actually timeout) waiting for an expected output frame,
which is not ideal. On the other hand, there are situations where we
definitely have to block to avoid deadlocks. (This mess is a
consequence of trying to map MMAL's asynchronous and flexible
dataflow to libavcodec, which is more static and rigid.)
Solve this by doing a blocking wait only if the amount of buffered data
is too big. The whole purpose of the blocking wait is to avoid excessive
buffering of input data, so we can skip it if it appears to be low. The
consequence is that libavcodec can gracefully return no frame to the
API user.
We want to track the number of full packets to make our heuristic work.
But MMAL buffers are fixed-size, requiring splitting large packets. This
is why the previous commit is needed. We use the ..._FRAME_END flag to
remember packet boundaries, but MMAL does not preserve these buffer
flags when returning buffers to the user.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The next commit needs 1 bit of additional information per MMAL buffer
sent to the MMAL input port. This information will be needed when the
buffer is recycled (i.e. returned by the input port's callback).
Normally, we could use MMAL_BUFFER_HEADER_FLAG_USER0, but that is
unexpectedly not preserved.
Do this by storing a pointer to FFBufferEntry in the MMAL buffer's
user data, instead of an AVBufferRef. This also changes the lifetime
of FFBufferEntry.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This affects Annex B streams (such as demuxed from .ts and others). It
also handles the format change in reinit-large_420_8-to-small_420_8.h264
correctly.
Instead of passing through the extradata, create it on the fly it from
the currently active SPS and PPS. Since reconstructing the PPS and SPS
NALs would be very complicated and verbose, we use the NALs as they
originally appeared in the bitstream.
The code for writing the extradata is somewhat derived from
libavformat/avc.c, but it's small and different enough that sharing it
is not really worth it.
We assume an upper bound of 4096 bytes for each raw SPS/PPS. It's hard
to determine an exact maximum size, but this value was was considered
high enough and safe.
Needed for the following VideotoolBox commit.
Commits 43bc5cf9 and c5371f77 add code for skipping initial zeros in mp3
packets. This code forgot to report to the user that data was skipped at
all.
Since audio codecs allow partial packet decoding, the user application
has to rely on the return value. It will remove the data reported as
consumed by the decoder, and feed it to the decoder again. This resulted
in the mp3 frame after the zero region to be decoded over and over
again, until the zero region was finally skipped by the application.
Fix this by including the amount of skipped bytes to the number of
consumed bytes returned by the decode call.
Fixes trac ticket #4890.
Note that convergence_duration had another meaning, one which was in
practice never used. The only real use for it was a 64 bit replacement
for the duration field. It's better just to make duration 64 bits, and
to get rid of it.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
It appears vdpau drivers can return constrained baseline as unsupported,
even if libvdpau knows about the symbol, and the main profile is
supported.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Note that this slightly changes behavior: it sets AVMEDIA_TYPE_UNKNOWN
if the codec type is unknown. This should be ok.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
If cmd_pos is broken, this would just keep accumulating packets in the
reassembly buffer, until it fails and flushes the buffer on overflow.
Since packets are usually rather small, this will take a lot of subtitle
packets. The perceived effect is that subtitles are not displayed
anymore after the faulty packet was passed to the decoder.
I'm not terribly sure about this, but on the other hand this code is
active only when fragmented packets need to be reassembled.
Fixes sample file in trac issue #4872.
Assuming the first and second packets are partial, this would append the
reassembly buffer (ctx->buf) to itself with the second
append_to_cached_buf() call, because buf is set to ctx->buf.
I do not know a valid sample file which triggers this, and do not know
if packets can be split into more than 2 sub-packets, but it triggered
with a (differently) broken sample file in trac issue #4872.
Broken by commit ba12ba859a. This only
happens with HLS streams which use encryption and require preserving
cookies sent by the server.
Fixes trac issue #4846.
Some .idx files actually contain duplicate subtitle events:
timestamp: 00:07:52:600, filepos: 00004e800
timestamp: 00:07:52:600, filepos: 00004f800
The second will be dropped, because it has same pts, duration, and text
(the text is just a dummy empty string; the real data is retrieved when
actually reading vobsub subtitle packets).
Dropping this is probably not intended/safe, so avoid it.
See trac issue #4872 for a sample. This patch doesn't fix decoding of
the sample, though.
The stream ID is essentially an arbitrary number defined by the .idx
file headers. They have to match the IDs in the .sub stream. The vobsub
demuxer assumed the IDs would just start from 0, increassing by 1 for
each stream. This is not correct. In the sample I had, the IDs were
starting from 1, leading to no subtitles being displayed at all.
Fix this by using the correct stream ID.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This also drops setting the frame->pts field. This is usually not set by
decoders, so this would be an inconsistency that's at worst a danger to
the API user.
It appears the buffer->dts field is normally not set by the MMAL
decoder, so don't use it. If it's ever going to be set by MMAL, we
don't know whether the value will be what we want.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The generic code in utils.c sets the AVFrame.pkt_dts field from the
packet it was supposedly decoded. This does not have to be true for a
fully asynchronous decoder like mmaldec. It could be overwritten with an
incorrect value. Even if the decoder doesn't determine the DTS (but sets
it to AV_NOPTS_VALUE), it's impossible to determine a correct value in
utils.c.
Decoders can now be marked with FF_CODEC_CAP_SETS_PKT_DTS, in which case
utils.c won't overwrite the field. The decoders are expected to set this
field (even if they only set it to AV_NOPTS_VALUE).
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This MMAL feature fills in missing timestamps from the framerate set on
the input port. This is generally unwanted, since libavcodec decoders
merely pass through timestamps without ever "fixing" them. The framerate
is also unknown, and even the timebase doesn't have to be set.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Don't try to do a blocking wait for MMAL output if we haven't even sent
a single real packet, but only flush packets. Obviously we can't expect
to get anything back.
Additionally, don't send a flush packet to MMAL in the same case. It
appears the MMAL decoder will sometimes hang in mmal_vc_port_disable()
(called from ffmmal_close_decoder()), waiting for a reply from the GPU
which never arrives. Either MMAL disallows sending flush packets without
preceding real data, or it's a MMAL bug.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
I can't come up with a nice way to handle this. It's hard to keep the
lock-stepped input/output in this case. You can't predict whether the
MMAL decoder will output a picture (because it's asynchronous), so
you have to assume in general that any packet could produce 0 or 1
frames. You can't continue to write input packets to the decoder,
because then you might get too many output frames, which you can't
get rid of because the lavc decoding API does not allow the decoder
to return an output frame without consuming an input frame (except
when flushing).
The ideal fix is a M:N decoding API (preferably asynchronous), which
would make this code potentially much cleaner. For now, this hack
will do.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Don't try to do a blocking wait for MMAL output if we haven't even sent
a single real packet, but only flush packets. Obviously we can't expect
to get anything back.
Additionally, don't send a flush packet to MMAL in the same case. It
appears the MMAL decoder will sometimes hang in mmal_vc_port_disable()
(called from ffmmal_close_decoder()), waiting for a reply from the GPU
which never arrives. Either MMAL disallows sending flush packets without
preceding real data, or it's a MMAL bug.
I can't come up with a nice way to handle this. It's hard to keep the
lock-stepped input/output in this case. You can't predict whether the
MMAL decoder will output a picture (because it's asynchronous), so
you have to assume in general that any packet could produce 0 or 1
frames. You can't continue to write input packets to the decoder,
because then you might get too many output frames, which you can't
get rid of because the lavc decoding API does not allow the decoder
to return an output frame without consuming an input frame (except
when flushing).
The ideal fix is a M:N decoding API (preferably asynchronous), which
would make this code potentially much cleaner. For now, this hack
will do.
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.
Fix this by propagating end_frame errors.
There can be other headers than "Content-Type:" (in this case, a
"Content-Length:" header was following), so checking for a trailing
newline is wrong.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This returns something like "v12_dev0-1332-g333a27c". This is much more
useful than the individual library versions, of which there are too
many, and which are very hard to map back to releases or git commits.
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
This returns something like "N-73264-gb54ac84". This is much more useful
than the individual library versions, of which there are too much and
which are very hard to map back to releases or git commits.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The APIC description must be unique, and some ID3v2 tag writers add
spaces to write several APIC entries with the same description. The
trailing spaces simply serve as a way to disambiguate the description.
Do this so that API users do not have to special-case mp3 to fix this
cosmetic issue.
Requested-by: wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Calling ffio_ensure_seekback() if ffio_init_checksum() has been called
on the same context can lead to out of bounds memory accesses and
crashes. The reason is that ffio_ensure_seekback() does not update
checksum_ptr after reallocating the buffer, resulting in a dangling
pointer.
This effectively fixes potential crashes when opening mp3 files.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
wm4
