This is a bit messy, mainly due to timestamp handling.
decode_video() relied on the fact that it could set dts on a flush/drain
packet. This is not possible with the old API, and won't be. (I think
doing this was very questionable with the old API. Flush packets should
not contain any information; they just cause a FIFO to be emptied.) This
is replaced with checking the best_effort_timestamp for AV_NOPTS_VALUE,
and using the suggested DTS in the drain case.
The modified tests (fate-cavs and others) still fails due to dropping
the last frame. This happens because the timestamp of the last frame
goes backwards (ffprobe -show_frames shows the same thing). I suspect
that this "worked" due to the best effort timestamp logic picking the
DTS over the decreasing PTS. Since this logic is in libavcodec (where
it probably shouldn't be), this can't be easily fixed. The timestamps
of the cavs samples are weird anyway, so I chose not to fix it.
Another strange thing is the timestamp handling in the video path of
process_input_packet (after the decode_video() call). It looks like
the code to increase next_dts and next_pts should be run every time
a frame is decoded - but it's needed even if output is skipped.
With the new decode API, doing this in ffmpeg.c is impractical. There
was resistance against removing the warning, so put it into libavcodec.
Not bothering with reducing the warning to verbose log level for
subsequent wanrings. The warning should be rare, and only happen when
developing new codecs for the old API.
Includes a change suggested by Michael Niedermayer.
This uses a new MMAL feature, which limits the number of extra frames
that can be buffered within the decoder. VIDEO_MAX_NUM_CALLBACKS can
be defined as positive or negative number. Positive numbers are
absolute, and can lead to deadlocks if the user underestimates the
number of required buffers. Negative numbers specify the number of extra
buffers, e.g. -1 means no extra buffer, (-1-N) means N extra buffers.
Set a gratuitous default of -11 (N=10). This is much lower than the
firmware default, which appears to be 96.
This is backwards compatible, but needs a symbol only present in newer
firmware headers. (It's an enum item, so it requires a check in
configure.)
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Slight simplification. The result is the same. Also, change the
wording of the message as requested in patch review.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Fixes apparent mmal_port_disable() freezes in ffmmal_stop_decoder() when
calling ffmmal_decode() with flush semantics a large number of times in
a row.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The assert in ffmmal_stop_decoder() could trigger sometimes. The
packets_buffered counter was indeed not correctly maintained, and
packets were not subtracted from it if they were still in the waiting
queue.
For some reason, this happened especially with VC-1.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
The deprecated avcodec_decode_video2() and avcodec_decode_audio4()
functions called av_packet_split_side_data() on the input packets. This
is required for packets produced by libavformat with the
AVFMT_FLAG_KEEP_SIDE_DATA flag unset (which is unfortunately the
default).
The new API didn't do this yet, although it didn't matter as no decoder
supports the new API yet. The emulation layer for the old API calls the
old API functions, which took care of the splitting. Add this code to
the new API codec entrypoints too, because we shouldn't send essentially
corrupted data to decoders.
Until now, the decoding API was restricted to outputting 0 or 1 frames
per input packet. It also enforces a somewhat rigid dataflow in general.
This new API seeks to relax these restrictions by decoupling input and
output. Instead of doing a single call on each decode step, which may
consume the packet and may produce output, the new API requires the user
to send input first, and then ask for output.
For now, there are no codecs supporting this API. The API can work with
codecs using the old API, and most code added here is to make them
interoperate. The reverse is not possible, although for audio it might.
From Libav commit 05f66706d1.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Until now, the decoding API was restricted to outputting 0 or 1 frames
per input packet. It also enforces a somewhat rigid dataflow in general.
This new API seeks to relax these restrictions by decoupling input and
output. Instead of doing a single call on each decode step, which may
consume the packet and may produce output, the new API requires the user
to send input first, and then ask for output.
For now, there are no codecs supporting this API. The API can work with
codecs using the old API, and most code added here is to make them
interoperate. The reverse is not possible, although for audio it might.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
This uses a new MMAL feature, which limits the number of extra frames
that can be buffered within the decoder. VIDEO_MAX_NUM_CALLBACKS can
be defined as positive or negative number. Positive numbers are
absolute, and can lead to deadlocks if the user underestimates the
number of required buffers. Negative numbers specify the number of extra
buffers, e.g. -1 means no extra buffer, (-1-N) means N extra buffers.
Set a gratuitous default of -11 (N=10). This is much lower than the
firmware default, which appears to be 96.
This is backwards compatible, but needs a symbol only present in newer
firmware headers. (It's an enum item, so it requires a check in
configure.)
Fixes a specific srt sample, which has an event with negative duration.
libavcodec will convert an event with negative duration to an ASS event
which will be displayed forever, which is not wanted here.
Treat negative duration always as unknown duration instead, and show it
until the next subtitle event.
If videotoolbox_common_end_frame failed, then the AVFrame was returned
to the API user with the dummy buffer (in AVFrame.buf[0]) still set, and
the decode call indicating success.
These "half-set" AVFrames with dummy buffer are a videotoolbox specific
hack, because the decoder requires an allocated AVFrame for its internal
logic. Videotoolbox on the other hand allocates its frame itself
internally, and outputs it only on end_frame. At this point, the dummy
buffer is replaced with the real frame (unless decoding fails).
Fixes apparent mmal_port_disable() freezes in ffmmal_stop_decoder() when
calling ffmmal_decode() with flush semantics a large number of times in
a row.
The assert in ffmmal_stop_decoder() could trigger sometimes. The
packets_buffered counter was indeed not correctly maintained, and
packets were not subtracted from it if they were still in the waiting
queue.
For some reason, this happened especially with VC-1.
I've got some m4a samples that had jpeg cover art marked as png. Since
these files were supposedly written by iTunes, and other software can
read it (e.g. clementine does), this should be worked around.
Since png has a very simple to detect header, while it's apparently a
real pain to detect jpeg in the general case, try to detect png and
assume jpeg otherwise. Not bothering with bmp, as I have no test case.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Commit 2b3e9bbfb5 caused problems for a
certain API user:
https://code.google.com/p/chromium/issues/detail?id=537725https://code.google.com/p/chromium/issues/detail?id=542032
The problem seems rather arbitrary, because if there's junk, anything
can happen. In this case, the imperfect junk skipping just caused it to
read different junk, from what I can see.
We can improve the accuracy of junk detection by a lot by checking if 2
consecutive frames use the same configuration. While in theory it might
be completely fine for the 1st frame to have a different format than the
2nd frame, it's exceedingly unlikely, and I can't think of a legitimate
use-case.
This is approximately the same mpg123 does for junk skipping. The
set of compared header bits is the same as the libavcodec mp3 parser
uses for similar purposes.
The AVCodecContext.get_format callback is not only used for pixel format
negotiation with the API user, but also for hwaccel init. For the
latter, it's required that some codec parameters, in particular the
codec profile, are set when the callback is invoked.
This patch removes a get_format invocation where this is not guaranteed.
The codec parameters, including the profile, are really set further
below. (The same code path that sets the profile also calls get_format
properly too.)
This just happened to work by coincidence in most cases. For example, if
the API user just copied or reused the AVStream's AVCodecContext when
decoding, the profile would be set properly. But in some cases it
fails., such as with the sample WolfensteinTwitch.mp4 on the samples
server.
Remove the redundant get_format call. Apparently it serves no purpose
anymore, although it is possible that this was different at the time it
was added in commit ffd77f94a2.
This fixes hwaccel usage for API users which do not set the profile
when setting up the AVCodecContext (which is allowed).
Work on the AVFrame references directly.
Instead of setting up a flipped/swapped "view" on the pictures,
flip/swap them when returning decoded frames to the API user.
In some situations, MMAL won't return a decoded frame for certain input
frames. This can happen if a frame fails to decode, or if a packet does
not actually contain a complete frame. In these situations, we would
deadlock (or actually timeout) waiting for an expected output frame,
which is not ideal. On the other hand, there are situations where we
definitely have to block to avoid deadlocks. (This mess is a
consequence of trying to map MMAL's asynchronous and flexible
dataflow to libavcodec, which is more static and rigid.)
Solve this by doing a blocking wait only if the amount of buffered data
is too big. The whole purpose of the blocking wait is to avoid excessive
buffering of input data, so we can skip it if it appears to be low. The
consequence is that libavcodec can gracefully return no frame to the
API user.
We want to track the number of full packets to make our heuristic work.
But MMAL buffers are fixed-size, requiring splitting large packets. This
is why the previous commit is needed. We use the ..._FRAME_END flag to
remember packet boundaries, but MMAL does not preserve these buffer
flags when returning buffers to the user.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The next commit needs 1 bit of additional information per MMAL buffer
sent to the MMAL input port. This information will be needed when the
buffer is recycled (i.e. returned by the input port's callback).
Normally, we could use MMAL_BUFFER_HEADER_FLAG_USER0, but that is
unexpectedly not preserved.
Do this by storing a pointer to FFBufferEntry in the MMAL buffer's
user data, instead of an AVBufferRef. This also changes the lifetime
of FFBufferEntry.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This affects Annex B streams (such as demuxed from .ts and others). It
also handles the format change in reinit-large_420_8-to-small_420_8.h264
correctly.
Instead of passing through the extradata, create it on the fly it from
the currently active SPS and PPS. Since reconstructing the PPS and SPS
NALs would be very complicated and verbose, we use the NALs as they
originally appeared in the bitstream.
The code for writing the extradata is somewhat derived from
libavformat/avc.c, but it's small and different enough that sharing it
is not really worth it.
We assume an upper bound of 4096 bytes for each raw SPS/PPS. It's hard
to determine an exact maximum size, but this value was was considered
high enough and safe.
Needed for the following VideotoolBox commit.
Commits 43bc5cf9 and c5371f77 add code for skipping initial zeros in mp3
packets. This code forgot to report to the user that data was skipped at
all.
Since audio codecs allow partial packet decoding, the user application
has to rely on the return value. It will remove the data reported as
consumed by the decoder, and feed it to the decoder again. This resulted
in the mp3 frame after the zero region to be decoded over and over
again, until the zero region was finally skipped by the application.
Fix this by including the amount of skipped bytes to the number of
consumed bytes returned by the decode call.
Fixes trac ticket #4890.
Note that convergence_duration had another meaning, one which was in
practice never used. The only real use for it was a 64 bit replacement
for the duration field. It's better just to make duration 64 bits, and
to get rid of it.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
It appears vdpau drivers can return constrained baseline as unsupported,
even if libvdpau knows about the symbol, and the main profile is
supported.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
Note that this slightly changes behavior: it sets AVMEDIA_TYPE_UNKNOWN
if the codec type is unknown. This should be ok.
Signed-off-by: Anton Khirnov <anton@khirnov.net>
If cmd_pos is broken, this would just keep accumulating packets in the
reassembly buffer, until it fails and flushes the buffer on overflow.
Since packets are usually rather small, this will take a lot of subtitle
packets. The perceived effect is that subtitles are not displayed
anymore after the faulty packet was passed to the decoder.
I'm not terribly sure about this, but on the other hand this code is
active only when fragmented packets need to be reassembled.
Fixes sample file in trac issue #4872.
Assuming the first and second packets are partial, this would append the
reassembly buffer (ctx->buf) to itself with the second
append_to_cached_buf() call, because buf is set to ctx->buf.
I do not know a valid sample file which triggers this, and do not know
if packets can be split into more than 2 sub-packets, but it triggered
with a (differently) broken sample file in trac issue #4872.
Broken by commit ba12ba859a. This only
happens with HLS streams which use encryption and require preserving
cookies sent by the server.
Fixes trac issue #4846.
Some .idx files actually contain duplicate subtitle events:
timestamp: 00:07:52:600, filepos: 00004e800
timestamp: 00:07:52:600, filepos: 00004f800
The second will be dropped, because it has same pts, duration, and text
(the text is just a dummy empty string; the real data is retrieved when
actually reading vobsub subtitle packets).
Dropping this is probably not intended/safe, so avoid it.
See trac issue #4872 for a sample. This patch doesn't fix decoding of
the sample, though.
The stream ID is essentially an arbitrary number defined by the .idx
file headers. They have to match the IDs in the .sub stream. The vobsub
demuxer assumed the IDs would just start from 0, increassing by 1 for
each stream. This is not correct. In the sample I had, the IDs were
starting from 1, leading to no subtitles being displayed at all.
Fix this by using the correct stream ID.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This also drops setting the frame->pts field. This is usually not set by
decoders, so this would be an inconsistency that's at worst a danger to
the API user.
It appears the buffer->dts field is normally not set by the MMAL
decoder, so don't use it. If it's ever going to be set by MMAL, we
don't know whether the value will be what we want.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
The generic code in utils.c sets the AVFrame.pkt_dts field from the
packet it was supposedly decoded. This does not have to be true for a
fully asynchronous decoder like mmaldec. It could be overwritten with an
incorrect value. Even if the decoder doesn't determine the DTS (but sets
it to AV_NOPTS_VALUE), it's impossible to determine a correct value in
utils.c.
Decoders can now be marked with FF_CODEC_CAP_SETS_PKT_DTS, in which case
utils.c won't overwrite the field. The decoders are expected to set this
field (even if they only set it to AV_NOPTS_VALUE).
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
This MMAL feature fills in missing timestamps from the framerate set on
the input port. This is generally unwanted, since libavcodec decoders
merely pass through timestamps without ever "fixing" them. The framerate
is also unknown, and even the timebase doesn't have to be set.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Don't try to do a blocking wait for MMAL output if we haven't even sent
a single real packet, but only flush packets. Obviously we can't expect
to get anything back.
Additionally, don't send a flush packet to MMAL in the same case. It
appears the MMAL decoder will sometimes hang in mmal_vc_port_disable()
(called from ffmmal_close_decoder()), waiting for a reply from the GPU
which never arrives. Either MMAL disallows sending flush packets without
preceding real data, or it's a MMAL bug.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
I can't come up with a nice way to handle this. It's hard to keep the
lock-stepped input/output in this case. You can't predict whether the
MMAL decoder will output a picture (because it's asynchronous), so
you have to assume in general that any packet could produce 0 or 1
frames. You can't continue to write input packets to the decoder,
because then you might get too many output frames, which you can't
get rid of because the lavc decoding API does not allow the decoder
to return an output frame without consuming an input frame (except
when flushing).
The ideal fix is a M:N decoding API (preferably asynchronous), which
would make this code potentially much cleaner. For now, this hack
will do.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Don't try to do a blocking wait for MMAL output if we haven't even sent
a single real packet, but only flush packets. Obviously we can't expect
to get anything back.
Additionally, don't send a flush packet to MMAL in the same case. It
appears the MMAL decoder will sometimes hang in mmal_vc_port_disable()
(called from ffmmal_close_decoder()), waiting for a reply from the GPU
which never arrives. Either MMAL disallows sending flush packets without
preceding real data, or it's a MMAL bug.
I can't come up with a nice way to handle this. It's hard to keep the
lock-stepped input/output in this case. You can't predict whether the
MMAL decoder will output a picture (because it's asynchronous), so
you have to assume in general that any packet could produce 0 or 1
frames. You can't continue to write input packets to the decoder,
because then you might get too many output frames, which you can't
get rid of because the lavc decoding API does not allow the decoder
to return an output frame without consuming an input frame (except
when flushing).
The ideal fix is a M:N decoding API (preferably asynchronous), which
would make this code potentially much cleaner. For now, this hack
will do.
At least the new videotoolbox decoder does not actually set a frame if
end_frame fails. This causes the API to return success and signals that
a picture was decoded, even though AVFrame->data[0] is NULL.
Fix this by propagating end_frame errors.
There can be other headers than "Content-Type:" (in this case, a
"Content-Length:" header was following), so checking for a trailing
newline is wrong.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This returns something like "v12_dev0-1332-g333a27c". This is much more
useful than the individual library versions, of which there are too
many, and which are very hard to map back to releases or git commits.
Signed-off-by: Janne Grunau <janne-libav@jannau.net>
This returns something like "N-73264-gb54ac84". This is much more useful
than the individual library versions, of which there are too much and
which are very hard to map back to releases or git commits.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
The APIC description must be unique, and some ID3v2 tag writers add
spaces to write several APIC entries with the same description. The
trailing spaces simply serve as a way to disambiguate the description.
Do this so that API users do not have to special-case mp3 to fix this
cosmetic issue.
Requested-by: wm4
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Calling ffio_ensure_seekback() if ffio_init_checksum() has been called
on the same context can lead to out of bounds memory accesses and
crashes. The reason is that ffio_ensure_seekback() does not update
checksum_ptr after reallocating the buffer, resulting in a dangling
pointer.
This effectively fixes potential crashes when opening mp3 files.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
GNUTLS_SHUT_RDWR means GnuTLS will keep waiting for the server's
termination reply. But since we don't shutdown the TCP connection at
this point yet, GnuTLS will just keep skipping actual data from the
server, which basically is perceived as hang.
Use GNUTLS_SHUT_WR instead, which doesn't have this problem.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
GNUTLS_SHUT_RDWR means GnuTLS will keep waiting for the server's
termination reply. But since we don't shutdown the TCP connection at
this point yet, GnuTLS will just keep skipping actual data from the
server, which basically is perceived as hang.
Use GNUTLS_SHUT_WR instead, which doesn't have this problem.
Signed-off-by: Martin Storsjö <martin@martin.st>
Some files have SeekHead elements with broken IDs. They mismatch with
the ID of the destination element. These files are written by
"IDMmkvlib0.1" (as identified by the MuxingApp and WritingApp elements),
and the SeekHead IDs are actually endian-swapped.
This confuses the SeekHead logic of the demuxer. It will read some
elements twice, because the SeekHead ID is used to identify and remember
already read elements. With the file at hand, the stream list was
duplicated by reading the Tracks element twice.
Fix this by rejecting invalid EBML IDs in SeekHead entries. (This fix is
relatively specific to the broken file at hand, and doesn't protect
against some other cases of broken SeekHead, such as valid but
mismatching target element IDs.)
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
OSX does not know MSG_NOSIGNAL. BSD (which OSX is based on) has got
the socket option SO_NOSIGPIPE (even if modern BSDs also support
MSG_NOSIGNAL).
Signed-off-by: Martin Storsjö <martin@martin.st>
OSX does not know MSG_NOSIGNAL, and provides its own non-standard
mechanism instead. I guess Apple hates standards.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
This was probably broken some time ago. The breakage is now part of the
ABI. For example, we have:
AV_PIX_FMT_XYZ12BE
AV_PIX_FMT_NV16
AV_PIX_FMT_NV20LE
AV_PIX_FMT_NV20LE is wrong. It has the value 113, but as little-endian
format it should be even. This must have been quite obvious when these
formats were added (because of the AV_PIX_FMT_XYZ12BE entry), but
nobody cared or knew about this.
The future libavutil major bump will also break this additionally,
because disabling FF_API_VDPAU will remove an odd number of entries from
the middle of the enum.
Signed-off-by: Luca Barbato <lu_zero@gentoo.org>
Otherwise it will never be reset, and remain "stuck" in this state
forever. Can happen when seeking: the decoder will receive fragments
from different file positions, which triggers the condition easily.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Move the OpenSSL and GnuTLS implementations to their own files. Other
than the connection code (including options) and some boilerplate, no
code is actually shared.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Apparently it can happen that a mp3 file has junk data between id3 tag
and actual mp3 data. Skip this to avoid outputting nonsense timestamps.
(Two packets had the same timestamps, because the mp3 parser failed to
compute a frame duration.)
In this case, the junk consisted of 1044 bytes of zero, which
incidentally is the same size as normal mp3 frames in this stream. I
suspect the mp3 was edited with some tool which wiped the Xing/LAME
headers. Data near the end of the file suggests it was encoded with
"LAME3.97", but the normal Xing/LAME headers are missing. So this could
be "normal". mpg123 also attempts to skip at least 64KB of junk data by
scanning for headers.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Move the OpenSSL and GnuTLS implementations to their own files. Other
than the connection code (including options) and some boilerplate, no
code is actually shared.
Signed-off-by: Martin Storsjö <martin@martin.st>
Personally, I need the decoder to back out if get_format() returns no
usable pixel format. This didn't work because the error code was not
propagated down the call chain. This in turn happened because the
variable declaration removed in this patch shadowed the variable, whose
value is returned at the end of the function. Consequently, failures of
decode_nal_unit() were ignored in this place.
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
Personally, I need the decoder to back out if get_format() returns no
usable pixel format. This didn't work because the error code was not
propagated down the call chain. This in turn happened because the
variable declaration removed in this patch shadowed the variable, whose
value is returned at the end of the function. Consequently, failures of
decode_nal_unit() were ignored in this place.
Reviewed-by: Andreas Cadhalpun <andreas.cadhalpun@googlemail.com>
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
APIC tags always have a description. Tag writers obviously leave it
empty if there is no description. In this case, libavformat would export
"" as title. Do not set the title instead.
APIC tags always have a description. Tag writers obviously leave it
empty if there is no description. In this case, libavformat would export
"" as title. Do not set the title instead.
Signed-off-by: Michael Niedermayer <michaelni@gmx.at>
wm4
