getauxval is marginally faster, and works even when procfs is not mounted
support on Linux was added in glibc 2.16
support on Android was added in 4.4 (API 20)
fixes#6578
Signed-off-by: Aman Karmani <aman@tmm1.net>
deinterlaces CVPixelBuffers, i.e. AV_PIX_FMT_VIDEOTOOLBOX frames
for example, an interlaced mpeg2 video can be decoded by avcodec,
uploaded into a CVPixelBuffer, deinterlaced by Metal, and then
encoded to h264 by VideoToolbox as follows:
ffmpeg \
-init_hw_device videotoolbox \
-i interlaced.ts \
-vf hwupload,yadif_videotoolbox \
-c:v h264_videotoolbox \
-b:v 2000k \
-c:a copy \
-y progressive.ts
(note that uploading AVFrame into CVPixelBuffer via hwupload
requires 504c60660d)
this work is sponsored by Fancy Bits LLC
Reviewed-by: Ridley Combs <rcombs@rcombs.me>
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Aman Karmani <aman@tmm1.net>
Teach AV_HWDEVICE_TYPE_VIDEOTOOLBOX to be able to create AVFrames of type
AV_PIX_FMT_VIDEOTOOLBOX. This can be used to hwupload a regular AVFrame
into its CVPixelBuffer equivalent.
ffmpeg -init_hw_device videotoolbox -f lavfi -i color=black:640x480 -vf hwupload -c:v h264_videotoolbox -f null -y /dev/null
Signed-off-by: Aman Karmani <aman@tmm1.net>
This can be used to receive the raw mpegts stream from a SAT>IP
server, by letting avformat handle the RTSP/RTP/UDP negotiation
and setup, but then simply passing the MP2T stream through
instead of demuxing it further.
For example, this command would demux/remux the mpegts stream:
SATIP_URL='satip://192.168.1.99:554/?src=1&freq=12188&pol=h&ro=0.35&msys=dvbs&mtype=qpsk&plts=off&sr=27500&fec=34&pids=0,17,18,167,136,47,71'
ffmpeg -i $SATIP_URL -map 0 -c copy -f mpegts -y remux.ts
Whereas this command will simply write out the raw stream, with
the original PAT/PMT/PIDs intact:
ffmpeg -rtsp_flags satip_raw -i $SATIP_URL -map 0 -c copy -f data -y raw.ts
Signed-off-by: Aman Karmani <aman@tmm1.net>
Without this ffmpeg will attempt to copy the dts from the
most recently enqueued packet into the most recently dequeued
frame, which does not account for the buffering inside v4l2
and is not accurate.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Frames are generally dequeued into capture buffers, so using
the output properties here was incorrect. It happened to work
fine for decoding, since the output/capture buffers have the same
dimensions.
For the v4l2 scaler, the dimensions can be different between output
and capture. Using the buffer's associated context makes this code
work correctly regardless of where the frame is coming from.
Signed-off-by: Aman Gupta <aman@tmm1.net>
On the RPI, three different /dev/video devices exist (decoder, scaler, encoder).
When probing the devices in order, the originally requested pix fmt
would be mutated causing the wrong one to be chosen when a matching
device was finally found.
Signed-off-by: Aman Gupta <aman@tmm1.net>
In rare circumstances, if the codec is not configured with the
proper parameters the input buffers can be allocated with a size
that's too small to hold an individual packet. Since MediaCodec
expects exactly one incoming buffer with a given PTS, it is not
valid to split data for a given PTS across two input buffers.
See https://developer.android.com/reference/android/media/MediaCodec#data-processing:
> Do not submit multiple input buffers with the same timestamp
Signed-off-by: Aman Gupta <aman@tmm1.net>
The loop may mutate the input buffer, so re-fetch it to ensure
the current one is always used.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
When compiling natively on an RPI where libomxil-bellagio-dev
was also installed, `check_headers OMX_Core.h` succeeded and
the -isystem compiler flag was never added to the build.
For non-native builds, the error message now mentions the
raspberrypi/firmware repository where the RPI specific
headers are available.
Signed-off-by: Aman Gupta <aman@tmm1.net>
ENOMEM indicates an allocation failure, and there are no allocations
happening here. The buffers are pre-allocated and there are simply
none available at this time.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Integer overflow in the Q16 framerate calculation was sending
invalid values to the OMX encoder.
On the RPI4, this manifested as bitrate controls being ignored
on video streams with 60000/1001 framerates. Video streams with
30000/1001 framerates were not affected.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Many ffmpeg + rpi compilation guides on the internet recommend
using `./configure --enable-omx --enable-omx-rpi`. This fails
to find the IL OMX headers on device because the omx require_headers
check happens first before the add_cflags in omx_rpi.
A workaround is to use `./configure --enable-omx-rpi` only, since
omx_rpi already implies omx. But because many users expect to use
existing scripts and commands, we swap the order here so omx_rpi
special cases are applied first.
In the past this wasn't an issue because users noticed the OMX_Core.h
missing error and installed libomxil-bellagio-dev. But since
76c82843cc, the rpi specific headers from /opt/vc/include/IL
are required.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Fixes crash noticed in the cbs_userdata patchset.
====ERROR: AddressSanitizer: heap-buffer-overflow on address 0x609000026c89 at pc 0x00010725d37b bp 0x7ffeea04e750 sp 0x7ffeea04e748
READ of size 4 at 0x609000026c89 thread T0
#0 0x10725d37a in ff_cbs_read_unsigned get_bits.h:274
#1 0x1072d2767 in ff_cbs_read_a53_user_data cbs_misc_syntax_template.c:119
#2 0x1078251a7 in h264_metadata_filter h264_metadata_bsf.c:595
#3 0x105c1321d in output_packet ffmpeg.c:853
0x609000026c89 is located 1 bytes to the right of 8-byte region [0x609000026c80,0x609000026c88)
allocated by thread T0 here:
#0 0x10aef08d7 in wrap_posix_memalign (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x578d7)
#1 0x10aca95e6 in av_malloc mem.c:87
#2 0x10ac545fe in av_buffer_allocz buffer.c:72
#3 0x107263b27 in cbs_h264_read_nal_unit cbs_h264_syntax_template.c:722
#4 0x10725b688 in cbs_read_fragment_content cbs.c:155
Signed-off-by: Aman Gupta <aman@tmm1.net>
The existing av_mediacodec_release_buffer allows the user to render
or discard the Surface-backed frame. This new method allows the user
to control exactly when the frame will be rendered to its SurfaceView.
Available since Android API 21.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Fixes a bug that would prevent using multiple comma-separated filters,
and allows options to be passed to each filter.
Based on similar loop in ffmpeg_opt.c's new_output_stream().
Signed-off-by: Aman Gupta <aman@tmm1.net>
In 9152c1e495, the mpegts parser was taught how to parse
PMT sections which contained multiple tables. That commit
fixed parsing of PMT packets from some cable providers,
which included a special SCTE table (0xc0) before the
standard program map table (0x2).
Sometimes, however, the combined 0xc0 and 0x2 tables are
larger than a single TS packet (188 bytes). The mpegts parser
already attempts to parse sections which span multiple packets,
but still assumed that the split section only contained one
table.
This patch fixes parsing of such a sample[1].
Before:
Input #0, mpegts, from 'combined-pmt-tids-split.ts':
Duration: 00:00:01.26, start: 39188.931756, bitrate: 597 kb/s
Program 1
No Program
Stream #0:0[0xeff]: Audio: ac3, 48000 Hz, mono, fltp, 64 kb/s
Stream #0:1[0xefd]: Audio: mp3, 0 channels, fltp
Stream #0:2[0xefe]: Unknown: none
After:
Input #0, mpegts, from 'combined-pmt-tids-split.ts':
Duration: 00:00:01.27, start: 39188.931756, bitrate: 589 kb/s
Program 1
Stream #0:0[0xefd]: Video: h264 ([27][0][0][0] / 0x001B), none, 59.94 fps, 59.94 tbr, 90k tbn, 180k tbc
Stream #0:1[0xefe](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream #0:2[0xeff](spa): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, mono, fltp, 64 kb/s
Stream #0:3[0xf00]: Data: scte_35
Stream #0:4[0xf01]: Unknown: none (ETV1 / 0x31565445)
Stream #0:5[0xf02]: Unknown: none (ETV1 / 0x31565445)
Stream #0:6[0xf03]: Unknown: none ([192][0][0][0] / 0x00C0)
With the patch, the PMT is parsed correctly so the streams are
created in the correct order, are associated with "Program 1",
and their codecs are set correctly.
[1] https://s3.amazonaws.com/tmm1/combined-pmt-tids-split.ts
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
@xyz reported a regression on his Sony Xperia Z3 Tablet Compact where
playback would intermittently fail to start, essentially deadlocking in
the decoder. Bisecting narrowed down the issue to this commit, which was
meant as an optimization but is not necessary.
This reverts commit a75bb5496a.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Both stream_id and stream_identifier are used in this file,
and have different meanings. The latter comes from the
stream_identifier_descriptor.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This new optional flag makes it easier to deal with mpegts
samples where the PMT is updated and elementary streams move
to different PIDs in the middle of playback.
Previously, new AVStreams were created per PID, and it was up
to the user to figure out which streams had migrated to a new PID
(by iterating over the list of AVProgram and making guesses), and
switch seamlessly to the new AVStream during playback.
Transcoding or remuxing these streams with ffmpeg on the CLI was
also quite painful, and the user would need to extract each set
of PIDs into a separate file and then stitch them back together.
With this new option, the mpegts demuxer will automatically detect
PMT changes and feed data from the new PID to the original AVStream
that was created for the orignal PID. For mpegts samples with
stream_identifier_descriptor available, the unique ID is used to
merge PIDs together. If the stream id is not available, the demuxer
attempts to map PIDs based on their position within the PMT.
With this change, I am able to playback and transcode/remux these
two samples which previously caused issues:
https://tmm1.s3.amazonaws.com/pmt-version-change.tshttps://kuroko.fushizen.eu/videos/pid_switch_sample.ts
I also have another longer sample in which the PMT changes
repeatedly and ES streams move to different pids three times
during playback:
https://tmm1.s3.amazonaws.com/multiple-pmt-change.ts
Demuxing this sample with the new option shows several new log
messages as the PMT changes are handled:
[mpegts] detected PMT change (program=1, version=3/6, pcr_pid=0xf98/0xfb7)
[mpegts] re-using existing video stream 0 (pid=0xf98) for new pid=0xfb7
[mpegts] re-using existing audio stream 1 (pid=0xf99) for new pid=0xfb8
[mpegts] re-using existing audio stream 2 (pid=0xf9a) for new pid=0xfb9
[mpegts] detected PMT change (program=1, version=6/3, pcr_pid=0xfb7/0xf98)
[mpegts] detected PMT change (program=1, version=3/4, pcr_pid=0xf98/0xf9b)
[mpegts] re-using existing video stream 0 (pid=0xf98) for new pid=0xf9b
[mpegts] re-using existing audio stream 1 (pid=0xf99) for new pid=0xf9c
[mpegts] re-using existing audio stream 2 (pid=0xf9a) for new pid=0xf9d
[mpegts] detected PMT change (program=1, version=4/5, pcr_pid=0xf9b/0xfa9)
[mpegts] re-using existing video stream 0 (pid=0xf98) for new pid=0xfa9
[mpegts] re-using existing audio stream 1 (pid=0xf99) for new pid=0xfaa
[mpegts] re-using existing audio stream 2 (pid=0xf9a) for new pid=0xfab
[mpegts] detected PMT change (program=1, version=5/6, pcr_pid=0xfa9/0xfb7)
Signed-off-by: Aman Gupta <aman@tmm1.net>
With these fields, the user has enough information to
detect PMT changes and switch to new streams when the PMT
is updated with new ES pids.
To do so, the user would monitor the AVProgram they're interested
in for changes to pmt_version. If the version changes, they would
iterate over the program's streams to find new streams added with
the updated version number.
If new versions of streams are found, then the user would first try
to replace existing streams where stream_identifier matched.
If stream_identifier is not available, then the user would compare
pmt_stream_idx instead to replace the stream that was previously
at the same position within the PMT.
Signed-off-by: Aman Gupta <aman@tmm1.net>
These fields will allow the mpegts demuxer to expose details about
the PMT/program which created the AVProgram and its AVStreams.
In mpegts, a PMT which advertises streams has a version number
which can be incremented at any time. When the version changes,
the pids which correspond to each of it's streams can also change.
Since ffmpeg creates a new AVStream per pid by default, an API user
needs the ability to (a) detect when the PMT changed, and (b) tell
which AVStream were added to replace earlier streams.
This has been a long-standing issue with ffmpeg's handling of mpegts
streams with PMT changes, and I found two related patches in the wild
that attempt to solve the same problem:
The first is in MythTV's ffmpeg fork, where they added a
void (*streams_changed)(void*); to AVFormatContext and call it from
their fork of the mpegts demuxer whenever the PMT changes.
The second was proposed by XBMC in
https://ffmpeg.org/pipermail/ffmpeg-devel/2012-December/135036.html,
where they created a new AVMEDIA_TYPE_DATA stream with id=0 and
attempted to send packets to it whenever the PMT changed.
Signed-off-by: Aman Gupta <aman@tmm1.net>
In a normal hwaccel, the AVHWFramesContext sets AVFrame.hw_frames_ctx
when it initializes a new AVFrame in av_hwframe_get_buffer().
But the VT hwaccel doesn't know what hw_frames_ctx to assign when
the AVFrame is first created, because it depends on the format of
the pixbuf that the decoder eventually decides to return. Thus
newly created AVFrames always have a NULL hw_frames_ctx, and the
hwaccel would only assign the ctx once a frame was done decoding.
This worked fine with the H264 decoder, but with the HEVC decoder
the frame's data may be moved to another empty AVFrame. Since the
empty AVFrame never had hw_frames_ctx set, a frame with a NULL
ctx could be returned to the API user.
This patch works around the issue by moving the derived
hw_frames_ctx from the AVFrame to a new VTHWFrame which now holds
both the CVPixelBufferRef and the AVBuffer. The hw_frames_ctx
is only copied to the AVFrame right before it is about to be
returned to the user in videotoolbox_postproc_frame() (since
in the case of VT, the hw_frames_ctx is only there for the API
user anyway).
Fixes playback on macOS and iOS of some hevc videos like
https://s3.amazonaws.com/tmm1/videotoolbox/germany-hevc-zdf.ts
Signed-off-by: Aman Gupta <aman@tmm1.net>
Some filtered mpegts streams may erroneously include PMTs for
programs that are not advertised in the PAT. This confuses ffmpeg
and most players because multiple audio/video streams are created
and it is unclear which ones actually contain data.
See for example https://tmm1.s3.amazonaws.com/unknown-pmts.ts
In this sample, the PAT advertises exactly one program. But the
pid it points to for the program's PMT contains PMTs for other
programs as well. This is because the broadcaster decided to
re-use the same pid for multiple program PMTs.
The hardware that filtered the original multi-program stream
into a single-program stream did so by rewriting the PAT to
contain only the program that was requested. But since it just
passed through the PMT pid referenced in the PAT, multiple PMTs
are still present for the other programs.
Before:
Input #0, mpegts, from 'unknown-pmts.ts':
Duration: 00:00:10.11, start: 80741.189700, bitrate: 9655 kb/s
Program 4
Stream #0:2[0x41]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 11063 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:3[0x44](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:4[0x45](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
No Program
Stream #0:0[0x31]: Video: mpeg2video ([2][0][0][0] / 0x0002), none(tv), 90k tbr, 90k tbn, 90k tbc
Stream #0:1[0x34](eng): Audio: ac3 (AC-3 / 0x332D4341), 0 channels, fltp
Stream #0:5[0x51]: Video: mpeg2video ([2][0][0][0] / 0x0002), none, 90k tbr, 90k tbn
Stream #0:6[0x54](eng): Audio: ac3 (AC-3 / 0x332D4341), 0 channels
With skip_unknown_pmt=1:
Input #0, mpegts, from 'unknown-pmts.ts':
Duration: 00:00:10.11, start: 80741.189700, bitrate: 9655 kb/s
Program 4
Stream #0:0[0x41]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 11063 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:1[0x44](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:2[0x45](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
Signed-off-by: Aman Gupta <aman@tmm1.net>
Parses the video_stream_descriptor (H.222 2.6.2) to look
for the still_picture_flag. This is exposed to the user
via a new AV_DISPOSITION_STILL_IMAGE.
See for example https://tmm1.s3.amazonaws.com/music-choice.ts,
whose video stream only updates every ~6 seconds.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
For seekable mpegts streams, duration is calculated from
pts by seeking to the end of the file for a pts and subtracting
the initial pts to compute a duration.
This can be expensive in terms of added latency during
probe, especially when streaming over a network. This new
option lets you skip the duration calculation, which is useful
when you don't care about the value and want to save some overhead.
This patch is particularly useful when dealing with live mpegts
streams. Normally such streams are not seekable, so durations
are not calculated. However in my case I am dealing with a seekable
live mpegts stream (networked access to a .ts file which is still
being appended to).
Signed-off-by: Aman Gupta <aman@tmm1.net>
Fixes PMT parsing in some mpegts streams which contain
multiple tables within the PMT pid. Previously, the parser
assumed only one table was present in each packet, and discarded
the rest of the section data after attempting to parse the first
table.
A similar issue was documented in the BeyondTV software[1], which
helped me diagnose the same bug in the ffmpeg mpegts demuxer. I also
tried DVBInspector, libdvbpsi's dvbinfo, and tstools' tsinfo to
help debug. The former two properly read PMTs with multiple tables,
whereas the last has the same bug as ffmpeg.
I've created a minimal sample[2] which contains the combined PMT.
Here's what ffmpeg probe shows before and after this patch:
Before:
Input #0, mpegts, from 'combined-pmt-tids.ts':
Duration: 00:00:01.08, start: 4932.966167, bitrate: 741 kb/s
Program 1
No Program
Stream #0:0[0xf9d]: Audio: ac3, 48000 Hz, mono, fltp, 96 kb/s
Stream #0:1[0xf9b]: Audio: mp3, 0 channels, fltp
Stream #0:2[0xf9c]: Unknown: none
After:
Input #0, mpegts, from 'combined-pmt-tids.ts':
Duration: 00:00:01.11, start: 4932.966167, bitrate: 718 kb/s
Program 1
Stream #0:0[0xf9b]: Video: mpeg2video ([2][0][0][0] / 0x0002), none(tv, top first), 29.97 fps, 29.97 tbr, 90k tbn, 90k tbc
Stream #0:1[0xf9c](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:2[0xf9d](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, mono, fltp, 96 kb/s
With the patch, the PMT is parsed correctly so the streams are
created in the correct order, are associated with "Program 1",
and their codecs are set correctly.
[1] http://forums.snapstream.com/vb/showpost.php?p=343816&postcount=201
[2] https://s3.amazonaws.com/tmm1/combined-pmt-tids.ts
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This allows remuxing streams from one mpegts container to another,
without requiring avformat_find_stream_info() (or using `ffmpeg
-probesize 32` on the cli).
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This mimics the logic flow in all the other callbacks
(pat_cb, sdt_cb, m4sl_cb), and avoids calling skip_identical()
for non PMT_TID packets.
Since skip_identical modifies internal state like
MpegTSSectionFilter.last_ver, this change prevents unnecessary
reprocessing on some streams which contain multiple tables in
the PMT pid. This can be observed with streams from certain US
cable providers, which include both tid=0x2 and another unspecified
tid=0xc0.
Signed-off-by: Aman Gupta <aman@tmm1.net>
No longer required since 63d875772d. The equivalent hack
for h264 was removed in that commit, but this one was missed.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Older iOS devices don't have a hardware HEVC decoder, but the
software decoder offered by VideoToolbox is well-optimized and
performs much better than the ffmpeg decoder.
Signed-off-by: Aman Gupta <aman@tmm1.net>
I tested the previous mediacodec changes on seven different Android
TV devices, with both mpeg2 and h264 content. All except one worked
as expected. The exception was the MiBox3 running Android 6.0.1,
where playback would freeze on a frame every few seconds. I tested
two other AMLogic devices with newer Android versions that did not
show the same problem. H264 decoding on the MiBox3 was also not affected,
so this workaround applies only to OMX.amlogic.mpeg2.decoder.awesome
on Android API22.
There is a rumor that Xiaomi is planning to release Android Oreo for
the MiBox3, so I will revisit in a few months to confirm whether this
is specific to os/driver version or the chipset used in that device.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
The output_buffer_count==0 special case is no longer required, and
can cause spurious EAGAIN to surface to the user when input buffers
are filled up. Since the caller now knows if the decoder is accepting
new input (via current_input_buffer>=0), let the wait parameter
control whether we block or not.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
The new logic follows a recommendation by @rcombs to use
dequeueInputBuffer with a timeout of 0 as a way to detect
whether the codec wants more data. The dequeued buffer index is
kept in MediaCodecDecContext until it can be used next.
A similar technique is also used by the Google's official media
player Exoplayer: see MediaCodecRenderer.feedInputBuffer().
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
Also fixes a bug where EOS buffer was sent with incorrect
pts when not using surface generation.
Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
Signed-off-by: Aman Gupta <aman@tmm1.net>
Aman KarmaniandPaul B Mahol
