These fields were added to support -merge_pmt_versions, but the mpegts demuxer
is also keeping track its programs internally, so that should be a better place
to handle it.
Also it is not a very good idea to keep fields like program_num or
pmt_stream_idx in an AVStream, because a single stream can be part of multiple
programs, multiple PMTs, so the stream attributes can refer to any program the
stream is part of.
Since they are not part of public API, lets simply remove them, or rather
replace them with placeholders for ABI compatibility with libavdevice.
Signed-off-by: Marton Balint <cus@passwd.hu>
Also make sure we are checking the old state of the streams because otherwise
some streams might already have the newly parsed stream identifiers which
corrupts matching.
Fixes streams having the same identifier mixed up on pmt version change.
Fixes ticket #9006.
Signed-off-by: Marton Balint <cus@passwd.hu>
Otherwise there can be a small period when the programs only contain the PMT
pid.
Also make sure skip_clear only affects AVProgram clear, and that pmt_pid is
always kept as the first entry of the PID list of the programs. Also reject
PMTs for programs on the wrong PID.
Signed-off-by: Marton Balint <cus@passwd.hu>
PID 0 was removed from the pid list when then PMT was parsed, it is better
to explictly avoid it from being discarded instead of keeing it in the list of
every program.
Signed-off-by: Marton Balint <cus@passwd.hu>
This is a pathological case where the fuzzer provides only 2 bytes per iteration.
Fixes: Timeout (>30 -> 0.9sec)
Fixes: 26488/clusterfuzz-testcase-minimized-ffmpeg_dem_MPEGTS_fuzzer-5911031077142528
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
mpegts_read_header stops parsing the file at the first PMT. However the check
that ensured this was wrong because streams can also be added before the first
PMT is received (e.g. EIT).
So let's make sure we are in the header reading phase by checking if ts->pkt is
unset instead of checking if the number of streams found so far is 0.
Signed-off-by: Marton Balint <cus@passwd.hu>
Fixes: out of array access
Fixes: 26816/clusterfuzz-testcase-minimized-ffmpeg_dem_MPEGTSRAW_fuzzer-6282861159907328.fuzz
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This takes the used values from ISO/IEC 13818-1 Table 2-45 and adds
them to the mpegts.h header. No functional changes.
Signed-off-by: Brad Hards <bradh@frogmouth.net>
Signed-off-by: Marton Balint <cus@passwd.hu>
While having the possibility of non-NOPTS values that can suddenly
jump in time due to adjustments to match PCR is not nice for DVB
subtitles, apparently the parser for this format bases its behavior on
whether the packets' timestamps are NOPTS or not. Thus while we can
adjust timestamps, we should exclude DVB subtitles from the timestamp
unsetting logic.
Fixes#8844
Up until now, opening a section filter works as follows: A filter is
opened and (on success) attached to the MpegTSContext. Then a buffer for
said filter is allocated and upon success attached to the section
filter; on error, the filter is simply freed without removing it from
the MpegTSContext, leaving the latter in an inconsistent state. This
leads to use-after-frees lateron.
This commit fixes this by allocating the buffer first; the filter is
only opened if the buffer could be successfully allocated.
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
The old resync logic had some bugs, for example the packet size could stuck
into 192 bytes, because pos47_full was not updated for every packet, and for
unseekable inputs the resync logic simply skipped some 0x47 sync bytes,
therefore the calculated distance between sync bytes was a multiple of 188
bytes.
AVIO only buffers a single packet (for UDP/mpegts, that usually means 1316
bytes), so for every ten consecutive 188-byte MPEGTS packets there was always a
seek failure, and that caused the old code to not find the 188 byte pattern
across 10 consecutive packets.
This patch changes the custom logic to the one which is used when probing to
determine the packet size. This was already proposed as a FIXME a long time
ago...
This avoids accessing an old, no longer valid buffer.
Fixes: out of array access
Fixes: crash_audio-2020
Found-by: le wu <shoulewoba@gmail.com>
Reviewed-by: Marton Balint <cus@passwd.hu>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
support DOVI Video Stream Descriptor from Dolby Vision Streams
Within the MPEG-2 Transport Stream Format V1.2
From the spec: https://www.dolby.com/us/en/technologies/\
dolby-vision/dolby-vision-bitstreams-in-mpeg-2-transport-\
stream-multiplex-v1.2.pdf.
export the DOVI information with sidedata.
Signed-off-by: vacingfang <vacingfang@tencent.com>
This brings a performance improvement when demuxing files, most of the
improvement comes from buffer pooling unbound packets.
time ffprobe -i samples/ffmpeg-bugs/trac/ticket6132/Samsung_HDR_-_Chasing_the_Light.ts -show_packets >/dev/null 2>&1
Before:
real 0m1.967s
user 0m1.471s
sys 0m0.493s
After:
real 0m1.497s
user 0m1.364s
sys 0m0.129s
Based on a patch of James Almer.
Signed-off-by: Marton Balint <cus@passwd.hu>
Since bae8844e the packet will always be unreferenced when a demuxer
returns an error, so that a lot of calls to av_packet_unref() in lots of
demuxers are now redundant and can be removed.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Marton Balint <cus@passwd.hu>
It is not uncommon to find code where the caller thinks to know better
what the return value should be than the callee. E.g. something like
"if (av_new_packet(pkt, size) < 0) return AVERROR(ENOMEM);". This commit
changes several instances of this to instead forward the actual error.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
On some DVB stream SCTE-35 data packet are available before the end of
MpegTSContext initialization. We have to check if it is the case to
avoid a SEGFAULT.
Signed-off-by: Marton Balint <cus@passwd.hu>
After a1b4f120c0 the teletext PTS values were set
to AV_NOPTS_VALUE if the stream of the PCR pid was discarded.
What actually matters is that if we parse the PCR of the PCR PID or not, so
let's use the cached discard value of the actual PCR PID instead of the stream
discard value, which may be different.
Also fixes ticket #7567, which was caused by the fact that teletext PTS values
were not touched if the PCR pid was discarded even before
a1b4f120c0.
Signed-off-by: Marton Balint <cus@passwd.hu>
merge_pmt_versions was not usable if multiple programs were present because
when it was searching for candidate streams it did not make sure that the PMT was
of the same program. This caused the streams of all programs to get merged into
a single (garbled) program.
This patch makes sure that the program number (service ID) is also matching
with the old streams when parsing the PMT making the feature useful for multi
program streams.
This change might cause issues for single program streams if the program number
changes, but I think it is acceptable because the goal of the option is to make
the parsing resilient to PID changes, and that is still working as expected.
Signed-off-by: Marton Balint <cus@passwd.hu>
Reduces buffering latency with low bitrate streams, where
8192 bytes can mean several seconds.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
ISO-10646 alone means UCS-4 for iconv, the specs refers to the Basic
Multilingual Plane (BMP), therefore we need UCS-2. VLC also using that.
Signed-off-by: Marton Balint <cus@passwd.hu>
This enables us to read the data coding type utilized for
a specific private data stream, of which we currently are
interested in ARIB caption streams.
The component tag limitations are according to ARIB TR-B14,
and the component IDs are defined in ARIB STD-B10.
Fixes issues when a subtitle packet is received before PCR for the
program has been received, leading to wildly jumping timestamps
on the lavf client side as well as in the re-ordering logic.
This usually happens in case of multiplexes where the PCR of a
program is not taken into account with subtitle tracks' DTS/PTS.
In 9152c1e495, the mpegts parser was taught how to parse
PMT sections which contained multiple tables. That commit
fixed parsing of PMT packets from some cable providers,
which included a special SCTE table (0xc0) before the
standard program map table (0x2).
Sometimes, however, the combined 0xc0 and 0x2 tables are
larger than a single TS packet (188 bytes). The mpegts parser
already attempts to parse sections which span multiple packets,
but still assumed that the split section only contained one
table.
This patch fixes parsing of such a sample[1].
Before:
Input #0, mpegts, from 'combined-pmt-tids-split.ts':
Duration: 00:00:01.26, start: 39188.931756, bitrate: 597 kb/s
Program 1
No Program
Stream #0:0[0xeff]: Audio: ac3, 48000 Hz, mono, fltp, 64 kb/s
Stream #0:1[0xefd]: Audio: mp3, 0 channels, fltp
Stream #0:2[0xefe]: Unknown: none
After:
Input #0, mpegts, from 'combined-pmt-tids-split.ts':
Duration: 00:00:01.27, start: 39188.931756, bitrate: 589 kb/s
Program 1
Stream #0:0[0xefd]: Video: h264 ([27][0][0][0] / 0x001B), none, 59.94 fps, 59.94 tbr, 90k tbn, 180k tbc
Stream #0:1[0xefe](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 384 kb/s
Stream #0:2[0xeff](spa): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, mono, fltp, 64 kb/s
Stream #0:3[0xf00]: Data: scte_35
Stream #0:4[0xf01]: Unknown: none (ETV1 / 0x31565445)
Stream #0:5[0xf02]: Unknown: none (ETV1 / 0x31565445)
Stream #0:6[0xf03]: Unknown: none ([192][0][0][0] / 0x00C0)
With the patch, the PMT is parsed correctly so the streams are
created in the correct order, are associated with "Program 1",
and their codecs are set correctly.
[1] https://s3.amazonaws.com/tmm1/combined-pmt-tids-split.ts
Signed-off-by: Aman Gupta <aman@tmm1.net>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Both stream_id and stream_identifier are used in this file,
and have different meanings. The latter comes from the
stream_identifier_descriptor.
Signed-off-by: Aman Gupta <aman@tmm1.net>
This new optional flag makes it easier to deal with mpegts
samples where the PMT is updated and elementary streams move
to different PIDs in the middle of playback.
Previously, new AVStreams were created per PID, and it was up
to the user to figure out which streams had migrated to a new PID
(by iterating over the list of AVProgram and making guesses), and
switch seamlessly to the new AVStream during playback.
Transcoding or remuxing these streams with ffmpeg on the CLI was
also quite painful, and the user would need to extract each set
of PIDs into a separate file and then stitch them back together.
With this new option, the mpegts demuxer will automatically detect
PMT changes and feed data from the new PID to the original AVStream
that was created for the orignal PID. For mpegts samples with
stream_identifier_descriptor available, the unique ID is used to
merge PIDs together. If the stream id is not available, the demuxer
attempts to map PIDs based on their position within the PMT.
With this change, I am able to playback and transcode/remux these
two samples which previously caused issues:
https://tmm1.s3.amazonaws.com/pmt-version-change.tshttps://kuroko.fushizen.eu/videos/pid_switch_sample.ts
I also have another longer sample in which the PMT changes
repeatedly and ES streams move to different pids three times
during playback:
https://tmm1.s3.amazonaws.com/multiple-pmt-change.ts
Demuxing this sample with the new option shows several new log
messages as the PMT changes are handled:
[mpegts] detected PMT change (program=1, version=3/6, pcr_pid=0xf98/0xfb7)
[mpegts] re-using existing video stream 0 (pid=0xf98) for new pid=0xfb7
[mpegts] re-using existing audio stream 1 (pid=0xf99) for new pid=0xfb8
[mpegts] re-using existing audio stream 2 (pid=0xf9a) for new pid=0xfb9
[mpegts] detected PMT change (program=1, version=6/3, pcr_pid=0xfb7/0xf98)
[mpegts] detected PMT change (program=1, version=3/4, pcr_pid=0xf98/0xf9b)
[mpegts] re-using existing video stream 0 (pid=0xf98) for new pid=0xf9b
[mpegts] re-using existing audio stream 1 (pid=0xf99) for new pid=0xf9c
[mpegts] re-using existing audio stream 2 (pid=0xf9a) for new pid=0xf9d
[mpegts] detected PMT change (program=1, version=4/5, pcr_pid=0xf9b/0xfa9)
[mpegts] re-using existing video stream 0 (pid=0xf98) for new pid=0xfa9
[mpegts] re-using existing audio stream 1 (pid=0xf99) for new pid=0xfaa
[mpegts] re-using existing audio stream 2 (pid=0xf9a) for new pid=0xfab
[mpegts] detected PMT change (program=1, version=5/6, pcr_pid=0xfa9/0xfb7)
Signed-off-by: Aman Gupta <aman@tmm1.net>
With these fields, the user has enough information to
detect PMT changes and switch to new streams when the PMT
is updated with new ES pids.
To do so, the user would monitor the AVProgram they're interested
in for changes to pmt_version. If the version changes, they would
iterate over the program's streams to find new streams added with
the updated version number.
If new versions of streams are found, then the user would first try
to replace existing streams where stream_identifier matched.
If stream_identifier is not available, then the user would compare
pmt_stream_idx instead to replace the stream that was previously
at the same position within the PMT.
Signed-off-by: Aman Gupta <aman@tmm1.net>
Some filtered mpegts streams may erroneously include PMTs for
programs that are not advertised in the PAT. This confuses ffmpeg
and most players because multiple audio/video streams are created
and it is unclear which ones actually contain data.
See for example https://tmm1.s3.amazonaws.com/unknown-pmts.ts
In this sample, the PAT advertises exactly one program. But the
pid it points to for the program's PMT contains PMTs for other
programs as well. This is because the broadcaster decided to
re-use the same pid for multiple program PMTs.
The hardware that filtered the original multi-program stream
into a single-program stream did so by rewriting the PAT to
contain only the program that was requested. But since it just
passed through the PMT pid referenced in the PAT, multiple PMTs
are still present for the other programs.
Before:
Input #0, mpegts, from 'unknown-pmts.ts':
Duration: 00:00:10.11, start: 80741.189700, bitrate: 9655 kb/s
Program 4
Stream #0:2[0x41]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 11063 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:3[0x44](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:4[0x45](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
No Program
Stream #0:0[0x31]: Video: mpeg2video ([2][0][0][0] / 0x0002), none(tv), 90k tbr, 90k tbn, 90k tbc
Stream #0:1[0x34](eng): Audio: ac3 (AC-3 / 0x332D4341), 0 channels, fltp
Stream #0:5[0x51]: Video: mpeg2video ([2][0][0][0] / 0x0002), none, 90k tbr, 90k tbn
Stream #0:6[0x54](eng): Audio: ac3 (AC-3 / 0x332D4341), 0 channels
With skip_unknown_pmt=1:
Input #0, mpegts, from 'unknown-pmts.ts':
Duration: 00:00:10.11, start: 80741.189700, bitrate: 9655 kb/s
Program 4
Stream #0:0[0x41]: Video: mpeg2video (Main) ([2][0][0][0] / 0x0002), yuv420p(tv, bt709, progressive), 1280x720 [SAR 1:1 DAR 16:9], Closed Captions, 11063 kb/s, 59.94 fps, 59.94 tbr, 90k tbn, 119.88 tbc
Stream #0:1[0x44](eng): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:2[0x45](spa): Audio: ac3 (AC-3 / 0x332D4341), 48000 Hz, stereo, fltp, 128 kb/s
Signed-off-by: Aman Gupta <aman@tmm1.net>
