The mapping of streams to the various variant streams to be created by
the HLS muxer is roughly as follows: Space and tab separate variant
stream group maps while the entries in each variant stream group map are
separated by ','.
The parsing process of each variant stream group proceeded as follows:
At first the number of occurences of "a:", "v:" and "s:" in each variant
stream group is calculated so that one can can allocate an array of
streams with this number of entries. Then each entry is checked and the
check for stream numbers was deficient: It did check that there is a
number beginning after the ":", but it did not check that the number
extends until the next "," (or until the end).
This means that an invalid variant stream group like v:0_v:1 will not be
rejected; the problem is that the variant stream in this example is
supposed to have two streams associated with it (because it contains two
"v:"), yet only one stream is actually associated with it (because there
is no ',' to start a second stream specifier). This discrepancy led to
segfaults (null pointer dereferencing) in the rest of the code (when the
nonexistent second stream associated to the variant stream was inspected).
Furthermore, this commit also removes an instance of using atoi() whose
behaviour on a range error is undefined.
Fixes ticket #8652.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
fix ticket: 8651
because the init fragment mp4 file name is without base url name,
so just modify it use the full url which splice after init function.
Tested-by: matclayton
Signed-off-by: Steven Liu <liuqi05@kuaishou.com>
This patch adds possibility to use 'periodic-rekey' option with
multi-variant streams to hlsenc muxer. All streams variants
use parameters from the same key_info_file.
There are 2 sets of encryption options that kind of overlaps and add
complexity, so I tried to do the thing without changing too much code.
There is a little duplication of the key_file, key_uri, iv_string, etc
in the VariantStream since we copy it from hls to each variant stream,
but generally all the code remains the same to minimise appearing
of unexpected bugs. Refactoring could be done as a separate patch then as needed.
Signed-off-by: Yaroslav Pogrebnyak <yyyaroslav@gmail.com>
segment duration is using vs duration which compute by frame per second,
that can not fix problem of VFR video stream, so compute the duration
when split the segment, set the segment target duration use
current packet pts minus the prev segment end pts..
Reported-by: Zhao Jun <barryjzhao@tencent.com>
Reviewed-by: Zhao Jun <barryjzhao@tencent.com>
Signed-off-by: Steven Liu <liuqi05@kuaishou.com>
Sequence numbers of segments should be unique, if an encoder is using shorter
than 1 second segments and it is restarted, then future segments will be using
already used sequence numbers if initial sequence number is based on the number
of seconds since epoch and not microseconds.
Signed-off-by: Marton Balint <cus@passwd.hu>
if the directory name of the segments contains "%v".
This memleak is caused by masking the pointer that will eventually
be freed by a variable of the same name in a smaller scope.
Therefore the pointer that gets freed is always NULL when it is
freed and the allocated data leaks.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
This patch fixes Bug #8469
If x264 baseline profile is used with other profiles,
start_pts will be initialized to audio stream's first pts,
while the duration is calculated based on video stream's pts.
In this patch the start_pts is initialized with the correct stream's first pts.
Signed-off-by: Hongcheng Zhong <sj.hc_Zhong@sjtu.edu.cn>
Reviewed-by: Steven Liu <liuqi05@kuaishou.com>
The default is not to write SDT and PAT periodically, only in the beginning of
every segment. After this patch the user might override this if needed.
Signed-off-by: Marton Balint <cus@passwd.hu>
Currently when hls_enc is active and there are multiple variant stream
outputs, default key file url construction does not work, because it is
based on the FormatContext' url field. But in case of multiple variant
streams, it contains the variant m3u8 output playlist url that contains
the %v placeholder. So the result key file url will hold the %v
placeholder causing run time error message about "could not write the
key file".
This patch correct this behaviour, and use the master playlist url for
constructing the output key file url when master playlist is vailable.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
When program_date_time flag is present, in m3u8 playlist file each
segment has a corresponding EXT-X-PROGRAM-DATE-TIME value. The intial
program-date-time value is the actual current time at init and
each new segment increments this value by its duration. When append_list
flags is also present, existing playlist parsing by hls_append_segment
treats existing segments as new segments regarding the program-date-time
calculation. But it should not do that, because this way all real the
new segments' EXT-X-PROGRAM-DATE-TIME values will be shifted erroneously
by the sum duration of existing segments. Instead it should have
decremented the initial program-date-time value by its duration. This
would ensure that
the first new segment's program-date-time value had the current time as
it is expected.
This patch corrects this behaviour and prevent existing segments to
increment the value of initial_prog_date_time variable but decrements
it.
Reviewed-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
fix ticket: 8461
there is no problem before commit 75aea52a10
Reviewed-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
When a parameter like e.g. language is contained more than once in the
part of var_stream_map pertaining to a single VariantStream, the later
one just overwrites the pointer to the earlier one, leading to a
memleak. This commit changes this by handling the situation gracefully:
The earlier string is silently freed first, so that the last one wins.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
"If an error is detected, localtime_r() shall return a null pointer
and set errno to indicate the error." Yet in case this happened in
hls_init(), AVERROR(ENOMEM) has been returned.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
The hls muxer allocates an array of VariantStreams, a structure that
contains pointers to objects that need to be freed on their own. This
means that the number of allocated VariantStreams needs to be correct
when they are freed; yet the number of VariantStreams is set in
update_variant_stream_info() resp. parse_variant_stream_mapstring()
before the allocation has been checked for success, so that upon error
an attempt would be made to free the objects whose pointers are
positioned at position NULL (the location of VariantStreams) +
offsetof(VariantStream, the corresponding pointer).
Furthermore d1fe1344 added another possibility for the first function
to leave an inconsistent state behind: If an allocation of one of the
objects referenced by the VariantStream fails, the VariantStream will be
freed, but the number of allocated VariantStreams isn't reset, leading
to the same problem as above. (This was done in the mistaken belief that
the VariantStreams array would leak otherwise.)
Essentially the same also happens for the number of cc-streams. It has
been fixed, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
Under certain circumstances hls_write_packet() would add options to an
AVDictionary. Said dictionary was never explicitly freed, instead it was
presumed that these options would be consumed when opening a new
IO-context. This left several possibilities for memleaks:
a) When no new IO-context would be opened at all. This is possible when
using both the flags temp_file and single_file together with a file
output.
b) When an error happens before one actually tries to open the new
IO-context.
c) When the new IO-context does not consume all options.
All three have been fixed; furthermore, the AVDictionary has been put
into a smaller scope (namely the only part of hls_write_packet() where
it is actually used).
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
hls_mux_init() currently leaks an AVDictionary if opening a dynamic
buffer fails or if avformat_init_output fails. This has been fixed by
moving the initialization resp. the freeing of the dictionary around:
In the former case to a place after opening the dynamic buffer, in the
latter to a place before the check for initialization failure so that it
is done unconditionally.
Furthermore, the dictionary is now only copied and freed if the options
in it are actually used (namely when in SEGMENT_TYPE_FMP4 mode).
Finally, a similar situation in hls_start() has been fixed, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
hls_init() would allocate a buffer, although it is only needed in one of
two branches that follow. This commit moves the allocation to the branch
that actually needs the buffer.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
Before ed897633, the hls muxer would free its child AVFormatContexts
and reset the pointer to these contexts to NULL immediately afterwards;
ed897633 moved the former to later (into a separate function), but kept
the resetting, ensuring that the child context leaks.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
Reviewed-by: Steven Liu <lq@onvideo.cn>
This commit fixes a memleak in the hls muxer when one uses a single file
as output. It has been forgotten to free the temporary buffers used to write
the packets so that the size of the leaks basically amounts to the size
of the output file. This commit adds the necessary free.
Reviewed-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@gmail.com>
fix ticket: 7975
Tested-by: Ian Klassen <ian@virtualfunc.com>
Suggested-by: Ian Klassen <ian@virtualfunc.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
free the old_filname to avoid memory leak in error handle
path.
Reviewed-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
write mpegts or fmp4 context into buffer, and flush the buffer into
output file when split fragment. merge two format split workflow into
one workflow
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Throughout hlsenc code, all filename related buffer lengths are set
hardcoded as 1024. This PATCH change it to general value as MAX_URL_SIZE
in internal.h
Reviewed-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
currently master playlist and subtitle playlist creation does not use
temporary files even when temp_file flag is set. Most of the use cases
it is not a problem because master playlist creation happens once on the
beginning of the whole process. But if master playlist is periodically
re-created because of master_pl_refresh_rate is set, non-atomic playlist
creation may cause problems in case of live streaming. This patch
correct this behavior by adding this functionality.
Reviewed-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
When multiple variant streams are specified by var_stream_map option,
%v is expected either in the filename or in the last sub-directory name,
but only in one of them. When both of them contains %v string, current
error message only states half of the truth.
And even %v may appears several times inside the last sub-directory name
or in filename pattern.
This patch clarifies this in the log message and in the doc also.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
When multiple variant streams are specified by var_stream_map option,
implementation assumes that each elementary stream is assigned only once
to any variant. But this is not checked. This patch makes this checking.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
Reviewed-by: Steven Liu<lq@onvideo.cn>
When multiple variant streams are specified by var_stream_map option, %v
placeholder in various names ensures that each variant has its unique
names. Currently %v is substituted by its variant index value (0, 1, 2
etc.). In some use cases it would be handy to specify names for variants
instead of numerical indexes. This patch makes it possible to use names
instead of default indexes. In var_stream_map option each or some of the
variant streams may use an optional name attributum (e.g.
-var_stream_map "v:0,a:0,name:sd v:1,a:1,name:720p") If a name is
specified for a variant, then this name value will be used as
substitution value of %v instead of the default index value.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
Signed-off-by: Steven Liu <lq@onvideo.cn>
Hi All,
When multiple variant streams are specified by var_stream_map option, %v
placeholder in various names ensures that each variant has its unique
names. Most of %v handlng is done in format_name function. Currently
in this function the result buffer is the same as the
input pattern buffer, so you must allocate it before calling format_name
function. It also means, that it is silently assumed that the result
string will NOT be
longer that the pattern string. It is true most of the time, because %v
may appear only once in the pattern string and number of variant streams
is less than 100 in practical cases. But theoretically it will fail if
specified number of variant streams is greater than 100 (i.e. longer
than 2 digits).
This patch fixes this behaviour by altering format_name function to
allocate the
result buffer and return it to the caller.
Please, review this patch.
best,
Bela
>From 6377ebee8a106a9684d41b270c7d6c8e57cd3e7b Mon Sep 17 00:00:00 2001
From: Bela Bodecs <bodecsb@vivanet.hu>
Date: Mon, 17 Jun 2019 14:31:36 +0200
Subject: [PATCH] av_format/hlsenc: fix %v handling by format_name function
When multiple variant streams are specified by var_stream_map option, %v
placeholder in various names ensures that each variant has its unique
names. Most of %v handlng is done in format_name function. Currently
in this function the result buffer is the same as the input pattern
buffer, so you must allocate it before calling format_name function. It
also means, that it is silently assumed that the result string will NOT
be longer that the pattern string. It is true most of the time, because
%v may appear only once in the pattern string and number of variant
streams is less than 100 in practical cases. But theoretically it will
fail if specified number of variant streams is greater than 100. This
patch fixes this behaviour by altering format_name function to allocate
the result buffer and return it to the caller.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
when set option fmp4_init_filename to init_%v.mp4
before patch:
the init file will be init_%v_0.mp4, init_%v_1.mp4
after patch:
the init file will be init_0.mp4, init_1.mp4
Reported-By: Gyan Doshi <ffmpeg@gyani.pro>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
This is a regression introduced by 223d2bde22.
It appears that regression was introduced in 4.1, 4.0.x does not share
this behaviour.
Temp files were not created for MPEG-TS segments options - HLS_TEMP_FILE
flag was never set on AVFormatContext, it is however set on HLSContext object.
In order to fix this issue, proper flags field must be checked. In addition,
renaming code was messed up and apparently was working only for MP4 files.
This patch fixes the segmentation fault issues due to
unhandled errors from ff_http_do_new_request function.
Reviewed-by: Steven Liu <lq@chinaffmpeg.org>
Fixes the following warnings:
libavformat/hlsenc.c: In function 'hls_write_trailer':
libavformat/hlsenc.c:2364:17: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
uint8_t *buffer = NULL;
^~~~~~~
libavformat/hlsenc.c:2372:17: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
int byterange_mode = (hls->flags & HLS_SINGLE_FILE) || (hls->max_seg_size > 0);
^~~
libavformat/hlsenc.c:2379:13: warning: ISO C90 forbids mixed declarations and code [-Wdeclaration-after-statement]
int range_length = 0;
^~~
fix ticket: 7527
check dirname before use it
refine webvtt code in the hls_delete_old_segments
Reported-by: caspy
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
In fmp4 & sub-range mode, the output's duration always smaller than expected,
because the size of the last #EXT-X-BYTERANGE is too small.
Signed-off-by: Charles Liu <liuchh83@gmail.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
The size of init.mp4 is zero in fmp4 mode,
when the input duraton smaller than the expected segment time.
fix ticket: 7166
Signed-off-by: Charles Liu <liuchh83@gmail.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
In fmp4 mode, the duration of the second m4s segment is
unusually smaller than the expected segment time.
Signed-off-by: Charles Liu <liuchh83@gmail.com>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Because it will be used by avformat/segment.c or other module which
need to automatically create sub-directories operation.
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
fix the build warning for "ISO C90 forbids mixed declarations and code"
Reviewed-by: Steven Liu <lq@chinaffmpeg.org>
Signed-off-by: Jun Zhao <mypopydev@gmail.com>
This fixes the creation of the hls manifest in hlsenc.c by writing the
entire manifest at the end for VOD playlists. Live & Event Playlists are unaffected.
This also fixes the behavior with HLS_TEMP_FILE to work correctly when
-hlsflags temp_file is specified, instead of always relying on use_rename, which caused these problems.
Files that would previously take over a week to fragment now take
1 minute on the same hardware. This was a 153 hour audio file (2.2GB of audio).
Signed-off-by: Ronak Patel <ronak2121@yahoo.com>
fix ticket: 7305
vs->sequence - hls->start_sequence - vs->nb_entries is the
after_init_list_dur fragment numbers
fix the wrong compute way vs->sequence - vs->nb_entries
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
and refine the warning message when use both hls_key_info_file and hls_enc
Reviewed-by: Gyan Doshi <ffmpeg@gyani.pro>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
When using hls_list_size with hls_flags delete_segments, currently
hls_list_size * 2 +- segments remain on disk. With this new option,
the amount of disk space used can be controlled by the user.
fix ticket: #7131
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Signed-off-by: Aman Gupta <aman@tmm1.net>
When var_stream_map option is used, %v must appear either in segment
name template or in the directory path. This latter case currently is
not handled and delete_segments flag of hls_flags is broken now. This
patch fix this. The root cause of the bug was that HLSSegment struct
only stores the final filename part, but not the final directory path.
Most of the cases, final path info is unneded, It only necessary when
you want to delete old segments (e.g in case of live streaming).
Without variant streams it was unnecessary to store the final directory
path, because all segment were stored into the same directory. But
introducing %v in directory names either require to store the final
directory path into HLSSegment or associate segments with their variant
streams to be able deleting them later. I have choosen the second
solution and introduced a variant index data member into the segment
struct.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
Signed-off-by: Steven Liu <lq@onvideo.cn>
av_strtok calls strspn on a non-NULL *saveptr, so not NULL initializing
it is an issue.
Fixes CID #1428568
Reviewed-by: Karthick Jeyapal <kjeyapal@akamai.com>
Signed-off-by: Steven Liu <lq@onvideo.cn>
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Enables having proper bit rate values being written into the master
playlist in case of hard-constrained VBR where the maximum bit
rate utilized is known before hand.
Does the same thing as movenc.c, for example.
Signed-off-by: Jan Ekström <jan.ekstrom@aminocom.com>
fix ticket: #7044
Get the first video stream to reference for split segment
when there have more than one video stream
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Reviewed-by: Karthick Jeyapal <kjeyapal@akamai.com>
Currently using HLS_TEMP together with HLS_SECOND_LEVEL_SEGMENT_DURATION
or HLS_SECOND_LEVEL_SEGMENT_SIZE gives error at end of each segment
writing and the final segment file names do not contain the desired
data. This patch fixes this bug by delaying the initilization of
original segment filename after actual temp file renaming will skip the
interfering.
Signed-off-by: Bela Bodecs <bodecsb@vivanet.hu>
If codecpar->extradata is not set (for example, when the stream goes
through the 'tee' muxer), then a segfault occurs.
This patch ensures the data variable is not null before attempting
to access it
Before the var_stream_map option was available - I was using the tee
muxer to create each resolution as an individual stream.
When running this configuration after the most recent hlsenc change
I hit a segfault
The most simple command which recreates the segfault is:
ffmpeg -i in.ts -map 0:a -map 0:v -c:a aac -c:v h264 -f tee [select=\'a,v\':f=hls]tv_hls_hd.m3u8
Signed-off-by: Brendan McGrath <redmcg@redmandi.dyndns.org>
Currently http end of chunk is signalled implicitly in hlsenc_io_open().
This mean playlists http writes would have to wait upto a segment duration to signal end of chunk causing delays.
This patch will fix that problem and improve performance.
Before this patch persistent http connections would work only for media segments.
The playlists were still opening a new connection everytime.
This patch extends persistent http connections to playlists as well.
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
Since close is not called, during http persistent connection,
flush needs to be called so that output is written on time.
Signed-off-by: Steven Liu <lq@chinaffmpeg.org>
This patch fixes Bug #6868
Sometimes end_pts is getting initialized to audio stream's
first pts, while the duration is calculated based on video stream's pts.
In this patch the end_pts is initialized with the correct stream's first pts.
Reviewed-by: Steven Liu <lq@onvideo.cn>
Tested-by: beloko
