This limits the bugs, speedloss and extra memory allocation to the case when
optimal tables are needed.
Fixes regressions with slice multi-threading
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
If this is wanted iam not against it but it must be designed to work with all cases
like slice threads, and a single growing buffer does not work very well with slices.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Only do this when building for a recent VAAPI version - initial
driver implementations were confused about the interpretation of the
framerate field, but hopefully this will be consistent everywhere
once 0.40.0 is released.
(cherry picked from commit ff35aa8ca4069bf1543adeec4c28e51e4a012eee)
Default to using VBR when a target bitrate is set, unless the max rate
is also set and matches the target. Changes to the Intel driver mean
that min_qp is also respected in this case, so set a codec default to
unset the value rather than using the current default inherited from
the MPEG-4 part 2 encoder.
(cherry picked from commit eddfb57210298a0a94472794485400a3a6c76196)
This includes a backward-compatibility hack to choose CBR anyway on
old drivers which have no CBR support, so that existing programs will
continue to work their options now map to VBR.
(cherry picked from commit f033ba470fbab1ff6838666d4d86411effa97b27)
Before this change, it was possible to overflow pic_order_cnt_lsb and
generate a stream with invalid POC numbering. This makes sure that
the field is large enough that a single IDR B* P sequence uses fewer
than half the available POC lsb values.
(cherry picked from commit 89725a8512721fffd190021ded2d3f5b42e20e2a)
This change makes the configured GOP size be respected exactly -
previously the value could be exceeded slightly due to flaws in the
frame type selection logic.
(cherry picked from commit 37fab0661a760b2a9d727939d72e629acee1a6ef)
Fixes out of array read
Fixes: 544/clusterfuzz-testcase-5936536407244800.f8bd9b24_8ba77916_70c2c7be_3df6a2ea_96cd9f14
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This fixes ubsan warnings in non debug builds by using unsigned operations
in debug builds the correct signed operations are retained so that overflows
(which should not occur in valid files and may indicate problems in the DSP code
or decoder) can be detected.
Alternatively they can be changed to unsigned unconditionally, then its
not possible though to detect overflows easily if someone wants to test
the DSP code for overflows.
The 2nd alternative would be to leave the code as it is and accept that
there are undefined operations in the DSP code and that ubsan output is
full of them in some cases.
Similar changes would be needed in some other DSP routines
Suggested-by: Matt Wolenetz <wolenetz@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Change the encoding of the original developer name from ISO-8859-1 to UTF-8.
Remove the stale/completed TODO list.
Fix two small typos.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Certain alpha run lengths (for SHQ1/SHQ3/SHQ5) could be stored in
both long and short versions, and we would only accept the short version,
returning -1 (invalid code) for the others. This could cause an
out-of-bounds write on malicious input, as discovered by
Andreas Cadhalpun during fuzzing.
Fix by simply allowing both versions, leaving no invalid codes
in the alpha VLC.
Signed-off-by: Andreas Cadhalpun <Andreas.Cadhalpun@googlemail.com>
