Fixes: left shift of 1073741824 by 1 places cannot be represented in type 'int32_t' (aka 'int')
Fixes: 21245/clusterfuzz-testcase-minimized-ffmpeg_AV_CODEC_ID_DIRAC_fuzzer-5683334274613248
Change to int16_t suggested by Lynne
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: Lynne <dev@lynne.ee>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This version is able to output multiple coefficients at a time and
is able to altogether remove actual golomb code parsing.
Its also able to partially recover the last coefficient in case
the packet is incomplete.
Total decoder performance gain for 8bit 420 1080p lossless: 40%.
Total decoder performance gain for 10bit 420 1080p lossless: 40%.
clang was able to vectorize the loop much better than
my handwritten assembly, but gcc was very naive and didn't.
Lookup table is a rewritten version of vc2hqdecode.
Fixes: runtime error: left shift of 1073741838 by 1 places cannot be represented in type 'int32_t' (aka 'int')
Fixes: 3279/clusterfuzz-testcase-minimized-4564805744590848
Suggested-by: <atomnuker>
Reviewed-by: <atomnuker>
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: shift exponent 64 is too large for 64-bit type 'residual' (aka 'unsigned long')
Fixes: 2838/clusterfuzz-testcase-minimized-6260066086813696
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: shift exponent 65 is too large for 64-bit type 'residual' (aka 'unsigned long')
Fixes: 2737/clusterfuzz-testcase-minimized-4968639147016192
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: shift exponent 64 is too large for 64-bit type 'residual' (aka 'unsigned long')
Fixes: 2674/clusterfuzz-testcase-minimized-4999700518273024
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Adding a check for bits == 0 would still make Coverity misdetect this,
so just revert to the normal way of setting the residue to 0.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Can also be used in future cleanups since 99% of the time the leftover
appending will just append to an already empty residue.
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Shifting by more than 63 bits is undefined behavior, athough any
compiler not returning 0 after shifting by any amount would be insane.
Found by Coverity, fixes CID1363959 and CID1363960
Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
Still much left to optimize, but it provides a significant performance
improvement - 10% for 300Mbps (1080p30), 25% for 1.5Gbps (4k 60fps) in
comparison with the default implementation.
Signed-off-by: Rostislav Pehlivanov <rpehlivanov@obe.tv>
