hw accelerated transcode (h264_cuvid -> h264_nvenc with -hwaccel cuvid) was
broken after the filtergraph initialization was changed to intialize decoder
first followed by encoder (commit af1761f7b5b1b72197dc40934953b775c2d951cc).
During initialzing encoder with bframes, local buffers are allocated
internally in encoder which fails since no cuda context is available. Now
pushing the correct cuda context before encoder initialization fixes the issue.
Also adding push/pop cuda ctx during create/destroy/map/unmap resources and
destroy encoder session.
Signed-off-by: Timo Rothenpieler <timo@rothenpieler.org>
This prevents part of one exploit leading to an information leak
Found-by: Emil Lerner and Pavel Cheremushkin
Reported-by: Thierry Foucu <tfoucu@google.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a5d849b149ca67ced2d271dc84db0bc95a548abb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1902/clusterfuzz-testcase-minimized-4762451407011840
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 87bddba43b725d43767f2a387cdea0936ac1b549)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: shift exponent 40 is too large for 32-bit type 'unsigned int'
Fixes: 1898/clusterfuzz-testcase-minimized-5970744880136192
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4020b009d1e88ff10abd25fb768165afa546851d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1894/clusterfuzz-testcase-minimized-4716739789062144
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d90c5bf10559554d6f9cd1dfb90767b991b76d5d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1892/clusterfuzz-testcase-minimized-4519341733183488
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit e091b9b3c7859030f2896ca2ae96faa3afc694a1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1891/clusterfuzz-testcase-minimized-6274417925554176
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c49fa2a51452eeba0cf2c14ce999ddeadf69aa4f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes part of: 1888/clusterfuzz-testcase-minimized-5237704826552320
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c901627918ff7480c1bb6f9cae507ee2c7c933d8)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes part of: 1888/clusterfuzz-testcase-minimized-5237704826552320
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c9e884f3d98df85bf7f2cf30d71877b22929fdcb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1885/clusterfuzz-testcase-minimized-5336328549957632
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c845450d2daa0d066045cf94ab51cb496f1b824)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1884/clusterfuzz-testcase-minimized-4637425835966464
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4c472c52525fcab4c80cdbc98b4625d318c84fcb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: shift exponent 1073741848 is too large for 32-bit type 'INTFLOAT' (aka 'int')
Fixes: 1880/clusterfuzz-testcase-minimized-4900645322620928
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 872bac81590ccbec40ba7ad203421d9e38d1b253)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1878/clusterfuzz-testcase-minimized-6441918630199296
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6b9cb5d26a2d9905093621d12785bc5903dce66d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 211633430 + 2147483647 cannot be represented in type 'int'
Fixes: 1874/clusterfuzz-testcase-minimized-5037763613163520
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 67b30decf7793523f7fdaef6fdf7f1179ef42b18)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1871/clusterfuzz-testcase-minimized-5719950331215872
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b9c032ebc0ad17ac0ffefb915ff96baf9d79cab1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1870/clusterfuzz-testcase-minimized-4686788029317120
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 781f88bb26534ececc76eaa972f02536ba2f0f55)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 9c1812491f7be2730351969f4abd9b99d300d604)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1851/clusterfuzz-testcase-minimized-5692607495667712
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 6c3a63fc3d1be7ac947e38a165a299c9e5d37764)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This requires at least 25% of a picture to allocate and decode it
Fixes: Timeout
Fixes: 1845/clusterfuzz-testcase-minimized-5075974343360512
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7f50c25124a015a539823077bb302ff0c7ce8963)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
the timeDataSize argument to aacDecoder_DecodeFrame() seems undocumented and until
2016 04 (203e3f28fbebec7011342017fafc2a0bda0ce530) unused.
after that commit libfdk-aacdec interprets it as size in sample units and memsets that on error.
FFmpeg as well as others (like GStreamer) did interpret it as size in bytes
Fixes: 1442/clusterfuzz-testcase-minimized-4540199973421056 (This requires recent libfdk to reproduce)
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ca6776a993903dbcfef5ae8a18556c40ecf83e1c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1770/clusterfuzz-testcase-minimized-5285511235108864
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 7c36ee216f1e668e2c2af1573bd9dbbb2a501f48)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1839/clusterfuzz-testcase-minimized-6238490993885184
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 357f2316a08478a4442e8051978c7b161e10281c)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Kevin Mark <kmark937@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 114e8716214d414d7965029ae5fe74668ed69e4a)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: index 12 out of bounds for type 'uint8_t [8]'
Fixes: 1832/clusterfuzz-testcase-minimized-6574546079449088
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ac8dfcbd89a818b786d05ebc1af70f7bf6aeb86e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1830/clusterfuzz-testcase-minimized-5828293733384192
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 53c0c637d36c1de9ea461a8d863e8703da090894)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1829/clusterfuzz-testcase-minimized-5527165321871360
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 77d98898211eeb0241e8411428b0b364a6231744)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Found-by: Roger Scott <rscott@grammatech.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 1a36354698fc0453ba4d337786d2cb4d3e374cfb)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This avoids potential undefined behavior in debug mode while still allowing
developers which want to check for potential additional overflows to do so
by manually enabling this.
Reviewed-by: wm4
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit a44b3abb4cf922e379fbac55452d0482a8223597)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1825/clusterfuzz-testcase-minimized-6002833050566656
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8e87d146d798ca25d8f3a4520a6deb7946b39d73)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: out of array read
Fixes: 1781/clusterfuzz-testcase-minimized-4617176877105152
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b946bd8ef2c7aeee09469a4901182a44f9b67189)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Found-by: jamrial
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 5782e0ba8cc30bb08a806cdeda1adfb89a0556b4)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 4096 - -2147483648 cannot be represented in type 'int'
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 89325417e7b33f4b08171d9d609c48662d96b2d3)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: poc.snm
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ca616b0f72c65b0ef5f9e1e6125698b15f50a26e)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 4bc3008d04451cd31818e21703ed7ed96b6ff074)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: fmvc-poc.avi
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 3766aa7343c43521c8ad67aaec26e3c91f6d91c7)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1734/clusterfuzz-testcase-minimized-5385630815092736
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 8fb00b3e858b7a5aeccfe6bdfc10290c2121c3ec)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1724/clusterfuzz-testcase-minimized-4842395432648704
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 40fa6a2fa2c255293a780a194eecae5df52644a1)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1721/clusterfuzz-testcase-minimized-4719352135811072
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit b5228e44c7f3a5eba537c8a39a45cfbf2961a28d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes multiple: runtime error: signed integer overflow: 2147115008 + 413696 cannot be represented in type 'int'
Fixes: 1723/clusterfuzz-testcase-minimized-5309409372667904
Fixes: 1727/clusterfuzz-testcase-minimized-5900685306494976
Fixes: 1737/clusterfuzz-testcase-minimized-5922321338466304
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit efeb47fd5d5cbf980e52a6d5e741c3c74b94b5e2)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1710/clusterfuzz-testcase-minimized-4837032931098624
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Reviewed-by: "Ronald S. Bultje" <rsbultje@gmail.com>
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit d4ee76780869c659a5d3b0815c56024ab260a81d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 65535 * 64256 cannot be represented in type 'int'
Fixes: 1717/clusterfuzz-testcase-minimized-5491696676634624
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit ec2b76aab44f55be22eb12d86eb0dfd2eff68581)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1630/clusterfuzz-testcase-minimized-6326111917047808
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 955db411929a9876d3cd016fbbb9c49b6362feba)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: 1716/clusterfuzz-testcase-minimized-4691012196761600
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit c3547dcbc326474745f02a618e01848a293f3f92)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
