Niklas Haas ce457bfccd swscale/slice: fix init of 32 bpc planes ...

In input.c and output.c and many other places, swscale follows the rule of using
15-bit intermediate if output bpc is <= 8, and 19-bit (inside int32_t)
intermediate otherwise. See e.g. the comments on hyScale() on
swscale_internal.h. These are also the coefficients that yuv2gbrpf32_full_X_c()
is using.

In contrast to this, the plane init code in slice.c (function fill_ones) is
assuming that we use 35-bit intermediates (inside 64-bit integers) for this
case, seemingly added by commit b4967fc71c with
no further justification.

This causes a mismatch whenever the implicitly initialized plane contents leak
out to the output, e.g. when converting from grayscale to RGB.

Fixes: ticket #10716
Signed-off-by: Niklas Haas <git@haasn.dev>
Sponsored-by: Sovereign Tech Fund

2024-12-16 12:21:55 +01:00

12 KiB

Raw Blame History

View Raw