PSEUDOPAL pixel formats are not paletted, but carried a palette with the
intention of allowing code to treat unpaletted formats as paletted. The
palette simply mapped the byte values to the resulting RGB values,
making it some sort of LUT for RGB conversion.
It was used for 1 byte formats only: RGB4_BYTE, BGR4_BYTE, RGB8, BGR8,
GRAY8. The first 4 are awfully obscure, used only by some ancient bitmap
formats. The last one, GRAY8, is more common, but its treatment is
grossly incorrect. It considers full range GRAY8 only, so GRAY8 coming
from typical Y video planes was not mapped to the correct RGB values.
This cannot be fixed, because AVFrame.color_range can be freely changed
at runtime, and there is nothing to ensure the pseudo palette is
updated.
Also, nothing actually used the PSEUDOPAL palette data, except xwdenc
(trivially changed in the previous commit). All other code had to treat
it as a special case, just to ignore or to propagate palette data.
In conclusion, this was just a very strange old mechnaism that has no
real justification to exist anymore (although it may have been nice and
useful in the past). Now it's an artifact that makes the API harder to
use: API users who allocate their own pixel data have to be aware that
they need to allocate the palette, or FFmpeg will crash on them in
_some_ situations. On top of this, there was no API to allocate the
pseuo palette outside of av_frame_get_buffer().
This patch not only deprecates AV_PIX_FMT_FLAG_PSEUDOPAL, but also makes
the pseudo palette optional. Nothing accesses it anymore, though if it's
set, it's propagated. It's still allocated and initialized for
compatibility with API users that rely on this feature. But new API
users do not need to allocate it. This was an explicit goal of this
patch.
Most changes replace AV_PIX_FMT_FLAG_PSEUDOPAL with FF_PSEUDOPAL. I
first tried #ifdefing all code, but it was a mess. The FF_PSEUDOPAL
macro reduces the mess, and still allows defining FF_API_PSEUDOPAL to 0.
Passes FATE with FF_API_PSEUDOPAL enabled and disabled. In addition,
FATE passes with FF_API_PSEUDOPAL set to 1, but with allocation
functions manually changed to not allocating a palette.
Without direct support of such pix_fmt, content is padded to 16-bit
and it is not possible to know that the source file was with a smaller bit depth
so framemd5 is different
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Fixes: runtime error: signed integer overflow: 2147483647 + 1 cannot be represented in type 'int'
Fixes: 3361/clusterfuzz-testcase-minimized-5065842955911168
Found-by: continuous fuzzing process https://github.com/google/oss-fuzz/tree/master/projects/ffmpeg
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
This is required for >= 16bit RGB support
I tried it without templates but its too much duplicated code
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
When checking pix_fmt mapping, some bitstreams are mapped to an
incorrect pix_fmt instead of being rejected (ENOSYS).
Actually, such bitstreams are not supported (FFmpeg encoder does not
produce such bitstream, such bitstream may come only from another
encoder for the moment).
- JPEG 2000 RCT 11/13/15/16 bit depths are mapped to a 8-bit FFmpeg
pix_fmt (e.g. bgr0), which is not expected.
- JPEG 2000 RCT 9/10/12/14 bit depths with alpha are mapped to a
FFmpeg pix_fmt without alpha (e.g. AV_PIX_FMT_GBRP9 for 9-bit with
alpha), which is not expected.
The order for choosing the pix_fmt is changed to the one used by YCbCr
selection (<=8 bit first).
" && !f->transparency" is added to the other lines.
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Libav, for some reason, merged this as a public API function. This will
aid in future merges.
A define is left for backwards compat, just in case some person
used it, since it is in a public header.
Signed-off-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
FFv1 uses two types of coders, golomb and range with two different
tables. This is exposed this in a rather convoluted way, for example
mentioning to set coder type 1 while initializing the variable 'ac' to 2,
because encoder does not use range coder with default table.
Appropriate internal coder type values have been added and used in any
check rather than using raw numbers.
Initialization of avctx.coder_type in ffv1dec is removed because this
field is encoder only. An unneeded validation check in the encoder
is dropped too.
Signed-off-by: Vittorio Giovara <vittorio.giovara@gmail.com>
When compiled with --disable-pthreads, e.g
http://fate.ffmpeg.org/report.cgi?time=20150917015044&slot=alpha-debian-qemu-gcc-4.7,
a bunch of -Wunused-functions are reported due to missing header guards
around threading related functions.
This patch should silence such warnings.
Signed-off-by: Ganesh Ajjanagadde <gajjanagadde@gmail.com>
