Michael Niedermayer da8a5e77c3 avcodec/hevc/ps: validate rep_format dimensions in multi-layer SPS ...

When an SPS uses the multi-layer extension (nuh_layer_id > 0 with
sps_max_sub_layers_minus1 == 7), width and height are taken from the
VPS rep_format without the av_image_check_size() validation that the
direct path performs.  HEVC F.7.4.3.1.1 requires rep_format pic
dimensions to satisfy the constraints in 7.4.3.2.1, including
"pic_width_in_luma_samples shall not be equal to 0".

Run the same av_image_check_size() check in the multi-layer-extension
path so the SPS is rejected before it reaches setup_pps().

Fixes: VS-FF-2026-0003/poc.flv
Fixes: out of array access

Found-by: Vuln Seeker Cyber Security Team
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
(cherry picked from commit 0f5705959d)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>

2026-05-03 19:57:05 +02:00

..

cabac.c

…

data.c

…

data.h

…

dsp_template.c

…

dsp.c

…

dsp.h

…

filter.c

…

hevc.h

…

hevcdec.c

…

hevcdec.h

…

Makefile

…

mvs.c

…

parse.c

…

parse.h

…

parser.c

…

pred_template.c

…

pred.c

…

pred.h

…

ps_enc.c

…

ps.c

avcodec/hevc/ps: validate rep_format dimensions in multi-layer SPS

2026-05-03 19:57:05 +02:00

ps.h

…

refs.c

…

sei.c

…

sei.h

…