This makes the filter output match that of the C version.
It was left intentionally while we figured out if it was better
or not, and while it makes certain samples better, it makes static
samples jump around slightly.
The issue is that while decode_slice is guaranteed to never get
called without start_frame, end_frame is not. Moreover, it is
not guaranteed it won't be called twice.
On a badly-broken sample, this is what happens, which leads to
a segfault, as vp->slices_buf doesn't exist, as it has been handed
off for decoding already and isn't owned by the frame.
Return an error as it's indicative that it's a corrupt stream rather
than just missing any slices.
Prevents a segfault.
As per the spec:
VUID-VkVideoDecodeH264PictureInfoKHR-sliceCount-arraylength
sliceCount must be greater than 0
VUID-VkVideoDecodeH265PictureInfoKHR-sliceSegmentCount-arraylength
sliceSegmentCount must be greater than 0
This particularly happens with seeking in field-coded H264.
This commit scraps a bool to signal to recreate the session parameters,
but instead destroys them, forcing them to be recreated.
As this can happen between start_frame and end_frame, do this
at both places.
Move the slice offsets buffer to the thread decode context.
It isn't part of the resources for frame decoding, the driver
has to process and finish with it at submission time.
That way, it doesn't need to be alloc'd + freed on every frame.
This requires using the new AVHWFramesContext.opaque field, as
otherwise, the profile attached to the decoder will be freed
before the frames context, rendering the frames context useless.
The issue is that with a threadsafe hwaccel and multiple enabled
frame threads, hwaccel->uninit() is never called.
Previously, the function was guaranteed to never have any threads
with hwaccel contexts, so it never bothered to uninit any.
Determined experimentally, on various videos and hardware.
On Intel, using less resources in-flight is around 15% faster,
with similar results on Nvidia hardware.
This reduces memory needed dramatically, as unneeded resources
can be immediately returned to the pool.
Although waitforfences is threadsafe, we add a mutex wait around
it, as the mutex fence in combination with waitforfences assures
us that no other thread will reset the fence in the meanwhile
whilst the mutex is locked. This allows is to call
ff_vk_exec_discard_deps.
It was introduced for Vulkan, but it is equivalent to
short_term_ref_pic_set_size when !short_term_ref_pic_set_sps_flag,
and when !!short_term_ref_pic_set_sps_flag, Vulkan hardcodes a zero
anyway.
According to Dave Airlie:
> <airlied> but I think ignoring it should be fine, I can't see any
> other way to get the imaeg extents correct for other usage
> <Lynne> what width/height should be used for the images?
> the final presentable dimensions, or the coded dimensions?
> <airlied> if you don't want noise I think the presentable dims
> <airlied> the driver should round up the allocations internally,
> but if you are going to sample from the images then w/h have to be
> the bounds of the image you want
> <airlied> since otherwise there's no way to stop the sampler from
> going outside the edges
Apparently, the alignment values are informative, rather than mandatory,
but the spec's wording makes it sound as if they're mandatory.
When seeking through MBAFF-coded H264, this can happen. Decoding calls end_frame
without calling start_frame. We are unable to decode this, as no frame
state has been set.
Happens for both VAAPI and Vulkan. Could be an issue elsewhere, hence
the individual commit.
The temporary AVFrame on staack enables us to use the common
dependency/dispatch code in prepare_frame().
The prepare_frame() function is used for both frame initialization
and frame import/export queue family transfer operations.
In the former case, no AVFrame exists yet, so, as this is purely
libavutil code, we create a temporary frame on stack. Otherwise,
we'd need to allocate multiple frames somewhere, one for each
possible command buffer dispatch.
The idea was that it's faster to map linear images and copy them
via regular memcpy. This is a very niche use, plus very inconsistently
useful, as it would only really be faster on a few Intel GPUs.
Even then, using the non-cached memcpy would've been better.
Instead, scrap this code. Drivers are better at figuring out
what copy to use, and if we're host-mapping, it should actually be
just as fast, if not faster.
This commit adds proper handling of multiplane images throughout
all of the hwcontext code. To avoid breakage of individual
components, the change is performed as a single commit.
This commit rewrites the majority of vulkan.c to enable its use
as a general-purpose high-level utility code, usable for decoding,
encoding, and filtering of video frames.
The dependency system was rewritten to simplify management of
execution.
The image handling system was rewritten to accomodate multiplane
images.
Due to how related all the new features were, this is a single
commit.
This just disables the vulkan headers from defining any symbols
like vkCmdPipelineBarrier2(). Instead, all functions must be loaded
via the loader and used as function pointers as vk->CmdPipelineBarrier2.
Mostly just forces developers to write correct code, as using the
symbols can be undesirable in case API users define their own
function wrappers via the loader API.
The hack was added to enable exporting of vulkan images to DRM.
On Intel hardware, specifically for DRM images, all planes must be
allocated next to each other, due to hardware limitation, so the hack
used a single large allocation and suballocated all planes from it.
By natively supporting multiplane images, the driver is what decides
the layout, so exporting just works.
It's a hack because it conflicted heavily with image allocation, and
with the whole ecosystem in general, before multiplane images were
supported, which just made it redundant.
This is also the commit which broke the hwcontext hardest and prompted
the entire rewrite in the first place.
This just bumps the required loader library version (libvulkan).
All device-related features, such as video decoding, atomics, etc.
are still optional and the code deals with their loss on a local level
(e.g. the decoder or filter checks for the features it needs, not
the hwcontext).
Bumping the required version essentially packs all maintenance
extensions which correct the spec rather than requiring to enable
them individually.
Lynne