Commit Graph
100 Commits
Author SHA1 Message Date
LynneandMichael Niedermayer aab4eb20ca configure: update copyright year
(cherry picked from commit b95ee2ec5f)
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
2024-07-24 16:52:20 +02:00
Lynne 8815d77532 avfft: avoid overreads with RDFT API users
The new API requires an extra array member at the very end,
which old API users did not do.

This disables in-place RDFT transforms and instead
does the transform out of place by copying once, there shouldn't
be a significant loss of speed as our in-place FFT requires a reorder
which is likely more expensive in the majority of cases to do.

(cherry picked from commit 90adef99ca)
2024-02-09 23:23:49 +01:00
Lynne 2c87aa0b23 lavc/Makefile: build vulkan decode code if vulkan_av1 has been enabled
Forgotten.

Reviewed-by: Neal Gompa <ngompa13@gmail.com>
Tested-by: Neal Gompa <ngompa13@gmail.com>
(cherry picked from commit 8c117b75af)
2023-12-04 07:59:03 +01:00
Lynne 86c4d04051 nlmeans_vulkan: fix offsets calculation and various stride issues
We calculated offsets as pairs, but addressed them in the shader
as single float values, while reading them as ivec2s.

Also removes unused code (was provisionally added if cooperative matrices
could be used, but that turned out to be impossible).

(cherry picked from commit 99fcdee5e8)
2023-11-09 09:16:55 +01:00
Lynne 4e5f3e6b8e bwdif_vulkan: fix artifacts on vulkan decode images
Due to making the decode frames context use the coded size, the
filter started to display those artifacts as it reused the input frame's size.

Change it to instead output the real image size for images, not the input.

(cherry picked from commit 0e8abf2698)
2023-10-31 21:40:42 +01:00
Lynne 1a8e766984 vulkan: return VK_NOT_READY when no queries are available
Fixes a validation issue.
The issue is that the function gets called before we've sumitted a frame
for decoding to that context. However, we cannot run queries before
they've been reset, which happens at submission time.
As we'd need to otherwise run a command queue at init-time, just check
if submissions have happened.
2023-10-28 21:16:15 +02:00
Lynne 70864e6adb vulkan_decode: correct flipped condition in image layout
Changed by the previous commit.
Caused validation issues on hardware with !reuse_dpb_dst but not layered_dpb.
2023-10-25 22:01:21 +02:00
Lynne 0b3616231d vulkan_decode: fix another validation issue
Surprising no one, the insane usage rule has a catch.
2023-10-25 20:51:55 +02:00
Lynne 467e411839 vulkan_decode: fix pedantic validation issue
"Validation Error: [ VUID-VkImageViewCreateInfo-imageViewType-04974 ] Object 0: handle = 0x9f9b41000000003c, type = VK_OBJECT_TYPE_IMAGE; | MessageID = 0xc120e150 | vkCreateImageView():
Using pCreateInfo->viewType VK_IMAGE_VIEW_TYPE_2D and the subresourceRange.layerCount VK_REMAINING_ARRAY_LAYERS=(17) and must 1 (try looking into VK_IMAGE_VIEW_TYPE_*_ARRAY).
The Vulkan spec states: If viewType is VK_IMAGE_VIEW_TYPE_1D, VK_IMAGE_VIEW_TYPE_2D, or VK_IMAGE_VIEW_TYPE_3D; and subresourceRange.layerCount is VK_REMAINING_ARRAY_LAYERS,
then the remaining number of layers must be 1"
2023-10-25 20:51:54 +02:00
Lynne 9ee4f47c94 vulkan_decode: use coded_width/height instead of the non-coded width and height
Partially fixes https://streams.videolan.org/issues/19938/20000_20180305-15.04.59.ts
The is coded as 1920x1080, meant to be rendered at 1440x1080 with cropping,
or 1680x1080 before cropping. Currently, the created DPB is 1440x1080, which results
in the image being decoded incorrectly, as the decoder overwrites output memory.
This commit fixes this.
2023-10-25 20:51:05 +02:00
Lynne c258623c0a hwcontext_vulkan: improve queue family init code
When users zero-init'd the struct, or left it as-is, the encode
queue family matched the graphics queue family, which led it to be
incorrectly logged as being used for encode.

This just improves the logging so this isn't printed anymore.
2023-10-24 06:07:09 +02:00
Lynne fec6e84b18 configure: disable libglslang/libshaderc if the vulkan is disabled
Fixes build failures when the Vulkan headers are too old and libglslang
or libshaderc are enabled.
2023-10-15 17:11:47 +02:00
Lynne f31d0f1141 nlmeans_vulkan: parallelize workgroup invocations 2023-10-11 16:53:06 +02:00
Lynne 658b01b5ee nlmeans_vulkan: reduce dispatches by parallelizing the planes 2023-10-11 16:53:04 +02:00
Lynne 6bc8ff7d93 nlmeans_vulkan: fix width/height for chroma plane weights calculation 2023-10-11 16:53:00 +02:00
Lynne 81cc0e1345 hwcontext_vulkan: properly support STORAGE usage for mutliplane images
Fixes multiplane support on Nvidia.

Also, remove the ENCODE usage, even if the driver signals it as supported.
Currently, it's not used, and when it is used, it'll be gated behind
two extension checks.
2023-10-05 23:50:30 +02:00
Lynne 9310ffc809 vulkan_decode: don't call get_proc_addr on every frame's destruction
The issue is that we cannot rely on any context existing when we free
frames. The Vulkan functions are loaded in each context separately,
so until now, we've just been loading them on every frame's destruction.

Rather than do this, just save the function pointers we need in each
frame. The function pointers are guaranteed to not change and exist.
2023-09-15 17:35:22 +02:00
Lynne 552a5fa496 vulkan_hevc: switch from a buffer pool to a malloc and simplify
Simpler and more robust now that contexts are not shared between threads.
2023-09-15 17:35:19 +02:00
Lynne 398467f519 vulkan_decode: convert max level from vulkan to av for comparisons 2023-09-08 06:56:43 +02:00
Lynne 00e77fd21a lavu/tx: add missing prints for the type of dctI/dstI 2023-09-08 06:56:39 +02:00
Lynne 139e54911c lavc/avfft: deprecate the API
This deprecates the currently unused API.
2023-09-01 23:59:14 +02:00
Lynne d40672e661 lavu/tx: fix scaling of R2R transforms
Still slightly inaccurate, but it's good enough now.
2023-09-01 23:59:13 +02:00
Lynne 59b39d241e lavu/tx: improve rdft table generation precision slightly 2023-09-01 23:59:13 +02:00
Lynne 86aa34d339 avfft: wrap lavu/tx instead of ff_dct 2023-09-01 23:59:13 +02:00
Lynne 83ede01bb0 avfft: wrap lavu/tx instead of ff_rdft 2023-09-01 23:59:12 +02:00
Lynne 517e4fcca6 avfft: wrap lavu/tx instead of ff_mdct 2023-09-01 23:59:12 +02:00
Lynne dfcd4bbf51 avfft: wrap lavu/tx instead of ff_fft 2023-09-01 23:59:11 +02:00
Lynne 4acd08be6c ffplay: port to lavu/tx 2023-09-01 23:59:11 +02:00
Lynne a810126501 wmavoice: convert DCT-I/DST-I to lavu/tx
This is the very last user of any lavc transform code.

This also *corrects* wmavoice decoding, as the previous DCT/DST
transforms were incorrect, bringing it closer to Microsoft's
own wmavoice decoder.
2023-09-01 23:59:11 +02:00
Lynne d895d3c8c7 wmavoice: convert RDFT to lavu/tx 2023-09-01 23:59:10 +02:00
Lynne ef8fd7bc3c lavu/tx: add DCT-I and DST-I transforms
These are true, actual DCT-I and DST-I transforms, unlike the
libavcodec versions, which are plainly not.
2023-09-01 23:59:10 +02:00
Lynne 11e22730e1 lavu/tx: add real to real and real to imaginary RDFT transforms
These are in-place transforms, required for DCT-I and DST-I.

Templated as the mod2 variant requires minor modifications, and is
required specifically for DCT-I/DST-I.
2023-09-01 23:59:08 +02:00
Lynne d0a64f9a81 vulkan: do not leak bound_buffer_indices 2023-08-28 22:29:36 +02:00
Lynne 5f8feddd6a lavfi/testsrc_vulkan: fix memory leaks 2023-08-28 22:29:36 +02:00
Lynne 3ef1e50c92 lavfi/transpose_vulkan: fix memory leaks 2023-08-28 22:29:36 +02:00
Lynne d0ab2e2f98 lavfi/scale_vulkan: fix memory leaks 2023-08-28 22:29:35 +02:00
Lynne 0f2ae1ba69 lavfi/nlmeans_vulkan: fix memory leaks 2023-08-28 22:29:35 +02:00
Lynne 9944e96c61 lavfi/chromaber_vulkan: fix memory leaks 2023-08-28 22:29:34 +02:00
Lynne a4673c9dff lavfi/bwdif_vulkan: fix memory leaks 2023-08-28 22:29:34 +02:00
Lynne b6cc53092a lavfi/avgblur_vulkan: fix memory leaks 2023-08-28 22:29:34 +02:00
Lynne f6cf3a40e4 vulkan: check for extension rather than function pointer
The loader ensures only that functions with tagged supported extensions
exist, rather than ensuring only those with supported extensions are
loaded.
As the init function uses Vulkan functions, whose loading requires them
to have the extension flags set, the extension flags are guaranteed
to also exist at this point.
2023-08-28 22:29:33 +02:00
Lynne 747871a42c vulkan: do not leak cooperative matrix properties 2023-08-28 22:29:29 +02:00
Lynne 358919506d vulkan: enable VK_KHR_cooperative_matrix
It's of interest to API users, and of interest to us,
as a DCT/DST can be implemented via matrix multiplies.
2023-08-26 23:14:53 +02:00
Lynne e6a12a5214 vulkan_av1: remove an invalid debug print
Leftover.
2023-07-21 20:04:21 +02:00
Lynne 3e3d46309b lavu/vulkan: remove unused field from the execution pool structure 2023-07-21 20:04:21 +02:00
Lynne 97890c2b55 lavu/vulkan: remove threadsafe buffer index load and fix a signed overflow
It's not needed anymore.
2023-07-21 20:04:20 +02:00
Lynne c06ad641ec lavc/vulkan_decode: use a single execution pool per thread
The spec says command buffer pools must be externally synchronized
objects.

This still lets us pool some, just not as much.
2023-07-21 20:04:15 +02:00
Lynne 9ff834c2a0 bwdif_vulkan: clamp the temporarily interpolated sample spatially
This makes the filter output match that of the C version.
It was left intentionally while we figured out if it was better
or not, and while it makes certain samples better, it makes static
samples jump around slightly.
2023-07-04 00:52:19 +02:00
Lynne c39e861a3d vulkan_h264: check if slices_buf exists on end_frame
The issue is that while decode_slice is guaranteed to never get
called without start_frame, end_frame is not. Moreover, it is
not guaranteed it won't be called twice.

On a badly-broken sample, this is what happens, which leads to
a segfault, as vp->slices_buf doesn't exist, as it has been handed
off for decoding already and isn't owned by the frame.

Return an error as it's indicative that it's a corrupt stream rather
than just missing any slices.

Prevents a segfault.
2023-06-25 03:11:38 +02:00
Lynne 997d8a7e73 vulkan_decode: reject decoding of frames with no slices
As per the spec:
VUID-VkVideoDecodeH264PictureInfoKHR-sliceCount-arraylength
sliceCount must be greater than 0

VUID-VkVideoDecodeH265PictureInfoKHR-sliceSegmentCount-arraylength
sliceSegmentCount must be greater than 0

This particularly happens with seeking in field-coded H264.
2023-06-22 18:18:53 +02:00
Lynne 4ff303a7b8 vulkan_decode: simplify and make session parameter generation more robust
This commit scraps a bool to signal to recreate the session parameters,
but instead destroys them, forcing them to be recreated.

As this can happen between start_frame and end_frame, do this
at both places.
2023-06-22 18:17:54 +02:00
Lynne ba8a803236 vulkan_decode: clean up slice handling
Move the slice offsets buffer to the thread decode context.
It isn't part of the resources for frame decoding, the driver
has to process and finish with it at submission time.
That way, it doesn't need to be alloc'd + freed on every frame.
2023-06-22 18:17:54 +02:00
Lynne 237c400727 vulkan_decode: remove unused fields 2023-06-22 18:17:53 +02:00
Lynne d9af84426b vulkan_decode: fix small memory leak
This requires using the new AVHWFramesContext.opaque field, as
otherwise, the profile attached to the decoder will be freed
before the frames context, rendering the frames context useless.
2023-06-22 18:17:53 +02:00
Lynne 13ff3aa9e7 vulkan_decode: use the hwfc->user_opaque field to store the profile 2023-06-22 18:17:47 +02:00
Lynne ca818ab51c vulkan_h264: filter out constrained/inter flags from the profile index
As the comment says, Vulkan signals all the constrant_set flags,
and does not want them OR'd onto the profile IDC.
So just unset them.
2023-06-15 22:00:42 +02:00
Lynne d0f1d937fe hwcontext_vulkan: free temporary array once unneeded
Fixes a small memory leak.
This also prevents leaks on malloc/mutex init errors.
2023-06-15 22:00:41 +02:00
Lynne b4d5baa8b0 hwcontext_vulkan: call ff_vk_uninit() on device uninit
This fixes three memory leaks from ff_vk_load_props().
2023-06-15 22:00:41 +02:00
Lynne f30b979ca2 pthread_frame: uninit the hwaccel of each frame thread
The issue is that with a threadsafe hwaccel and multiple enabled
frame threads, hwaccel->uninit() is never called.
Previously, the function was guaranteed to never have any threads
with hwaccel contexts, so it never bothered to uninit any.
2023-06-15 22:00:40 +02:00
Lynne eff565dc19 hwcontext_vulkan: tune execution pools
Having less in-flight resources is better in this case.
2023-06-07 23:59:17 +02:00
Lynne 24c4307b80 vulkan_decode: halve execution pool size
Determined experimentally, on various videos and hardware.
On Intel, using less resources in-flight is around 15% faster,
with similar results on Nvidia hardware.
2023-06-07 23:59:17 +02:00
Lynne 5f1be341c2 vulkan: discard dependencies when explicitly waiting for execution
This reduces memory needed dramatically, as unneeded resources
can be immediately returned to the pool.
Although waitforfences is threadsafe, we add a mutex wait around
it, as the mutex fence in combination with waitforfences assures
us that no other thread will reset the fence in the meanwhile
whilst the mutex is locked. This allows is to call
ff_vk_exec_discard_deps.
2023-06-07 23:59:16 +02:00
Lynne 975cd48bb3 vulkan: synchronize access to execution pool fences
vkResetFences is specified as being user-synchronized
(yet vkWaitFences, is not).
2023-06-07 23:59:16 +02:00
Lynne 697382168d hevcdec: remove redundant bits_used_for_short_term_rps field
It was introduced for Vulkan, but it is equivalent to
short_term_ref_pic_set_size when !short_term_ref_pic_set_sps_flag,
and when !!short_term_ref_pic_set_sps_flag, Vulkan hardcodes a zero
anyway.
2023-06-07 23:59:12 +02:00
Lynne 9f9534f5b6 vulkan_decode: fix typo when setting AV1 capabilities
All pNext chained structs in Vulkan are defined as void *, so it doesn't
help catch this.
2023-05-29 23:26:10 +02:00
Lynne e71cd18049 vulkan_decode: do not align the image dimensions
According to Dave Airlie:

> <airlied> but I think ignoring it should be fine, I can't see any
> other way to get the imaeg extents correct for other usage
> <Lynne> what width/height should be used for the images?
> the final presentable dimensions, or the coded dimensions?
> <airlied> if you don't want noise I think the presentable dims
> <airlied> the driver should round up the allocations internally,
> but if you are going to sample from the images then w/h have to be
> the bounds of the image you want
> <airlied> since otherwise there's no way to stop the sampler from
> going outside the edges

Apparently, the alignment values are informative, rather than mandatory,
but the spec's wording makes it sound as if they're mandatory.
2023-05-29 05:12:27 +02:00
Lynne bae92361ed vulkan_decode: check if yuv_sampler exists before freeing it
This prevents multiple NULL accesses - if yuv_sampler exists, then
everything required for it to be destroyed also exists.
2023-05-29 03:23:06 +02:00
Lynne 58f82fc26a vulkan: replace usage of %lu with %"SIZE_SPECIFIER" 2023-05-29 03:22:58 +02:00
Lynne 6018f87599 doc/filtering: add documentation for Vulkan filters
This commit documents most of the Vulkan filters. Some of this was
copy-pasted from equivalent OpenCL filters.
2023-05-29 00:42:04 +02:00
Lynne ea0394fd24 changelog: add new Vulkan features 2023-05-29 00:42:03 +02:00
Lynne f3795e1857 APIchanges: add AVHWAccel changes and bump lavc minor 2023-05-29 00:42:03 +02:00
Lynne db1d022781 APIchanges: add hwcontext_vulkan changes and bump lavu minor 2023-05-29 00:42:02 +02:00
Lynne bef86ba86c APIchanges: add new pixel formats supported and bump lavu minor 2023-05-29 00:42:02 +02:00
Lynne b3fb73af6b swscale: bump minor for implementing support for the new pixfmts 2023-05-29 00:42:02 +02:00
Lynne f701f088f4 vulkan_h264: reject end_frame being called without valid session parameters
When seeking through MBAFF-coded H264, this can happen. Decoding calls end_frame
without calling start_frame. We are unable to decode this, as no frame
state has been set.

Happens for both VAAPI and Vulkan. Could be an issue elsewhere, hence
the individual commit.
2023-05-29 00:42:01 +02:00
Lynne 160a415e22 lavfi: add nlmeans_vulkan filter 2023-05-29 00:42:01 +02:00
Lynne dfff3877b7 vulkan: add support for the atomic float ops extension 2023-05-29 00:42:01 +02:00
Lynne 88e2cca3db tools/cl2c: change to tools/source2c and allow non-OpenCL source files 2023-05-29 00:42:00 +02:00
Lynne 7cfd7e4af4 lavfi: add color_vulkan filter 2023-05-29 00:42:00 +02:00
Lynne 77478f6793 av1dec: add Vulkan hwaccel 2023-05-29 00:42:00 +02:00
Lynne 5dc047716a fate-source: exclude Apache-2 licensed files 2023-05-29 00:41:59 +02:00
Lynne 6ec58ff820 cbs_av1: expose tile col/row starts in SBs 2023-05-29 00:41:59 +02:00
Lynne 36c16a06b2 hevcdec: add Vulkan hwaccel
Thanks to Dave Airlie for figuring out a lot of the parameters.
2023-05-29 00:41:58 +02:00
Lynne a9fbe8b472 h264dec: add Vulkan hwaccel
Thanks to Dave Airlie for figuring out a lot of the parameters.
2023-05-29 00:41:58 +02:00
Lynne 1e8fefff93 libavcodec: add Vulkan common video decoding code 2023-05-29 00:41:57 +02:00
Lynne 023ae6103f libavcodec: add Vulkan common video code 2023-05-29 00:41:57 +02:00
Lynne 6733a1a456 avcodec: add AVHWAccel.flush callback 2023-05-29 00:41:57 +02:00
Lynne be07145109 avcodec: add AVHWAccel.free_frame_priv callback 2023-05-29 00:41:56 +02:00
Lynne 09dc9193ea lavfi: add bwdif_vulkan 2023-05-29 00:41:56 +02:00
Lynne f70f873272 transpose_vulkan: port for the rewrite 2023-05-29 00:41:54 +02:00
Lynne d4b51b5085 scale_vulkan: port for the rewrite 2023-05-29 00:41:54 +02:00
Lynne 8e9ceb1efb overlay_vulkan: port for the rewrite 2023-05-29 00:41:54 +02:00
Lynne 59f21c051d gblur_vulkan: port for the rewrite 2023-05-29 00:41:53 +02:00
Lynne d26cbae3e1 flip_vulkan: port for the rewrite 2023-05-29 00:41:53 +02:00
Lynne 125636608d chromaber_vulkan: port for the rewrite 2023-05-29 00:41:52 +02:00
Lynne 40bf1d525e blend_vulkan: port for the rewrite 2023-05-29 00:41:52 +02:00
Lynne f29850e98f avgblur_vulkan: port for the rewrite 2023-05-29 00:41:52 +02:00
Lynne 05ce6473ac lavfi: add lavfi-only Vulkan infrastructure 2023-05-29 00:41:51 +02:00
Lynne 51b7fe81be hwcontext_vulkan: enable additional device properties 2023-05-29 00:41:51 +02:00
Lynne 33fc919bb7 hwcontext_vulkan: remove duplicate code, port to use generic vulkan utils
The temporary AVFrame on staack enables us to use the common
dependency/dispatch code in prepare_frame().
The prepare_frame() function is used for both frame initialization
and frame import/export queue family transfer operations.
In the former case, no AVFrame exists yet, so, as this is purely
libavutil code, we create a temporary frame on stack. Otherwise,
we'd need to allocate multiple frames somewhere, one for each
possible command buffer dispatch.
2023-05-29 00:41:51 +02:00