1
0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2024-12-07 11:13:41 +02:00
Commit Graph

58 Commits

Author SHA1 Message Date
Anton Khirnov
09ca2f1950 lavu/hwcontext: clarify behavior on av_hwframe_map() failure
Clear anything that av_hwframe_map() might have done to the destination
frame, but leave caller-provided fields unchanged.
2022-02-17 11:05:44 +01:00
Haihao Xiang
7c6f9b9d63 Revert "avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions"
This reverts commit a428949775.
There were objections on ML (see
https://ffmpeg.org/pipermail/ffmpeg-devel/2021-December/290530.html)

Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
2022-01-05 11:56:58 +08:00
Soft Works
a428949775 avutils/hwcontext: When deriving a hwdevice, search for existing device in both directions
The test /libavutil/tests/hwdevice checks that when deriving a device
from a source device and then deriving back to the type of the source
device, the result is matching the original source device, i.e. the
derivation mechanism doesn't create a new device in this case.

Previously, this test was usually passed, but only due to two different
kind of flaws:

1. The test covers only a single level of derivation (and back)

It derives device Y from device X and then Y back to the type of X and
checks whether the result matches X.

What it doesn't check for, are longer chains of derivation like:

CUDA1 > OpenCL2 > CUDA3 and then back to OpenCL4

In that case, the second derivation returns the first device (CUDA3 ==
CUDA1), but when deriving OpenCL4, hwcontext.c was creating a new
OpenCL4 context instead of returning OpenCL2, because there was no link
from CUDA1 to OpenCL2 (only backwards from OpenCL2 to CUDA1)

If the test would check for two levels of derivation, it would have
failed.

This patch fixes those (yet untested) cases by introducing forward
references (derived_device) in addition to the existing back references
(source_device).

2. hwcontext_qsv didn't properly set the source_device

In case of QSV, hwcontext_qsv creates a source context internally
(vaapi, dxva2 or d3d11va) without calling av_hwdevice_ctx_create_derived
and without setting source_device.

This way, the hwcontext test ran successful, but what practically
happened, was that - for example - deriving vaapi from qsv didn't return
the original underlying vaapi device and a new one was created instead:
Exactly what the test is intended to detect and prevent. It just
couldn't do so, because the original device was hidden (= not set as the
source_device of the QSV device).

This patch properly makes these setting and fixes all derivation
scenarios.

(at a later stage, /libavutil/tests/hwdevice should be extended to check
longer derivation chains as well)

Reviewed-by: Lynne <dev@lynne.ee>
Reviewed-by: Anton Khirnov <anton@khirnov.net>
Tested-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: softworkz <softworkz@hotmail.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
2022-01-05 11:05:06 +08:00
Andreas Rheinhardt
1ea3650823 Replace all occurences of av_mallocz_array() by av_calloc()
They do the same.

Reviewed-by: Paul B Mahol <onemda@gmail.com>
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2021-09-20 01:03:52 +02:00
Lynne
2e08b39444
hwcontext: add av_hwdevice_ctx_create_derived_opts
This allows for users who derive devices to set options for the
new device context they derive.
The main use case of this is to allow users to enable extensions
(such as surface drawing extensions) in Vulkan while deriving from
the device their frames are on. That way, users don't need to write
any initialization code themselves, since the Vulkan spec invalidates
mixing instances, physical devices and active devices.
Apart from Vulkan, other hwcontexts ignore the opts argument since they
don't support options at all (or in VAAPI and OpenCL's case, options are
currently only used for device selection, which device_derive overrides).
2020-05-23 19:07:26 +01:00
Anton Khirnov
f30a41a608 Stop hardcoding align=32 in av_frame_get_buffer() calls.
Use 0, which selects the alignment automatically.
2020-05-22 14:38:57 +02:00
Timo Rothenpieler
b06ae06e6a avutil/hwcontext: correctly set extended_data on hwframe_get_buffer 2020-03-28 17:59:29 +01:00
Lynne
a88449ffb2 lavu: add Vulkan hwcontext code
This commit adds the necessary code to initialize and use a Vulkan device
within the hwcontext libavutil framework.
Currently direct mapping to VAAPI and DRM frames is functional, and
transfers to CUDA and native frames are supported.

Lets hope the future Vulkan video decode extension fits well within this
framework.
2020-02-04 23:19:48 +00:00
Philip Langdale
d7210ce7f5 lavu/hwcontext: Add support for HW -> HW transfers
We are beginning to consider scenarios where a given HW Context
may be able to transfer frames to another HW Context without
passing via system memory - this would usually be when two
contexts represent different APIs on the same device (eg: Vulkan
and CUDA).

This is modelled as a transfer, as we have today, but where both
the src and the dst are hardware frames with hw contexts. We need
to be careful to ensure the contexts are compatible - particularly,
we cannot do transfers where one of the frames has been mapped via
a derived frames context - we can only do transfers for frames that
were directly allocated by the specified context.

Additionally, as we have two hardware contexts, the transfer function
could be implemented by either (or indeed both). To handle this
uncertainty, we explicitly look for ENOSYS as an indicator to try
the transfer in the other direction before giving up.
2020-02-04 23:19:48 +00:00
Rostislav Pehlivanov
7062e4dbc8 hwcontext_internal: add ff_hwframe_map_replace
Used to fix unmapping when no direct interop exists between APIs.

Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2018-06-21 16:34:06 +01:00
Mark Thompson
136260eb88 hwcontext: Do not call device_init again when deriving an existing device
The change in 309d660775 to call device_init
when doing derivation missed this case - we should only call it if we
actually made a new device.
2018-05-15 22:16:03 +01:00
James Almer
0a320f7e7a Merge commit '2eb396b175e55e515aa6a13c5b1789a2a18d3935'
* commit '2eb396b175e55e515aa6a13c5b1789a2a18d3935':
  hwcontext: Fix memory leak on derived frame allocation failure

Merged-by: James Almer <jamrial@gmail.com>
2018-02-11 22:50:57 -03:00
Mark Thompson
2eb396b175 hwcontext: Fix memory leak on derived frame allocation failure 2018-02-04 13:47:25 +00:00
Aman Gupta
e4d9f05ca7 lavu/hwcontext: add AV_HWDEVICE_TYPE_MEDIACODEC
Signed-off-by: Matthieu Bouron <matthieu.bouron@gmail.com>
2017-12-16 00:52:00 +01:00
Mark Thompson
0f93cef2d6 hwcontext: Perform usual uninitialisation on derived frames contexts 2017-11-22 23:20:32 +00:00
Mark Thompson
b25d8ef0a7 lavu: OpenCL hwcontext implementation 2017-11-22 23:02:29 +00:00
James Almer
4e9dc52a97 Merge commit '1bd986ed4b0e95ded368a8eeb5c044853c090f9b'
* commit '1bd986ed4b0e95ded368a8eeb5c044853c090f9b':
  hwcontext: Move NONE to the be the first member of AVHWDeviceType

Merged-by: James Almer <jamrial@gmail.com>
2017-10-24 19:14:22 -03:00
Mark Thompson
309d660775 hwcontext: Perform usual initialisation on derived device contexts
The initialisation should be common.  For libmfx, it was previously
happening in the derivation function and this moves it out.  For VAAPI,
it fixes some failures when deriving from a DRM device because this
initialisation did not run.
2017-10-09 00:11:53 +01:00
James Almer
318778de9e Merge commit 'fd9212f2edfe9b107c3c08ba2df5fd2cba5ab9e3'
* commit 'fd9212f2edfe9b107c3c08ba2df5fd2cba5ab9e3':
  Mark some arrays that never change as const.

Merged-by: James Almer <jamrial@gmail.com>
2017-09-26 16:02:40 -03:00
Mark Thompson
9cb23cd9fe lavu: Add DRM hwcontext 2017-09-13 22:03:27 +01:00
wm4
3303511f33 lavu: add new D3D11 pixfmt and hwcontext
To be used with the new d3d11 hwaccel decode API.

With the new hwaccel API, we don't want surfaces to depend on the
decoder (other than the required dimension and format). The old D3D11VA
pixfmt uses ID3D11VideoDecoderOutputView pointers, which include the
decoder configuration, and thus is incompatible with the new hwaccel
API. This patch introduces AV_PIX_FMT_D3D11, which uses ID3D11Texture2D
and an index. It's simpler and compatible with the new hwaccel API.

The introduced hwcontext supports only the new pixfmt.

Frame upload code untested.

Significantly based on work by Steve Lhomme <robux4@gmail.com>, but with
heavy changes/rewrites.

Merges Libav commit fff90422d1.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
2017-06-27 18:05:02 +02:00
Mark Thompson
4993a68f0f hwcontext: Mark local table static const 2017-06-15 00:02:06 +01:00
Mark Thompson
d59c6a3aeb hwcontext: Improve allocation in derived contexts
Use the flags argument of av_hwframe_ctx_create_derived() to pass the
mapping flags which will be used on allocation.  Also, set the format
and hardware context on the allocated frame automatically - the user
should not be required to do this themselves.

(cherry picked from commit c5714b51aa)
2017-06-14 22:27:32 +01:00
Mark Thompson
ec3dbeae81 hwcontext: Add frame context mapping for nontrivial contexts
Some frames contexts are not usable without additional format-specific
state in hwctx.  This change adds new functions frames_derive_from and
frames_derive_to to initialise this state appropriately when deriving
a frames context which will require it to be set.

(cherry picked from commit 27978155bc)
2017-06-14 22:27:07 +01:00
Mark Thompson
0b1794a43e hwcontext: Make it easier to work with device types
Adds functions to convert to/from strings and a function to iterate
over all supported device types.  Also adds a new invalid type
AV_HWDEVICE_TYPE_NONE, which acts as a sentinel value.

(cherry picked from commit b7487f4f3c)
2017-06-14 22:26:29 +01:00
Mark Thompson
b22172f6f3 hwcontext: Add device derivation
Creates a new device context from another of a different type which
refers to the same underlying hardware.

(cherry picked from commit b266ad56fe)
2017-06-14 22:25:57 +01:00
wm4
fff90422d1 lavu: add new D3D11 pixfmt and hwcontext
To be used with the new d3d11 hwaccel decode API.

With the new hwaccel API, we don't want surfaces to depend on the
decoder (other than the required dimension and format). The old D3D11VA
pixfmt uses ID3D11VideoDecoderOutputView pointers, which include the
decoder configuration, and thus is incompatible with the new hwaccel
API. This patch introduces AV_PIX_FMT_D3D11, which uses ID3D11Texture2D
and an index. It's simpler and compatible with the new hwaccel API.

The introduced hwcontext supports only the new pixfmt.

Frame upload code untested.

Significantly based on work by Steve Lhomme <robux4@gmail.com>, but with
heavy changes/rewrites.

Signed-off-by: Diego Biurrun <diego@biurrun.de>
2017-06-08 20:55:36 +02:00
wm4
532b23f079 videotoolbox: add hwcontext support
This adds tons of code for no other benefit than making VideoToolbox
support conform with the new hwaccel API (using hw_device_ctx and
hw_frames_ctx).

Since VideoToolbox decoding does not actually require the user to
allocate frames, the new code does mostly nothing.

One benefit is that ffmpeg_videotoolbox.c can be dropped once generic
hwaccel support for ffmpeg.c is merged from Libav.

Does not consider VDA or VideoToolbox encoding.

Fun fact: the frame transfer functions are copied from vaapi, as the
mapping makes copying generic boilerplate. Mapping itself is not
exported by the VT code, because I don't know how to test.
2017-05-15 11:30:36 +02:00
Mark Thompson
c5714b51aa hwcontext: Improve allocation in derived contexts
Use the flags argument of av_hwframe_ctx_create_derived() to pass the
mapping flags which will be used on allocation.  Also, set the format
and hardware context on the allocated frame automatically - the user
should not be required to do this themselves.
2017-04-30 17:33:18 +01:00
Mark Thompson
27978155bc hwcontext: Add frame context mapping for nontrivial contexts
Some frames contexts are not usable without additional format-specific
state in hwctx.  This change adds new functions frames_derive_from and
frames_derive_to to initialise this state appropriately when deriving
a frames context which will require it to be set.
2017-04-30 16:13:56 +01:00
Clément Bœsch
4cda23f1f1 Merge commit 'd06aa24ba583ad08025da9e1b29afcd8218ff9b0'
* commit 'd06aa24ba583ad08025da9e1b29afcd8218ff9b0':
  hwcontext: Hardware frame mapping

Merged-by: Clément Bœsch <cboesch@gopro.com>
2017-03-30 10:36:37 +02:00
Mark Thompson
1bd986ed4b hwcontext: Move NONE to the be the first member of AVHWDeviceType
Also use that to fix a warning in av_hwdevice_get_type_name().
2017-03-27 21:54:06 +01:00
Mark Thompson
b7487f4f3c hwcontext: Make it easier to work with device types
Adds functions to convert to/from strings and a function to iterate
over all supported device types.  Also adds a new invalid type
AV_HWDEVICE_TYPE_NONE, which acts as a sentinel value.
2017-03-20 23:15:43 +00:00
Mark Thompson
b266ad56fe hwcontext: Add device derivation
Creates a new device context from another of a different type which
refers to the same underlying hardware.
2017-03-20 23:15:43 +00:00
Anton Khirnov
fd9212f2ed Mark some arrays that never change as const. 2017-02-01 10:42:59 +01:00
Hendrik Leppkes
7071924cf2 Merge commit 'fdfe01365d579189d9a55b3741dba2ac46eb1df8'
* commit 'fdfe01365d579189d9a55b3741dba2ac46eb1df8':
  hwcontext: allocate the destination frame for the pool size

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2016-11-13 22:35:49 +01:00
Mark Thompson
d06aa24ba5 hwcontext: Hardware frame mapping
Adds the new av_hwframe_map() function, which allows mapping between
hardware frames and normal memory, along with internal support for
implementing it.

Also adds av_hwframe_ctx_create_derived(), for creating a hardware
frames context associated with one device using frames mapped from
another by some hardware-specific means.
2016-11-03 23:49:01 +00:00
James Almer
13dd5edb88 avutil/hwcontext: use CONFIG_QSV instead of CONFIG_LIBMFX for qsv
See "[FFmpeg-devel] [PATCH] hwcontext: add a QSV implementation"

Suggested-by: nablet developer <sdk@nablet.com>
Signed-off-by: James Almer <jamrial@gmail.com>
2016-09-28 15:13:10 -03:00
James Almer
eba0414768 Merge commit '59e7361cc791e5103be1712dc59a2055f118d0da'
* commit '59e7361cc791e5103be1712dc59a2055f118d0da':
  hwcontext: add a QSV implementation

Conflicts:
    doc/APIchanges
    libavutil/version.h

Merged-by: James Almer <jamrial@gmail.com>
2016-09-28 13:30:27 -03:00
Anton Khirnov
fdfe01365d hwcontext: allocate the destination frame for the pool size
The source frame may be cropped, so that its dimensions are smaller than
the pool dimensions. The transfer_data API requires the allocated size
of the destination frame to be the same as the pool size.
2016-06-28 08:31:28 +02:00
Hendrik Leppkes
481f320aa3 Merge commit '1c9e8616c535ef496e7ee8a5cbc5e9e972a6977d'
* commit '1c9e8616c535ef496e7ee8a5cbc5e9e972a6977d':
  hwcontext: add a function for opening devices

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2016-06-26 15:23:24 +02:00
Hendrik Leppkes
3689efea3d Merge commit 'c46db38cde8e8fd8ecb1c6602f10ec0e002f29a8'
* commit 'c46db38cde8e8fd8ecb1c6602f10ec0e002f29a8':
  hwcontext: add a dxva2 implementation

Merged-by: Hendrik Leppkes <h.leppkes@gmail.com>
2016-06-22 22:38:49 +02:00
Anton Khirnov
59e7361cc7 hwcontext: add a QSV implementation 2016-06-21 19:53:37 +02:00
Anton Khirnov
1c9e8616c5 hwcontext: add a function for opening devices 2016-05-26 15:40:32 +02:00
Anton Khirnov
c46db38cde hwcontext: add a dxva2 implementation 2016-05-17 09:11:25 +02:00
Derek Buitenhuis
c71c3b9ff4 Merge commit 'a0f469da744db83db32f3fe13186ee4aa2bc7dc5'
* commit 'a0f469da744db83db32f3fe13186ee4aa2bc7dc5':
  hwcontext: initialize sw_format in av_hwframe_ctx_alloc()

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2016-05-11 19:46:17 +01:00
Anton Khirnov
a0f469da74 hwcontext: initialize sw_format in av_hwframe_ctx_alloc() 2016-04-15 10:15:54 +02:00
Derek Buitenhuis
28abb216cb Merge commit '551c6775abb5e0ad34c26d7e23bc6fbbe8ccc9d4'
* commit '551c6775abb5e0ad34c26d7e23bc6fbbe8ccc9d4':
  lavu: VAAPI hwcontext implementation

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2016-04-14 13:49:55 +01:00
Derek Buitenhuis
afccfaf26a Merge commit 'b1f01e85a92d401a9b29c79f23db36b7685e8c09'
* commit 'b1f01e85a92d401a9b29c79f23db36b7685e8c09':
  lavu: add a way to query hwcontext frame constraints

Merged-by: Derek Buitenhuis <derek.buitenhuis@gmail.com>
2016-04-14 13:33:37 +01:00
Mark Thompson
551c6775ab lavu: VAAPI hwcontext implementation
Signed-off-by: Anton Khirnov <anton@khirnov.net>
2016-03-19 15:40:22 +01:00