User may set color range / matrix coefficient set / primaries / transfer
characteristics for output.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
libavutil/hwcontext_qsv.c: In function ‘qsv_map_to’:
libavutil/hwcontext_qsv.c:1905:47: warning: cast from pointer to integer
of different size [-Wpointer-to-int-cast]
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
glcontext was added under CONFIG_SDL2
libavdevice/opengl_enc.c: In function ‘opengl_draw’:
libavdevice/opengl_enc.c:1204:15: error: ‘OpenGLContext’ has no member
named ‘glcontext’
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The HDR metadata should be removed after HDR to SDR conversion,
otherwise the output frame still has HDR side data.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
We will postpone the vpp session initialization to when input and output
frames are ready, this copy of the sequence parameters will be used to
initialize vpp session.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Intel MediaSDK and oneVPL expect continuous allocation for data[i],
however there are mandatory padding bytes between data[i] and data[i+1].
when calling av_frame_get_buffer. This patch removes all extra padding
bytes.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The data copy is unnecessary for packed formats when frame width and
height are aligned
For example:
$ ffmpeg -f lavfi -i testsrc=size=1920x1088 -vf "format=yuyv422" -c:v hevc_qsv -f null -
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The mfx implementation based on D3D11 is expected for an internal
session on Windows, however sometimes this implemntation is not
supported [1]. A fallback to the default mfx implementation is added in
this patch.
[1] https://github.com/intel/cartwheel-ffmpeg/issues/246
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Otherwise main and overlay videos share the same input region. Note NULL
pointer imples the whole overlay video will be processed.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Otherwise the output format is not changed when output is in system
memory. For example, the output format is still p010le in the following
case:
$ ffmpeg -qsv_device /dev/dri/renderD128 -f lavfi -i testsrc -vf
"format=p010le,vpp_qsv=extra_hw_frames=8:format=nv12" -f null -
...
Output #0, null, to 'pipe:':
Metadata:
encoder : Lavf60.4.100
Stream #0:0: Video: wrapped_avframe, p010le(tv, progressive), 320x240
[SAR 1:1 DAR 4:3], q=2-31, 200 kb/s, 25 fps, 25 tbn
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
It only works on Linux
$ ffmpeg -loglevel verbose -init_hw_device qsv=intel -f lavfi -i \
yuvtestsrc -vf "format=uyvy422,vpp_qsv=format=nv12" -f null -
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
QSVDeintContext and VPPContext have the same base context, and all
features in deinterlace_qsv are implemented in vpp_qsv filter, so
deinterlace_qsv can be taken as a special case of vpp_qsv filter, and we
may use VPPContext with a different option array, preinit callback and
support pixel formats to implement deinterlace_qsv, then remove
QSVDeintContext.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Like what we did for scale_qsv filter, we use QSVVPPContext as a base
context to manage MFX session for deinterlace_qsv filter.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This is used to control the output at frame rate or field rate when
deinterlace is expected and framerate is not specified.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
QSVScaleContext and VPPContext have the same base context, and all
features in scale_qsv are implemented in vpp_qsv filter, so scale_qsv
can be taken as a special case of vpp_qsv filter, and we may use
VPPContext with a different option array, preinit callback and supported
pixel formats to implement scale_qsv then remove QSVScaleContext
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Use QSVVPPContext as a base context of QSVScaleContext, hence we may
re-use functions defined for QSVVPPContext to manage MFX session for
scale_qsv filter.
In addition, system memory has been taken into account in
QSVVVPPContext, we may add support for non-QSV pixel formats in the
future.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The same members between QSVVPPContext and VPPContext are removed from
VPPContext, and async_depth is moved from QSVVPPParam to QSVVPPContext
so that all QSV filters using QSVVPPContext may support async depth.
In addition, we may use QSVVPPContext as base context in other QSV
filters in the future so that we may re-use functions defined in
qsvvpp.c for other QSV filters.
This commit shouldn't change the functionality of vpp_qsv / overlay_qsv.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This is in preparation for reusing the code for other QSV filters. E.g.
deinterlacing_qsv may have an option array without format option
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
QSV filters may set this flag in preinit callback to turn on / off pass
through mode
This is in preparation for reusing the code for other QSV filters. E.g.
scale_qsv filter doesn't support pass through mode.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Set the expected default value for options in this callback, hence we
have the right values even if these options are not included in the
option arrray.
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Special values are:
0 = original width/height
-1 = keep original aspect
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
This patch provides default value if the expression is NULL.
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Also fix the naming style in enum var_name.
This is in preparation for reusing the code for other QSV filters.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
PI, PHI and E are defined in libavutil/eval.c, and user may use these
constants for scale_qsv filter, so we needn't re-define these variables
in vf_scale_qsv.c
Reviewed-by: Gyan Doshi <ffmpeg@gyani.pro>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
VPP in the SDK requires the frame rate to be set to a valid value,
otherwise init will fail, so always set a default framerate when the
input link doesn't have a valid framerate.
Reviewed-by: Soft Works <softworkz@hotmail.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The SDK will return error if feeding h264_qsv encoder p010 input.
$ ffmpeg -f lavfi -i testsrc -vf "format=p010" -c:v h264_qsv -f null -
[...]
[h264_qsv @ 0x55584888b080] Current pixel format is unsupported
[h264_qsv @ 0x55584888b080] some encoding parameters are not supported
by the QSV runtime. Please double check the input parameters.
Error initializing output stream 0:0 -- Error while opening encoder for
output stream #0:0 - maybe incorrect parameters such as bit_rate, rate,
width or height
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The SDK may provides HDR metadata for HDR streams via mfxExtBuffer
attached on output mfxFrameSurface1
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The next commit and other commits in future will use more ExtParam
buffers.
And combine 2 free functions into single one
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
qsvenc makes a copy when the input in system memory is not padded as the
SDK requires, however the padding area is not filled with right data
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
HEVC spec has CRA frame which allows random access with open GOP, hence
it can achieve higher compression efficiency.
Removing the entry was suggested by Andreas
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
XV30 is used for 10bit 4:4:4 content in FFmpeg VAAPI, so XV30 should be
used for 10bit 4:4:4 content in FFmpeg QSV too because QSV is based on
VAAPI on Linux. However the SDK only declares support for Y410 but does
nothing with the alpha in Y410, so this commit fudged a mapping between
AV_PIX_FMT_XV30 and MFX_FOURCC_Y410.
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
We can't get Shift from bit depth for some formats in the SDK. For
example, bit depth is 10, however Shift is 0 for Y410 (XV30 in FFmpeg).
In order to support these formats in the next commits, this patch
specified Shift for each format
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The current pbc might be small for an obu frame, so a new pbc is
required then parse this obu frame again. Because
CodedBitstreamAV1Context has already been updated for this obu frame, we
need to restore CodedBitstreamAV1Context, otherwise
CodedBitstreamAV1Context doesn't match this obu frame when parsing obu
frame again, e.g. CodedBitstreamAV1Context.order_hint.
$ ffmpeg -i input.ivf -c:v copy -f null -
[...]
[av1_frame_merge @ 0x558bc3d6f880] ref_order_hint[i] does not match
inferred value: 20, but should be 22.
[av1_frame_merge @ 0x558bc3d6f880] Failed to write unit 1 (type 6).
[av1_frame_merge @ 0x558bc3d6f880] Failed to write packet.
[obu @ 0x558bc3d6e040] av1_frame_merge filter failed to send output
packet
Reviewed-by: James Almer <jamrial@gmail.com>
Reviewed-by: Wenbin Chen <wenbin.chen@intel.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
AV_PIX_FMT_VUYX is used in FFmpeg and MFX_FOURCC_AYUV is used in the SDK
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
AV_PIX_FMT_VUYX is used for 8bit 4:4:4 content in FFmpeg VAAPI, so
AV_PIX_FMT_VUYX should be used for 8bit 4:4:4 content in FFmpeg QSV too
because QSV is based on VAAPI on Linux. However the SDK only declares
support for AYUV and does nothing with the alpha, so this commit fudged
a mapping between AV_PIX_FMT_VUYX and MFX_FOURCC_AYUV.
Reviewed-by: Philip Langdale <philipl@overt.org>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Currently AVBR is disabled and VBR is the default method if maxrate is
not specified on Linux, but AVBR is the default one if maxrate is not
specified on Windows. In order to make user experience better accross
Linux and Windows, use VBR by default on Windows if maxrate is not
specified. User need to set both avbr_accuracy and avbr_convergence to
non-zero explicitly and not to specify maxrate if AVBR is expected.
In addition, AVBR works for H264 and HEVC only in the SDK.
$ ffmpeg.exe -v verbose -f lavfi -i yuvtestsrc -vf "format=nv12" -c:v
vp9_qsv -f null -
This allows user to build FFmpeg against Intel oneVPL. oneVPL 2.6
is the required minimum version when building Intel oneVPL code.
It will fail to run configure script if both libmfx and libvpl are
enabled.
It is recommended to use oneVPL for new work, even for currently available
hardwares [1]
Note the preferred child device type is d3d11va for libvpl on Windows.
The commands below will use d3d11va if d3d11va is available on Windows.
$ ffmpeg -hwaccel qsv -c:v h264_qsv ...
$ ffmpeg -qsv_device 0 -hwaccel qsv -c:v h264_qsv ...
$ ffmpeg -init_hw_device qsv=qsv:hw_any -hwaccel qsv -c:v h264_qsv ...
$ ffmpeg -init_hw_device qsv=qsv:hw_any,child_device=0 -hwaccel qsv -c:v h264_qsv ...
User may use child_device_type option to specify child device type to
dxva2 or derive a qsv device from a dxva2 device
$ ffmpeg -init_hw_device qsv=qsv:hw_any,child_device=0,child_device_type=dxva2 -hwaccel qsv -c:v h264_qsv ...
$ ffmpeg -init_hw_device dxva2=d3d9:0 -init_hw_device qsv=qsv@d3d9 -hwaccel qsv -c:v h264_qsv ...
[1] https://www.intel.com/content/www/us/en/develop/documentation/upgrading-from-msdk-to-onevpl/top.html
If qsv hwdevice is available, use the mfxLoader handle in qsv hwdevice
to create mfx session. Otherwise create mfx session with a new mfxLoader
handle.
This is in preparation for oneVPL support
The following Cflags has been added to libmfx.pc, so mfx/ prefix is no
longer needed when including mfx headers in FFmpeg.
Cflags: -I${includedir} -I${includedir}/mfx
Some old versions of libmfx have the following Cflags in libmfx.pc
Cflags: -I${includedir}
We may add -I${includedir}/mfx to CFLAGS when running 'configure
--enable-libmfx' for old versions of libmfx, if so, mfx headers without
mfx/ prefix can be included too.
If libmfx comes without pkg-config support, we may do a small change to
the settings of the environment(e.g. set -I/opt/intel/mediasdk/include/mfx
instead of -I/opt/intel/mediasdk/include to CFLAGS), then the build can
find the mfx headers without mfx/ prefix
After applying this change, we won't need to change #include for mfx
headers when mfx headers are installed under a new directory.
This is in preparation for oneVPL support (mfx headers in oneVPL are
installed under vpl directory)
The data structures for VP9 in mfxvp9.h is wrapped by
MFX_VERSION_NEXT, which means those data structures have never been used
in a public release. Actually MFX_CODEC_VP9 and other VP9 stuffs are
added in mfxstructures.h. In addition, mfxdefs.h is included in
mfxvp9.h, so we may use the check in this patch for MFX_CODEC_VP9
This is in preparation for oneVPL support because mfxvp9.h is removed
from oneVPL [1]
[1]: https://github.com/oneapi-src/oneVPL
Intel's oneVPL is a successor to MediaSDK, but removed some obsolete
features of MediaSDK[1], some early versions of oneVPL still use libmfx
as library name[2]. However some of obsolete features, including OPAQUE
memory, multi-frame encode, user plugins and LA_EXT rate control mode
etc, have been enabled in QSV, so user can not use --enable-libmfx to
enable QSV if using an early version of oneVPL SDK. In order to ensure
user builds FFmpeg against a right version of libmfx, this patch added a
check for version < 2.0 and warning message about the used obsolete
features.
[1] https://spec.oneapi.io/versions/latest/elements/oneVPL/source/VPL_intel_media_sdk.html
[2] https://github.com/oneapi-src/oneVPL
Usually a HW decoder is expected when user specifies a HW acceleration
method via -hwaccel option, however the current implementation doesn't
take HW acceleration method into account, it is possible to select a SW
decoder.
For example:
$ ffmpeg -hwaccel vaapi -i av1.mp4 -f null -
$ ffmpeg -hwaccel nvdec -i av1.mp4 -f null -
$ ffmpeg -hwaccel vdpau -i av1.mp4 -f null -
[...]
Stream #0:0 -> #0:0 (av1 (libdav1d) -> wrapped_avframe (native))
libdav1d is selected in this case even if vaapi, nvdec or vdpau is
specified.
After applying this patch, the native av1 decoder (with vaapi, nvdec or
vdpau support) is selected for decoding(libdav1d is still used for
probing format).
$ ffmpeg -hwaccel vaapi -i av1.mp4 -f null -
$ ffmpeg -hwaccel nvdec -i av1.mp4 -f null -
$ ffmpeg -hwaccel vdpau -i av1.mp4 -f null -
[...]
Stream #0:0 -> #0:0 (av1 (native) -> wrapped_avframe (native))
Tested-by: Mario Roy <marioeroy@gmail.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
After applying this patch, the desired HW acceleration method is known
before selecting decoder, so we may take HW acceleration method into
account when selecting decoder for input stream in the next commit
There should be no functional changes in this patch
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Signed-off-by: Anton Khirnov <anton@khirnov.net>
When AV_CODEC_EXPORT_DATA_FILM_GRAIN is present, AV1 decoder should
disable film grain application and export the corresponding side data
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The SDK may insert picture timing SEI for hevc and the code to set mfx
parameter has been added in qsvenc, however the corresponding option is
missing in the hevc option array
Reviewed-by: Limin Wang <lance.lmwang@gmail.com>
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The string for AV_OPT_TYPE_STRING AVOption gets freed by av_opt_free()
when closing the AVCodecContext
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
The SDK checks Data.V when using system memory for VP9 encoding. This
fixed the error below:
$ ffmpeg -qsv_device /dev/dri/renderD129 -f lavfi -i yuvtestsrc -c:v
vp9_qsv -f null -
[vp9_qsv @ 0x55b8387cbe90] Error during encoding: NULL pointer (-2)
Video encoding failed
Signed-off-by: Haihao Xiang <haihao.xiang@intel.com>
Haihao Xiang
