virtualenv/FFmpeg
1
0
Fork 0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2024-11-26 19:01:44 +02:00
Code Issues Releases Activity
7bf08a5b2d
FFmpeg/libavcodec/dovi_rpu.h

160 lines
5.1 KiB
C
Raw Normal View History

lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
/*
* Dolby Vision RPU decoder
*
* Copyright (C) 2021 Jan Ekström
* Copyright (C) 2021 Niklas Haas
*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
#ifndef AVCODEC_DOVI_RPU_H
#define AVCODEC_DOVI_RPU_H
#include "libavutil/dovi_meta.h"
#include "libavutil/frame.h"
avcodec/dovi_rpuenc: add ff_dovi_configure() We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. Behavior can be controlled by a new DOVIContext.enable flag.
2024-04-12 13:24:22 +02:00
#include "avcodec.h"
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
#define DOVI_MAX_DM_ID 15
typedef struct DOVIContext {
void *logctx;
avcodec/dovi_rpuenc: add ff_dovi_configure() We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. Behavior can be controlled by a new DOVIContext.enable flag.
2024-04-12 13:24:22 +02:00
/**
* Enable tri-state. For encoding only. FF_DOVI_AUTOMATIC enables Dolby
* Vision only if avctx->decoded_side_data contains an AVDOVIMetadata.
*/
#define FF_DOVI_AUTOMATIC -1
int enable;
avcodec/dovi_rpu: store entire config record And make it public. For encoding, users may also be interested in the configured level and compatibility ID. So generalize the dv_profile field and just expose the whole configuration record. This makes the already rather reductive ff_dovi_update_cfg() function almost wholly redundant, since users can just directly assign DOVIContext.cfg.
2024-03-22 22:24:43 +02:00
/**
* Currently active dolby vision configuration, or {0} for none.
avcodec/dovi_rpuenc: add ff_dovi_configure() We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. Behavior can be controlled by a new DOVIContext.enable flag.
2024-04-12 13:24:22 +02:00
* Set by the user when decoding. Generated by ff_dovi_configure()
* when encoding.
avcodec/dovi_rpu: store entire config record And make it public. For encoding, users may also be interested in the configured level and compatibility ID. So generalize the dv_profile field and just expose the whole configuration record. This makes the already rather reductive ff_dovi_update_cfg() function almost wholly redundant, since users can just directly assign DOVIContext.cfg.
2024-03-22 22:24:43 +02:00
*
* Note: sizeof(cfg) is not part of the libavutil ABI, so users should
* never pass &cfg to any other library calls. This is included merely as
* a way to look up the values of fields known at compile time.
*/
AVDOVIDecoderConfigurationRecord cfg;
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
/**
avcodec/dovi_rpuenc: add ff_dovi_rpu_generate() This function takes a decoded AVDOVIMetadata struct and turns it back into a binary RPU. Verified using existing tools, and matches the bitstream in official reference files. I decided to just roll the EMDF and NAL encapsulation into this function because the end user will need to do it otherwise anyways.
2024-03-30 19:12:35 +02:00
* Currently active RPU data header, updates on every ff_dovi_rpu_parse()
* or ff_dovi_rpu_generate().
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
*/
AVDOVIRpuDataHeader header;
/**
* Currently active data mappings, or NULL. Points into memory owned by the
* corresponding rpu/vdr_ref, which becomes invalid on the next call to
avcodec/dovi_rpuenc: add ff_dovi_rpu_generate() This function takes a decoded AVDOVIMetadata struct and turns it back into a binary RPU. Verified using existing tools, and matches the bitstream in official reference files. I decided to just roll the EMDF and NAL encapsulation into this function because the end user will need to do it otherwise anyways.
2024-03-30 19:12:35 +02:00
* ff_dovi_rpu_parse() or ff_dovi_rpu_generate().
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
*/
const AVDOVIDataMapping *mapping;
const AVDOVIColorMetadata *color;
avcodec/dovi_rpu: add ext_blocks array to DOVIContext
2024-03-23 17:52:41 +02:00
/**
* Currently active extension blocks, updates on every ff_dovi_rpu_parse()
avcodec/dovi_rpuenc: add ff_dovi_rpu_generate() This function takes a decoded AVDOVIMetadata struct and turns it back into a binary RPU. Verified using existing tools, and matches the bitstream in official reference files. I decided to just roll the EMDF and NAL encapsulation into this function because the end user will need to do it otherwise anyways.
2024-03-30 19:12:35 +02:00
* or ff_dovi_rpu_generate().
avcodec/dovi_rpu: add ext_blocks array to DOVIContext
2024-03-23 17:52:41 +02:00
*/
AVDOVIDmData *ext_blocks;
int num_ext_blocks;
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
/**
* Private fields internal to dovi_rpu.c
*/
avcodec/dovi_rpu: fix dm_metadata_id handling Despite the suggestive size limits, this metadata ID has nothing to do with the VDR metadata ID used for the data mappings. Actually, the specification leaves them wholly unexplained, other than acknowleding their existence. Must be some secret dolby sauce. They're not even involved in DM metadata compression, which is handled using an entirely separate ID. That leaves us with a lack of anything sensible to do with these IDs. Since we unfortunately only expose one `dm_metadata_id` field to the user, just ensure that they match; which appears to always be the case in practice. If somebody ever hits this error, I would really much rather like to see the triggering file.
2024-06-09 15:04:45 +02:00
AVDOVIColorMetadata *dm; ///< RefStruct
avcodec/dovi_rpu: simplify vdr type Storing the color metadata alongside the data mapping is no longer needed, so we can simplify this array's type.
2024-06-09 16:43:29 +02:00
AVDOVIDataMapping *vdr[DOVI_MAX_DM_ID+1]; ///< RefStruct references
avcodec/dovi_rpu: strip container in separate step This ensures that `gb` in the following section is fully byte-aligned, points at the start of the actual RPU, and ends on the CRC terminator. This is important for both calculation of the CRC, as well as dovi extension block parsing (which aligns to byte boundaries in various places).
2024-03-23 16:59:44 +02:00
uint8_t *rpu_buf; ///< temporary buffer
unsigned rpu_buf_sz;
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
} DOVIContext;
avcodec/dovi_rpu: Use RefStruct API for Vdr data It avoids allocations and the corresponding error checks. Also avoids casts and indirections. Reviewed-by: Anton Khirnov <anton@khirnov.net> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-08-04 12:53:28 +02:00
void ff_dovi_ctx_replace(DOVIContext *s, const DOVIContext *s0);
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
/**
* Completely reset a DOVIContext, preserving only logctx.
*/
void ff_dovi_ctx_unref(DOVIContext *s);
/**
avcodec/dovi_rpu: store entire config record And make it public. For encoding, users may also be interested in the configured level and compatibility ID. So generalize the dv_profile field and just expose the whole configuration record. This makes the already rather reductive ff_dovi_update_cfg() function almost wholly redundant, since users can just directly assign DOVIContext.cfg.
2024-03-22 22:24:43 +02:00
* Partially reset the internal state. Resets per-frame state, but preserves
* the stream-wide configuration record.
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
*/
void ff_dovi_ctx_flush(DOVIContext *s);
/**
* Parse the contents of a Dovi RPU NAL and update the parsed values in the
* DOVIContext struct.
*
* Returns 0 or an error code.
avcodec/dovi_rpu: expose guess_profile(), clarify semantics To allow internally re-using it for both the encoder and decoder. This is based on HEVC only, H.264/AV1 use their own (hopefully correctly signalled) profiles (and in particular, the AV1 decoders implicitly default the correct profile in the absence of a configuration record).
2024-03-22 22:25:35 +02:00
*
* Note: `DOVIContext.cfg` should be initialized before calling into this
* function. If not done, the profile will be guessed according to HEVC
* semantics.
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
*/
avcodec/dovi_rpu: verify RPU data CRC32 The Dolby Vision RPU contains a CRC32 to validate the payload against. The implementation is CRC32/MPEG-2. The CRC is only verified with the AV_EF_CRCCHECK flag. Co-authored-by: quietvoid <tcChlisop0@gmail.com>
2024-03-23 17:29:12 +02:00
int ff_dovi_rpu_parse(DOVIContext *s, const uint8_t *rpu, size_t rpu_size,
int err_recognition);
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
/**
* Attach the decoded AVDOVIMetadata as side data to an AVFrame.
*/
int ff_dovi_attach_side_data(DOVIContext *s, AVFrame *frame);
avcodec/dovi_rpuenc: add ff_dovi_configure() We need to set up the configuration struct appropriately based on the codec type, colorspace metadata, and presence/absence of an EL (though, we currently don't support an EL). When present, we use the signalled RPU data header to help infer (and validate) the right values. Behavior can be controlled by a new DOVIContext.enable flag.
2024-04-12 13:24:22 +02:00
/**
* Configure the encoder for Dolby Vision encoding. Generates a configuration
* record in s->cfg, and attaches it to avctx->coded_side_data. Sets the correct
* profile and compatibility ID based on the tagged AVCodecContext colorspace
* metadata, and the correct level based on the resolution and tagged framerate.
*
* Returns 0 or a negative error code.
*/
int ff_dovi_configure(DOVIContext *s, AVCodecContext *avctx);
avcodec/dovi_rpu: split into dovi_rpu.c and dovi_rpudec.c To allow compiling the decoding objects without the encoding objects and vice versa. Common helpers that users of both APIs need are put into the shared dovi_rpu.c.
2024-04-12 13:11:10 +02:00
/***************************************************
* The following section is for internal use only. *
***************************************************/
enum {
RPU_COEFF_FIXED = 0,
RPU_COEFF_FLOAT = 1,
};
avcodec/dovi_rpuenc: add ff_dovi_rpu_generate() This function takes a decoded AVDOVIMetadata struct and turns it back into a binary RPU. Verified using existing tools, and matches the bitstream in official reference files. I decided to just roll the EMDF and NAL encapsulation into this function because the end user will need to do it otherwise anyways.
2024-03-30 19:12:35 +02:00
/**
* Synthesize a Dolby Vision RPU reflecting the current state. Note that this
* assumes all previous calls to `ff_dovi_rpu_generate` have been appropriately
* signalled, i.e. it will not re-send already transmitted redundant data.
*
* Mutates the internal state of DOVIContext to reflect the change.
* Returns 0 or a negative error code.
*
* This generates a fully formed RPU ready for inclusion in the bitstream,
* including the EMDF header (profile 10) or NAL encapsulation (otherwise).
*/
int ff_dovi_rpu_generate(DOVIContext *s, const AVDOVIMetadata *metadata,
uint8_t **out_rpu, int *out_size);
avcodec/dovi_rpu: expose guess_profile(), clarify semantics To allow internally re-using it for both the encoder and decoder. This is based on HEVC only, H.264/AV1 use their own (hopefully correctly signalled) profiles (and in particular, the AV1 decoders implicitly default the correct profile in the absence of a configuration record).
2024-03-22 22:25:35 +02:00
/**
* Internal helper function to guess the correct DV profile for HEVC.
*
* Returns the profile number or 0 if unknown.
*/
int ff_dovi_guess_profile_hevc(const AVDOVIRpuDataHeader *hdr);
avcodec/dovi_rpu: properly handle vdr_dm_metadata_present When this is 0, the metadata is explicitly inferred to stated default values from the spec, rather than inferred from the previous frame's values. Likewise, when encoding, instead of checking if the value changed since the last frame, we need to check if it differs from the default.
2024-06-09 14:58:12 +02:00
/* Default values for AVDOVIColorMetadata */
extern const AVDOVIColorMetadata ff_dovi_color_default;
lavc: Implement Dolby Vision RPU parsing Based on a mixture of guesswork, partial documentation in patents, and reverse engineering of real-world samples. Confirmed working for all the samples I've thrown at it. Contains some annoying machinery to persist these values in between frames, which is needed in theory even though I've never actually seen a sample that relies on it in practice. May or may not work. Since the distinction matters greatly for parsing the color matrix values, this includes a small helper function to guess the right profile from the RPU itself in case the user has forgotten to forward the dovi configuration record to the decoder. (Which in practice, only ffmpeg.c and ffplay do..) Notable omissions / deviations: - CRC32 verification. This is based on the MPEG2 CRC32 type, which is similar to IEEE CRC32 but apparently different in subtle enough ways that I could not get it to pass verification no matter what parameters I fed to av_crc. It's possible the code needs some changes. - Linear interpolation support. Nothing documents this (beyond its existence) and no samples use it, so impossible to implement. - All of the extension metadata blocks, but these contain values that seem largely congruent with ST2094, HDR10, or other existing forms of side data, so I will defer parsing/attaching them to a future commit. - The patent describes a mechanism for predicting coefficients from previous RPUs, but the bit for the flag whether to use the prediction deltas or signal entirely new coefficients does not seem to be present in actual RPUs, so we ignore this subsystem entirely. - In the patent's spec, the NLQ subsystem also loops over num_nlq_pivots, but even in the patent the number is hard-coded to one iteration rather than signalled. So we only store one set of coefs. Heavily influenced by https://github.com/quietvoid/dovi_tool Documentation drawn from US Patent 10,701,399 B2 and ETSI GS CCM 001 Signed-off-by: Niklas Haas <git@haasn.dev> Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
2022-01-03 02:27:25 +02:00
#endif /* AVCODEC_DOVI_RPU_H */
Reference in New Issue Copy Permalink