Instead of SWS_UOP_PERMUTE/SWS_UOP_COPY.
No real measurable difference in performance (it just eliminates a few
practically free register renames), but definitely simpler.
Signed-off-by: Niklas Haas <git@haasn.dev>
This decomposes a swizzle mask into a series of optimal register-register
moves, using at most two temporary scratch registers.
This is a better match for ASM-style backends than the existing PERMUTE/COPY
uops that are designed for the needs of the C backend (or other backends which
either apply the swizzle mask directly or permute pointers).
I originally had logic equivalent to this written in NASM macros, but it was
just such a complicated mess that I think it's better to rewrite it in C and
have the resulting metadata be an explicit part of the uop definition.
This commit only adds the uop, I'll update the x86 implementation in the
next step.
Co-authored-by: Ramiro Polla <ramiro.polla@gmail.com>
Signed-off-by: Niklas Haas <git@haasn.dev>
The old x86 backend was the only backend that actually mutated the ops list.
With this gone, we can constify this parameter.
Signed-off-by: Niklas Haas <git@haasn.dev>
This is no longer needed now that both C and x86 are ported to uops.
The other ff_sws_setup_*() functions are still used by the aarch64 backend.
Signed-off-by: Niklas Haas <git@haasn.dev>
This is a ground-up refactor of the existing x86 ops code, using the new
uops macros to auto-generate every single kernel instance without guesswork.
While I was at it, I also cleaned up the file a bit and made sure we have only
a single, consistent way of writing/defining the kernels. This also gets rid
of some of the old boilerplate like decl_pattern.
Most kernels are trivial ports, but a few deserve attention or note:
- SWS_UOP_LINEAR is now generated more efficiently, thanks to the distinction
between 0/1/arbitrary components. I also rewrote the code to keep track of
whether the output was initialized yet or not, which lets us skip the
initial `xorps` and `addps` for the first component.
- SWS_UOP_PERMUTE is generated automatically by using some NASM logic to
detect permutation cycles and emit the minimal sequence of `mova`
instructions. SWS_UOP_COPY, on the other hand, is implemented naively. I
originally had a more complex implementation that could handle both, but
I decided it really isn't worth the complication just to save 2-3 cycles.
- SWS_UOP_SCALE now has a native 8-bit implementation, which is faster than
falling back to C code.
- SWS_UOP_SWAP_BYTES is no longer compiled as a type-agnostic pshufb, instead
we hard-code the shuffle mask
- SWS_UOP_DITHER is now much simpler and avoids branching etc. entirely
Signed-off-by: Niklas Haas <git@haasn.dev>
Rather than hard-coding a separate set of NASM macros, or generating them
with a separate function, we can just leverage the C preprocessor to generate
a NASM source file *from* the existing ops macros.
This is maybe a bit unorthodox, but it avoids unnecessary overhead from
re-generating the macros twice, avoids manual updating of the NASM macros,
and generally does not come with any real downside except being a bit ugly.
The main source of ugliness is the fact that the C preprocessor expands
everything into a single line, whereas NASM expects separate statements to
be on separate lines. Very fortunately, we can work around this by writing a
another NASM macro to take its arguments and dump them onto multiple lines.
It may seem premature, but I went ahead and defined all the macros, since
it was easy enough to do.
I added the %include in this commit to trigger build errors that occur only
as a result of introducing this file in the same commit that introduces it.
Signed-off-by: Niklas Haas <git@haasn.dev>
The ops.h infrastructure currently hard-codes this as SWS_PIXEL_F32,
but I want to at least properly parametrize this in case we ever
decide to revisit this decision in the future. In particular, it
may become relevant for trivial kernels or kernels whose intermediates
are bounded, exact integers (which could possibly be output directly
as e.g. U16 or U32).
The FATE change is just because the filter op names gained a suffix.
Sponsored-by: Sovereign Tech Fund
Signed-off-by: Niklas Haas <git@haasn.dev>
Analog of SWS_UOP_READ_PLANAR_FV for FMA-enabled backends.
The logic for determining when we can safely use FMA is maybe a bit
obtuse, given that a `return type == SWS_PIXEL_U8` would have just done
the trick as well, but better to be safe than sorry, if we ever decide to
tune this constant in the future.
Signed-off-by: Niklas Haas <git@haasn.dev>
This is like SWS_UOP_LINEAR but parametrized by which matrix entries can use
FMA instead of bitexact IEEE mul/add instructions.
I decided to make these a separate uop to avoid bogging down the reference
backend with arch-specific details like FMA. However, I think FMA ops are quite
common/universal so I pre-emptively split it into its own separate flag rather
than defining something like SWS_UOP_FLAG_X86.
Signed-off-by: Niklas Haas <git@haasn.dev>
And SWS_BITEXACT|SWS_ACCURATE_RND, for completeness. This roughly doubles
the runtime of the uops macros generation. Let's hope it doesn't explode
further.
Signed-off-by: Niklas Haas <git@haasn.dev>
This list is currently empty but will be expanded by the following commit.
I briefly tested whether it would be worth avoiding the free/realloc on
the uops array, but found the performance difference to be negligible.
Signed-off-by: Niklas Haas <git@haasn.dev>
This ensures 100% coverage of all uop primitives by generating the set of
tests exactly from the list of seen primitives, using the uops macros.
There are some annoying quirks still because of the fact that we have to
essentially "untranslate" the UOPs back to SwsOps that result back in the
intended uop after the translation, but overall it's not too bad and still
much better than the status quo of hand-rolling the list of test cases.
Signed-off-by: Niklas Haas <git@haasn.dev>
Removes the 1x1 dither fast path, mirroring the previous commit.
This is not really needed nor useful but it will make the transition to
the uops architecture slightly easier, as 1x1 dither gets reinterpreted
as SWS_UOP_ADD there.
Signed-off-by: Niklas Haas <git@haasn.dev>
This is broken because it fails to check dither.y_offset[] to determine if
dithering for a channel is requested or not.
This is unnecessary because the generic dither code already jumps over unused
components, which is cheap enough not to worry about this special case for
now.
This code will, in any case, soon be replaced by a uops_macros.h-derived
approach. This commit is only needed as a stopgap to make checkasm continue
working after the sws_uops refactor.
Signed-off-by: Niklas Haas <git@haasn.dev>
As well as the packed shuffle solver. These don't really interact with
the rest of the code in ops_int.asm, which is, by name at least, intended for
integer op kernels.
More importantly, these functions will be shared with the uops rewrite.
Signed-off-by: Niklas Haas <git@haasn.dev>
Instead of choosing by hand which kernels to implement, this rewrite focuses
on leveraging the power of uops_macros.h to auto-generate all needed kernels.
This not only simplifies maintenance, but also improves performance.
I have decided to develop the replacement backend as a separate file, under
a separate prefix, for the explicit purpose of being able to verify the
correctness of the rewrite using the current backend as a checkasm reference.
The code for the kernels themselves has been largely copied from the old
C backend, modified slightly to conform to the uop template style. This does
result in some code duplication, but a following commit will clean it up.
I nonetheless want to preserve this commit for bisection purposes, to ensure
we have one commit that contains both backends side-by-side.
Overall speedup=1.182x faster, min=0.197x max=3.450x
The big slowdowns are flukes caused by tiny deviations in the runtime of
a noop memcpy conversion.
As a nice side benefit, the compiled binary is now also ~10% smaller, and
the code ~50% smaller.
Signed-off-by: Niklas Haas <git@haasn.dev>
This will eventually replace the existing op_match() and
ff_sws_op_compile_tables(), but I've decided to introduce it separately first
so that I can incrementally update the backends to use the new API, at the
cost of some temporary code duplication.
Signed-off-by: Niklas Haas <git@haasn.dev>
This follows the same approach as is used currently by ops_entries_aarch64,
except I decided to have the generation logic live directly in uops.c
to allow re-using internal helpers and move it closer to the other helpers
that depend on the exact set of uops and their fields.
Unlike libswscale/tests/sws_ops.c, we make an effort to actually test all
relevant flag combinations, since these can affect the generated op lists.
I will use these macros to auto-generate both the C template-based kernels,
as well as the entire x86 backend, in the near future, hence their excessive
flexibility.
Re-use the libswscale/tests/sws_ops.c that we already compile. We could put it
in its own file but this is just as convenient, and it's easily moved anyways.
Having it be a FATE test ensures that it is always up-to-date.
Signed-off-by: Niklas Haas <git@haasn.dev>
This will replace the fuzzy matching logic in op_match() that is used by the
C and x86 implementations, as well as the translation to AARCH64_OP_* that is
used by the NEON asmgen backend.
Down the line, this function will also take a set of flags to enable
backend-specific kernels like FMA variants, but I also decided to keep it
initially simple to ease the transition.
Signed-off-by: Niklas Haas <git@haasn.dev>
Taken from AARCH64_OP_*, but generalized/simplified a bit and updated to add
missing op types, especially for special cases that already have dedicated
implementations on x86.
This initial definition is kept intentionally simple and close to SwsOp, to
make it easier to port the existing ops backends to the new infrastructure.
However, in the future, this will be refactored dramatically - distinctions
like convert vs expand will cease to exist on the SwsOp level, and will
instead be introduced by separate optimization passes on the uops level.
SWS_UOP_LINEAR in particular will most likely be broken up into multiple
uops. I also took this opportunity to redefine the mask in a more useful way.
I decided to split up SWS_OP_CONVERT as well, because it was making x86
codegen unnecessarily difficult due to the strong interaction between exact
pixel sizes.
Signed-off-by: Niklas Haas <git@haasn.dev>
Forming what will be the start of a larger helper file for backend-internal
translation of higher-level ops into lower level kernels. This header file
needs to be includable from independent source files, as it will be used to
provide definitions for build-time code generation (e.g. ops_asmgen.c), so
it must be self-contained.
Pulling in all of ops.h from uops.h would be too large dependency, since
ops.h pulls in graph.h, refstruct, bprint, etc. It's easier to start from a
fresh file that is documented as being usable at compile time.
For now, just declare the common types that will be needed by the uops layer.
Signed-off-by: Niklas Haas <git@haasn.dev>
This suppresses the addition of #line directives in the preprocessed output,
which is what we want when we're invoking the hostcc just to preprocess some
files. (Currently, this variable is only used for configure-internal checks
anyways, but I want to use it to preprocess a NASM file)
On MSVC/Intel, /EP is the equivalent syntax, though we use -EP instead for
consistency.
Signed-off-by: Niklas Haas <git@haasn.dev>
Add NEON-optimized implementations for HEVC angular intra prediction
modes 10 (pure horizontal) and 26 (pure vertical) at 8-bit depth.
Mode 10 (Horizontal):
- Broadcasts left[y] to fill each row using ld2r/ld4r for efficiency
- Applies edge smoothing for luma blocks smaller than 32x32
Mode 26 (Vertical):
- Copies top reference row to all output rows
- Applies edge smoothing for luma blocks smaller than 32x32
Edge smoothing uses uhsub+usqadd to compute the filtered result
directly in 8-bit, avoiding widening to 16-bit intermediates.
The C pred_angular wrappers are made non-static with ff_ prefix to
allow the NEON dispatch to fall back to C for modes not yet optimized.
This will be reverted once all angular modes are implemented.
Note: since pred_angular[] is a per-size function pointer (not
per-mode), checkasm benchmarks will show '_neon' for all 33 modes
even though only modes 10/26 are truly accelerated; unoptimized
modes show ~1.0x speedup as they pass through the NEON wrapper to
the C fallback with negligible overhead.
Speedup over C on Apple M4 (checkasm --bench, 15-run average):
Mode 10 (Horizontal):
4x4: 4.66x 8x8: 5.80x 16x16: 16.86x 32x32: 24.89x
Mode 26 (Vertical):
4x4: 1.16x 8x8: 1.83x 16x16: 2.45x 32x32: 4.50x
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Replace plain memcmp+fail() with checkasm_check_pixel_padded() for
DC, planar, and angular prediction tests. Use PIXEL_RECT for output
buffers instead of flat arrays.
This enables:
- Detailed per-pixel difference output when run with 'checkasm -v'
- Detection of out-of-bounds writes beyond the NxN block area
- Padding violation reporting (writes past block boundary)
Previously, a test failure would only report "FAILED" with no
information about which pixels were wrong, making assembly debugging
difficult. Follows the pattern established in 4d4b301e4a (checkasm:
hevc_pel: Use helpers for checking for writes out of bounds).
Suggested-by: Martin Storsjö <martin@martin.st>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Set sps->vui.sar to {0,1} (unspecified) before the VUI parsing
block, matching the HEVC pattern at hevc_ps.c. The old
zero-init-to-1 workaround is now unreachable and is removed.
Suggested-by: James Almer <jamrial@gmail.com>
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Per ITU-T H.264 (ISO/IEC 14496-10) Annex E.2.1 and ITU-T H.265
(ISO/IEC 23008-2) Annex E.3.1, when sar_width or sar_height is zero
the sample aspect ratio shall be considered unspecified. Internally
ffmpeg represents an unspecified SAR as 0/1, while fractions with a
zero denominator are not handled properly (den=0 is silently changed
to den=1 in h264_ps.c, turning an invalid 20480/0 into a "valid" but
impossibly extreme 20480/1); so we bridge the gap by replacing x/0
with 0/1 at the VUI parsing layer.
An av_log warning is added so an invalid SAR in the bitstream is
diagnosed rather than silently overwritten.
This fixes a problem with some video files provided by game
OddBallers when executed with Wine/Proton, which report SAR 20480/0.
Based on patch by Giovanni Mascellani <gmascellani@codeweavers.com>.
Fixes: ticket #23321
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
If this were to be checked, it should be checked generically,
not in every single encoder.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Instead use CODEC_PIXFMTS. Avoids deprecation warnings
from Clang and simplifies the removal of AVCodec.pix_fmts.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
Lynne
Niklas Haas
haoyuLiu
Jun Zhao
Michael Niedermayer
Andreas Rheinhardt