virtualenv/FFmpeg
1
0
Fork 0
mirror of https://github.com/FFmpeg/FFmpeg.git synced 2024-12-23 12:43:46 +02:00
Code Issues Releases Activity
3f3be1c07a
FFmpeg/libavcodec/aacenc_tns.c

220 lines
8.0 KiB
C
Raw Normal View History

aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
/*
* AAC encoder TNS
* Copyright (C) 2015 Rostislav Pehlivanov
*
* This file is part of FFmpeg.
*
* FFmpeg is free software; you can redistribute it and/or
* modify it under the terms of the GNU Lesser General Public
* License as published by the Free Software Foundation; either
* version 2.1 of the License, or (at your option) any later version.
*
* FFmpeg is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* Lesser General Public License for more details.
*
* You should have received a copy of the GNU Lesser General Public
* License along with FFmpeg; if not, write to the Free Software
* Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
*/
/**
* @file
* AAC encoder temporal noise shaping
* @author Rostislav Pehlivanov ( atomnuker gmail com )
*/
#include "aacenc.h"
#include "aacenc_tns.h"
#include "aactab.h"
#include "aacenc_utils.h"
#include "aacenc_quantization.h"
aacenc_tns: encode coefficients directly and reenable compression This commit was made possible with the earlier commits since the new quantization method basically means we're working always with unsigned values. The specifications mention to use compression when the first 2 bits are identical but they didn't mention if this should happen before or after the conversion to signed values. Actually they said nothing about conversion to signed values. With this commit, coefficient compression usually always happens which saves a lot of space, especially at extremely low bitrates and doesn't change the quality at all. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:10:28 +02:00
/*
* Shifts the values as well if compression is possible.
*/
static inline int compress_coeffs(int *coef, int order, int c_bits)
{
int i, res = 0;
const int low_idx = c_bits ? 4 : 2;
const int shift_val = c_bits ? 8 : 4;
const int high_idx = c_bits ? 11 : 5;
for (i = 0; i < order; i++)
aacenc_tns: fix coefficient compression condition Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-16 18:48:02 +02:00
if (coef[i] < low_idx || coef[i] > high_idx)
aacenc_tns: encode coefficients directly and reenable compression This commit was made possible with the earlier commits since the new quantization method basically means we're working always with unsigned values. The specifications mention to use compression when the first 2 bits are identical but they didn't mention if this should happen before or after the conversion to signed values. Actually they said nothing about conversion to signed values. With this commit, coefficient compression usually always happens which saves a lot of space, especially at extremely low bitrates and doesn't change the quality at all. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:10:28 +02:00
res++;
if (res == order)
for (i = 0; i < order; i++)
coef[i] -= (coef[i] > high_idx) ? shift_val : 0;
return res == order;
}
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
/**
* Encode TNS data.
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
* Coefficient compression saves a single bit per coefficient.
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
*/
aacenc: Add missing ff_ prefixes Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com> Reviewed-by: Ganesh Ajjanagadde <gajjanag@mit.edu>
2015-08-22 07:11:23 +02:00
void ff_aac_encode_tns_info(AACEncContext *s, SingleChannelElement *sce)
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
{
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
int i, w, filt, coef_len, coef_compress = 0;
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
const int is8 = sce->ics.window_sequence[0] == EIGHT_SHORT_SEQUENCE;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
TemporalNoiseShaping *tns = &sce->tns;
aacenc_tns: encode coefficients directly and reenable compression This commit was made possible with the earlier commits since the new quantization method basically means we're working always with unsigned values. The specifications mention to use compression when the first 2 bits are identical but they didn't mention if this should happen before or after the conversion to signed values. Actually they said nothing about conversion to signed values. With this commit, coefficient compression usually always happens which saves a lot of space, especially at extremely low bitrates and doesn't change the quality at all. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:10:28 +02:00
const int c_bits = is8 ? TNS_Q_BITS_SHORT == 4 : TNS_Q_BITS == 4;
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
if (!sce->tns.present)
return;
for (i = 0; i < sce->ics.num_windows; i++) {
put_bits(&s->pb, 2 - is8, sce->tns.n_filt[i]);
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
if (tns->n_filt[i]) {
aacenc_tns: encode coefficients directly and reenable compression This commit was made possible with the earlier commits since the new quantization method basically means we're working always with unsigned values. The specifications mention to use compression when the first 2 bits are identical but they didn't mention if this should happen before or after the conversion to signed values. Actually they said nothing about conversion to signed values. With this commit, coefficient compression usually always happens which saves a lot of space, especially at extremely low bitrates and doesn't change the quality at all. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:10:28 +02:00
put_bits(&s->pb, 1, c_bits);
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
for (filt = 0; filt < tns->n_filt[i]; filt++) {
put_bits(&s->pb, 6 - 2 * is8, tns->length[i][filt]);
put_bits(&s->pb, 5 - 2 * is8, tns->order[i][filt]);
if (tns->order[i][filt]) {
aacenc_tns: encode coefficients directly and reenable compression This commit was made possible with the earlier commits since the new quantization method basically means we're working always with unsigned values. The specifications mention to use compression when the first 2 bits are identical but they didn't mention if this should happen before or after the conversion to signed values. Actually they said nothing about conversion to signed values. With this commit, coefficient compression usually always happens which saves a lot of space, especially at extremely low bitrates and doesn't change the quality at all. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:10:28 +02:00
coef_compress = compress_coeffs(tns->coef_idx[i][filt],
tns->order[i][filt], c_bits);
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
put_bits(&s->pb, 1, !!tns->direction[i][filt]);
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
put_bits(&s->pb, 1, !!coef_compress);
aacenc_tns: encode coefficients directly and reenable compression This commit was made possible with the earlier commits since the new quantization method basically means we're working always with unsigned values. The specifications mention to use compression when the first 2 bits are identical but they didn't mention if this should happen before or after the conversion to signed values. Actually they said nothing about conversion to signed values. With this commit, coefficient compression usually always happens which saves a lot of space, especially at extremely low bitrates and doesn't change the quality at all. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:10:28 +02:00
coef_len = c_bits + 3 - coef_compress;
for (w = 0; w < tns->order[i][filt]; w++)
put_bits(&s->pb, coef_len, tns->coef_idx[i][filt][w]);
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
}
}
}
}
}
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
/* Apply TNS filter */
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
void ff_aac_apply_tns(AACEncContext *s, SingleChannelElement *sce)
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
{
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
TemporalNoiseShaping *tns = &sce->tns;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
IndividualChannelStream *ics = &sce->ics;
int w, filt, m, i, top, order, bottom, start, end, size, inc;
const int mmm = FFMIN(ics->tns_max_bands, ics->max_sfb);
float lpc[TNS_MAX_ORDER];
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
for (w = 0; w < ics->num_windows; w++) {
bottom = ics->num_swb;
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
for (filt = 0; filt < tns->n_filt[w]; filt++) {
top = bottom;
bottom = FFMAX(0, top - tns->length[w][filt]);
order = tns->order[w][filt];
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
if (order == 0)
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
continue;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
// tns_decode_coef
compute_lpc_coefs(tns->coef[w][filt], order, lpc, 0, 0, 0);
start = ics->swb_offset[FFMIN(bottom, mmm)];
end = ics->swb_offset[FFMIN( top, mmm)];
aacenc_tns: rework the way coefficients are calculated This commit abandons the way the specifications state to quantize the coefficients, makes use of the new LPC float functions and is much better. The original way of converting non-normalized float samples to int32_t which out LPC system expects was wrong and it was wrong to assume the coefficients that are generated are also valid. It was essentially a full garbage-in, garbage-out system and it definitely shows when looking at spectrals and listening. The high frequencies were very overattenuated. The new LPC function performs the analysis directly. The specifications state to quantize the coefficients into four bit index values using an asin() function which of course had to have ugly ternary operators because the function turns negative if the coefficients are negative which when encoding causes invalid bitstream to get generated. This deviates from this by using the direct TNS tables, which are fairly small since you only have 4 bits at most for index values. The LPC values are directly quantized against the tables and are then used to perform filtering after the requantization, which simply fetches the array values. The end result is that TNS works much better now and doesn't attenuate anything but the actual signal, e.g. TNS removes quantization errors and does it's job correctly now. It might be enabled by default soon since it doesn't hurt and helps reduce nastyness at low bitrates. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 07:47:31 +02:00
if ((size = end - start) <= 0)
continue;
if (tns->direction[w][filt]) {
inc = -1;
start = end - 1;
} else {
inc = 1;
}
start += w * 128;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
// ar filter
for (m = 0; m < size; m++, start += inc)
for (i = 1; i <= FFMIN(m, order); i++)
sce->coeffs[start] += lpc[i-1]*sce->pcoeffs[start - i*inc];
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
}
}
}
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
/*
* c_bits - 1 if 4 bit coefficients, 0 if 3 bit coefficients
*/
static inline void quantize_coefs(double *coef, int *idx, float *lpc, int order,
int c_bits)
{
int i;
const float *quant_arr = tns_tmp2_map[c_bits];
for (i = 0; i < order; i++) {
idx[i] = quant_array_idx((float)coef[i], quant_arr, c_bits ? 16 : 8);
lpc[i] = quant_arr[idx[i]];
}
}
/*
* 3 bits per coefficient with 8 short windows
*/
aacenc: Add missing ff_ prefixes Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com> Reviewed-by: Ganesh Ajjanagadde <gajjanag@mit.edu>
2015-08-22 07:11:23 +02:00
void ff_aac_search_for_tns(AACEncContext *s, SingleChannelElement *sce)
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
{
TemporalNoiseShaping *tns = &sce->tns;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
int w, w2, g, count = 0;
const int mmm = FFMIN(sce->ics.tns_max_bands, sce->ics.max_sfb);
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
const int is8 = sce->ics.window_sequence[0] == EIGHT_SHORT_SEQUENCE;
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
const int c_bits = is8 ? TNS_Q_BITS_SHORT == 4 : TNS_Q_BITS == 4;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
int sfb_start = av_clip(tns_min_sfb[is8][s->samplerate_index], 0, mmm);
int sfb_end = av_clip(sce->ics.num_swb, 0, mmm);
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
aacenc_tns: fix triggering an assertion with assert-level=2 It also made no sense to actually make the filter span the entire window including the first band of the next window. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-29 20:01:59 +02:00
for (w = 0; w < sce->ics.num_windows; w++) {
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
int use_tns;
int order = is8 ? 5 : s->profile == FF_PROFILE_AAC_LOW ? 12 : TNS_MAX_ORDER;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
int coef_start = w*sce->ics.num_swb + sce->ics.swb_offset[sfb_start];
int coef_len = sce->ics.swb_offset[sfb_end] - sce->ics.swb_offset[sfb_start];
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
float e_ratio = 0.0f, threshold = 0.0f, spread = 0.0f, en[2] = {0.0, 0.0f};
double gain = 0.0f, coefs[MAX_LPC_ORDER] = {0};
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
for (g = 0; g < sce->ics.num_swb; g++) {
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
if (w*16+g < sfb_start || w*16+g > sfb_end)
continue;
for (w2 = 0; w2 < sce->ics.group_len[w]; w2++) {
FFPsyBand *band = &s->psy.ch[s->cur_channel].psy_bands[(w+w2)*16+g];
if ((w+w2)*16+g > sfb_start + ((sfb_end - sfb_start)/2))
aacenc_tns: adjust coefficient calculation, add double filter support This commit improves the TNS implementation to the point where it's actually usable and very rarely results in nastyness (in all bitrates except extremely low bitrates it's increasing the quality and prevents some distortions from the coder being audiable). Also adds a double filter support which is only used if the energy difference between the top and bottom of the SFBs is above the thresholds defined in the header file. Looking at the bitstream that fdk_aac generates it sometimes used a double filter despite the specs stating that a single filter should be enough for almost all cases and purposes. Unlike FAAC or fdk_aac we sometimes use a reverse filter in case the energy difference isn't enought to use a double filter. This actually works better. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-05 10:08:30 +02:00
en[1] += band->energy;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
else
aacenc_tns: adjust coefficient calculation, add double filter support This commit improves the TNS implementation to the point where it's actually usable and very rarely results in nastyness (in all bitrates except extremely low bitrates it's increasing the quality and prevents some distortions from the coder being audiable). Also adds a double filter support which is only used if the energy difference between the top and bottom of the SFBs is above the thresholds defined in the header file. Looking at the bitstream that fdk_aac generates it sometimes used a double filter despite the specs stating that a single filter should be enough for almost all cases and purposes. Unlike FAAC or fdk_aac we sometimes use a reverse filter in case the energy difference isn't enought to use a double filter. This actually works better. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-05 10:08:30 +02:00
en[0] += band->energy;
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
threshold += band->threshold;
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
spread += band->spread;
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
}
}
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
if (coef_len <= 0 || (sfb_end - sfb_start) <= 0)
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
continue;
/* LPC */
aacenc_tns: rework coefficient quantization and filter application This commit reworks the TNS implementation to a hybrid between what the specifications say, what the decoder does and what's the best thing to do. The filter application function was copied from the decoder and modified such that it applies the inverse AR filter to the coefficients. The LPC coefficients themselves are fed into the same quantization expression that the specifications say should be used however further processing is not done, instead they're converted to the form that the decoder expects them to be in and are sent off to the compute_lpc_coeffs function exactly the way the decoder does. This function does all conversions and will return the exact coefficients that the decoder will generate, which are then applied to the coefficients. Having the exact same coefficients on both the encoder and decoder is a must since otherwise the entire sfb's over which the filter is applied will be attenuated. Despite this major rework, TNS might not work fine on some audio types at very low bitrates (e.g. sub 90kbps) as it can attenuate some coefficients too much. Users are advised to experiment with TNS at higher bitrates if they wish to use this tool or simply wait for the implementation to be improved. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-01 07:44:07 +02:00
gain = ff_lpc_calc_ref_coefs_f(&s->lpc, &sce->coeffs[coef_start],
coef_len, order, coefs);
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
if (!order || gain < TNS_GAIN_THRESHOLD_LOW || gain > TNS_GAIN_THRESHOLD_HIGH)
use_tns = 0;
else if ((en[0]+en[1]) < TNS_GAIN_THRESHOLD_LOW*threshold || spread < TNS_SPREAD_THRESHOLD)
use_tns = 0;
else
use_tns = 1;
if (use_tns) {
e_ratio = en[0]/en[1];
aacenc_tns: adjust coefficient calculation, add double filter support This commit improves the TNS implementation to the point where it's actually usable and very rarely results in nastyness (in all bitrates except extremely low bitrates it's increasing the quality and prevents some distortions from the coder being audiable). Also adds a double filter support which is only used if the energy difference between the top and bottom of the SFBs is above the thresholds defined in the header file. Looking at the bitstream that fdk_aac generates it sometimes used a double filter despite the specs stating that a single filter should be enough for almost all cases and purposes. Unlike FAAC or fdk_aac we sometimes use a reverse filter in case the energy difference isn't enought to use a double filter. This actually works better. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-05 10:08:30 +02:00
if (is8 || order < 2 || (e_ratio > TNS_E_RATIO_LOW && e_ratio < TNS_E_RATIO_HIGH)) {
tns->n_filt[w] = 1;
for (g = 0; g < tns->n_filt[w]; g++) {
tns->length[w][g] = sfb_end - sfb_start;
tns->direction[w][g] = en[0] < en[1];
tns->order[w][g] = order;
quantize_coefs(coefs, tns->coef_idx[w][g], tns->coef[w][g],
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
order, c_bits);
aacenc_tns: adjust coefficient calculation, add double filter support This commit improves the TNS implementation to the point where it's actually usable and very rarely results in nastyness (in all bitrates except extremely low bitrates it's increasing the quality and prevents some distortions from the coder being audiable). Also adds a double filter support which is only used if the energy difference between the top and bottom of the SFBs is above the thresholds defined in the header file. Looking at the bitstream that fdk_aac generates it sometimes used a double filter despite the specs stating that a single filter should be enough for almost all cases and purposes. Unlike FAAC or fdk_aac we sometimes use a reverse filter in case the energy difference isn't enought to use a double filter. This actually works better. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-05 10:08:30 +02:00
}
} else { /* 2 filters due to energy disbalance */
tns->n_filt[w] = 2;
for (g = 0; g < tns->n_filt[w]; g++) {
tns->direction[w][g] = en[g] < en[!g];
tns->order[w][g] = !g ? order/2 : order - tns->order[w][g-1];
tns->length[w][g] = !g ? (sfb_end - sfb_start)/2 : \
(sfb_end - sfb_start) - tns->length[w][g-1];
quantize_coefs(&coefs[!g ? 0 : order - tns->order[w][g-1]],
tns->coef_idx[w][g], tns->coef[w][g],
aacenc_tns: redo coefficient quantization and decision making This finally (and again) gets rid of basically everything the specifications say about how TNS should be done. The main problem used to be that a single filter was used for all coefficients which despite being explicitly recommended by the specifications usually sounds wrong, therefore it's a corner case in the current TNS implementation. This commit also changes the coefficient bit size, as apparently it's better to use lower precision in case the windows are eight short. This is apparently what fdk_aac uses, looking at the bit stream and makes sense. Also the order when 8 SHORT windows happen is important as 7 was too much and according to PSNR was worse while 5 is just about correct. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-12 14:04:27 +02:00
tns->order[w][g], c_bits);
aacenc_tns: adjust coefficient calculation, add double filter support This commit improves the TNS implementation to the point where it's actually usable and very rarely results in nastyness (in all bitrates except extremely low bitrates it's increasing the quality and prevents some distortions from the coder being audiable). Also adds a double filter support which is only used if the energy difference between the top and bottom of the SFBs is above the thresholds defined in the header file. Looking at the bitstream that fdk_aac generates it sometimes used a double filter despite the specs stating that a single filter should be enough for almost all cases and purposes. Unlike FAAC or fdk_aac we sometimes use a reverse filter in case the energy difference isn't enought to use a double filter. This actually works better. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-09-05 10:08:30 +02:00
}
aacenc_tns: implement temporal noise shaping This commit implements temporal noise shaping support in the encoder, along with an -aac_tns option to toggle it on or off (off by default for now). TNS will increase audio quality and reduce quantization noise by applying a multitap FIR filter across allowed coefficients and transmit side information to the decoder so it could create an inverse filter. Users are encouraged to test the new functionality by enabling -aac_tns 1 during encoding. No major bugs are observable at this time so after a while if no new problems appear and if the current implementation is deemed of high enough quality and stability it will be enabled by default, possibly at the same time the encoder has its experimental flag removed and becomes the standard aac encoder in ffmpeg. Signed-off-by: Rostislav Pehlivanov <atomnuker@gmail.com>
2015-08-21 20:27:38 +02:00
}
count++;
}
}
sce->tns.present = !!count;
}
Reference in New Issue Copy Permalink