FFmpeg 9.1's new AAC encoder

lynne and 17 Guests are viewing this topic. 2026-06-30 10:31:23 Hi,I recently did a full rewrite of FFmpeg's AAC encoder.  Everything from rate control, RDO, and all coding tools (PNS, TNS, I/S and M/S) was reengineered. Metrics-wise (Google's new Zimtohrli, ViSQOL, and my own hearing) it's apparently the best AAC encoder, compared to qaac and fdk-aac:kbps8.1 fast8.1 twoloopnmrfdk-aacapplelibopus640.01315 / 2.650.00696 / 3.240.00309 / 3.830.00322 / 3.690.00612 / 3.290.00100 / 4.59960.00338 / 3.770.00268 / 3.990.00134 / 4.040.00153 / 3.980.00175 / 3.870.00039 / 4.621280.00229 / 4.100.00170 / 4.280.00072 / 4.470.00143 / 4.270.00081 / 4.440.00020 / 4.681600.00129 / 4.300.00108 / 4.440.00051 / 4.560.00065 / 4.310.00117

/ 4.510.00084 / 4.682560.00105 / 4.410.00121 / 4.550.00031 / 4.610.00103 / 4.450.00067 / 4.630.00002 / 4.73Zim/ViS, lower-is-better/higher-is-better respectively.The rewrite will be merged today. You can either compile it from source or grab one of BtbN's latest nightly builds (once merged and they update) to test it.Some specifics:the new encoder is strictly CBR, with very little variation in rate. Having a bit budget target helps coding a lot. I don't recommend using -q:a (real VBR mode).Other encoders don't use any coding tools except TNS. So first, I beat them fair, using only TNS, then reimplemented PNS, I/S and M/S on top to crush them conclusively.I RE'd qaac and found out it does no perceptual optimizations. It simply uses band energy with a bit allocation curve that favours high frequencies. I improved on that by having a similar curve, and using masked band energy for the RDO.FFmpeg's AAC DEcoder is busted with regards to stereo PNS, and the bug may be in other AAC decoders too, so we work around it in the encoder. Since no other encoder used PNS, the bug was not found until now.Special attention was paid to high bitrates. Coding tools like I/S and PNS get turned off by themselves if the encoder does good on its own to maintain rate.All coding tools, PNS, TNS, I/S and M/S are part of the RDO loop. We use no parroted heuristics or arbitrary bitrate cutoffs. If a tool can be used, it will be used.If you intend to downmix or expect the output to get downmixed, use -aac_is 0 -aac_pns 0 to maintain the original signal phase.The encoder was mainly optimized for 48Khz audio.

Get over it. It's 2026, resampling is free, 48Khz is the standard. 44.1Khz will work, and so will 96Khz but use 48Khz if you want the best quality.If you look at spectrograms, we leave a lot of holes. This is by design. Masked bands get zero'd or PNS'd, since a neighbouring band is loud enough to avoid noticing the missing bands. It's better to only code audible bands well, than code everything badly.I look forward to A/B testing by users. I tested this on 3000 tracks (my music collection), though I had very little speech content to test on, so it may need some optimizations there.The encoder prints extra stats on uninit:Qavg: 207.975  Tr: 5.3%  TNS(L): 4.8%  TNS(S): 36.9%  M/S: 3.9%  I/S: 10.0%  PNS: 5.1%Here's how to read them.the average lambda value (higher values == encoder struggles more to keep rate)Short blocksTNS usage on long framesTNS usage on short framesMid/Side coding usageIntensity stereo coding usagePerceptual Noise Substitution usageIf you find annoying artifacts, post the original input samples, and that line so I can look into it. Re: FFmpeg 9.1's new AAC encoder Reply #1 – 2026-06-30 10:37:12 "The encoder was mainly optimized for 48Khz audio. Get over it. It's 2026, resampling is free, 48Khz is the standard. 44.1Khz will work, and so will 96Khz but use 48Khz if you want the best quality.

"Yet most of the worlds audio is 44KHz... Re: FFmpeg 9.1's new AAC encoder Reply #2 – 2026-06-30 10:53:44 "The encoder was mainly optimized for 48Khz audio. Get over it. It's 2026, resampling is free, 48Khz is the standard. 44.1Khz will work, and so will 96Khz but use 48Khz if you want the best quality."Yet most of the worlds audio is 44KHz...The benchmarks I posted were done mainly on 44.1Khz. I tuned by ear on 48Khz data though, so some of the windowing/transient logic is tied to 48Khz. It translated to 44.1Khz well enough that I left it as-is, since the timing difference isn't that large.And if you're encoding lossily, may as well resample to 48 and not require decoders to do so on the other end. Re: FFmpeg 9.1's new AAC encoder Reply #3 – 2026-06-30 11:28:45 What do you mean, you tuned it by ear?

Tuned what, exactly? Re: FFmpeg 9.1's new AAC encoder Reply #4 – 2026-06-30 12:14:21 I have some killer FLAC samples that clearly shows how old AAC encoder is low quality.I hope the new encoder is much better. Re: FFmpeg 9.1's new AAC encoder Reply #5 – 2026-06-30 16:00:40 Oh, Lynne is on HA! Welcome!For those unaware: Lynne also wrote the native USAC decoder for FFmpeg, all of it AFAIR. Bit off-topic, but: Lynne, are you aware of the USAC playback glitches during seeking? Discussed a lot here, e.g. https://hydrogenaudio.org/index.php/topic,129641.msg1083064.html#msg1083064 and https://hydrogenaudio.org/index.php/topic,129641.msg1083087.html#msg1083087I agree with Mycroft that FFmpeg's old AAC encoder performs quite "suboptimally", and I think this is also confirmed in the results of one of Kamedo2's listening tests, https://hydrogenaudio.org/index.php/topic,119861.0.htmlChris Last Edit: 2026-06-30 16:03:22 by C.R.Helmrich Re: FFmpeg 9.1's new AAC encoder

Reply #6 – 2026-06-30 16:55:06 Too bad, I am currently testing the FFmpeg AAC encoder version N-124995-gbb6de744cc, Sat, 13 Jun 2026 19:08:10 version. Your wholly-rewritten version won't be included in the upcoming ABC/HR listening test results(now 19% done).aac_mf is AAC via MediaFoundation (Available on Windows 10 and 11)$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a libopus -b:a 89k out.opus$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a opus -strict experimental -b:a 96k out.opus$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a libfdk_aac -b:a 96k out.mp4$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a aac -b:a 96k out.mp4$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a aac_mf -b:a 96k out.mp4$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a libmp3lame -b:a 96k out.mp3$ ffmpeg124995 -i in.44100Hz.stereo.wav -c:a ac3 -b:a 96k out.ac3 Re:

FFmpeg 9.1's new AAC encoder Reply #7 – Today at 13:33 The rewrite will be merged today.well, turns out it did actually merge today.just... not the today that marked June 30, 2026.yeah. Re: FFmpeg 9.1's new AAC encoder Reply #8 – Today at 14:52 Oh, Lynne is on HA! Welcome!For those unaware: Lynne also wrote the native USAC decoder for FFmpeg, all of it AFAIR. Bit off-topic, but: Lynne, are you aware of the USAC playback glitches during seeking? Discussed a lot here, e.g. https://hydrogenaudio.org/index.php/topic,129641.msg1083064.html#msg1083064 and https://hydrogenaudio.org/index.php/topic,129641.msg1083087.html#msg1083087I agree with Mycroft that FFmpeg's old AAC encoder performs quite "suboptimally", and I think this is also confirmed in the results of one of Kamedo2's listening tests, https://hydrogenaudio.org/index.php/topic,119861.0.htmlChrisIt shouldn't be worse than seeking on HE-AAC, right? We don't handle preroll properly at FFmpeg yet, since the preroll grows by a lot due to the delay HE-AAC has.Too bad, I am currently testing the FFmpeg AAC encoder version N-124995-gbb6de744cc, Sat, 13 Jun 2026 19:08:10 version.

Your wholly-rewritten version won't be included in the upcoming ABC/HR listening test results(now 19% done).I don't think you should cancel the tests you're running, but if you could, would you be able to run quick comparisons rather than full A/B testing?By the way, about the metrics, Zim is the better metric, but saturates at high bitrates, so ViS is used as a tiebreaker. Using this rule, we win across all comparisons (except Opus, because it's the best). Re: FFmpeg 9.1's new AAC encoder Reply #9 – Today at 15:35 tested new AAC encoder.solid encoding quality indeed.i tested this with only one song (Burn the Boats from Marathon [Vol. II]: They are Waiting (Original Game Soundtrack)), but did so with -b:a mode as lynne requested.following bitrates tested are 64kbps, 134kbps, 200kbps.64k is good, albeit with a bit of artifacting. not sure if ffmpeg AAC PNS decoder bug (playing the resulting file with ffplay btw) or not.134k and 200k makes for a sublime listen. Re: FFmpeg 9.1's new AAC encoder Reply #10 – Today at 16:08 Some command line help pls for fb2k? Highly interested, just not having much time and only few love for ffmpeg cmdline. Re: FFmpeg 9.1's new AAC encoder Reply #11 – Today at