Advanced Audio Coding

Advanced Audio Coding is an audio coding standard for lossy digital audio compression. Designed to be the successor of the MP3 format, AAC generally achieves better sound quality than MP3 at similar bit rates AAC has been standardized by ISO and IEC, as part of the MPEG-2 and MPEG-4 specifications Part of the AAC known as High Efficiency Advanced Audio Coding which is part of MPEG-4 Audio is also adopted into digital radio standards like DAB+ and Digital Radio Mondiale, as well as mobile television standards DVB-H and ATSC-M/H AAC supports inclusion of 48 full-bandwidth audio channels in one stream plus 16 low frequency effects channels, up to 16 “coupling” or dialog channels, and up to 16 data streams. The quality for stereo is satisfactory to modest requirements at 96 kbit/s in joint stereo mode; however, hi-fi transparency demands data rates of at least 128 kbit/s. The MPEG-2 audio tests showed that AAC meets the requirements referred to as “transparent” for the ITU at 128 kbit/s for stereo, and 320 kbit/s for 5.1 audio AAC is the default or standard audio format for YouTube, iPhone, iPod, iPad, Nintendo DSi, Nintendo 3DS, iTunes, DivX Plus Web Player and PlayStation 3. It is supported on PlayStation Vita, Wii, Sony Walkman MP3 series and later, Sony Ericsson; Nokia, Android, BlackBerry, and webOS-based mobile phones, with the use of a converter. AAC also continues to enjoy increasing adoption by manufacturers of in-dash car audio systems History AAC was developed with the cooperation and contributions of companies including AT&T Bell Laboratories, Fraunhofer IIS, Dolby Laboratories, Sony Corporation and Nokia. It was officially declared an international standard by the Moving Picture Experts Group in April 1997 It is specified both as Part 7 of the MPEG-2 standard, and Subpart 4 in Part 3 of the MPEG-4 standard Standardization In 1997, AAC was first introduced as MPEG-2 Part 7, formally known as ISO/IEC 13818-7:1997 This part of MPEG-2 was a new part, since MPEG-2 already included MPEG-2 Part 3, formally known as ISO/IEC 13818-3: MPEG-2 BC. Therefore, MPEG-2 Part 7 is also known as MPEG-2 NBC, because it is not compatible with the MPEG-1 audio formats MPEG-2 Part 7 defined three profiles: Low-Complexity profile, Main profile and Scalable Sampling Rate profile. AAC-LC profile consists of a base format very much like AT&T’s Perceptual Audio Coding coding format, with the addition of temporal noise shaping, the Dolby Kaiser Window, a nonuniform quantizer, and a reworking of the bitstream format to handle up to 16 stereo channels, 16 mono channels, 16 low-frequency effect channels and 16 commentary channels in one bitstream. The Main profile adds a set of recursive predictors that are calculated on each tap of the filterbank. The SSR uses a 4-band PQMF filterbank, with four shorter filterbanks following, in order to allow for scalable sampling rates In 1999, MPEG-2 Part 7 was updated and included in the MPEG-4 family of standard and became known as MPEG-4 Part 3, MPEG-4 Audio or ISO/IEC 14496-3:1999. This update included several improvements. One of these improvements was the addition of Audio Object Types which are used to allow interoperability with a diverse range of other audio formats such as TwinVQ, CELP, HVXC, Text-To-Speech Interface and MPEG-4 Structured Audio. Another notable addition in this version of the AAC standard is Perceptual Noise Substitution. In that regard, the AAC profiles are combined with perceptual noise substitution and are defined in the MPEG-4 audio standard as Audio Object Types. MPEG-4 Audio Object Types are combined in four MPEG-4 Audio profiles: Main, Scalable, Speech and

Low Rate Synthesis The reference software for MPEG-4 Part 3 is specified in MPEG-4 Part 5 and the conformance bit-streams are specified in MPEG-4 Part 4 MPEG-4 Audio remains backward-compatible with MPEG-2 Part 7 The MPEG-4 Audio Version 2 defined new audio object types: the low delay AAC object type, bit-sliced arithmetic coding object type, parametric audio coding using harmonic and individual line plus noise and error resilient versions of object types. It also defined four new audio profiles: High Quality Audio Profile, Low Delay Audio Profile, Natural Audio Profile and Mobile Audio Internetworking Profile The HE-AAC Profile and AAC Profile were first standardized in ISO/IEC 14496-3:2001/Amd 1:2003 The HE-AAC v2 Profile was first specified in ISO/IEC 14496-3:2005/Amd 2:2006. The Parametric Stereo audio object type used in HE-AAC v2 was first defined in ISO/IEC 14496-3:2001/Amd 2:2004 The current version of the AAC standard is defined in ISO/IEC 14496-3:2009 AAC+ v2 is also standardized by ETSI as TS 102005 The MPEG-4 Part 3 standard also contains other ways of compressing sound. These include lossless compression formats, synthetic audio and low bit-rate compression formats generally used for speech AAC’s improvements over MP3 Advanced Audio Coding is designed to be the successor of the MPEG-1 Audio Layer 3, known as MP3 format, which was specified by ISO/IEC in 11172-3 and 13818-3 Blind tests in the late 1990s showed that AAC demonstrated greater sound quality and transparency than MP3 for files coded at the same bit rate, but since that time numerous codec listening tests have shown that the best encoders in each format are often of similar quality and that the quality is often dependent on the encoder used even within the same format. As an approximation, when using the best encoders, AAC’s advantage over MP3 tends to be evident below around 100 kbit/s, but certain AAC encoders are not as good as the best MP3 encoder as they do not take optimal advantage of the additional encoding tools that AAC makes available Improvements include: More sample frequencies than MP3 Up to 48 channels Arbitrary bit-rates and variable frame length Standardized constant bit rate with bit reservoir Higher efficiency and simpler filter bank Higher coding efficiency for stationary signals Higher coding accuracy for transient signals Can use Kaiser-Bessel derived window function to eliminate spectral leakage at the expense of widening the main lobe Much better handling of audio frequencies above 16 kHz More flexible joint stereo Adds additional modules to increase compression efficiency: TNS, Backwards Prediction, PNS etc… These modules can be combined to constitute different encoding profiles Overall, the AAC format allows developers more flexibility to design codecs than MP3 does, and corrects many of the design choices made in the original MPEG-1 audio specification This increased flexibility often leads to more concurrent encoding strategies and, as a result, to more efficient compression. However, in terms of whether AAC is better than MP3, the advantages of AAC are not entirely decisive, and the MP3 specification, although antiquated, has proven surprisingly robust in spite of considerable flaws. AAC and HE-AAC are better than MP3 at low bit rates. This is especially true at very low bit rates where the superior stereo coding, pure MDCT, and better transform window sizes leave MP3 unable to compete While the MP3 format has near-universal hardware and software support, primarily due to MP3 being the format of choice during the crucial first few years of widespread music file-sharing/distribution over the internet, AAC is a strong contender due to some unwavering industry support How AAC works AAC is a wideband audio coding algorithm that exploits two primary coding strategies to dramatically reduce the amount of data needed

to represent high-quality digital audio Signal components that are perceptually irrelevant are discarded; Redundancies in the coded audio signal are eliminated The actual encoding process consists of the following steps: The signal is converted from time-domain to frequency-domain using forward modified discrete cosine transform. This is done by using filter banks that take an appropriate number of time samples and convert them to frequency samples The frequency domain signal is quantized based on a psychoacoustic model and encoded Internal error correction codes are added; The signal is stored or transmitted In order to prevent corrupt samples, a modern implementation of the Luhn mod N algorithm is applied to each frame The MPEG-4 audio standard does not define a single or small set of highly efficient compression schemes but rather a complex toolbox to perform a wide range of operations from low bitrate speech coding to high-quality audio coding and music synthesis The MPEG-4 audio coding algorithm family spans the range from low bitrate speech encoding to high-quality audio coding AAC offers sampling frequencies between 8 kHz and 96 kHz and any number of channels between 1 and 48 In contrast to MP3’s hybrid filter bank, AAC uses the modified discrete cosine transform together with the increased window lengths of 1024 or 960 points AAC encoders can switch dynamically between a single MDCT block of length 1024 points or 8 blocks of 128 points If a signal change or a transient occurs, 8 shorter windows of 128/120 points each are chosen for their better temporal resolution By default, the longer 1024-point/960-point window is otherwise used because the increased frequency resolution allows for a more sophisticated psychoacoustic model, resulting in improved coding efficiency Modular encoding AAC takes a modular approach to encoding Depending on the complexity of the bitstream to be encoded, the desired performance and the acceptable output, implementers may create profiles to define which of a specific set of tools they want to use for a particular application The MPEG-2 Part 7 standard was first published in 1997 and offers three default profiles: Low Complexity – the simplest and most widely used and supported; Main Profile – like the LC profile, with the addition of backwards prediction; Scalable Sample Rate – a.k.a. Sample-Rate Scalable; The MPEG-4 Part 3 standard defined various new compression tools and their usage in brand new profiles. AAC is not used in some of the MPEG-4 Audio profiles. The MPEG-2 Part 7 AAC LC profile, AAC Main profile and AAC SSR profile are combined with Perceptual Noise Substitution and defined in the MPEG-4 Audio standard as Audio Object Types. These are combined with other Object Types in MPEG-4 Audio profiles Here is a list of some audio profiles defined in the MPEG-4 standard: Main Audio Profile – defined in 1999, uses most of the MPEG-4 Audio Object Types Scalable Audio Profile – defined in 1999, uses AAC-LC, AAC-LTP, AAC Scalable, TwinVQ, CELP, HVXC, TTSI Speech Audio Profile – defined in 1999, uses CELP, HVXC, TTSI Synthetic Audio Profile – defined in 1999, TTSI, Main synthesis High Quality Audio Profile – defined in 2000, uses AAC-LC, AAC-LTP, AAC Scalable, CELP, ER-AAC-LC, ER-AAC-LTP, ER-AAC Scalable, ER-CELP Low Delay Audio Profile – defined in 2000, uses CELP, HVXC, TTSI, ER-AAC-LD, ER-CELP, ER-HVXC Mobile Audio Internetworking Profile – defined in 2000, uses ER-AAC-LC, ER-AAC-Scalable, ER-TwinVQ, ER-BSAC, ER-AAC-LD AAC Profile – defined in 2003, uses AAC-LC High Efficiency AAC Profile – defined in 2003, uses AAC-LC, SBR

High Efficiency AAC v2 Profile – defined in 2006, uses AAC-LC, SBR, PS (One of many improvements in MPEG-4 Audio is the Object Type – Long Term Prediction, which is an improvement of the Main profile using a forward predictor with lower computational complexity.) AAC error protection toolkit Applying error protection enables error correction up to a certain extent. Error correcting codes are usually applied equally to the whole payload However, since different parts of an AAC payload show different sensitivity to transmission errors, this would not be a very efficient approach The AAC payload can be subdivided into parts with different error sensitivities Independent error correcting codes can be applied to any of these parts using the Error Protection tool defined in MPEG-4 Audio standard This toolkit provides the error correcting capability to the most sensitive parts of the payload in order to keep the additional overhead low The toolkit is backwardly compatible with simpler and pre-existing AAC decoders. A great deal of the tool kit’s error correction functions are based around spreading information about the audio signal more evenly in the datastream Error Resilient AAC Error Resilience techniques can be used to make the coding scheme itself more robust against errors For AAC, three custom-tailored methods were developed and defined in MPEG-4 Audio Huffman Codeword Reordering to avoid error propagation within spectral data; Virtual Codebooks to detect serious errors within spectral data; Reversible Variable Length Code to reduce error propagation within scale factor data AAC Low Delay The MPEG-4 Low Delay Audio Coder is designed to combine the advantages of perceptual audio coding with the low delay necessary for two-way communication. It is closely derived from the MPEG-2 Advanced Audio Coding format Licensing and patents No licenses or payments are required to be able to stream or distribute content in AAC format. This reason alone can make AAC a much more attractive format to distribute content than its predecessor MP3, particularly for streaming content depending on the use case However, a patent license is required for all manufacturers or developers of AAC codecs For this reason, free and open source software implementations such as FFmpeg and FAAC may be distributed in source form only, in order to avoid patent infringement Extensions and improvements Some extensions have been added to the first AAC standard: Perceptual Noise Substitution, added in MPEG-4 in 1999. It allows the coding of noise as pseudorandom data; Long Term Predictor, added in MPEG-4 in 1999 It is a forward predictor with lower computational complexity Error Resilience, added in MPEG-4 Audio version 2 in 2000, used for transport over error prone channels; AAC-LD, defined in 2000, used for real-time conversation applications; High Efficiency AAC, a.k.a. aacPlus v1 or AAC+, the combination of SBR and AAC LC; used for low bitrates; defined in 2003; HE-AAC v2, a.k.a. aacPlus v2 or eAAC+, the combination of Parametric Stereo and HE-AAC; used for even lower bitrates; defined in 2004 and 2006; MPEG-4 Scalable To Lossless, defined in 2006, can supplement an AAC stream to provide a lossless decoding option, such as in Fraunhofer IIS’s “HD-AAC” product; Container formats In addition to the MP4, 3GP and other ISO base media file format-based container formats for file storage, AAC audio data was first packaged in file based form for the MPEG-2 standard by using a more basic format called Audio Data Interchange Format, consisting of a single header followed by the raw AAC audio data blocks. However, if the data is to be streamed within an MPEG-2 transport stream, a self-synchronizing format called an Audio Data Transport Stream is used, consisting of a series of frames, each frame having a header followed by the AAC audio data. This file and streaming-based format are defined in MPEG-2 Part 7, but are only considered informative by MPEG-4, so an MPEG-4 decoder

does not need to support either format. These containers, as well as a raw AAC stream, may bear the .aac file extension. MPEG-4 Part 3 also defines its own self-synchronizing format called a Low Overhead Audio Stream that encapsulates not only AAC, but any MPEG-4 audio compression scheme such as TwinVQ and ALS. This format is what was defined for use in DVB transport streams when encoders use either SBR or parametric stereo AAC extensions However, it is restricted to only a single non-multiplexed AAC stream. This format is also referred to as a Low Overhead Audio Transport Multiplex, which just an interleaved multiple stream version of a LOAS Products that support AAC HDTV Standards Japanese ISDB-T In December 2003, Japan started broadcasting terrestrial DTV ISDB-T standard that implements MPEG-2 video and MPEG-2 AAC audio. In April 2006 Japan started broadcasting the ISDB-T mobile sub-program, called 1seg, that was the first implementation of video H.264/AVC with audio HE-AAC in Terrestrial HDTV broadcasting service on the planet International ISDB-Tb In December 2007, Brazil started broadcasting terrestrial DTV standard called International ISDB-Tb that implements video coding H.264/AVC with audio AAC-LC on main program and video H.264/AVC with audio HE-AACv2 in the 1seg mobile sub-program DVB The ETSI, the standards governing body for the DVB suite, supports AAC, HE-AAC and HE-AAC v2 audio coding in DVB applications since at least 2004. DVB broadcasts which use the H.264 compression for video normally use HE-AAC for audio Hardware iTunes and iPod In April 2003, Apple brought mainstream attention to AAC by announcing that its iTunes and iPod products would support songs in MPEG-4 AAC format. Customers could download music in a closed-source Digital Rights Management-restricted form of AAC via the iTunes Store or create files without DRM from their own CDs using iTunes. In later years, Apple began offering music videos and movies, which also use AAC for audio encoding On May 29, 2007, Apple began selling songs and music videos free of DRM from participating record labels. These files mostly adhere to the AAC standard and are playable on many non-Apple products but they do include custom iTunes information such as album artwork and a purchase receipt, so as to identify the customer in case the file is leaked out onto peer-to-peer networks. It is possible, however, to remove these custom tags to restore interoperability with players that conform strictly to the AAC specification. As of January 6, 2009, nearly all music on the USA regioned iTunes Store became DRM-free, with the remainder becoming DRM-free by the end of March 2009 iTunes supports a “Variable Bit Rate” encoding option which encodes AAC tracks in an “Average Bit Rate” scheme. As of September 2009, Apple has added support for HE-AAC only for radio streams, not file playback, and iTunes still lacks support for true VBR encoding. The underlying QuickTime API does offer a true VBR encoding profile however Other portable players Archos Cowon Creative Zen Portable Microsoft Zune Samsung Yepp SanDisk Sansa Sony PlayStation Portable with firmware 2.0 or greater Sony Walkman Nintendo DSi Nintendo 3DS Any portable player that fully supports the Rockbox third party firmware Mobile phones For a number of years, many mobile phones from manufacturers such as Nokia, Motorola, Samsung, Sony Ericsson, BenQ-Siemens and Philips have supported AAC playback. The first such phone was the Nokia 5510 released in 2002 which also plays MP3s. However, this phone was a commercial failure and such phones with integrated music players did not gain mainstream popularity until 2005 when the trend of having AAC as well as MP3 support continued. Most new smartphones and music-themed phones support playback of these formats Sony Ericsson phones support various AAC formats in MP4 container. AAC-LC is supported in all

phones beginning with K700, phones beginning with W550 have support of HE-AAC. The latest devices such as the P990, K610, W890i and later support HE-AAC v2 Nokia XpressMusic and other new generation Nokia multimedia phones like N- and E-Series: also support AAC format in LC, HE, M4A and HEv2 profiles BlackBerry: RIM’s latest series of Smartphones such as the 8100, 9500 and 8800 support AAC bada OS Apple’s iPhone supports AAC and FairPlay protected AAC files formerly used as the default encoding format in the iTunes store until the removal of DRM restrictions in March 2009 All recent Android phones support AAC-LC, HE-AAC and HE-AAC v2 in MP4 or M4A containers along with several other audio formats. From Android 3.1 also raw ADTS files are supported Android 4.0 can also encode these kind of files WebOS by HP/Palm supports AAC, AAC+, eAAC+, and .m4a containers in its native music player as well as several third-party players. However, it does not support Apple’s FairPlay DRM files downloaded from iTunes Windows Phone: Windows Phone’s Silverlight runtime supports AAC-LC, HE-AAC and HE-AAC v2 decoding Other devices Apple’s iPad: Supports AAC and FairPlay protected AAC files used as the default encoding format in the iTunes store Palm OS PDAs: Many Palm OS based PDAs and smartphones can play AAC and HE-AAC with the 3rd party software Pocket Tunes. Version 4.0, released in December 2006, added support for native AAC and HE-AAC files. The AAC codec for TCPMP, a popular video player, was withdrawn after version 0.66 due to patent issues, but can still be downloaded from sites other than corecodec.org. CorePlayer, the commercial follow-on to TCPMP, includes AAC support Other Palm OS programs supporting AAC include Kinoma Player and AeroPlayer Microsoft Windows Mobile platforms support AAC either by the native Windows Media Player or by third-party products Epson supports AAC playback in the P-2000 and P-4000 Multimedia/Photo Storage Viewers This support is not available with their older models, however The Sony Reader portable eBook plays M4A files containing AAC, and displays metadata created by iTunes. Other Sony products, including the A and E series Network Walkmans, support AAC with firmware updates while the S series supports it out of the box Nearly every major car stereo manufacturer offers models that will play back .m4a files recorded onto CD in a data format. This includes Pioneer, Sony, Alpine, Kenwood, Clarion, Panasonic, and JVC The Sonos Digital Media Player supports playback of AAC files The Barnes & Noble Nook Color electronic-book reader supports playback of AAC encoded files The Roku SoundBridge network audio player supports playback of AAC encoded files The Squeezebox network audio player supports playback of AAC files The PlayStation 3 supports encoding and decoding of AAC files The Xbox 360 supports streaming of AAC through the Zune software, and of supported iPods connected through the USB port The Wii video game console supports AAC files through version 1.1 of the Photo Channel as of December 11, 2007. All AAC profiles and bitrates are supported as long as it is in the.m4a file extension. This update removed MP3 compatibility, but users who have installed this may freely downgrade to the old version if they wish The Livescribe Pulse and Echo Smartpens record and store audio in AAC format. The audio files can be replayed using the pen’s integrated speaker, attached headphones, or on a computer using the Livescribe Desktop software. The AAC files are stored in the user’s “My Documents” folder of the Windows OS and can be distributed and played without specialized hardware or software from Livescribe Software

Almost all current computer media players include built-in decoders for AAC, or can utilize a library to decode it. On Microsoft Windows, DirectShow can be used this way with the corresponding filters to enable AAC playback in any DirectShow based player. Mac OS X supports AAC via the QuickTime libraries Adobe Flash Player, since version 9 update 3, can also play back AAC streams. Since Flash Player is also a browser plugin, it can play AAC files through a browser as well The Rockbox open source firmware also offers support for AAC to varying degrees, depending on the model of player and the AAC profile Optional iPod support for the Xbox 360 is available as a free download from Xbox Live Following, a non-comprehensive list of other software player applications: 3ivx MPEG-4 – A suite of DirectShow and QuickTime plugins which support AAC encoding or AAC/ HE-AAC decoding in any DirectShow application CorePlayer also supports LC and HE AAC Easy CD-DA Extractor for Windows, CD Ripper and audio converter, which includes an AAC encoder that supports LC and HE AAC ffdshow is a free open source DirectShow filter for Microsoft Windows operating systems that uses FAAD2 to support AAC decoding foobar2000 is a freeware audio player for Windows that supports LC and HE AAC JAAD is an open-source decoder written in Java K-Multimedia Player also supports AAC KSP Sound Player also supports AAC MediaMonkey AIMP Media Player Classic Home Cinema MPlayer or xine are often used as AAC decoders on Linux or Macintosh MusicBee – An advanced music manager / player that also supports encoding / ripping RealPlayer includes RealNetworks’ RealAudio 10 AAC encoder Songbird for Windows, Linux and Mac OS X supports AAC, including the DRM rights management encoding used for purchased music from the iTunes Store, with a plug-in Sony SonicStage also support AAC VLC media player supports playback and encoding of MP4 and AAC files Winamp for Windows, which includes an AAC encoder that supports LC and HE AAC; Windows Media Player 12, released with Windows 7, supports playback of AAC files natively Another Real product, Rhapsody supports the RealAudio AAC codec, in addition to offering subscription tracks encoded with AAC XBMC supports AAC XMMS supports mp4 playback using a plugin provided by the faad2 library Some of these players also support the decoding of ADTS using the SHOUTcast protocol. Plug-ins for Winamp and foobar2000 enable the creation of such streams Nero Digital Audio In May 2006, Nero AG released an AAC encoding tool free of charge, Nero Digital Audio, which is capable of encoding LC-AAC, HE-AAC and HE-AAC v2 streams. The tool is a Command Line Interface tool only. A separate utility is also included to decode to PCM WAV Various tools including the foobar2000 audio player, MediaCoder, MeGUI encoding front end and dBpoweramp can provide a GUI for this encoder FAAC and FAAD2 FAAC and FAAD2 stand for Freeware Advanced Audio Coder and Decoder 2 respectively. FAAC supports audio object types LC, Main and LTP FAAD2 supports audio object types LC, Main, LTP, SBR and PS. Although FAAD2 is free software, FAAC is not free software FFmpeg FFmpeg’s libavcodec library contains free software codecs for both encoding and decoding AAC. See also here for a list of other encoder/decoder libraries available See also Comparison of audio coding formats AAC-LD MPEG-4 Part 14 ALAC — Apple’s own lossless codec Vorbis — the main open, royalty-free competitor to AAC and MP3 Opus — an open, royalty-free codec for both pre-encoded and interactive use, standardized

in 2012 References External links Fraunhofer audio codecs AudioCoding.com – home of FAAC and FAAD2 Official MPEG web site AAC improvements and extensions RFC 3016 – RTP Payload Format for MPEG-4 Audio/Visual Streams RFC 3640 – RTP Payload Format for Transport of MPEG-4 Elementary Streams RFC 4281 – The Codecs Parameter for “Bucket” Media Types RFC 4337 – MIME Type Registration for MPEG-4