Have you ever bought a storage upgrade for your iPod, iPhone, or Mac to have more room for your Apple Music (formerly iTunes) digital music collection? You may have been filling it with gigabytes worth of empty space.
The music files you purchase from the Apple Music Store (AMS; formerly the iTunes Music Store) contain approximately 6 % of the digital equivalent of nothingness. I’m not referring to silence, but continuous blocks of empty space set aside inside the files. It serves no purpose other than to pad the files to make them 0,5 MB larger. So, why is Apple making music files unnecessarily large? and how do you recover the wasted space?
This article is fairly technical, and the topics in it require a fair bit of explanation. You can skip the first 14 paragraphs if you’re only interested in the “why”, or skip the first 17 paragraphs for the “how.”
Music files purchased from AMS is encoded with the Advanced Audio Codec (AAC) using the Apple AAC Encoder. The files are delivered in an ISOBMFF (ISO 14496-1) container. This container format is used for different media files including audio, pictures, and movies. The container format has lots of room for implementations to develop their own weird quirks.
ISOM containers are broken up into different data blocks including the actual multimedia stream, metadata (encoding details, artist name, album art, volume normalization hints, etc.), and empty space. There are other blocks too, but these three are the ones we care about in this article.
These sections can technically be arranged in any order in the file However, the exact order causes various side effects in some situations. For example, placing the metadata block at the end of the file means you must download (or read from local storage) the entire song before you can play it. To avoid this, encoders tend to place it before the much larger multimedia stream block. The metadata block refers to absolute positions inside the multimedia stream block measured as an offset from the start of the file. You can’t make changes to the metadata (e.g. change the spelling of an artist’s name) without reassembling the entire file to recalculate the offsets.
To get around this, encoders tend to set aside some empty space between the metadata and multimedia stream blocks. In ISOM containers, this block is known as a “free” block. These blocks contain nothing but null-bytes. If you change the title of an album, Apple Music can expand the metadata block and shrink the free block without having to move the multimedia stream block. So far, so good.
If you rip a CD with Apple Music using the Apple AAC Encoder (the default option), it will reserve approximately 5 kB of free space inside each file for this purpose. That’s enough for you to add about 1250 emojis to the metadata of each of your songs. It should be plenty of room for you to make whatever changes you find necessary to the metadata. If you really need to add more data, the music file can be reassembled later to grow the metadata space. The operation would take slightly longer, but this would be an acceptable compromise as to not waste too much storage space.
However, the music files you’ve purchased from the Apple Music Store contains approximately 500 kB (0,5 MB) of free space. That’s 100 times more than Apple Music sets aside inside songs you’ve ripped yourself. It’s also a lot more space for metadata than anyone will ever need.
My Apple Music library consists of 2500 songs that I’ve purchased from AMS, and about the same number of home-rips. They’re encoded with the same quality settings. After analyzing my collection, I found that AMS purchased songs are on average 7 % larger than my home-rips. 6 % of that difference is empty space (1,25 GB in total.) Only 0,005 % of the storage space for home-rips is empty space.
I’ve analyzed samples of iTunes Store music files purchased in 2010 and 2020, and they’re all structured in the same way with the same amount of wasted space. Broadband speeds and storage was more expensive ten years ago, so this would have been an even bigger issue back then.
Apple’s classic HFS+ file system didn’t support sparse storage. Sparse storage is a file system storage technique that can be used to compact consecutive chunks of null bytes when writing files. The file system records how many repeating null-bytes there are instead of writing the individual null-bytes to the drive.
The new Apple File System (APFS) does support sparse storage, but Apple Music doesn’t take advantage of it and writes all the null-bytes out on the drive. I tried rewriting my music collection in a sparse-aware fashion, and was able to shrink my music collection by 5,2 %. Sparse storage can reduce storage requirements on your local computer. However, it won’t lower the bill from your online backup provider, or help you when non-sparse aware programs (like Apple Music) modifies the files or syncs them with your iProducts.
I’ve mentioned that you can reassemble the music files a couple of times already. If you’re interested in recovering the space wasted discussed in this article, you can do so by reassembling the files using an encoder that doesn’t set aside as much free space as AMS. Note that reassembling a music file — sometimes called remuxing — doesn’t involve re-encoding the multimedia stream block. Practically, this means that the operation is lossless in terms of the actual multimedia portion of the file.
Before I tell you how to remux the empty space out of your music, I need to discuss why it’s there in the first place. As discussed, encoders often reserve a small chunk of empty space to allow for metadata changes without reassembling the entire file for each change. I’ve also mentioned that the album art is stored in the metadata section.
However, AMS doesn’t store album art inside each music file. Starting with iTunes 7 (released ), it began storing album art files separately from the music files. The change had to do with the implementation of iTunes’ fancy new album-flipping feature called Cover Flow, according to TUAW.
I can only speculate here, but I believe the 500 kB block was originally reserved for album art. Apple’s servers would inject it on-the-fly into the metadata at the time of delivery. Album art can vary by region for the same album, so I guess this makes sense? The same song can be part of different albums too. Maybe Apple only stores one copy and injects metadata on-the-fly? When album art was removed, no one at Apple seems to have remembered to shrink the metadata block size.
I don’t know for sure, but this is the only reasonable explanation I can come up with. A more sinister explanation is that Apple wanted to upsell people on more expensive iPods and Macs with more storage space. Separating the artwork from music files can also be seen as a move to make it less convenient for customers to move their music collections to competing players.
You should make sure to first re-embed album artwork into your music files before you proceed. The below process will alter the music files and it may make iTunes confused about which album art goes with which files. Embedding them solves this problem, and will also make sure you can play your music in any other music application. You can use Re-Embed Artwork for MacOS to fix your entire music collection in one go.
You may notice that most songs don’t grow in size after re-embedding the album art. The album art nicely slots into the empty space that is already reserved inside each file. To recover the remaining empty space inside your music files, you’ll need to remux each file using FFmpeg.
Start by installing Homebrew, and then use it to install findutils and ffmpeg. Once you’ve installed those dependencies, you can run the following script chain in the Terminal app. Make sure to select the entire line and copy it exactly into the terminal.
find ~/Music/ "(" -name "*.m4a" -and -not -name ".*" ")" -print0 | xargs -0 -P4 -n1 sh -c 'ffmpeg -y -i "$0" -movflags +faststart -map 0 -c copy "$0.tmp.m4a" && mv "$0.tmp.m4a" "$0"'It will locate every music file in your Music directory (using the find command), and apply the right processing with ffmpeg. This is a lossless operation and you won’t lose any audio quality. Note that some iTunes-specific metadata (purchase information) will be lost from the files in the process. This is what may cause iTunes to have trouble later unless you embedded the album art into the files first.
I’ve always chosen the low-storage capacity Apple products over the years, and filled them to the brim with music. It’s frustrating to realize I could have fit 6 % more music onto those devices if Apple hadn’t filled them with null bytes! I don’t believe this was a malicious or sinister act by Apple. It was probably just a mistake by an engineer that has gone unnoticed for over ten years and never got fixed.