Revisiting Apple Notes (5): Encrypted Notes

TL;DR: Apple Notes allows users to encrypt note contents at rest and the Apple Cloud Notes Parser now supports parsing of encrypted content.

Background

Apple Notes has allowed users to encrypt their note’s contents at rest in the NoteStore database since iOS 9.3. While some commercial forensics tools can unlock notes, I am unaware of free, open source tools in the community which do so and adding this functionality was a challenge posed to me by Heather Mahalik a few years ago during a SANS FOR585. The foundations which Apple uses to carry out the encryption are well documented standards, but initial attempts at putting them together when the parser was written in Perl failed. The struggles with debugging are what led to completely rewriting the parser into an object-oriented language in late 2019 and early 2020. With the parser rewritten, each of the foundational blocks could be implemented discretely and built into the overall program, leading to decryption of encrypted notes in July of 2020.

This article is a detailed look at what is encrypted, how it is encrypted, and how to decrypt it. If you are just looking at Apple notes for the first time, I would recommend starting with the other entries in the Apple Notes category to understand how it works under normal circumstances before tackling encrypted notes. Throughout this article, I will mainly refer to the ZICCLOUDSYNCINGOBJECT table when I reference the NoteStore.sqlite database and hence will not be writing that table name every time. I will intentionally include a table name when referencing other tables, such as ZICNOTEDATA.ZDATA.

Password Recovery

It must be said from the outset that the Apple Cloud Notes Parser is not intended to be a password cracker for Apple Notes. This software is to be used to backup or recover notes which you legally have the password to. With that said, Apple’s documentation says that “if you forgot your password, Apple can’t help you regain access to your locked notes.” While true, if you do not have the password for the encrypted note, but do still have the database, you could use a program like HashCat or John the Ripper to potentially recover it. Both of those programs support password recovery on Apple Notes and Apple File System (APFS) encryption. The rest of this article assumes you either created the note and know the password for the note or have a legal reason for reading the note and have the password.

Face ID and Touch ID can also be used to unlock notes, but these do not change the underlying password. All they do is unlock the password you set up in Settings->Notes->Password and enter it automatically, so the password is still able to be recovered from the database. .

Decrypting with Apple Cloud Notes Parser

While I hope the detailed explanations below allow many others to implement note decryption, Apple Cloud Notes Parser aims to make it easy. To tell the parser which passwords to try, a new argument has been added which expects a file path pointing to a file with one password on each line.

-w, --password-file FILE         File with plaintext passwords, one per line.

For example:

notta@cuppa ~/apple_cloud_notes_parser $ cat passwords.txt
root
Summer!2018
password
password1

notta@cuppa ~/apple_cloud_notes_parser $ ruby notes_cloud_ripper.rb --itunes-dir ~/phone_rips/iphone/notes_2019_12_05/device_id --password-file passwords.txt

Starting Apple Notes Parser at Wed Jul 29 19:48:43 2020
Storing the results in ./output/2020_07_29-19_48_43

Created a new AppleBackup from iTunes backup: /home/notta/phone_rips/iphone/notes_2019_12_05/device_id/
Guessed Notes Version: 13
Guessed Notes Version: 8
Added 4 passwords to the AppleDecrypter from /home/notta/apple_could_notes_parser/passwords.txt
Updated AppleNoteStore object with 70 AppleNotes in 11 folders belonging to 2 accounts.
Updated AppleNoteStore object with 0 AppleNotes in 1 folders belonging to 1 accounts.
Adding the ZICNOTEDATA.ZPLAINTEXT and ZICNOTEDATA.ZDECOMPRESSEDDATA columns, this takes a few seconds

Successfully finished at Wed Jul 29 19:48:48 2020

What is Encrypted?

When I say that users can encrypt their note’s contents, I need to be careful to be precise. For the most part, what is encrypted is what is found in the ZICNOTEDATA.ZDATA column in the NoteStore.sqlite database. This leaves most of the note metadata unencrypted, and even some of the content (specifically the ZTITLE1 column) unencrypted. Aside from the ZICNOTEDATA.ZDATA column, contents of attached files end up encrypted and some of the metadata about objects ends up encypted in the ZENCRYPTEDVALUESJSON column, such as filenames and URLs. Notice in the below screenshot how, even though the notes are all locked, you can see the day they were created and the title.

According to Apple¹, users can encrypt notes containing “images, sketches, tables, maps, and websites.” Each of those functions slightly differently in how they encrypt, but the general rules I’ve seen are:

1. If the normal version of the attachment writes to disk, such as an image or a thumbnail of a webpage, that file will contain encrypted data
2. If the normal version of the attachment has a user-generated filename, such as an image, that filename will be encrypted and stored in the ZENCRYPTEDVALUESJSON column but the file will use the ZIDENTIFER column as its filename
3. If the normal version of the attachment has a Notes-generated filename, such as a sketch, that filename will remain the same
4. If the normal version of the attachment uses the ZMERGEABLEDATA1 column, such as a table, that blob will be encrypted and stored in the ZENCRYPTEDVALUESJSON column
5. Files are encrypted on disk using the ZASSETCRYPTOTAG and ZASSETCRYPTOINITIALIZATIONVECTOR columns
6. Fallback images, such as for sketches, are encrypted on disk using the ZFALLBACKIMAGECRYPTOTAG and ZFALLBACKIMAGECRYPTOINITIALIZATIONVECTOR columns

How is it Encrypted?

Like much of Apple’s products, specific details on the inner workings of Apple Notes are hard to come by and they use functionality that is not part of the larger libraries Apple makes available to most developers². The bulk of what is officially known comes from a brief blurb that has been copied over a few different Apple pages over the years¹. The blurb describes the underlying foundation of their encryption, but doesn’t get quite far enough for someone unfamiliar with the implementation of these foundations to implement it on their own. That is possibly why this feature shows up in commercial tools and not homegrown ones. Here is the most relevant section of the blurb:

When a user secures a note, a 16-byte key is derived from the user’s passphrase using PBKDF2 and SHA256. The note and all of its attachments are encrypted using AES-GCM. New records are created in Core Data and CloudKit to store the encrypted note, attachments, tag, and initialization vector. After the new records are created, the original unencrypted data is deleted. Attachments that support encryption include images, sketches, tables, maps, and websites. Notes containing other types of attachments can’t be encrypted, and unsupported attachments can’t be added to secure notes.

…

To change the passphrase on a secure note, the user must enter the current passphrase, as Touch ID and Face ID aren’t available when changing the passphrase. After choosing a new passphrase, the Notes app rewraps the keys of all existing notes in the same account that are encrypted by the previous passphrase.

What Changes During Encryption?

During testing of how to decrypt, we wanted to see what exactly happened when a user encrypted a note. I used a MacOS computer to create a note, copied the NoteStore.sqlite database off, then encrypted the note and copied it off again. This is the output of sqldiff’ing the files:

notta@cuppa ~/notestores $ sqldiff MacNoteStore_preencryption.sqlite MacNoteStore_postencryption.sqlite | grep -e "ZICCLOUDSYNCINGOBJECT\|ZICNOTEDATA"
UPDATE ZICCLOUDSYNCINGOBJECT SET Z_OPT=8, ZMARKEDFORDELETION=1, ZSNIPPET=NULL, ZTITLE1=NULL WHERE Z_PK=121;
UPDATE ZICCLOUDSYNCINGOBJECT SET Z_OPT=4 WHERE Z_PK=123;
INSERT INTO ZICCLOUDSYNCINGOBJECT(Z_PK,Z_ENT,Z_OPT,ZCRYPTOITERATIONCOUNT,ZISPASSWORDPROTECTED,ZMARKEDFORDELETION,ZMINIMUMSUPPORTEDNOTESVERSION,ZNEEDSINITIALFETCHFROMCLOUD,ZNEEDSTOBEFETCHEDFROMCLOUD,ZNEEDSTOSAVEUSERSPECIFICRECORD,ZCLOUDSTATE,ZACCOUNT,ZCHECKEDFORLOCATION,ZFILESIZE,ZHANDWRITINGSUMMARYVERSION,ZHASMARKUPDATA,ZIMAGECLASSIFICATIONSUMMARYVERSION,ZIMAGEFILTERTYPE,ZOCRSUMMARYVERSION,ZORIENTATION,ZSECTION,ZLOCATION,ZMEDIA,ZNOTE,ZNOTEUSINGTITLEFORNOTETITLE,ZPARENTATTACHMENT,ZAPPEARANCETYPE,ZSCALEWHENDRAWING,ZVERSION,ZVERSIONOUTOFDATE,ZATTACHMENT,ZSTATE,ZACCOUNT1,ZTYPE,ZACCOUNT2,ZATTACHMENT1,ZATTACHMENTVIEWTYPE,ZISPINNED,ZLEGACYNOTEWASPLAINTEXT,ZNOTEHASCHANGES,ZPAPERSTYLETYPE,ZPREFERREDBACKGROUNDTYPE,ZACCOUNT3,ZFOLDER,ZNOTEDATA,ZTITLESOURCEATTACHMENT,ZISHIDDENNOTECONTAINER,ZSORTORDER,ZOWNER,ZACCOUNTTYPE,ZDIDCHOOSETOMIGRATE,ZDIDFINISHMIGRATION,ZDIDMIGRATEONMAC,ZSTOREDATASEPARATELY,ZACCOUNTDATA,ZCUSTOMNOTESORTTYPEVALUE,ZFOLDERTYPE,ZIMPORTEDFROMLEGACY,ZACCOUNT4,ZPARENT,ZCREATIONDATE,ZCROPPINGQUADBOTTOMLEFTX,ZCROPPINGQUADBOTTOMLEFTY,ZCROPPINGQUADBOTTOMRIGHTX,ZCROPPINGQUADBOTTOMRIGHTY,ZCROPPINGQUADTOPLEFTX,ZCROPPINGQUADTOPLEFTY,ZCROPPINGQUADTOPRIGHTX,ZCROPPINGQUADTOPRIGHTY,ZDURATION,ZMODIFICATIONDATE,ZORIGINX,ZORIGINY,ZPREVIEWUPDATEDATE,ZSIZEHEIGHT,ZSIZEWIDTH,ZHEIGHT,ZMODIFIEDDATE,ZSCALE,ZWIDTH,ZSTATEMODIFICATIONDATE,ZMODIFICATIONDATEATIMPORT,ZCREATIONDATE1,ZFOLDERMODIFICATIONDATE,ZLASTNOTIFIEDDATE,ZLASTVIEWEDMODIFICATIONDATE,ZLEGACYMODIFICATIONDATEATIMPORT,ZMODIFICATIONDATE1,ZCUSTOMNOTESORTTYPEMODIFICATIONDATE,ZDATEFORLASTTITLEMODIFICATION,ZPARENTMODIFICATIONDATE,ZIDENTIFIER,ZPASSWORDHINT,ZZONEOWNERNAME,ZADDITIONALINDEXABLETEXT,ZFALLBACKSUBTITLEIOS,ZFALLBACKSUBTITLEMAC,ZFALLBACKTITLE,ZHANDWRITINGSUMMARY,ZIMAGECLASSIFICATIONSUMMARY,ZOCRSUMMARY,ZREMOTEFILEURLSTRING,ZSUMMARY,ZTITLE,ZTYPEUTI,ZURLSTRING,ZUSERTITLE,ZDEVICEIDENTIFIER,ZCONTENTHASHATIMPORT,ZFILENAME,ZLEGACYCONTENTHASHATIMPORT,ZLEGACYIMPORTDEVICEIDENTIFIER,ZLEGACYMANAGEDOBJECTIDURIREPRESENTATION,ZSELECTEDINKCOLORSTRING,ZSELECTEDINKIDENTIFIER,ZSNIPPET,ZTHUMBNAILATTACHMENTIDENTIFIER,ZTITLE1,ZACCOUNTNAMEFORACCOUNTLISTSORTING,ZNESTEDTITLEFORSORTING,ZNAME,ZUSERRECORDNAME,ZTITLE2,ZASSETCRYPTOINITIALIZATIONVECTOR,ZASSETCRYPTOTAG,ZCRYPTOINITIALIZATIONVECTOR,ZCRYPTOSALT,ZCRYPTOTAG,ZCRYPTOWRAPPEDKEY,ZENCRYPTEDVALUESJSON,ZSERVERRECORDDATA,ZSERVERSHAREDATA,ZUNAPPLIEDENCRYPTEDRECORD,ZUSERSPECIFICSERVERRECORDDATA,ZMERGEABLEDATA,ZFALLBACKIMAGECRYPTOINITIALIZATIONVECTOR,ZFALLBACKIMAGECRYPTOTAG,ZMARKUPMODELDATA,ZMERGEABLEDATA1,ZMETADATADATA,ZCRYPTOMETADATAINITIALIZATIONVECTOR,ZCRYPTOMETADATATAG,ZENCRYPTEDMETADATA,ZMETADATA,ZLASTNOTIFIEDTIMESTAMPDATA,ZLASTVIEWEDTIMESTAMPDATA,ZREPLICAIDTOUSERIDDICTDATA,ZCRYPTOVERIFIER,ZMERGEABLEDATA2) VALUES(124,9,1,20000,1,0,0,0,NULL,NULL,124,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,0,0,NULL,NULL,0,0,4,115,44,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,617486270.51991,617486482.450767,NULL,-541232580,NULL,617486280.859228,NULL,NULL,NULL,'E147C9ED-C0C3-455B-8CA6-24D957DFAF8F','password',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,'Unencrypted',NULL,NULL,NULL,NULL,NULL,NULL,NULL,x'cb9bdaa5d303ed5b38ee53be3ddaa449',x'c195e60cd045facb026d148a27a6c347',x'41f9f4c637b07e9f7429a99754d0a0a9',x'1a4de80b87fd7791d6ab1c934e003a363e1a1d23faf048aa',NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL,NULL);
UPDATE ZICNOTEDATA SET Z_OPT=4, ZDATA=x'1f8b0800000000000013e36010ea67e460106090ea6014629012e0620171805c30adc12825041461048a304b81680e05460d26a818b30013588c0128c62c250c16631598c90b57c82225c6c50134e63f10f0038d84b39564b9a4b9042e24865c76feea56d2542131bf4175f71921668e79bc424c1c9c003f72e71694000000' WHERE Z_PK=41;
INSERT INTO ZICNOTEDATA(Z_PK,Z_ENT,Z_OPT,ZNOTE,ZCRYPTOINITIALIZATIONVECTOR,ZCRYPTOTAG,ZDATA) VALUES(44,15,1,124,x'cb9bdaa5d303ed5b38ee53be3ddaa449',x'41f9f4c637b07e9f7429a99754d0a0a9',x'017d9bfb981ed74fef8b6f71c2cacf59c061874d4715cafed0052f0aee07fde270cda3da7a8656b5369df3efc33c131952e775007fd527547204aad9bacb2f62d795369e8bf6de218e86691bb0ab3c4c3bc006f9824401c6cb2814947bb4aafb73ab5788d173cd8cfbebd16214b2a31cf36b19cc2112da32478cf1270d6d554017cc989c00a35f9a5feb076bda55553cb758dabefda65359fc5d01843a709e74d11b6f7c6c5f51a023d0736b04e48d2e8615d04dc0eba1fc04c2e2f3eb73070cd66fe4933b543f6d81e5966ede2717228adbcc8c8af586866d46d1c0cd4b353a1ac716d66b6974d478bcfa755835b6e7016559a10af0d488752b70d3362a36c484a143ce92a0484bfa03afb8c0e0ac4aab866c2f383808c40ea7227f17716123079307e1897ea54c3943e6e5271e1ab41424507253af6145297b1b21c51032400c924538ffed61b66460a8fd6aee24c71446766c27918c3e47ecbc4a2d8e54754efd86eba9efa646f78887abb09aaae7a61ed57ead131ff71ec095c36f96d256fab880eacf0365ed13e7d002dedaad5391e3324dc73b6b01270e4b611b500e39d55631ea444157db2eaf013ae50404f1e04496921086df4b3c4a6b32b691012e49a67e60b60c9c59a89be3f8510df099129b8fab249b645c08b614317663cb449687fe8cb737f72dbd95e62569fa9b145f1034867cb7ef12ac5415d1975b7bdbc418be7be5f6b446d19cf6c433b1f94464430343dc7a42da82346d09918a51f64d8e7e81014953445b8bd743d219ff9680c39af90fc92936ed3d64e5edac43a56416335c01979b130a4561e92cfe0f503ad0f10ec9543139b4dce35585df35604015c74f118929d0e7a72697e928d3c738a958a512622d1f63d310fe50cd97ae98ed43b5fb33782fb2ccaccb62730be9909ab52cbfd76dad511d08e6e1411add5d131ba81ce61b71b503630e335d67251926f6b88f17a0af967e0028a16ab5a3fd1efb11c3ee57e121c08160f6705a271e9a48aaa28e11cbbc1b29078af73829e06a855800c0970d69626753a683f7a8f7b6670c132094b5d106c2e25845f33340fa95235238590800978c49cd47f8b0c8cf62fef2193d7970af36eed7943d48a435c9120d5032aa06e5cb7180d1cdf807599075118f33803dfb6900eeef310fe77f040498482bf653d8ad4246dfeeb21356ea6d608deee1e40d22482215f71050663d4f6a121f0dfd047bbacf57115e8d7812b7eb5fd6b82f03d3c4432900b81bd3212f8ce7d4193c5420c9c913522390b5e93c3a48302c3f372f51f6fcb904b167fa62c11670b600a4361659993084db6bafc5f720c63fefa1d45c27a54a15a602b2c0')

The note I created initially was ZICCLOUDSYNCINGOBJECT.Z_PK=121 and ZICNOTEDATA=41 so you can see how Apple marks the note for deletion and clears out the fields which might have information that should be protected. You can also see the overwriting of ZICNOTEDATA.ZDATA with a new protobuf and the insertion of a new entry into ZICCLOUDSYNCINGOBJECT and ZICNOTEDATA with the encrypted information.

How is it Decrypted?

The above probably is more than sufficient for someone that knows cryptography, but for the layman such as myself, I found I needed a lot of time reading the RFCs and looking at common implementations to understand it. Below I will describe the three basic steps that are taken to decrypt an Apple note and in doing so, how it works. Through this example, I will use a note from my test database with ID 17 (meaning ZICNOTEDATA.ZNOTE=17 and ZICCLOUDSYNCINGOBJECT.Z_PK=17) as my test data.

Step 1: Derive the Password-Based Key

Apple makes the starting point clear in their description, the use of Password-Based Key Derivation Function 2 (PBKDF2) with the SHA-256 digest algorithm to make a 16-byte key. However, PBKDF2 requires two other parameters to run, the number of iterations and the password salt. Both of those values are found in the ZICCLOUDSYNCINGOBJECT table, in the obviously named ZCRYPTOITERATIONCOUNT³ and ZCRYPTOSALT columns.

This SQLite query would let you would pull the necessary values out of the database to carry out step 1 for all encrypted notes:

SELECT ZICCLOUDSYNCINGOBJECT.Z_PK, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT 
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.ZISPASSWORDPROTECTED=1

In Ruby, this is what a method might look like⁴ if you wanted to pass in the user’s password and the crypto_salt and crypto_iterations from the database to derive the key:

require 'openssl'

def derive_key(password, crypto_salt, crypto_iterations)
  return OpenSSL::PKCS5.pbkdf2_hmac(password, 
                                    crypto_salt, 
                                    crypto_iterations, 
                                    16, # Apple uses 16-byte keys
                                    OpenSSL::Digest::SHA256.new) # Apple uses SHA-256
end

key_encrypting_key = derive_key("password", 
                                "\x11\x65\x10\x6b\x6b\x28\x8b\xda\x1e\x6e\xcb\x18\xe6\x5c\x78\x76", 
                                20000)
# key_encrypting_key => "\x65\x2b\xe8\x61\x43\x34\x8a\x6e\x01\x06\x09\x58\x80\xbc\xf3\x1b"

Step 2: Unwrap the Encryption Key

In the last part of the quoted text above, Apple says it “rewraps” all of the keys for existing notes when you change the password. This indicates that, similar to the APFS protections, Apple isn’t reencrypting all the data, they simply are rewrapping the key to decrypt the data. There are advantages to this, including being able to “delete” data very quickly by simply deleting the decryption key. One downside is a layman such as myself could easily waste some time trying to “decrypt” the key, instead of “unwrap” it⁵. Another “downside” (from Apple’s perspective) or advantage (from the perspective of someone who lost their password) is that it is relatively easy to try offline password attacks if you have the database, since you never have to actually decrypt all the data, just unwrap the key.

To unwrap the decryption key, you need an implementation of the AES Key Wrap algorithm. Do not use AES-GCM, even though it is mentioned in the Apple specs as that is not a key wrapping algorithm and will take you down many, wrong, rabbit holes⁵. Do not be confused with the 24-byte wrapped key in your database when the Apple blurb says there should be a 16-byte key, the AES Key Wrap adds an extra 8-bytes on to the key material during wrapping.

The only things the AES Key Wrap algorithm needs to unwrap a key are the wrapped key and the key-encrypting key (KEK). The wrapped key is stored in the ZICCLOUDSYNCINGOBJECT table in the obviously named ZCRYPTOWRAPPEDKEY column and the KEK is what we generated in step 1 using PBKDF2.

This SQLite query would let you would pull the necessary values out of the database to carry out step 2 for all encrypted notes:

SELECT ZICCLOUDSYNCINGOBJECT.Z_PK, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.ZISPASSWORDPROTECTED=1

Building on the Ruby example above, this is how one could use the aes_key_wrap gem to unwrap the wrapped key:

require 'aes_key_wrap'

# From Step 1
key_encrypting_key = "\x65\x2b\xe8\x61\x43\x34\x8a\x6e\x01\x06\x09\x58\x80\xbc\xf3\x1b"

def unwrap_key(wrapped_key, key_encrypting_key)
  return AESKeyWrap.unwrap(wrapped_key, key_encrypting_key)
end

unwrapped_key = unwrap_key("\x98\xc0\xe5\x6b\x43\xb5\x07\xe6\x0c\x54\x65\xec\x5e\x1b\xb0\xc7\x4b\x75\x6f\x7d\x4f\x4a\x9b\xff", 
                           key_encrypting_key)

# unwrapped_key = "\x02\x3a\xae\x7c\x45\x0a\x28\x3b\x23\xe3\xd7\xc1\x41\x6a\xd6\x44"

Side Quest 2.5: Get the Right Library

While I wasted a lot of time learning the difference between an AES Key Wrap and AES-GCM in step 2, a very specific Ruby versioning issue burned about a week of my time for step 3. My development box has been around a long time and it still runs Ruby 2.3 and if you read the main page for the OpenSSL gem it clearly says:

NOTE: If you are using Ruby 2.3 (and not Bundler), you must activate the gem version of openssl, otherwise the default gem packaged with the Ruby installation will be used

The layman who glossed over that warning to get into the specifics for the gem would likely pay for that mistake later⁵. While the first two steps worked fine, step 3 kept generating bad decrypts and throwing an OpenSSL::Cipher::CipherError because the authentication tag seemed not to be right. My troubleshooting involved all sorts of crazy changes, odd assumptions about maybe Apple using this value instead of that value, and wasting hours of a good friend’s time trying to get the code to work. Finally, I was going down the path of openssl being too old and yet again was googling for specific Ruby and OpenSSL issues when one of the cached pages had the note about Ruby 2.3 on it. As soon as I actually read that warning and added the appropriate line to the AppleDecrypter class to explicitly used the OpenSSL gem instead of the built-in library, things worked immediately.

The most important thing you can look at if you implement this in another language is the IV size. My code was failing because the AES-GCM specifications state an IV of 96-bits should be used, but the IV in Apple Notes is 128-bits. This would not be a problem except the built-in openssl library in Ruby’s standard libraries from Ruy 2.3 appears to only take the first 12-bytes of the IV if you give it a longer value. Whatever library or language you use, make sure you can tell your algorithm that you are giving it a 128-bit IV. Using only part of the IV will, guaranteed, waste a week of your life⁵.

Step 3: Decrypt the Note

The third and final step of decrypting is to actually decrypt the content. Per Apple’s blurb, the note content is encrypted with AES-GCM. Apple says it stores the encrypted note, the note’s attachments, the tag, and the initialization vector before deleting the original note. Functionally, what that means in the database is that a new entry is created in ZICNOTEDATA and ZICCLOUDSYNCINGOBJECT for a new note (as well as for any attachments) and the old ones are “marked for deletion.”⁶ Even though the row is not immediately deleted (just marked as such for future cleanup), the ZICNOTEDATA.ZDATA column for the original note is overwritten as well.

In order to decrypt this new value that is stored, you need to have the unwrapped_key that was produced in step 2, the tag which is stored in ZCRYPTOTAG, and the initialzation vector (IV) which is stored in ZCRYPTOINITIALIZATIONVECTOR. Both the tag and IV are also stored in the ZICNOTEDATA table, and appear to be consistent in both places.

This SQLite query would let you would pull the necessary values out of the database to carry out step 3 for all encrypted notes:

SELECT ZICCLOUDSYNCINGOBJECT.Z_PK, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG,
  ZICNOTEDATA.ZDATA
FROM ZICCLOUDSYNCINGOBJECT, ZICNOTEDATA
WHERE ZICCLOUDSYNCINGOBJECT.ZISPASSWORDPROTECTED=1 AND 
  ZICNOTEDATA.ZNOTE=ZICCLOUDSYNCINGOBJECT.ZCLOUDSTATE

Continuing on in our basic Ruby example, this is how you could use the openssl gem to perform the final decryption.

gem 'openssl'
require 'openssl'

# From Step 2
unwrapped_key = "\x02\x3a\xae\x7c\x45\x0a\x28\x3b\x23\xe3\xd7\xc1\x41\x6a\xd6\x44"

def decrypt_aes_gcm(key, iv, tag, data)
  decrypter = OpenSSL::Cipher.new('aes-128-gcm').decrypt
  decrypter.iv_len = 16
  decrypter.key = key
  decrypter.iv = iv
  decrypter.auth_tag = tag
  plaintext = decrypter.update(data) + decrypter.final
end

iv = "\x15\x1f\x64\xde\x7b\xe3\x4d\x15\xda\xcd\xae\xa9\xb3\x34\x71\xf9"
tag = "\x80\x6b\xf2\xbb\xd3\xbf\x83\xcf\x12\x40\xb0\x3e\x7c\x4d\x6a\xb1"
encrypted_data = "\x13\x1b\x03\x57\x1f\xc9\xec\x47\xef\x58\xe5\x8e\x21\xfc\xe5\xc1" + 
                 "\x0a\xa7\x3a\x62\xb9\xe5\x8a\x74\x3b\xcd\xcc\x3a\xff\x1e\xa8\xab" + 
                 "\x99\x64\xf4\x53\x5b\x85\x97\x73\x5f\x3d\xa5\xf6\xae\x63\xb9\x37" + 
                 "\x06\x25\xa2\x0d\x63\x3e\x9c\xf2\x98\x6d\x4d\x11\x89\x89\x12\x4f" + 
                 "\x0d\xdf\xee\x95\x6e\x47\xcb\x5c\xbc\x36\x17\xc5\x20\xb0\x75\x62" + 
                 "\x0b\x37\xae\x40\x56\xf3\xa1\xaf\x83\x35\x1f\xda\x63\x4d\xfb\x44" + 
                 "\x60\x55\xc7\x5f\x71\x43\xa5\x60\x01\x49\xdb\x33\x38\x93\xc0\xec" + 
                 "\xb0\xef\x39\x44\xe2\xa6\x45\x42\xe9\xa4\x37\x5b\xf1\x52\x68\x98" + 
                 "\x58\xfe\xd8\xb2\x1a\xde\xd0\xea\xb0\xaf\xb1\x11\x90"

plaintext = decrypt_aes_gcm(unwrapped_key, iv, tag, encrypted_data)

# plaintext = "\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x13\xe3\x60\x10\x9a" + 
#             "\xc1\xc8\xc1\x20\xc0\x20\x35\x91\x51\x48\xde\x35\x2f\xb9" + 
#             "\xa8\xb2\xa0\x24\x35\x45\xa1\x24\xb3\x24\x27\x95\x8b\x0b" + 
#             "\x21\x90\x94\x9f\x52\x29\x25\xc0\xc5\x02\x52\x0b\x54\x0d" + 
#             "\xa6\x35\x18\xc1\x22\x8c\x40\x11\x79\x29\x30\xad\xc1\x24" + 
#             "\x25\xc6\xc5\x01\x94\xfb\x0f\x04\xfc\x40\x75\x70\xb6\x92" + 
#             "\x0c\x97\x14\x97\xc0\xbb\x7f\x02\xb7\xa2\x2a\x9d\x55\x3b" + 
#             "\x76\xe5\x9e\x7a\xf4\x72\xfb\x1b\x21\x26\x0e\x79\x20\x66" + 
#             "\xd4\xe2\xe0\x10\x10\x02\x9a\x29\xc1\xa8\x05\xe2\xb1\x71" + 
#             "\xf0\x09\x31\x49\x30\x02\x00\xd1\x69\x5a\x2d\x9d\x00\x00\x00"

Look at that output! 0x1f 0x8b, that’s the start of a GZip file! At this point, our normal processing could take over as we would be expecting a GZipped protobuf in the ZICNOTEDATA.ZDATA column.

Step 4: Repeat

Ok, I lied, our normal processing can’t quite take over yet because as it parses the protobuf it will start to hit encrypted attachments and run into problems. As you hit attachments in an encrypted note, you’ll need to do the same 3 steps above for that object. Each type of attachment, as noted at the top is different in how you have to decryp it. The attachment will have different cryptographic variables from the note, but the same underlying password. Here are brief rundowns of how each encrypted attachment behaves:

How are Attachments Handled?

Tables

Tables are the most straight forward as all they do is take the value that would normally be in ZMERGEABLEDATA1, base64 encoded it, put it into a JSON object along with the value that used to be in the ZSUMMARY column, encrypt that JSON, and insert it into the ZENCRYPTEDVALUESJSON column. The same row will have all of your cryptographic settings and the content.

{
  "summary":"This\nIs\nFantastic\nEncryption\n", 
  "mergeableData":"H4sIAAAAAAAAE7VU30/TUBReu63r7oaUwvhx5YWCc6mBYE2M8Q0HKMjGLEONiQ+ju0CXrsOuU+AJJSHwZnww8UFNlPjgE6JR4y8SeTGSSEJiTEwIalQ0Maj/gHpXNsLWLezFm3Wn5ztf7rnf15tDW9hvZbSFscCPZYACNvxqZy0wFAvSTsjTFtbLNbVmVnOBv+yCFE2wZCuBoxVHK452HGkcKRzLoBsAY/c0i4TNsX00AffSJNvINfjFcGRAQf6Ekoqr7bKGJF1OqD1oUA8nRHloWIfLxCViiQC3CDAOeljq58Iz/IOMsWH67Eb0EQZCYISERvSRsBrQuPYXr3LM23rnSZpIP2wVlsjY3XdezlxvU546Oj5ffBd+hVGCZeC1ld7Jqd7wQuRBQ6ey6mGbAAPI1rQsG9y0ijAQB0aoLLKNZTexHFkEumOAJrE5NpbMy8iczJqT2f63E+t198evhgJvJuqveA893Li56cQ0Il3uiba7L5iJ6PLiWymjEWBFzhyNboy48pxIs4CJ5S7ohD0no3IyR05GQ6+wuX8l3m1XZjergdRhpCqDkFs8BqPlOTwWIxUZxA67xWMs2ZUs6iVVgpcUvtBkbf7XFcUQCzpUSRsbSV/soh0qGFBCB2DqYM3zMKu4GuvzZH0wkFqM1GQQG+wRu1lbeFgurjl943Y+ka2A5hNiL+vsjKh6JKnLUtEGNOMsoYFzB8kOjpajSNVlfYyrlrRC04SzJZEyyFGSJiYuJDlnf39Xe5caRaOcU9I2uUnOJSFFySR8jZSIt0RGRhTU4hfbwy3BvmAqPoC0AoU+XZPVId5jKqS78NV5MB53KZzwPhMuoiE5qSMtkFJ0+WRESaEehP3jG0tgmrrjNhpCfF1BOJiIou1C1ISOki2ZQWwudPmNgsBs/GHenxnz75l+En/96fujHwLDq6mxg7NHPPOtofjXQGxJME1TgVn5PTO7vha4fGNtv2uOOD0qmGarwNSc//Wl8uzRtsdNU75o6Pa8YJo6AnP83OLq5Kng83tDBz7MRb2KYJpXh+sBBKZTsiS9Gz/EP16BfnLtBgAA"
}

The right SQL statement to pull all the variables necessary to decrypt this object is:

SELECT ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR, ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY,
  ZICCLOUDSYNCINGOBJECT.ZENCRYPTEDVALUESJSON 
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER="[your table's UUID]"

URLs

URLs are also fairly easy and behave much like tables in that they have the sensitive information in a JSON object in the ZENCRYPTEDVALUESJSON column. The exception is instead of having mergeable data they have the actual URL with its summary and title.

{
  "summary":"Washington St\nWellesley Hills MA 02481\nUnited States", 
  "title":"Caffè Nero", 
  "urlString":"https://maps.apple.com/?address=339%20Washington%20St,%20Wellesley%20Hills,%20MA%20%2002481,%20United%20States&auid=8412910740992417343&ll=42.310336,-71.276833&lsp=9902&q=Caff%C3%A8%20Nero&\_ext=ChkKBQgEEOIBCgQIBRADCgQIBhAICgQIChAAEiYpQjl25iUnRUAx5mc3JRvSUcA5wA6cQkwoRUBBvhEkH1TRUcBQAw%3D%3D&t=m"
}

The right SQL statement to pull all the variables necessary to decrypt this object is:

SELECT ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR, ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY,
  ZICCLOUDSYNCINGOBJECT.ZENCRYPTEDVALUESJSON 
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER="[your URL's UUID]"

Images

Images are a bit harder to deal with because of how many rows they go across in the ZICCLOUDSYNCINGOBJECT table. You’ll have the attachment’s row, then a row for the media, then potentially a lot of rows for thumbnails. While the image data is encrypted within the file on disk, the image’s filename is kept in the ZENCRYPTEDVALUESJSON column on the media’s row. Importantly, the correct cryptographic settings to decrypt the contents within the file on disk are in the ZASSETCRYPTOTAG and ZASSETCRYPTOINITIALIZATIONVECTOR columns for the media row whereas to decrypt the JSON you would continue to use the ZCRYPTOTAG and ZCRYPTOINITIALIZATIONVECTOR columns.

On the Image object itself

The right SQL statement to pull all the variables necessary to decrypt this object’s JSON is:

SELECT ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR, ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY,
  ZICCLOUDSYNCINGOBJECT.ZENCRYPTEDVALUESJSON 
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER="[your image's UUID]"

{
  "summary":"   ", 
  "title":"CF-Favicon.tiff"
}

On the Image’s ZMedia row

The right SQL statements to pull all the variables necessary to decrypt the actual file on disk and the JSON is as follows. Note, the image’s filename will be the ZIDENTIFIER from the second query.

SELECT ZICCLOUDSYNCINGOBJECT.ZMEDIA
FROM ZICCLOUDSYNCINGOBJECT
WHERE ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER="[your image's UUID]"

SELECT ZICCLOUDSYNCINGOBJECT.ZASSETCRYPTOINITIALIZATIONVECTOR, 
  ZICCLOUDSYNCINGOBJECT.ZASSETCRYPTOTAG, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR, ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY,
  ZICCLOUDSYNCINGOBJECT.ZENCRYPTEDVALUESJSON, ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.Z_PK=[your ZMedia result above]

{
  "filename":"CF-Favicon.tiff"
}

Thumbnails

After discussing images, thumbnails make a lot more sense. They follow basically the same process, except they only have the one row of information, which simplifies things. Their files on disk are also encrypted, but using the normal ZCRYPTOTAG and ZCRYPTOINITIALIZATIONVECTOR columns.

The right SQL statement to pull all the variables necessary to decrypt this object’s JSON is:

SELECT ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR, ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY 
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER="[your thumbnail's UUID]"

Sketches

Sketches can be done either like tables parsing a protobuf from ZENCRYPTEDVALUESJSON, or images decrypting a file on disk from the ZMEDIA row, or both if you want to get everything. The only big difference is when you are parsing a file from disk using fallback images the tag and IV are in the ZFALLBACKIMAGECRYPTOTAG and ZFALLBACKIMAGECRYPTOINITIALIZATIONVECTOR columns.

The right SQL statements to pull all the variables necessary to decrypt the actual file on disk and the JSON are as follows.

SELECT ZICCLOUDSYNCINGOBJECT.ZMEDIA
FROM ZICCLOUDSYNCINGOBJECT
WHERE ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER="[your sketch's UUID]"

SELECT ZICCLOUDSYNCINGOBJECT.ZFALLBACKIMAGECRYPTOINITIALIZATIONVECTOR, 
  ZICCLOUDSYNCINGOBJECT.ZFALLBACKIMAGECRYPTOTAG, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR, ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT, ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT,
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOVERIFIER, ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY,
  ZICCLOUDSYNCINGOBJECT.ZENCRYPTEDVALUESJSON, ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZICCLOUDSYNCINGOBJECT.Z_PK=[your ZMedia result above]

Conclusion

Apple’s use of well-documented, public algorithms for its encryption of data at rest in the Notes application allows for straight forward decryption of notes, if the password is known. Even if the password is not known, public tools such as John the Ripper can easily recover the password you need to use your Apple notes. It is my hope that this article is useful for others in implementing the decrypting of Apple notes for your personal use on your own data.

Footnotes