TL;DR: This post looks at Apple CloudKit within the context of Apple Notes to help you understand how Apple stores data in iCloud and why you may be surprised to find CloudKit data shoved into a random database field.

Background

I recently ran into an odd bug in Apple Cloud Notes Parser. As I was testing what information appeared when users shared notes back and forth, decrypting notes resulted in a crash every time. The odd thing was I really hadn’t changed my test corpus much, just moved existing notes into albums shared via iCloud. This stack trace began a long trip down an interesting rabbit hole as it indicated we were trying to decrypt something using an initialization vector which was nil.

Traceback (most recent call last):
  39: from notes_cloud_ripper.rb:122:in `<main>`
  [snip] ...
  11: from ./lib/AppleNote.rb:178:in `block (2 levels) in replace_embedded_objects`
  10: from ./lib/AppleNote.rb:178:in `new`
   9: from ./lib/AppleNotesEmbeddedTable.rb:48:in `initialize`
   8: from ./lib/AppleNotesEmbeddedTable.rb:206:in `rebuild_table`
  [snip] ...
   2: from ./lib/AppleNotesEmbeddedTable.rb:210:in `block in rebuild_table`
   1: from ./lib/AppleDecrypter.rb:166:in `decrypt_with_password`
./lib/AppleDecrypter.rb:130:in `aes_gcm_decrypt`: undefined method `length` for nil:NilClass

The Hunt Begins

Because we are failing when we try to rebuild the table in this encrypted note, my first stop was looking at the end of the debug_log.txt file.

[notta@cuppa apple_cloud_notes_parser]$ tail output/2020_10_14-06_53_09/debug_log.txt 
DEBUG -- : Apple Decrypter: Apple Note: 83 Attempting decryption with password password
DEBUG -- : Apple Decrypter: Apple Note: 83 generated a decrypt using password password
DEBUG -- : Rip Note: Ripping note from Note ID 84
DEBUG -- : Rip Note: Looking up account for 9
DEBUG -- : Rip Note: Looking up folder for 67
DEBUG -- : Apple Decrypter: Apple Note: 84 Attempting decryption with password password
DEBUG -- : Apple Decrypter: Apple Note: 84 generated a decrypt using password password
DEBUG -- : AppleNote: Note 84 replacing attachment C93C5BE6-9212-46F6-939F-331844B9FEA0
DEBUG -- : Note 84: Created a new Embedded Object of type com.apple.notes.table
DEBUG -- : Apple Decrypter: AppleNotesEncryptedTable C93C5BE6-9212-46F6-939F-331844B9FEA0

From the log file, we can see the last note I was trying to read was note #84 and it specifically died trying to deal with the table kept in ZICCLOUDSYNCINGOBJECT.ZIDENTIFIER='C93C5BE6-9212-46F6-939F-331844B9FEA0'. The next step is to check on the encryption variables we are expecting from that row in NoteStore.sqlite.

SELECT hex(ZICCLOUDSYNCINGOBJECT.ZCRYPTOINITIALIZATIONVECTOR) as iv, 
  hex(ZICCLOUDSYNCINGOBJECT.ZCRYPTOTAG) as tag, 
  hex(ZICCLOUDSYNCINGOBJECT.ZCRYPTOSALT) as salt, 
  ZICCLOUDSYNCINGOBJECT.ZCRYPTOITERATIONCOUNT as iterations, 
  hex(ZICCLOUDSYNCINGOBJECT.ZCRYPTOWRAPPEDKEY) as wrapped_key
FROM ZICCLOUDSYNCINGOBJECT 
WHERE ZIDENTIFIER='C93C5BE6-9212-46F6-939F-331844B9FEA0'

iv	tag	salt	iterations	wrapped_key
NULL	NULL	C15DFD1B4D95A86EA3A16334528F619A	20000	7EDB7DD3B1531D9E34E4A34F9D86D77AF92170BD96894A87

Sure enough, the initialization vector is null, and what’s odd about that result is you can’t decrypt the object without knowing the iv as a starting point. I had run into a similar issue when dealing with other embedded objects as they sometimes had values in other columns I had not expected. In this case, however, I was certain that I was accurately pulling the values from the right columns to decrypt a table because other tables were decrypting just fine. I also was certain that the values had to exist, because I was looking at the table on my iPhone in all its glory, so somehow Apple Notes knew what the iv was.

Forza Bruta

At this point, I fell back to my favorite solution, brute force. I had already looked at all of the columns which were related to either of these values, since there is fairly consistent naming between the columns that hold the cryptographic variables, and they were all null. That meant that either I was looking in completely the wrong place, or it was embedded within something inside a different blob on that row. This was somewhat the situation I created SQLite Miner for and I ran it against the example NoteStore.sqlite to see what might turn up¹.

[notta@cuppa apple_cloud_notes_parser]$ perl ~/perl/sqlite_miner/sqlite_miner.pl \
--file=output/2020_10_14-06_53_09/NoteStore.sqlite \
--export \
--protobufs

SQLite Miner 1.2.4 - Copyright (C) 2017 Jon Baumann, Ciofeca Forensics
  This program comes with ABSOLUTELY NO WARRANTY;
  This is free software, and you are welcome to redistribute it under certain conditions.
  See http://www.gnu.org/licenses/

Mining: output/2020_10_14_NoteStore.sqlite/NoteStore.investigated.sqlite

#######################################################
1 SQLite file mined, 234 potential blob files identified and exported in 18.0503 seconds.
Result file: output/2020_10_14_NoteStore.sqlite/results.csv

output/2020_10_14_NoteStore.sqlite/NoteStore.investigated.sqlite
  ACHANGE table:
    ZCOLUMNS column: 6 protobuf
  ZICCLOUDSYNCINGOBJECT table:
    ZCRYPTOINITIALIZATIONVECTOR column: 1 protobuf
    ZSERVERSHAREDATA column: 4 BPLIST
    ZUNAPPLIEDENCRYPTEDRECORD column: 6 BPLIST
    ZREPLICAIDTOUSERIDDICTDATA column: 51 protobuf
    ZLASTVIEWEDTIMESTAMPDATA column: 9 protobuf
    ZMERGEABLEDATA column: 1 GZIP
    ZUSERSPECIFICSERVERRECORDDATA column: 13 BPLIST
    ZMERGEABLEDATA1 column: 4 GZIP
    ZSERVERRECORDDATA column: 101 BPLIST
  ZICNOTEDATA table:
    ZDATA column: 36 GZIP, 1 ZLIB
  Z_METADATA table:
    Z_PLIST column: 1 BPLIST
#######################################################

While there are a few false hits on protobufs, there are some interesting binary plists in columns I had not investigated yet. Specifically, the ZSERVERSHAREDATA, ZUNAPPLIEDENCRYPTEDRECORD, ZUSERSPECIFICSERVERRECORDDATA, and ZSERVERRECORDDATA columns seemed interesting as they all had binary plists and names that seemed interesting. Since the table we were looking at before has a ZICCLOUDSYNCINGOBJECT.Z_PK of 82, we can check if any of the exports from those columns potentially held our missing data.

[notta@cuppa apple_cloud_notes_parser]$ ls output/2020_10_14_NoteStore.sqlite/exports/ \
| grep 82.blob

ZICCLOUDSYNCINGOBJECT-ZSERVERRECORDDATA-Z_PK-82.blob.bplist
ZICCLOUDSYNCINGOBJECT-ZUNAPPLIEDENCRYPTEDRECORD-Z_PK-82.blob.bplist

Digging In

That’s interesting! We have two blobs that were binary plists on our troublesome record, but what do they hold? It turns out, a significant amount of information. Here is what the first one looks like:

[notta@cuppa apple_cloud_notes_parser]$ ruby ~/ruby/bplister/plist_parse.rb \
output/2020_10_14_NoteStore.sqlite/exports/\
ZICCLOUDSYNCINGOBJECT-ZSERVERRECORDDATA-Z_PK-82.blob.bplist

{"$version"=>100000,
 "$archiver"=>"NSKeyedArchiver",
 "$top"=>
  {"TombstonedPublicKeyIDs"=>0,
   "HasUpdatedParentReference"=>false,
   "ChainProtectionData"=>0,
   "KnownToServer"=>true,
   "DisplayedHostname"=>0,
   "BaseToken"=>0,
   "WantsChainPCSKey"=>false,
   "RecordCtime"=>9,
   "RoutingKey"=>0,
   "ProtectionDataEtag"=>0,
   "PreviousProtectionDataEtagFromUnitTest"=>0,
   "ConflictLoserEtags"=>0,
   "PreviousProtectionDataEtag"=>0,
   "RecordType"=>1,
   "CreatorUserRecordID"=>11,
   "ParentReference"=>21,
   "ShareEtag"=>0,
   "PCSKeyID"=>0,
   "ZoneishKeyID"=>0,
   "MutableEncryptedPublicSharingKey"=>0,
   "ETag"=>19,
   "PreviousShareReference"=>0,
   "ModifiedByDevice"=>18,
   "ProtectionData"=>0,
   "UseLightweightPCS"=>false,
   "ShareReference"=>0,
   "URL"=>0,
   "ChainParentPublicKeyID"=>20,
   "LastModifiedUserRecordID"=>16,
   "RecordMtime"=>9,
   "WantsPublicSharingKey"=>false,
   "ZoneProtectionDataEtag"=>0,
   "WasCached"=>false,
   "ChainPrivateKey"=>0,
   "Permission"=>1,
   "RecordID"=>2,
   "HasUpdatedShareReference"=>false,
   "PreviousParentReference"=>25},
 "$objects"=>
  ["$null",
   "Attachment",
   {"$class"=>8, "RecordName"=>3, "ZoneID"=>4},
   "C93C5BE6-9212-46F6-939F-331844B9FEA0",
   {"ZoneName"=>5, "ownerName"=>6, "$class"=>7},
   "Notes",
   "__defaultOwner__",
   {"$classname"=>"CKRecordZoneID",
    "$classes"=>["CKRecordZoneID", "NSObject"]},
   {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
   {"NS.time"=>617585043.778, "$class"=>10},
   {"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]},
   {"$class"=>8, "RecordName"=>12, "ZoneID"=>13},
   "__defaultOwner__",
   {"ZoneName"=>14, "ownerName"=>15, "$class"=>7},
   "_defaultZone",
   "__defaultOwner__",
   {"$class"=>8, "RecordName"=>17, "ZoneID"=>13},
   "__defaultOwner__",
   "Mark’s iPhone",
   "8b",
   "\x08\xA3\x1F\xDF\v\xB7\x87\x81,\x9A\xFFD\xC7/\x8B\xF4\x14\x95\x9E\xF0",
   {"$class"=>24, "referenceAction"=>0, "recordID"=>22},
   {"$class"=>8, "RecordName"=>23, "ZoneID"=>4},
   "7D35FAE7-E05A-49D9-B05A-B372E211FEC6",
   {"$classname"=>"CKReference", "$classes"=>["CKReference", "NSObject"]},
   {"$class"=>24, "referenceAction"=>0, "recordID"=>26},
   {"$class"=>8, "RecordName"=>27, "ZoneID"=>4},
   "7D35FAE7-E05A-49D9-B05A-B372E211FEC6"]}

While this doesn’t look very nice, you can already see some interesting values, such as “Mark’s iPhone” and the ZIDENTIFIER of the table we were looking into, C93C5BE6-9212-46F6-939F-331844B9FEA0. This led to the first hurdle, understanding and parsing NSKeyedArchives.

The First Hurdle: NSKeyedArchives

Because I do not write code for any Apple platform, I’m often playing from behind slightly on recognizing obvious answers. In this case, I’m sure someone who regularly writes for the iOS platform is slapping their head saying “That’s easy!” but it took me a minute. With the keyword “NSKeyedArchiver” near the top of the file, I googled that and came across the Apple Developer Documentation for the structure as the first result. A bit further in the results I also came across a great entry from mac4n6 in 2016 (showing how late I am to this game). While that article gave me enough knowledge to know what I wanted to do, it revolved around using XCode or plutil to view or convert files, not actual interaction with the object itself.

Don’t Reinvent The Wheel

Apple Cloud Notes Parser is written in Ruby, so I was debating either writing classes in Ruby to include in this codebase, but first decided to look for an existing gem. As it turns out, keyed_archive² existed as a gem, but had not been updated in 7 years. During that time, the underlying formats had been updated and the gem did not fare well when trying to parse this example. It also did not allow for loading data directly from a variable, only from disk, and I did not want to have to save all the NSKeyedArchives on disk as temporary files. Thankfully, in open source every problem is just a pull request away from a solution, so I did just that and ended up a new maintainer of the code on Github and Ruby Gems.

Once keyed_archive supported current formats, I added a method to unpack the archive and spit out the pairing of keys to values, KeyedArchive#unpacked_top(). With that and a helper script I was able to get a much better look into the structure.

[notta@cuppa apple_cloud_notes_parser]$ ruby ~/ruby/bplister/nskeyed_archive_list.rb \
output/2020_10_14_NoteStore.sqlite/exports/\
ZICCLOUDSYNCINGOBJECT-ZSERVERRECORDDATA-Z_PK-82.blob.bplist

{"TombstonedPublicKeyIDs"=>nil,
 "HasUpdatedParentReference"=>false,
 "ChainProtectionData"=>nil,
 "KnownToServer"=>true,
 "DisplayedHostname"=>nil,
 "BaseToken"=>nil,
 "WantsChainPCSKey"=>false,
 "RecordCtime"=>
  {"NS.time"=>617585043.778,
   "$class"=>{"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]}},
 "RoutingKey"=>nil,
 "ProtectionDataEtag"=>nil,
 "PreviousProtectionDataEtagFromUnitTest"=>nil,
 "ConflictLoserEtags"=>nil,
 "PreviousProtectionDataEtag"=>nil,
 "RecordType"=>"Attachment",
 "CreatorUserRecordID"=>
  {"$class"=>
    {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
   "RecordName"=>"__defaultOwner__",
   "ZoneID"=>
    {"ZoneName"=>"_defaultZone",
     "ownerName"=>"__defaultOwner__",
     "$class"=>
      {"$classname"=>"CKRecordZoneID",
       "$classes"=>["CKRecordZoneID", "NSObject"]}}},
 "ParentReference"=>
  {"$class"=>
    {"$classname"=>"CKReference", "$classes"=>["CKReference", "NSObject"]},
   "referenceAction"=>nil,
   "recordID"=>
    {"$class"=>
      {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
     "RecordName"=>"7D35FAE7-E05A-49D9-B05A-B372E211FEC6",
     "ZoneID"=>
      {"ZoneName"=>"Notes",
       "ownerName"=>"__defaultOwner__",
       "$class"=>
        {"$classname"=>"CKRecordZoneID",
         "$classes"=>["CKRecordZoneID", "NSObject"]}}}},
 "ShareEtag"=>nil,
 "PCSKeyID"=>nil,
 "ZoneishKeyID"=>nil,
 "MutableEncryptedPublicSharingKey"=>nil,
 "ETag"=>"8b",
 "PreviousShareReference"=>nil,
 "ModifiedByDevice"=>"Mark’s iPhone",
 "ProtectionData"=>nil,
 "UseLightweightPCS"=>false,
 "ShareReference"=>nil,
 "URL"=>nil,
 "ChainParentPublicKeyID"=>
  "\x08\xA3\x1F\xDF\v\xB7\x87\x81,\x9A\xFFD\xC7/\x8B\xF4\x14\x95\x9E\xF0",
 "LastModifiedUserRecordID"=>
  {"$class"=>
    {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
   "RecordName"=>"__defaultOwner__",
   "ZoneID"=>
    {"ZoneName"=>"_defaultZone",
     "ownerName"=>"__defaultOwner__",
     "$class"=>
      {"$classname"=>"CKRecordZoneID",
       "$classes"=>["CKRecordZoneID", "NSObject"]}}},
 "RecordMtime"=>
  {"NS.time"=>617585043.778,
   "$class"=>{"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]}},
 "WantsPublicSharingKey"=>false,
 "ZoneProtectionDataEtag"=>nil,
 "WasCached"=>false,
 "ChainPrivateKey"=>nil,
 "Permission"=>"Attachment",
 "RecordID"=>
  {"$class"=>
    {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
   "RecordName"=>"C93C5BE6-9212-46F6-939F-331844B9FEA0",
   "ZoneID"=>
    {"ZoneName"=>"Notes",
     "ownerName"=>"__defaultOwner__",
     "$class"=>
      {"$classname"=>"CKRecordZoneID",
       "$classes"=>["CKRecordZoneID", "NSObject"]}}},
 "HasUpdatedShareReference"=>false,
 "PreviousParentReference"=>
  {"$class"=>
    {"$classname"=>"CKReference", "$classes"=>["CKReference", "NSObject"]},
   "referenceAction"=>nil,
   "recordID"=>
    {"$class"=>
      {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
     "RecordName"=>"7D35FAE7-E05A-49D9-B05A-B372E211FEC6",
     "ZoneID"=>
      {"ZoneName"=>"Notes",
       "ownerName"=>"__defaultOwner__",
       "$class"=>
        {"$classname"=>"CKRecordZoneID",
         "$classes"=>["CKRecordZoneID", "NSObject"]}}}}}

With clearer output, we can see that the ZIDENTIFIER we were looking for is in the RecordID section and “Mark’s iPhone” is identified as ModifiedByDevice. There are other potentially interesting keys in here, such as ParentReference, which includes a UUID that happens to be the ZIDENTIFIER for the actual note this table was in: 7D35FAE7-E05A-49D9-B05A-B372E211FEC6.

Getting to the Bottom of it

Now that we have a good way of viewing the NSKeyedArchive, we can look at the other potentially interesting blob, ZUNAPPLIEDENCRYPTEDRECORD.

[notta@cuppa apple_cloud_notes_parser]$ ruby ~/ruby/bplister/nskeyed_archive_list.rb \
output/2020_10_14_NoteStore.sqlite/exports/\
ZICCLOUDSYNCINGOBJECT-ZUNAPPLIEDENCRYPTEDRECORD-Z_PK-82.blob.bplist

{"root"=>
  {"TombstonedPublicKeyIDs"=>nil,
   "HasUpdatedParentReference"=>false,
   "ChainProtectionData"=>nil,
   "KnownToServer"=>true,
   "DisplayedHostname"=>nil,
   "BaseToken"=>nil,
   "WantsChainPCSKey"=>false,
   "RecordCtime"=>
    {"NS.time"=>617585043.778,
     "$class"=>{"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]}},
   "RoutingKey"=>nil,
   "ProtectionDataEtag"=>nil,
   "PreviousProtectionDataEtagFromUnitTest"=>nil,
   "ConflictLoserEtags"=>nil,
   "PreviousProtectionDataEtag"=>nil,
   "RecordType"=>"Attachment",
   "CreatorUserRecordID"=>
    {"$class"=>
      {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
     "RecordName"=>"__defaultOwner__",
     "ZoneID"=>
      {"ZoneName"=>"_defaultZone",
       "ownerName"=>"__defaultOwner__",
       "$class"=>
        {"$classname"=>"CKRecordZoneID",
         "$classes"=>["CKRecordZoneID", "NSObject"]}}},
   "ParentReference"=>
    {"$class"=>
      {"$classname"=>"CKReference", "$classes"=>["CKReference", "NSObject"]},
     "referenceAction"=>nil,
     "recordID"=>
      {"$class"=>
        {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
       "RecordName"=>"7D35FAE7-E05A-49D9-B05A-B372E211FEC6",
       "ZoneID"=>
        {"ZoneName"=>"Notes",
         "ownerName"=>"__defaultOwner__",
         "$class"=>
          {"$classname"=>"CKRecordZoneID",
           "$classes"=>["CKRecordZoneID", "NSObject"]}}}},
   "ShareEtag"=>nil,
   "PCSKeyID"=>nil,
   "ZoneishKeyID"=>nil,
   "MutableEncryptedPublicSharingKey"=>nil,
   "ETag"=>"8b",
   "PreviousShareReference"=>nil,
   "ModifiedByDevice"=>"Mark’s iPhone",
   "ProtectionData"=>nil,
   "UseLightweightPCS"=>false,
   "ShareReference"=>nil,
   "$class"=>{"$classname"=>"CKRecord", "$classes"=>["CKRecord", "NSObject"]},
   "URL"=>nil,
   "ChainParentPublicKeyID"=>
    "\x08\xA3\x1F\xDF\v\xB7\x87\x81,\x9A\xFFD\xC7/\x8B\xF4\x14\x95\x9E\xF0",
   "LastModifiedUserRecordID"=>
    {"$class"=>
      {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
     "RecordName"=>"__defaultOwner__",
     "ZoneID"=>
      {"ZoneName"=>"_defaultZone",
       "ownerName"=>"__defaultOwner__",
       "$class"=>
        {"$classname"=>"CKRecordZoneID",
         "$classes"=>["CKRecordZoneID", "NSObject"]}}},
   "RecordMtime"=>
    {"NS.time"=>617585043.778,
     "$class"=>{"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]}},
   "WantsPublicSharingKey"=>false,
   "EncryptedValueStore"=>
    {"ChangedKeys"=>
      {"NS.objects"=>[],
       "$class"=>
        {"$classname"=>"NSMutableSet",
         "$classes"=>["NSMutableSet", "NSSet", "NSObject"]}},
     "OriginalValues"=>
      {"NS.keys"=>[],
       "NS.objects"=>[],
       "$class"=>
        {"$classname"=>"NSMutableDictionary",
         "$classes"=>["NSMutableDictionary", "NSDictionary", "NSObject"]}},
     "RecordValues"=>
      {"NS.keys"=>["UTIEncrypted", "PasswordHint"],
       "NS.objects"=>
        [{"EncryptedData"=>"com.apple.notes.table",
          "$class"=>
           {"$classname"=>"CKEncryptedData",
            "$classes"=>["CKEncryptedData", "NSObject"]}},
         {"EncryptedData"=>"password",
          "$class"=>
           {"$classname"=>"CKEncryptedData",
            "$classes"=>["CKEncryptedData", "NSObject"]}}],
       "$class"=>
        {"$classname"=>"NSMutableDictionary",
         "$classes"=>["NSMutableDictionary", "NSDictionary", "NSObject"]}},
     "$class"=>
      {"$classname"=>"CKEncryptedRecordValueStore",
       "$classes"=>
        ["CKEncryptedRecordValueStore", "CKRecordValueStore", "NSObject"]}},
   "ValueStore"=>
    {"ChangedKeys"=>
      {"NS.objects"=>[],
       "$class"=>
        {"$classname"=>"NSMutableSet",
         "$classes"=>["NSMutableSet", "NSSet", "NSObject"]}},
     "OriginalValues"=>
      {"NS.keys"=>[],
       "NS.objects"=>[],
       "$class"=>
        {"$classname"=>"NSMutableDictionary",
         "$classes"=>["NSMutableDictionary", "NSDictionary", "NSObject"]}},
     "RecordValues"=>
      {"NS.keys"=>
        ["CryptoIterationCount",
         "CroppingQuadBottomLeftY",
         "UTI",
         "CroppingQuadBottomRightY",
         "FallbackSubtitleMac",
         "MinimumSupportedNotesVersion",
         "LastModificationDate",
         "EncryptedValues",
         "CroppingQuadTopRightX",
         "OcrSummaryVersion",
         "HandwritingSummaryVersion",
         "CryptoWrappedKey",
         "CryptoInitializationVector",
         "CreationDate",
         "CroppingQuadTopRightY",
         "FallbackSubtitleIOS",
         "CryptoTag",
         "ImageClassificationSummaryVersion",
         "CroppingQuadBottomRightX",
         "Deleted",
         "CroppingQuadBottomLeftX",
         "CryptoSalt",
         "Note",
         "FallbackTitle",
         "CroppingQuadTopLeftX",
         "CroppingQuadTopLeftY",
         "Orientation",
         "ImageFilterType"],
       "NS.objects"=>
        [20000,
         0.0,
         "com.apple.notes.table",
         0.0,
         "Upgrade macOS to view this table.",
         2,
         {"NS.time"=>617584971.683204,
          "$class"=>
           {"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]}},
         "\xA7\xCEC\xFF\xF7IO\xCA\x8A\xC1\xAC\x13/\xC90\xA4\x95\xB5\x91_\b_p\xDE" + 
         "a[\x10J\xB0h\a\xB7\xCE\x9Bx\xFA\xC1s\x8D\xE9\x8D\b\xEB=\xE8\xC4\xB3R\xAC" + 
         "\xBE\x1AV\xA6\xF9\xB4\x13)1\xB2\xF2\x15z\xFE\xFE\x1A\xD7\xB8\xAAH\x96IO" + 
         "\xD1\xBA\x11\x06\xCD \x1A\x16v\xCF\xA5\a\xEF-\e\xD5+\xBD\xA2\xE8\xE5\x02" + 
         "\xB90\xA0\xB8\"\xE1\x06\xFA\x94\x18z\xFD\x85\xEC\xED\"\xD1{b\\\xE4L\xF9" + 
         "\x9A:W\xF6\xE7\n" +
         "\x00D(!\xF4\xDE\xF8\x8B|\x7Ft\xC7A\x10/V\xC6|E\xB1\xF41\x83\e\xC8\x06\x87" + 
         "\x13\xF2\x15G1\xD1|\x1F\xC8\x18\xF5A\e6+h\xEE\x9CH\x13p\x13_R\f\x87m!e\x14" + 
         "\xD4]U\x9C:\x10O\xAF1\e\x13\x0F\xE5Y\xB7 -\xFF\x04z\x86\xD3\xE2\xAA\xB7B" + 
         "\xA9n\xD8B\x10Ng~\xB0s\x00\xDE\xD8\"YE\xCF2\xB4\xFE=\x1E?\xEDIN\xCAM\x85" + 
         "c!\xE5\xBBa\f\x95\xDB\xF6Eh\xBC_\xC7\xF4\xE9\xCD\xE8\x0EU\x13r\xA9&9)TW>@" + 
         "\x0F<0\xB8Y\xE8\x95\x05$\x8F(\xE8\xAEZ\xFB\xC8}yl\xE9\xFEH\xCA\xBF\xE9\`" + 
         "\x9E\xF10\x19?\enV\x17\x03\xD6\xAC\xCE._\xD6\xE7\x9E&\`:\x8C\xFC\x89\x06" + 
         "\x02\x88\x18j0\xD2\x0E{\xBC\x1A\xFB\b\xB6\xC9\xD7\xA0\xF5k\xDC\xCF\x14Q" + 
         "\x97\xFA\xB0h\xC2\xBC\xCD\x85\xFB\x96\v+m\xBB\xDD\xA1h-4\xA6{\xCA\xCE\xA8" + 
         "\xE5&\xFB8\x7F\xC2\a\xF3x;\xD7+\x03\xB8\x95\xACB\xA7\x991\x9D\xFD\xEF\xCB" + 
         "\xD6Q\xFC\xD4\xAD\xC9\`/\xB98\xD6\xD3r\x13dH\xAC\x8B\xF3\xF43e\xDD\xFFH\x80" + 
         "E \x19\xA2\x8FS\tg\x93~E'\xB4\xC9\xCC\xDD\x86\x95\n" +
         "\xEETx7B\x9C\x9B4mg\x90=f\x1F\xC9\x12G\xE2\xC2\x00\x0E0{\x93\x97\x02\xB8" + 
         "\x87L\xD6\x9E\xFA\xBB6\x9A\xAFNs\xEE\xD0\xF9\x9DN\x00\xC1\xD3}\xDB\xD9" + 
         "\xF3N\xEF\xBC{\x13\x05;v\xB8\x11L\xD2\x91\x8D\x9B\x9C\xA6\xCAh\xFE\xF6" + 
         "\x8B\x9F\xB3\x8C\x85\xD4\x05\x13\xDDvz\xB3\xF58G\x8E\xA3r\x15\xAD\x95"+ 
         "\x98H\xE9\xA3\xA0\x88\`d3\x87q\xE4\x91\xEA>\f\x85\vt\xD7\x94\xDA7L5f\x90"+ 
         "C\xCA}\xF8\xED\x14\x8E\xC8er\xCD\xCAD\x00,\x95\x19\xCFY\xA9L)\x94\x12\xC2"+ 
         "\xC7[t:\xCA\x16\xBAe\xCD\xE6\xD6\xD0\x1E\x13\xC2\xBAa\xFF\xF1\x9Bk.\xCE:" + 
         "\xF5V\xC4\xB7\xF9\x85\x06\xF2A\xD7\ts7\xA3\xACb\x15_\xE9\xA7\xFD\xAE_"+ 
         "\xDE\xF0\x02=\xA4(\xD9\x9C\x95\xFF'\x82\xCC\x1E\xB14=\xDD\xC3@\xC1\\\xF8" + 
         "\x10\xDE\xABX\x9C|\x95b7\xD8o\x10\x7F>\xC0'\a&{\\\x9D\x0F&\x12\x04q\x0EO" + 
         "2J\x9A\x93 \eIgM\xDF\xCE\xF7v\xD5\xE8\x96\x919H\x03gS\xFC\xC0\xB4\x00\x93"+ 
         "\x99\xFC(\xF4\xE3n\x05\xA5\x00\xF9X\xCF\xF2\x93\xE3\xBD\x9CG\"xl>~~\x02"+ 
         "\x91\xF3+\x13\xF0\xED\x84RrK\x16E\x16\xBB\x7FS\xC7\x97\xECf[\x99-\xAA|Y"+ 
         "\xFDs\xB0\xEA\xC6}\x1F\xE05\xD9\xF5_\xD1\xC4s\x89\xAE>L\xBE(p\xD2+\x00" + 
         "bt&\xB3\x04QE\xAB\x90\xCD\x05\xE7\x91g\x1C\x8F\xF5\x9DG\xDF\x92n\x98\x10" + 
         "\xA7\x8D\xDB\xF2J]\xE9\x86\x10{\xAA\x1A_y'\xAB\xBCt\xD9\xB9\xD2\x1F\x00"+ 
         "ah\xA9}\xAC)8\xA5(\x90k\xAC;Y\x8F:\xA1\x90/\xC2>\x81w\x02R\x9B\xC0\xC9z"+ 
         "\xC5\"\xE5\xC2\x8D\xA0\xEE\x11\xF1\x98\x82Re\xA3<%\xF0\x8D\xA3\xE55g\\" + 
         "\x05\xBFh\xE1\x81\xDC\x1A\x13\a\xDF\xA5\x8C\xE4\xFC\x88\xE6H\\\xB0H\xCB"+ 
         "V\xEE\x13\x1Fk/\x83\xC2\xDE\x95\x8A:\xDFz\xE7\xEB\xAF\xE9\xB7\x13\t\n" +
         "K\xA6\x1A\xAC\\\xF1\xD99\x94\x06\xBC\x82\xD6;\xE1\x01\xE6CZ\x17<\x14" + 
         "\xF7O\xE3\xA7\t4]\f%\xD9=\xE7\xF2\x1D\xA7\b\xDE\xDB\xD2C\x80\xE7\xF6" + 
         "\xD4\xF5\xA7\tX\x06\xD5\e\xDB\x9D\xA9\"f\x04~*\x00 \xCA\x12\x1As9\x93"+ 
         "\xFEcm\xD9[\x88\xAFv\xF8\xD8\xE0zKN\xA0\x8B.\xEB\xB7(\x99\xCD\xAE!\xA6" + 
         "\xE1\x03\x1A\xE1\r}\x96\xFC\xF4x\xFA\xAAd\x05\xCBB\x9FZW\xA2\xD2\xB1" + 
         "\x9D\xA9\x02\x9Dy\a\x04\xA6\xCB\x84\xBB\xBF\xC4\fPu\xD3\xB0\xD3n\xDF" + 
         "\x91F!\x13\xA3\x17\x9E\xC0\xD7\xEF\xEA%\xA7:-'\xB9\a\x7F\x852\x8A\xFD"+ 
         "*=NEw\xB7o \xE9\xDC\xB5HW\xB2|\xFAO\xE7\x8D\xBF\xEB\x1E\xFB\xFE\xD94" + 
         "\xEA6\xA4\x12\xA6D\xA8\x9C\x16E\b\x91\x82\x0252\xEE\xB6e\xA0\xF3.\x8A"+ 
         "V \x1DM\xC6\xC5\xC2]m\x9A\xA4\xB7hm\x8B4'\xAAw\xDFU",
         1.0,
         0,
         0,
         "~\xDB}\xD3\xB1S\x1D\x9E4\xE4\xA3O\x9D\x86\xD7z\xF9!p\xBD\x96\x89J\x87",
         "\xDD\x00\x0E\x9E)w\xCEU\xF8['l\xC9J\eK",
         {"NS.time"=>617584960.062718,
          "$class"=>
           {"$classname"=>"NSDate", "$classes"=>["NSDate", "NSObject"]}},
         1.0,
         "Upgrade iOS to view this table.",
         "\x83\xF04\xB8\xDF\x1FG\xBF\x01\x10HR\x98\xE9\xAF6",
         0,
         1.0,
         0,
         0.0,
         "\xC1]\xFD\eM\x95\xA8n\xA3\xA1c4R\x8Fa\x9A",
         {"$class"=>
           {"$classname"=>"CKReference",
            "$classes"=>["CKReference", "NSObject"]},
          "referenceAction"=>"Attachment",
          "recordID"=>
           {"$class"=>
             {"$classname"=>"CKRecordID",
              "$classes"=>["CKRecordID", "NSObject"]},
            "RecordName"=>"7D35FAE7-E05A-49D9-B05A-B372E211FEC6",
            "ZoneID"=>
             {"ZoneName"=>"Notes",
              "ownerName"=>"__defaultOwner__",
              "$class"=>
               {"$classname"=>"CKRecordZoneID",
                "$classes"=>["CKRecordZoneID", "NSObject"]}}}},
         "Table",
         0.0,
         1.0,
         0,
         0],
       "$class"=>
        {"$classname"=>"NSMutableDictionary",
         "$classes"=>["NSMutableDictionary", "NSDictionary", "NSObject"]}},
     "$class"=>
      {"$classname"=>"CKRecordValueStore",
       "$classes"=>["CKRecordValueStore", "NSObject"]}},
   "PluginFields"=>nil,
   "ZoneProtectionDataEtag"=>nil,
   "WasCached"=>false,
   "ChainPrivateKey"=>nil,
   "Permission"=>1,
   "RecordID"=>
    {"$class"=>
      {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
     "RecordName"=>"C93C5BE6-9212-46F6-939F-331844B9FEA0",
     "ZoneID"=>
      {"ZoneName"=>"Notes",
       "ownerName"=>"__defaultOwner__",
       "$class"=>
        {"$classname"=>"CKRecordZoneID",
         "$classes"=>["CKRecordZoneID", "NSObject"]}}},
   "HasUpdatedShareReference"=>false,
   "PreviousParentReference"=>
    {"$class"=>
      {"$classname"=>"CKReference", "$classes"=>["CKReference", "NSObject"]},
     "referenceAction"=>nil,
     "recordID"=>
      {"$class"=>
        {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
       "RecordName"=>"7D35FAE7-E05A-49D9-B05A-B372E211FEC6",
       "ZoneID"=>
        {"ZoneName"=>"Notes",
         "ownerName"=>"__defaultOwner__",
         "$class"=>
          {"$classname"=>"CKRecordZoneID",
           "$classes"=>["CKRecordZoneID", "NSObject"]}}}}}}

We see a lot more of the ZICCLOUDSYNCINGOBJECTS values in here, including our encryption variables! Specifically, as you look at the ValueStore section, we can find these values under RecordValues:

NS.key	NS.object
CryptoIterationCount	20000
CryptoWrappedKey	”~\xDB}\xD3\xB1S\x1D\x9E4\xE4\xA3O\x9D\x86\xD7z\xF9!p\xBD\x96\x89J\x87”
CryptoInitializationVector	“\xDD\x00\x0E\x9E)w\xCEU\xF8[‘l\xC9J\eK”
CryptoTag	“\x83\xF04\xB8\xDF\x1FG\xBF\x01\x10HR\x98\xE9\xAF6”
CryptoSalt	“\xC1]\xFD\eM\x95\xA8n\xA3\xA1c4R\x8Fa\x9A”

With these values, we can now decrypt the mergeable data, which is also found in the same section, in the EncryptedValues object. Once decrypted, this is a JSON object which contains the encrypted mergeable data that represents the table:

{
  "summary": "This\nIs\nFantastic\nEncryption\n", 
  "mergeableData": "H4sIAAAAAAAAE7VU30/TUBReu63r7oaUwvhx5YWCc6mBYE2M8Q0HKMjGLEONiQ+ju0CXrsOuU+AJJSHwZnww8UFNlPjgE6JR4y8SeTGSSEJiTEwIalQ0Maj/gHpXNsLWLezFm3Wn5ztf7rnf15tDW9hvZbSFscCPZYACNvxqZy0wFAvSTsjTFtbLNbVmVnOBv+yCFE2wZCuBoxVHK452HGkcKRzLoBsAY/c0i4TNsX00AffSJNvINfjFcGRAQf6Ekoqr7bKGJF1OqD1oUA8nRHloWIfLxCViiQC3CDAOeljq58Iz/IOMsWH67Eb0EQZCYISERvSRsBrQuPYXr3LM23rnSZpIP2wVlsjY3XdezlxvU546Oj5ffBd+hVGCZeC1ld7Jqd7wQuRBQ6ey6mGbAAPI1rQsG9y0ijAQB0aoLLKNZTexHFkEumOAJrE5NpbMy8iczJqT2f63E+t198evhgJvJuqveA893Li56cQ0Il3uiba7L5iJ6PLiWymjEWBFzhyNboy48pxIs4CJ5S7ohD0no3IyR05GQ6+wuX8l3m1XZjergdRhpCqDkFs8BqPlOTwWIxUZxA67xWMs2ZUs6iVVgpcUvtBkbf7XFcUQCzpUSRsbSV/soh0qGFBCB2DqYM3zMKu4GuvzZH0wkFqM1GQQG+wRu1lbeFgurjl943Y+ka2A5hNiL+vsjKh6JKnLUtEGNOMsoYFzB8kOjpajSNVlfYyrlrRC04SzJZEyyFGSJiYuJDlnf39Xe5caRaOcU9I2uUnOJSFFySR8jZSIt0RGRhTU4hfbwy3BvmAqPoC0AoU+XZPVId5jKqS78NV5MB53KZzwPhMuoiE5qSMtkFJ0+WRESaEehP3jG0tgmrrjNhpCfF1BOJiIou1C1ISOki2ZQWwudPmNgsBs/GHenxnz75l+En/96fujHwLDq6mxg7NHPPOtofjXQGxJME1TgVn5PTO7vha4fGNtv2uOOD0qmGarwNSc//Wl8uzRtsdNU75o6Pa8YJo6AnP83OLq5Kng83tDBz7MRb2KYJpXh+sBBKZTsiS9Gz/EP16BfnLtBgAA"
}

I need to do more testing to know why values would be stored in the actual columns, vice the ZUNAPPLIEDENCRYPTEDRECORD column. However, once I accounted for this possiblity and checked this column for data, Apple Cloud Notes Parser stopped erroring out. I had fixed my original problem but still had one significant issue: I had no idea what this was.

The Second Hurdle: “CK”

The abbreviation “CK” appeared a lot of times in these NSKeyedArchives, such as CKRecordZoneID, CKRecordID, and CKReference. Googling for any of these quickly brings you to the Apple Developer Documentation for CloudKit (“CK”). CloudKit is the mechanism to store data in iCloud for any application that wants to take advantage of that backend. Because Apple Notes has allowed users to share data since iOS 9, anything that can go into a shared note must be able to take advantage of CloudKit for that purpose. This includes notes, folders, and attachments.

CloudKit Records

The heart of CloudKit is an individual record, represented by the CKRecord class. A CKRecord is a dictionary of key-value pairs, which is what we are seeing in the ZUNAPPLIEDENCRYPTEDRECORD example, hence the term “RECORD”. Apple suggests creating a record type for each different type of information you need. These objects support a variety of field types and at a minimum, the initializer needs a RecordType and a RecordID, which we can see in our example:

{"root"=>
  {
   ...
   "RecordType"=>"Attachment",
   "RecordID"=>
    {"$class"=>
      {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
     "RecordName"=>"C93C5BE6-9212-46F6-939F-331844B9FEA0",
     "ZoneID"=>
      {"ZoneName"=>"Notes",
       "ownerName"=>"__defaultOwner__",
       "$class"=>
        {"$classname"=>"CKRecordZoneID",
         "$classes"=>["CKRecordZoneID", "NSObject"]}}},
    ...
  }
}

Here we see the RecordType is a simple String containing “Attachment”. This makes sense because the example comes from an embedded object. The RecordID is more complex and appears to be another dictionary, telling us the class of the object is a CKRecordID with a RecordName corresponding to our ZIDENTIFIER of C93C5BE6-9212-46F6-939F-331844B9FEA0. We also can see a ZoneID of type CKRecordZoneID.

CloudKit Zones

In CloudKit, CKRecordZones are how the specific application can organize its records. In our example, the ZoneName is “Notes” and the ownerName is the String “__defaultOwner__” indicating it is the currently logged in iCloud user. That means we are looking at record C93C5BE6-9212-46F6-939F-331844B9FEA0 inside of the currently logged in user’s “Notes” zone. It is important to note that in all of this, we are not getting above the Zone level, however the overall CloudKit architecture also has the concept of Containers and Databases above this³. In other words, there might be another app that could have the same RecordID and ZoneID for the same user, but in a different database and application.

Since the CloudKit record has to remember which user the note actually belongs to, does that mean we can get more data from iCloud about users who share notes? This question (sans the CloudKit-specific aspect) was what I was testing when the table originally broke, so let’s see what is in a shared note’s record. If you will permit some handwavy magic, I already know from my dataset that the folder which was created by another account to test sharing is ZICCLOUDSYNCINGOBJECT.Z_PK=138 in my database⁴. This query will pull out pertinent details for it:

SELECT ZIDENTIFIER as uuid,
  ZOWNER as owner,
  ZZONEOWNERNAME as zone_owner,
  ZACCOUNTNAMEFORACCOUNTLISTSORTING as list_as,
  ZTITLE2 as folder_name,
  ZSERVERRECORDDATA as record,
  ZSERVERSHAREDATA as share
FROM ZICCLOUDSYNCINGOBJECT
WHERE Z_PK=138

UUID	Owner	Zone Owner	List As	Folder Name
57B3EC94-2846-4E16-8E17-D8B1E8B8B5EA	9	_b291456806da1837211e8a60c9abe865b	1_iCloud	shared_testing

In the output, we see some interesting tidbits about the folder name shared_testing. To start with, the “Owner” of the folder is listed as 9, which when queried as a Z_PK in the same table corresponds to my iCloud account. Similarly, the “List As” is displayed how my iCloud account is: 1_iCloud. That would ordinarily give me the belief that the user of this Notes database created this folder and is responsible for the contents therein.

However, the “Zone Owner” gives the iCloud ID as recorded for Notes who actually created the folder, in this case _b291456806da1837211e8a60c9abe865b. This is the actual folder creator and because iCloud sharing rules are strict, you know for sure that the user intentionally shared this folder with the owner of this Notes database and the sharing request was accepted. That seems meaningful if there was something incriminating in that folder, but somewhat useless as _b291456806da1837211e8a60c9abe865b is fairly anonymous at this point.

ZSERVERRECORDDATA

Digging into the ZSERVERRECORDDATA object shows not much more information than we find within the columns above, save the fact that the “ModifiedByDevice” field contains the hostname of the device used by _b291456806da1837211e8a60c9abe865b.

[notta@cuppa apple_cloud_notes_parser]$ ruby ~/ruby/bplister/nskeyed_archive_list.rb \
138_server_record.bin | grep Device
 
  "ModifiedByDevice"=>"Forza's iPhone 12"

ZSERVERSHAREDDATA

Digging into the ZSERVERSHAREDATA object is much different, however, because these share objects are how CloudKit tracks who can see what. Within them is information about the users involved (and a lot more, I had to rip this down a ton to make it legible):

[notta@cuppa apple_cloud_notes_parser]$ ruby ~/ruby/bplister/nskeyed_archive_list.rb \
138_share_record.bin 

{"AllowsAnonymousAccess"=>false,
 "LastFetchedParticipants"=>[similar information as "Participants"]
 "Permission"=>nil,
 "ShareEtag"=>nil,
 "ParticipantVisibility"=>nil,
 "UseLightweightPCS"=>false,
 "PublicPermission"=>"cloudkit.share",
 "AddedParticipantIDs"=>
  {"NS.objects"=>[],
   "$class"=>
    {"$classname"=>"NSMutableSet",
     "$classes"=>["NSMutableSet", "NSSet", "NSObject"]}},
 "RemovedParticipantIDs"=>
  {"NS.objects"=>[],
   "$class"=>
    {"$classname"=>"NSMutableSet",
     "$classes"=>["NSMutableSet", "NSSet", "NSObject"]}},
 "DisplayedHostname"=>"www.icloud.com",
 "RecordID"=>
  {"$class"=>
    {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
   "RecordName"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
   "ZoneID"=>
    {"ZoneName"=>"Notes",
     "ownerName"=>"_b291456806da1837211e8a60c9abe865b",
     "$class"=>
      {"$classname"=>"CKRecordZoneID",
       "$classes"=>["CKRecordZoneID", "NSObject"]}}},
 "Participants"=>
  {"NS.objects"=>
    [{"Permission"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
      "OriginalAcceptanceStatus"=>"cloudkit.share",
      "OriginalPermission"=>
       {"$class"=>
         {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
        "RecordName"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
        "ZoneID"=>
         {"ZoneName"=>"Notes",
          "ownerName"=>"_b291456806da1837211e8a60c9abe865b",
          "$class"=>
           {"$classname"=>"CKRecordZoneID",
            "$classes"=>["CKRecordZoneID", "NSObject"]}}},
      "wantsNewInvitationToken"=>false,
      "$class"=>
       {"$classname"=>"CKShareParticipant",
        "$classes"=>["CKShareParticipant", "NSObject"]},
      "mutableInvitationTokenStatus"=>"cloudkit.share",
      "ParticipantID"=>"F3A75D2E-C5DC-4E77-94BB-8D803A42E137",
      "AcceptanceStatus"=>
       {"$class"=>
         {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
        "RecordName"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
        "ZoneID"=>
         {"ZoneName"=>"Notes",
          "ownerName"=>"_b291456806da1837211e8a60c9abe865b",
          "$class"=>
           {"$classname"=>"CKRecordZoneID",
            "$classes"=>["CKRecordZoneID", "NSObject"]}}},
      "InviterID"=>nil,
      "CreatedInProcess"=>false,
      "Type"=>"cloudkit.share",
      "ProtectionInfoPublicKey"=>nil,
      "IsOrgAdminUser"=>false,
      "UserIdentity"=>
       {"IsCached"=>false,
        "ProtectionData"=>nil,
        "UserRecordID"=>
         {"$class"=>
           {"$classname"=>"CKRecordID",
            "$classes"=>["CKRecordID", "NSObject"]},
          "RecordName"=>"_b291456806da1837211e8a60c9abe865b",
          "ZoneID"=>
           {"ZoneName"=>"_defaultZone",
            "ownerName"=>"__defaultOwner__",
            "$class"=>
             {"$classname"=>"CKRecordZoneID",
              "$classes"=>["CKRecordZoneID", "NSObject"]}}},
        "HasICloudAccount"=>true,
        "NameComponents"=>
         {"$class"=>
           {"$classname"=>"NSPersonNameComponents",
            "$classes"=>["NSPersonNameComponents", "NSObject"]},
          "NS.nameComponentsPrivate"=>
           {"NS.middleName"=>nil,
            "$class"=>
             {"$classname"=>"_NSPersonNameComponentsData",
              "$classes"=>["_NSPersonNameComponentsData", "NSObject"]},
            "NS.familyName"=>"Bruta",
            "NS.nickname"=>nil,
            "NS.givenName"=>"Forza",
            "NS.namePrefix"=>nil,
            "NS.nameSuffix"=>nil,
            "NS.phoneticRepresentation"=>nil}},
        "EncryptedPersonalInfo"=>[potentially interesting binary],
        "$class"=>
         {"$classname"=>"CKUserIdentity",
          "$classes"=>["CKUserIdentity", "NSObject"]},
        "OONProtectionData"=>nil,
        "LookupInfo"=>
         {"$class"=>
           {"$classname"=>"CKUserIdentityLookupInfo",
            "$classes"=>["CKUserIdentityLookupInfo", "NSObject"]},
          "PhoneNumber"=>nil,
          "RecordID"=>nil,
          "ReportsMissing"=>false,
          "EncryptedPersonalInfo"=>[potentially interesting binary],
          "EmailAddress"=>"forza_bruta@ciofecaforensics.com"},
        "ContactIdentifiers"=>
         {"NS.objects"=>[],
          "$class"=>
           {"$classname"=>"NSArray", "$classes"=>["NSArray", "NSObject"]}}},
      "InvitationToken"=>nil,
      "OriginalParticipantType"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
      "AcceptedInProcess"=>false,
      "ProtectionInfo"=>nil,
      "IsCurrentUser"=>false,
      "EncryptedPersonalInfo"=>[potentially interesting binary]},
     {"Permission"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
      "OriginalAcceptanceStatus"=>"cloudkit.share",
      "OriginalPermission"=>
       {"$class"=>
         {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
        "RecordName"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
        "ZoneID"=>
         {"ZoneName"=>"Notes",
          "ownerName"=>"_b291456806da1837211e8a60c9abe865b",
          "$class"=>
           {"$classname"=>"CKRecordZoneID",
            "$classes"=>["CKRecordZoneID", "NSObject"]}}},
      "wantsNewInvitationToken"=>false,
      "$class"=>
       {"$classname"=>"CKShareParticipant",
        "$classes"=>["CKShareParticipant", "NSObject"]},
      "mutableInvitationTokenStatus"=>"cloudkit.share",
      "ParticipantID"=>"937B5585-A6C1-4192-8373-18BEE7D2C538",
      "AcceptanceStatus"=>
       {"$class"=>
         {"$classname"=>"CKRecordID", "$classes"=>["CKRecordID", "NSObject"]},
        "RecordName"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
        "ZoneID"=>
         {"ZoneName"=>"Notes",
          "ownerName"=>"_b291456806da1837211e8a60c9abe865b",
          "$class"=>
           {"$classname"=>"CKRecordZoneID",
            "$classes"=>["CKRecordZoneID", "NSObject"]}}},
      "InviterID"=>"_b291456806da1837211e8a60c9abe865b",
      "CreatedInProcess"=>false,
      "Type"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
      "ProtectionInfoPublicKey"=>nil,
      "IsOrgAdminUser"=>false,
      "UserIdentity"=>
       {"IsCached"=>false,
        "ProtectionData"=>[potentially interesting binary],
        "UserRecordID"=>
         {"$class"=>
           {"$classname"=>"CKRecordID",
            "$classes"=>["CKRecordID", "NSObject"]},
          "RecordName"=>"__defaultOwner__",
          "ZoneID"=>
           {"ZoneName"=>"_defaultZone",
            "ownerName"=>"__defaultOwner__",
            "$class"=>
             {"$classname"=>"CKRecordZoneID",
              "$classes"=>["CKRecordZoneID", "NSObject"]}}},
        "HasICloudAccount"=>true,
        "NameComponents"=>
         {"$class"=>
           {"$classname"=>"NSPersonNameComponents",
            "$classes"=>["NSPersonNameComponents", "NSObject"]},
          "NS.nameComponentsPrivate"=>
           {"NS.middleName"=>nil,
            "$class"=>
             {"$classname"=>"_NSPersonNameComponentsData",
              "$classes"=>["_NSPersonNameComponentsData", "NSObject"]},
            "NS.familyName"=>"Cuppa",
            "NS.nickname"=>nil,
            "NS.givenName"=>"Notta",
            "NS.namePrefix"=>nil,
            "NS.nameSuffix"=>nil,
            "NS.phoneticRepresentation"=>nil}},
        "EncryptedPersonalInfo"=>[potentially interesting binary],
        "$class"=>
         {"$classname"=>"CKUserIdentity",
          "$classes"=>["CKUserIdentity", "NSObject"]},
        "OONProtectionData"=>nil,
        "LookupInfo"=>
         {"$class"=>
           {"$classname"=>"CKUserIdentityLookupInfo",
            "$classes"=>["CKUserIdentityLookupInfo", "NSObject"]},
          "PhoneNumber"=>nil,
          "RecordID"=>nil,
          "ReportsMissing"=>false,
          "EncryptedPersonalInfo"=>[potentially interesting binary],
          "EmailAddress"=>"notta_cuppa@ciofecaforensics.com"},
        "ContactIdentifiers"=>
         {"NS.objects"=>[],
          "$class"=>
           {"$classname"=>"NSArray", "$classes"=>["NSArray", "NSObject"]}}},
      "InvitationToken"=>nil,
      "OriginalParticipantType"=>"Share-E9CED921-6ADF-4D24-869C-08F0B2678AAE",
      "AcceptedInProcess"=>false,
      "ProtectionInfo"=>[potentially interesting binary],
      "IsCurrentUser"=>true,
      "EncryptedPersonalInfo"=>[potentially intersting binary]}],
   "$class"=>
    {"$classname"=>"NSMutableArray",
     "$classes"=>["NSMutableArray", "NSArray", "NSObject"]}},
 "ContainerID"=>
  {"$class"=>
    {"$classname"=>"CKContainerID", "$classes"=>["CKContainerID", "NSObject"]},
   "ContainerIdentifier"=>"com.apple.notes",
   "Environment"=>"cloudkit.share"},
 "AllPCSKeyIDs"=>nil}

Even after snipping the output down considerably, there is a lot there of interest. To begin with, every time you see “potentially interesting binary”, there was binary data I was removing since I wanted to sanitize the data. I am not yet completely sure of what is present in them, but figure I eventually have to just cut this research, ship what I have, and dive in later. Beyond that, you can see that personal information is being presented about the users who have access.

CloudKit Account Shared Account Information

For example, all of the data I have filled in on my iCloud account and the other test account can be found in the UserIdentity sections:

      "UserIdentity"=>
       {"UserRecordID"=>
         {"RecordName"=>"__defaultOwner__"}},
        "NameComponents"=>
         {"NS.nameComponentsPrivate"=>
           {"NS.middleName"=>nil,
            "NS.familyName"=>"Cuppa",
            "NS.nickname"=>nil,
            "NS.givenName"=>"Notta",
            "NS.namePrefix"=>nil,
            "NS.nameSuffix"=>nil,
            "NS.phoneticRepresentation"=>nil}},
        "LookupInfo"=>
         {"PhoneNumber"=>nil,
          "RecordID"=>nil,
          "ReportsMissing"=>false,
          "EmailAddress"=>"notta_cuppa@ciofecaforensics.com"}}
      ...
      "UserIdentity"=>
       {"UserRecordID"=>
         {"RecordName"=>"_b291456806da1837211e8a60c9abe865b",
        "NameComponents"=>
         {"NS.nameComponentsPrivate"=>
           {"NS.middleName"=>nil,
            "NS.familyName"=>"Bruta",
            "NS.nickname"=>nil,
            "NS.givenName"=>"Forza",
            "NS.namePrefix"=>nil,
            "NS.nameSuffix"=>nil,
            "NS.phoneticRepresentation"=>nil}},
        "LookupInfo"=>
         {"PhoneNumber"=>nil,
          "RecordID"=>nil,
          "ReportsMissing"=>false,
          "EmailAddress"=>"forza_bruta@ciofecaforensics.com"},

So now, instead of just knowing that _b291456806da1837211e8a60c9abe865b created a folder and shared it with our user, we know the name being used and the email account tied to their iCloud account. Potentially, we also have a phone number to correlate with other databases in the forensic image. Finally, based on the ZSERVERRECORDDATA contents, we know the hostname of at least a device used by that other user⁵.

Much Improved Analysis

Putting that all together, I would be able to say completely confidently that the iCloud user identified by forza_bruta@ciofecaforensics.com created the folder called shared_testing and shared that folder with the iCloud user identified by notta_cuppa@ciofecaforensics.com, who explicitly accepted the sharing request. Further, the iCloud user identified by forza_bruta@ciofecaforensics.com asserted that their name was Forza Bruta to iCloud and would be identified throughout this database by _b291456806da1837211e8a60c9abe865b. The iCloud user identified by forza_bruta@ciofecaforensics.com likely had a device named Forza's iPhone 12 which probably modified this shared folder (note the weasel wording here, I’m not yet confident I understand all of this).

Improved Notes Parsing

Apple Cloud Notes Parser Changes

This discovery led to a surprising amount of changes to the Apple Cloud Notes Parser. This will sound dumb, since “cloud” is in the name, but I didn’t realize how pervasive these changes woule be when I started in on them a month ago. Because all of these objects can be added to a Note saved to iCloud, everything has to support it. To fix this, I created a class called AppleCloudKitRecord which most of the other classes are children of. It handles the CloudKit data parsing for all of the other objects uniformly and most of the CSV and HTML output tries to bring the pertinent information forward.

Obviously, now that we know where the variables are hidden, I’ve also fixed decryption for objects with ZUNAPPLIEDENCRYPTEDRECORD data. Bigger picture, we’ve also started bringing keyed_archive up to date, which was critical to making any of the above happen quickly.

But Who Cares?

This has been a long journey and it would be pretty easy to say “Criminals likely won’t share notes” and walk away from this topic. I strongly suspect a greater appreciation for CloudKit will make things far more apparent, since almost any iOS application can use CloudKit. In this case Notes just ended up being the vehicle to start the research.

For example, I snagged one of Josh Hickman’s test images, and did just a simple grep for CKRecordZone, which I thought wouldn’t have too many false positives other than things which were actually CloudKit related. Unsurprisingly, there are a lot of Protected Cloud Storage (PCS) files, however there also are other files I wouldn’t have expected, such as Safari’s bookmark syncing and Apple’s HomeKit stuff.

[notta@cuppa mobile]$ sudo grep -r CKRecordZone | grep -v "wal"
Binary file Library/Caches/com.apple.routined/Cloud-V2.sqlite matches
Binary file Library/Caches/CloudKit/com.apple.suggestd/9689277fc800e022bb77d1141f289a17802a0e5b/Records/pcs.db matches
Binary file Library/Caches/CloudKit/CloudKitOperationInfo matches
Binary file Library/Caches/CloudKit/com.apple.routined/c5263141ed845f03fe972d5a292cc4c7e3717549/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.accessibility.AccessibilityUIServer/e3ef713a17f01b98667b63ad80d9cb50a3db5444/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.protectedcloudstorage.protectedcloudkeysyncing/aa73de30954c5cf0c85ed82c798fc05ce34a64b6/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.amsaccountsd/fae2b83855910dfd69b0ea3405999d7c621adf19/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.email.maild/8d4420900c0114f4061c04f73c872e91b36024d7/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.coreduetd/c71f032347d6cae25a3164e9ffda52413530d895/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.imagent/595e8bb0c8c5bca9dd197a05597a294fb2d5d1bb/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.avatarsd/e3218955454ddfc3c690909bd6d691c9f2d0b139/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.stocks.widget/9ca101754d46d930fbac9a0fa57c9e7c48c7a3ec/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.syncdefaultsd/46997fc7f15bd5b9b28f9415b5f510378f86f047/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.cloudphotod/com.apple.photos.cloud/4d4df0a526653b9317b428548a9695da13fa5b0c/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.cloudphotod/com.apple.upload-request-proxy.com.apple.photos.cloud/c7de0805e567c11f7528c440602793babb4da687/FrameworkCaches/com.apple.cloudkit.uploadrequest.com.apple.photos.cloud.cache matches
Binary file Library/Caches/CloudKit/com.apple.backupd/32e95e402d965071c4748db5ec0cdb0a05271933/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.cloudpaird/ce49f3500a597aa2dca9af15112d99e9363cd778/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.iad-cloudkit/com.apple.AdSheetPhone/e5ed812d72be56d0c37538766f5fd4d7eabdb84c/Records/pcs.db matches
Binary file Library/Caches/CloudKit/com.apple.iad-cloudkit/com.apple.AdSheetPhone/dc9f57b5d00dfc35b7bfc8ffd7dab5add6d780d7/Records/pcs.db matches
Binary file Library/Safari/Bookmarks.db matches
Binary file Library/Safari/CloudTabs.db matches
Binary file Library/Application Support/CloudDocs/session/db/client.db matches
Binary file Library/Application Support/com.apple.ProtectedCloudStorage/KeysSyncingVersion3-17193901029-ProtectedCloudStorage.db matches
Binary file Library/homed/datastore.sqlite matches
Binary file Library/homed/datastore3.sqlite matches
Binary file Library/SMS/sms.db matches
Binary file Library/Preferences/com.apple.SafariBookmarksSyncAgent.plist matches
Binary file Library/Preferences/com.apple.suggestions.plist matches
Binary file Library/ClassKit/db/ClassKit.db matches
Binary file Media/PhotoData/CPL/storage/store.cloudphotodb matches
Binary file Containers/Data/Application/9059248A-7FA9-455C-A212-6805F124F180/CloudKit/d548e5089fa75a02a5d8877fd1d20c222e8eb37f/Records/pcs.db matches
Binary file Containers/Data/Application/9059248A-7FA9-455C-A212-6805F124F180/Library/Caches/com.apple.MapsSupport.MSPCloudContainerCache-MSPHistoryContainer matches
Binary file Containers/Data/Application/9059248A-7FA9-455C-A212-6805F124F180/Library/Caches/com.apple.MapsSupport.MSPCloudContainerCache-MSPCollectionsContainer matches
Binary file Containers/Data/Application/9059248A-7FA9-455C-A212-6805F124F180/Library/Caches/com.apple.MapsSupport.MSPCloudContainerCache-MSPPinnedPlacesContainer matches
Binary file Containers/Data/Application/842C9702-4CA9-4A73-A315-62A1F4FD7537/CloudKit/bf597ca5949b365ff9ac52a5db7b5a3637fd34f1/Records/pcs.db matches
Binary file Containers/Data/Application/842C9702-4CA9-4A73-A315-62A1F4FD7537/CloudKit/2f499a61d18492de6bd996804e7fa35f3f2b5869/Records/pcs.db matches
Binary file Containers/Shared/AppGroup/1AA2421A-3D36-4566-B577-0BAC64154976/NoteStore.sqlite matches

Conclusion

Never, ever, let yourself close “just one more bug report” after 10:00 PM, it will inevitably lead to a rabbit hole which chews up a month of time and leads to many more rabbit holes to dig down. In the case of CloudKit, trying to figure out why the encryption variables I was expecting were buried away in another column led to my having to add a whole bunch of extra checks to Apple Cloud Notes Parser, start handling CloudKit data, and realize there might be a lot more data sitting around than we realize. It is certainly worth considering if an application in question is supported by CloudKit and, if so, digging into what data is available as a result.

Finally, this tool and research is entirely supported by my free time. This update, for example was probably around 30-40 hours of work between the research, dumping, diving, discovering, developing, and documenting. If it is meaningful to you, please consider supporting it through GitHub sponsors or Ko-Fi. I have plans for this, but need support to reach them, especially if I build more of a CloudKit lab.

Footnotes

Some of the output from these commands has been edited slightly for readability on the page, in particular to ensure blocks don’t need a scrollbar. ↩
The Github repo is here and the Ruby gem is hosted here. ↩
Check out the Apple CloudKit talk from WWDC 2016 for some background. ↩
I usually don’t like editing the data I present, but as the accounts in question are not entirely test data, I feel the need to remove certain aspects of personal information. I may not edit this consistently, where that happens I apologize in advance. ↩
I’m still testing exactly how and why the device hostname gets filled in. So far it seems like there can be false positives, but at a minimum you know this hostname is used by that account. ↩

Revisiting Apple Notes (7): Cloudkit Data