Gone

So, uhhhh. This isn’t good. I really hope I’m just confused and not sounding a false alarm, but…

Back in April I wrote a quick post about how I was backing up the shared iCloud Photo albums that my friends and family all use to send pictures and videos of our kids back and forth.

The conclusion of the post was that all of those shared pictures are not stored inside your primary Photos.app library. Instead, they were squirreled away deep inside here…

~/Library/Containers/com.apple.cloudphotosd/Data/Library/Application Support/com.apple.cloudphotosd/services/com.apple.photo.icloud.sharedstreams/assets/

Fair enough. And for someone who is crazy paranoid about backing up all of their data like myself, that was fine. I just pointed Arq at that folder and rested comfortably knowing all the photos of my kids and my nephew and my best friend’s kids were safely being backed up.

Not so much with Catalina.

A reader emailed me today to ask if I knew where that folder had been moved to after upgrading to 10.15. I’ll admit – I had no idea it had been moved. I didn’t even think to check.

So, I looked, and, sure enough, that sharedstreams folder is gone. But where? All said, the shared albums had previously been taking up 112 GB, so I turned to DaisyDisk again to look for another large folder that size, and…

Nothing.

But here’s the thing. Photos.app still had the Shared Albums preference enabled. And I can still see all of those shared photos and videos in the app.

I put on my developer hat and started watching the filesystem while I browsed my shared albums. I found that when I double-clicked a photo to open it to its full size, Photos.app downloaded the file and cached it locally here:

~/Pictures/Photos Library.photoslibrary/resources/cloudsharing/data/<PERSON ID>/<ALBUM UUID>/

(Where PERSON ID is I think an identifier assigned to your iCloud account? Maybe? Not sure. And ALBUM UUID is the unique identifier for a shared album.)

Ok. That’s great. It actually makes more sense to have shared photos also live inside your Photos.app library instead of somewhere random in ~/Library.

But here’s the shitty part I’m trying to figure out. Of the 112 GB my shared albums previously took up, only 250 MB (MB!!!) are cached locally.

Don’t get me wrong. This is likely an awesome thing for folks low on hard disk space. Keeping possibly hundreds of gigs locally for shared photos was probably a silly idea to begin with. But…

and that’s a big but…

I don’t trust iCloud with my photos. And I’m not trying to single out Apple. I don’t trust Google Photos either. I don’t trust any cloud with my data. I love the convenience of iCloud photo sharing with friends and family and using Google Photos as a source of truth for my own family archives. But I want a backup of my data – just in case – that I’m in control of.

And now I don’t know how to do that. Here’s the Preferences window for Photos.app on Catalina…

There’s an option to download the originals of my own photos/videos, but not for shared albums? And, try as I might, I couldn’t find a way to “select all” and tell Photos.app to download everything like you can with your music stored in iTunes Match.

So, uh, Apple? How can I get the full-res versions of my five years worth of shared photos and videos? I obviously have the originals of the pictures that my wife and myself took. But there are hundreds if not thousands of images of my kids taken by their grandparents as well as all the photos of my sister’s kid, our friends’ kids, etc. that I would very much like to keep safe.

And, just to make sure I’m not absolutely crazy or mis-remembering something, I took a look at my historical Arq backups. I upgraded my iMac to Catalina on October 7th. On October 6th, that folder of shared albums was 112 GB…

By October 8th, it was gone…

And they certainly weren’t moved inside my Photos.app library because that hasn’t grown in an equivalent size.

Just 112 GB of memories.

Gone.

(Note for the HN crowd: Obviously, as this post talks about, I have backups. I haven’t technically lost anything. But this new caching strategy does my backups no good going forward as I won’t reliably have the full-res content automatically cached locally and available to be backed up.)

Followup…