Uncategorized

Quick question. Take a look at these two screenshots…

and

That’s my Mac right now at work with Spark and Slack running fullscreen in Split View.

In the first screenshot, one of the apps has focus. And in the second screenshot the other app has focus.

Can you tell which is which?

I’ve never been a big fan of Spaces or full screen on macOS, but I’ve been giving them a try again recently just to see if I can improve my workflow. And after a day or two of having a dedicated space for Slack + email, I found it maddening.

It turns out that I really enjoy having my noisy, communication windows segregated from the apps and windows I actively use to do work, but every time I’d switch to that space and either try and type into Slack or use the keyboard for navigation, I’d invariably realize the wrong app has focus.

Why? Because there’s no way to tell which is which.

Ok, technically that’s not true. You can tell. I’ve highlighted the difference below…

When that little 4px by 64px, gray bar is visible it means Slack is focused. Or something. I don’t know. Probably no one does.

Like I wrote about on iOS a few months ago, there’s been a trend in the industry ever since the iOS 7 redesign to get rid of UI affordances – or as Apple puts it, “subtle and appropriate” “adornments”. I don’t know if these changes are deliberate, due to a lack of empathy for the user, or just inexperience. But as apps deviate further and further away from the HIG with custom UI, whether for design reasons or in the pursuit of a mythical, cross-platform code base that management thinks will cost less, we lose the benefits of a well reasoned platform that was formerly easy to work with and a joy to use. And if we’re gonna go down that road, then let’s just give up and put everything inside a web browser. At least then we can see which tab has focus.

Last May I wrote about how I was keeping alive a persistent SSH tunnel back to my home network. Specifically, it connects to an iMac Pro that I use as a home media/automation server. I frequently need to screen share, view a web service hosted on it, or just access its command line. It also serves as our build server at work. An SSH tunnel is basically a poor man’s VPN. And I feel more secure about using that then just outright opening up ports in my firewall.

Anyway, nine months later, it’s proven to be a great solution. However, there is an occasional quirk.

The launchd job that controls the tunnel is set to automatically restart the process if it dies. And, sure, when it does die launchd does the right thing and restarts it. But in some situations the tunnel won’t actually terminate when it fails – it just hangs or becomes unresponsive. I was able to reproduce the issue a few times by making my WiFi flake out. So, I turned off the iMac’s wireless and hardwired it into my router. That was a big improvement, but I still saw the occasional oddness and found myself locked out a number of times when trying to connect back from work, the coffee shop, etc.

Thankfully, before I wasted too much time debugging, I recognized this problem for what it was: a giant rabbit hole.

I knew damned well that I could absolutely lose (and enjoy!) a whole evening or entire weekend running tests, debugging, and trying to figure out the cause of the intermittent failures. I’ve done it a thousand times with other piddly little problems. But, as Oliver Wood might say, “I am wiser now.” I’ve got better ways to spend my time. So, let’s just fix it.

My thinking went like this:

1. When the network goes down, the tunnel will close, the ssh process will quit, and launchd will keep restarting it until it connects again. That part works fine.
2. The problem is figuring out when the connection has stalled or experienced some other type of unresponsive fuck-up. How do I detect that?
3. Me! I’m a smart human. It should be me detecting the issue instead of wasting hours trying to find a way to make the computer catch it. When I try and connect to the tunnel and it doesn’t work, that means something is broken. And since I only care if it’s broken when I’m actually trying to use it, that’s when I need to fix it. Automatically detecting any and all failures simply isn’t necessary.
4. So, when I notice the tunnel is down, how do I tell my iMac to kill the process and let launchd restart it when I don’t have access to the machine in the first place? I could just kill the process automatically on a set schedule. But that might interrupt me when I was actually using it. Also, if it were down, I’d have to wait until the next interval for it to come back up. I need to send a command while remote and without having access to the machine – preferably in a way that I can also do from my phone.
5. <Spidey sense starts tingling>
6. Hazel! Dropbox! And a small shell script! My three favorite tools.

First, let’s figure out how to kill the existing (stalled) ssh process. From my original blog post, you can see that launchd runs this command to start the tunnel:

/usr/bin/ssh -N imacvpn

So, we can figure out the pid of that process by greping for the imacvpn keyword like this:

ps -ax | grep imacvpn | head -n1 | cut -f1 -d " "

That’s a four part command

1. First, it lists all the running processes under my account.
2. Filter to just those that contain the name of the SSH tunnel. That will typically return two results. The real SSH tunnel command and also the grep command itself.
3. So, grab just the first line which will be (should always be?) the real tunnel process.
4. Split that line of text using a space as the delimiter. The first token will be the pid which we’ll use to identify and kill the tunnel.

With that pid, we can then kill the process like this:

That’ll find the ID of the SSH tunnel process, kill it, then launchd will take over and start it back up again.

Great. But how to make that script run on command? That’s where Hazel and Dropbox come in.

If you don’t know Hazel, go here. What we’re going to do is tell Hazel to monitor a specific folder inside my ~/Dropbox/. Whenever a new file is added to it, Hazel will detect the change, delete the file (since we don’t actually care about the file), and then run the above shell script.

Here’s how the configured Hazel rule looks:

Basically, we’re using Dropbox as a way to trigger a filesystem change that Hazel will detect and run whatever command we ask it to.

It’s great because if I’m on my computer at work or my laptop in a coffee shop, I can just save a blank text file into that folder. Dropbox will sync it to my iMac. Hazel will see the change and run the script.

And if I’m somewhere with just my phone, I’ve found the easiest thing to do is open up Photos.app and save a photo into Dropbox.app using Files.app via the system share sheet.

So that’s the dumb solution to my problem that I came up with. I’m still not sure what the actual technical reason is for the SSH tunnel occasionally crapping out on me. I’m just glad I’m to the point in my nerd existence where I can be happy applying a fix and not caring about the real underlying issues that don’t concern me.

Update:

After reading this post, Leo Kennis wrote to me:

A slightly safer option would be to change the command to:

ps -ax | grep imacvpn | grep -v grep | cut -f1 -d " "

The part with grep -v is a negative grep (find all lines that do not match “grep”)

This way it doesn’t matter which of the two results is listed first.

That’s one of my favorite things about shell scripts – there’s a million approaches to every problem. The above is a great way to make sure the correct pid is chosen. Thanks, Leo!

If you’ve been following along at home, you might remember that I started building a Mac app for managing my personal finances last April. Think of it as a powerful, privacy-focused, native alternative to Mint.com. Quicken, but not awful.

Since then, I’ve helped shipped a huge redesign to the app at my 9-5 job as well as two major updates to VirtualHostX and CommandQ. Complicating matters, VHX Pro is my first product ever to use a subscription pricing model. So, in addition to all the dev work that went into the app itself, I also had to write the server infrastructure to handle recurring billing, etc. I’ve been busy.

Nonetheless, I continued tinkering with my finance app all year. It’s basically my white whale at this point. And I’m happy to say it’s reached a point where I rarely if ever log into my bank’s website. Everything is done through my app.

But it’s not nearly ready yet. The UI and feature set is still too much in flux. However, while experimenting with different ways of managing and forecasting a budget within the app, I decided to prototype a few ideas as small, standalone Mac apps so I could build and test them in isolation. One of them took hold and became incredibly useful to me in real life as my wife and I made adjustments to our finances and tried to repair the damage from 2018.

I shared the app with a few friends and received some encouraging feedback. And then last month, with the end of the decade just a few weeks away, Twitter blew up with people posting about how they spent the 2010s and what they accomplished. I sure had a hell of a decade. But I wasn’t in the mood for reminiscing. I wanted to look ahead. So, I got the idea to put something new out into the world on January 1st and turned to my little budgeting app as it was the closest thing I had ready to go.

And here it is.

It’s called Rebudget, and it’s a wonderfully simple way to visualize your personal budget over time. See the natural ebb and flow of your account balance as paychecks arrive, recurring bills go out, and unexpected expenses arise.

The app isn’t meant to be an identical mirror of your bank statement. (That’s what my other, larger finance app is for.) Rebudget is a tool for planning and forecasting. Use it to see if your money is heading in the right direction. Use it to see if you can afford to take on a new car payment this Summer or if you should wait till next year. Figure out if you need to ask your new freelance client for a 50% deposit up-front or if you have enough cushion to wait for a final payment when the project is finished.

And that’s kind of the idea. If you’re already financially secure and never stress about your bank account, great. Rebudget isn’t for you. But if you freelance for a living or have other income that doesn’t arrive on a strict once or twice a month schedule – or if you’re recovering from a financially disastrous year where it seemed like everything was spiraling out of control, I think Rebudget can be a great way to plan or at least sanity check your finances. It’s certainly helped me in recent months.

I built the app to work the way I think. And that meant focusing on flexibility. I like to work with and visualize different scenarios at once. So rather than making Rebudget a typical shoebox Mac app with a canonical datasource, I went with our good friend NSDocument. That means you can have as many budgets as you want. Open them up side-by-side in different windows and compare how certain financial decisions effect things eighteen months from now. Or, quickly toggle a recurring bill on or off to see if it’s worth cutting that expense.

I have an early iOS version in the works, but until then I still wanted a way to have my data somewhat available on the go. So, I added an option to sync your bills and their due dates with Reminders.app. That way your bills show up on your calendar and you get alerts when they’re due. Also, if you check off a bill in Reminders, Rebudget sees the change and marks that bill as paid in its own database.

Here’s a quick, two-minute demo.

That’s Rebudget. I wanted to put a brand new thing out into the world to begin the new decade, and that’s what I came up with. I find it useful. I hope you enjoy the app, too.

Two things prompted this post:

1. Last week I moved my oldest and best selling Mac app to a subscription model.
2. Ryan Christoffel wrote about The Iconfactory announcing that their amazing app Linea Sketch will be doing the same in 2020.

I work for a company that primarily builds hardware, but software is my true love. I’ve been programming in some form or another for twenty-seven years. It’s my job and my hobby. It’s the thing in life I’m most passionate about (other than my family, which I guess I’m required to say). I love the high I get from solving a difficult problem in my own apps and seeing the even more amazing accomplishments other developers pull off.

And when I see a well-crafted, beloved indie app (or entire company!) go under, it kills me. So, here’s my unsolicited thoughts on the state of software subscriptions both as a business owner and as a customer.

The first dollar I ever made from a piece of software I wrote myself was from a website I built in 2002 called IHateClaire.com. (Fun story: In 2009 someone bought the domain and turned it into a revenge-porn site.) For a one-time payment of $5, you could upload as many photos to the website as you wanted, and using a specially crafted file format that I reverse engineered, my server would send the photo to your SprintPCS flip-phone for you to use as a wallpaper or caller ID image. It certainly didn’t make me rich, but I was able to buy the occasional sandwich on the way to my college job at RadioShack.

Other small projects like that followed, but my first, real commercial endeavor (at least as I view it) was VirtualHostX in 2007. Originally costing $7 for version 1.0, over the next six years it would mature and gain new features until finally settling at $49 in 2013; a price that lasted until November 2019.

VHX followed the typical up-front pricing model. Your one-time payment got you all minor updates until the next major release. At that point you could upgrade at a discounted price or stick with your current version as long as it kept working for you.

Over the course of twelve years I released seven paid upgrades – each time offering existing customers a discount of 15 to 50% off. And even with those discounts, my upgrade revenue typically dwarfed my income from new customers more and more with each passing year.

Here’s a chart of total new customers gained each month from 2007 through last month. You’ll see that I’ve had to rely on revenue from existing customers as the number of new users buying my app has fallen.

From 2012 through 2014 it was my only job. But with the birth of my son (and then daughter), my wife left her job to raise them while they were babies and I went back to a 9 to 5 job plus my software business.

I won’t bore you with the gory details – because they’re both personal and quite boring – but in late 2017 the business came crashing down. And by May 2018, my one-person indie company was essentially bankrupt. Burnt-out and slightly devastated, I called it a loss and tried to move on.

But like I said at the top of this post, my heart is in software and I wasn’t getting that type of satisfaction from my day job. With the help of a small angel investment, I got back to work later that year and put out version 8.0 in October 2018.

Since that time, I’ve been trying to win back my customers’ trust and make the business viable again. The plan for doing so started this past January and finally came to fruition with me ditching version numbers entirely and releasing what would have been version 9 as a rebranded VirtualHostX Pro with a subscription model (kind of).

As I said, I rely on upgrade revenue to stay afloat. And I typically put out a paid release every 18 to 24 months. But that simply wasn’t sustainable.

(Ok, now let’s get to the part about other developers and our industry in general.)

I have it extremely easy. My apps aren’t in the App Store. They wouldn’t be allowed even if I wanted to go that route. And that’s awesome! That means I have to do everything myself – from online storefronts, to payment processing, licensing and order fulfillment, etc. I’m not under Apple’s control (for now), and I don’t have to throw away 30% of every sale. My fees, all inclusive, are just under 9%.

But for developers who only (or primarily) target iOS? Fuck. 30%? Capricious App Review enforcement? Competing against scam artists? (Admittedly better) review times? Getting locked out of your developer account? And, the point of this whole blog post, a trillion dollar tech company that refuses to work with its most passionate and loyal customer base to figure out a sustainable business model? (Haha, just kidding. There’s no great mystery to figure out. Just fucking allow paid upgrades already.)

In the App Store your options are:

1. Release a paid up-front app and ship new features and bug fixes free, forever.
2. Release a paid up-front app and let it die a slow painful death due to abandonment and/or bankruptcy.
3. Riddle the app with in-app purchases.
4. Ship version N+1 as a brand new SKU every so often and hope (with zero promotional help from Apple or the App Store) that your customers buy again.
5. Turn to a subscription model.

Options 1 and (obviously) 2 aren’t sustainable – especially in today’s increasingly cloud-connected and service-oriented world where nearly every app has a web service component and simply costs the developer money for the app to continue to function.

But let’s play Devil’s advocate. Let’s ignore the recurring costs of the backend infrastructure or the time (which equals money) it takes to build, test, and ship new features.

iOS (and now macOS) are not Windows XP. Apple (and I don’t truly blame or criticize them for this) doesn’t give two shits about backwards compatibility. They move fast because, one, it’s in their financial interest to keep shipping new, shiny things, and, two, they have a genuine love for the cutting edge – old software be damned. (Again, as a macOS developer, I have this way easier than my iOS counterparts.)

We kill ourselves for four months every year just making sure our software continues to work with what Apple announces in June and ships in September. (Most recently in my day job, one developer spent about three weeks just focused on implementing UIScene support and dark mode.)

Option 3 is a possibility for some developers (Launch Center Pro attempted this with some success I think), but gating new features behind IAP paywalls is confusing (and sometimes deceptive) to customers and a nightmare for developers who have to if/else the crap out of their code on top of dealing with the ridiculous struggle of verifying more and more complex App Store receipts.

And say you do go the feature flag route. That still means you have to keep features that customers previously paid a one-time price for updated and compatible with iOS changes. And then you’re right back in the same unsustainable boat as up-front pricing except now you’re managing lots of a small purchases instead of one larger one.

And as for Option 4, if you’re going to regularly ship entirely new SKUs as a de-facto way of doing paid upgrades, well then you might as well go the subscription route (Option 5) because customers are going to have to keep paying for the app to continue working in either case.

The Iconfactory writes:

We spent over 200 hours on the Linea 2.7 update. A majority of that time was not even spent adding new features, instead it was spent making sure that everything looked right with the operating system’s new Dark Mode!

Changes to the operating system are obviously outside of our control but they must be dealt with or else the app risks becoming unusable. For example, prior to our work to properly handle dark mode, if you went to select a new template, you were presented with a menu that had black text on a black background.

Apple has recently begun to make a distinction between “iOS” for the iPhone and “iPadOS” for iPads. We expect that we are entering a period where new iPad-specific features will require a lot of maintenance in the coming years. If we don’t keep up-to-date, you’ll be working with a broken or unusable version of Linea sooner rather than later.

For a small software company whose product really is the app – as opposed to a SaaS with a companion app, or some other type of business that can bankroll an app by virtue of their real source of revenue – I don’t see any other sustainable path forward than subscriptions.

At least here on the macOS side of things outside the Mac App Store, Apple hasn’t willfully destroyed the perceived value of software as a way to juice hardware sales. So charging a premium (fair?) price is still realistically possible. But it is becoming more and more difficult.

So what can be done? What can literally keep food on developers’ tables and still be fair to our customers? (Other than paid upgrades in the App Store, of course, but that’s just silly, right, Apple?)

A common refrain from customers is they just want to pay for an app one time. I GET IT. But if we, developers, don’t get paid for our time, it is simply unsustainable to ship updates. And every app out there will 100% eventually die and cease to work without maintenance work. The breakneck pace of iOS ensures that.

And so I view it as a very simple calculus for customers. If you buy an app for $0.99 with the intention of only using it a few times for a specific use case, then who cares if it breaks next September? But if it’s an app you regularly use or – gasp! – actually depend on? Why would you not want to do your part to ensure it has a future?

Here’s a short Twitter conversation between Drew McCormack (Agenda) and Matthias Tretter (MindNode):

This made me think about the subscriptions I have for apps. It’s a grand tally of one: Tower. It’s a tool I need for work, and it cannot hold my data hostage. I don’t trust any other creational tool with a subscription. Perpetually renting access to my own data doesn’t appeal.

@drewmccormack agreed. which is why we will offer a free editor and always allow to export your documents, with our without an active subscription

@myell0w Sounds like the right way to do a subscription. IMO a lot depends on the “lock down” that is applied. If you can’t do any edits at all, it could be very frustrating. Eg. see a typo and have no choice by to pony up for another year to fix it.

Drew makes a great point. It’s the “lock down” that can be the key to a sustainable balance between what’s fair to customers and you staying in business. VirtualHostX’s new pricing structure follows the Sketch model: the app works forever – just no new updates after your subscription expires.

But, still, subscription fatigue can certainly be a real thing for people. My biggest gripe, however, as a developer, are those who still balk at paying anything for apps they use – up-front price or subscription.

I’m incredibly fortunate to earn a good living. And so I try very hard not to project my financial prosperity on to other people who may or not be in a situation I understand. But our customers are walking around with an – at a minimum – $500 supercomputer in their pocket. In addition to a larger, equally expensive slab of smart glass in their bag – if not also a laptop or smart watch, too. All of those combining for at least $60/month in service fees to the cell carriers.

The Iconfactory announced Linea will be $0.99/month or $9.99/year. If you love that app, why would you not fork over less than a dollar a month to continue using it?

I target a smaller, niche market and have way less sales than they do, so I have to adjust my pricing for a smaller set of customers. VirtualHostX Pro is $5/month or $49/year. I have a major advantage in that for a large percentage of my user base, my app is a business write-off because they rely on it to do their job. (Not knocking Linea or other apps. I’m sure many of their customers use it professionally, too.)

So when I did my big release last week and announced my new pricing model, I was extremely nervous and wary of what the response would be. But so far I’m very lucky that I’ve have had few complaints. (I’m sure there are lots of folks who are pissed and will just never tell me or buy from me again.)

Here’s two unsolicited quotes I received from customers last week:

Sorry, I don’t do subscriptions. It’s not a pricing model I support as a user. I’ll be moving to alternative software.

I replied to this user that while, yes, VHX is a subscription now, it’s following the Sketch model. If you let your subscription expire, the app keeps working forever. You just don’t get new updates.

Anyway, they never replied back. So I looked up their previous order history and their only purchase was three years ago when they bought the app on sale from a bundle website for $0.99. So, you know what? Fuck that guy. You can’t please everyone.

Even more from The Iconfactory:

The only way for us to continuously preserve access to your work is for us to keep the app up to date as best we can. To be able to afford to do that, we need your support.

If you still don’t agree, then feel free to export your work and move on.

But back to my customers, here’s the type of engaged customer that makes my job a joy:

Now a subscriber. I am happy to support a sustainable future for the project. I had to jump ship for a while when you had decided to no longer support the earlier version, so it was great to be back in the fold when you brought it back to life! I think you are making a good choice with the subscription option.

Will the Sketch model pan out and keep my tiny company afloat? I have no idea. I don’t know if it will save Linea or anyone else. The rise of advertising supported software, VC funded apps that can operate at a loss for years, and companies who finance their apps via alternative streams of revenue have conditioned users to believe that software is a cheap commodity not worth paying for. We started heading down that path with the Web 2.0 bubble, and the rise of the App Stores have only accelerated that trend.

So, now it’s subscriptions or bust. Because, obviously, Apple is smarter than all of us and knows that paid upgrades aren’t a viable solution to a problem that was solved thirty years ago. That, or they’re simply not self-aware enough to realize that they, too, rely on upgrades – just in the form of hardware sales instead of software.

Addendum

And just to prove that I eat my own dog food, here’s a screenshot of my App Store Subscriptions screen:

Installing a CarPlay system into my ten year-old Subaru is easily one of the best purchases I’ve made in years in terms of how it’s changed the way I go about my day. Right up there with the original iPhone. I know lots of folks who swear by their iPad, and, sure, I like mine. But iPad has always seemed like a product in search of a problem. CarPlay feels like a product that is a solution.

CarPlay has made my morning and afternoon commutes far more enjoyable. And with so much dead time between home and work, I’m now trying to see if I can make some of that a bit more productive, too.

I’ve got a big ass post about using Siri Shortcuts with OmniFocus coming soon (I hope), but for now I want to talk about email.

I understand that Apple wants to limit what’s possible while driving for safety reasons, but Siri’s constant “I’m sorry, I can’t help you with that while you’re driving” responses are slightly maddening. All I want is for Siri to read my unread emails.

If I cheat and leave my phone unlocked while barreling down the interstate at 75mph, Siri will read my recent email. But that’s kind of it. I can’t perform common actions like archiving, deleting, marking as read/unread, etc. I can’t even reply. (Or if you can, I haven’t discovered the correct voice incantation to make Siri do so.) You can compose and send a new email though, so there’s that at least.

Nonetheless, what I’m really after during my morning commute is the ability to triage my inbox and prepare for my daily standup call that happens as soon as I arrive at work. Between the time I wake up and when I finish dropping my kids off at school and start my drive in, I simply don’t have time to read everything that came in overnight or from my coworkers in earlier timezones.

What I want is to process and clear out all the junk (not in the literal spam email sense) and get a handle on any messages that need action before the people who sent them blindside me about their status during our phone call.

So, I built something to do that. I call it Voxmail, and it’s free on GitHub.

There are two components.

1. A 49-step Siri Shortcut that you install on your iPhone.
2. A small PHP script that you can throw onto most any web server that the Shortcut communicates with. Other than two Composer requirements that deal with speaking to email servers, it’s plain, vanilla PHP. It doesn’t even need a database.

When you run the Shortcut, it connects to the PHP script, which fetches your email and returns it in a format optimized for the Shortcut to read, parse, and let you take action on.

Siri will speak a summary of your unread messages and then allow you to take action on each individual email. You can listen to the full email body. Or, you can archive, delete, mark as spam, mark as read or unread, and send a reply.

I’ve been using the script for a week or two and it’s fast, reliable, and awesome. At least with Fastmail. I have no idea how it will perform against a Frankensteined IMAP implementation like Gmail. But I’d love feedback! Also, it doesn’t support Exchange. So sorry. Pull requests are welcome, though!

How does it work in practice? Here’s a demo video…

And here’s another showing CarPlay in action…

Note: In this example I’m propping my iPhone precariously on top of my gear shift against the air vents. I’m just doing this so you can see the status of the emails updating live from the Siri Shortcut, to the server, and back down to my phone’s mail client. My voice commands and Siri’s output are going through CarPlay – not the phone. This all works with your phone happily locked and in your pocket. (And also on your watch and AirPods, too!)

And here’s the ridiculous Shortcut in all its glory.

Of course, you may be thinking: “Hey, you’re an app developer! Why didn’t you write an app to do this?”

I thought about it. I’ve actually written an iOS email client before, so I knew MailCore would be up to the task. And I’ve seen amazingly deep Siri integrations from other iOS developers like OmniFocus.

But I’ve done some basic Siri programming for my clients, and it was a royal pain in the ass to setup and debug. There’s no way I could have prototyped and built this project to the point it’s at in 7h 56m over the course of a weekend like I did. (I know the time down to the minute because Timing.app is amazing.)

And that’s not because of Swift or Xcode, etc. Swift’s static typing would have actually saved my butt a few times when I did some dumb PHP things. It’s all the damn hoops Apple makes you jump through just to run a bespoke app on your own device.

I wanted to build this for myself – not to make money or distribute through the App Store. And being an open source project, I sure wasn’t going to make pro-but-not-actually-developer-users who want to use it deal with certificates, provisioning profiles, etc. This whole Siri Shortcut / PHP script solution is a complete hack, but it’s easier than the alternative. Ok. Rant over.

So, go visit Voxmail on GitHub and give it a try. I’d love your feedback.

And if all of this was somehow already possible natively with Siri and I just wasn’t asking the right questions, boy do I feel dumb.

So, uhhhh. This isn’t good. I really hope I’m just confused and not sounding a false alarm, but…

Back in April I wrote a quick post about how I was backing up the shared iCloud Photo albums that my friends and family all use to send pictures and videos of our kids back and forth.

The conclusion of the post was that all of those shared pictures are not stored inside your primary Photos.app library. Instead, they were squirreled away deep inside here…

~/Library/Containers/com.apple.cloudphotosd/Data/Library/Application Support/com.apple.cloudphotosd/services/com.apple.photo.icloud.sharedstreams/assets/

Fair enough. And for someone who is crazy paranoid about backing up all of their data like myself, that was fine. I just pointed Arq at that folder and rested comfortably knowing all the photos of my kids and my nephew and my best friend’s kids were safely being backed up.

Not so much with Catalina.

A reader emailed me today to ask if I knew where that folder had been moved to after upgrading to 10.15. I’ll admit – I had no idea it had been moved. I didn’t even think to check.

So, I looked, and, sure enough, that sharedstreams folder is gone. But where? All said, the shared albums had previously been taking up 112 GB, so I turned to DaisyDisk again to look for another large folder that size, and…

Nothing.

But here’s the thing. Photos.app still had the Shared Albums preference enabled. And I can still see all of those shared photos and videos in the app.

I put on my developer hat and started watching the filesystem while I browsed my shared albums. I found that when I double-clicked a photo to open it to its full size, Photos.app downloaded the file and cached it locally here:

~/Pictures/Photos Library.photoslibrary/resources/cloudsharing/data/<PERSON ID>/<ALBUM UUID>/

(Where PERSON ID is I think an identifier assigned to your iCloud account? Maybe? Not sure. And ALBUM UUID is the unique identifier for a shared album.)

Ok. That’s great. It actually makes more sense to have shared photos also live inside your Photos.app library instead of somewhere random in ~/Library.

But here’s the shitty part I’m trying to figure out. Of the 112 GB my shared albums previously took up, only 250 MB (MB!!!) are cached locally.

Don’t get me wrong. This is likely an awesome thing for folks low on hard disk space. Keeping possibly hundreds of gigs locally for shared photos was probably a silly idea to begin with. But…

and that’s a big but…

I don’t trust iCloud with my photos. And I’m not trying to single out Apple. I don’t trust Google Photos either. I don’t trust any cloud with my data. I love the convenience of iCloud photo sharing with friends and family and using Google Photos as a source of truth for my own family archives. But I want a backup of my data – just in case – that I’m in control of.

And now I don’t know how to do that. Here’s the Preferences window for Photos.app on Catalina…

There’s an option to download the originals of my own photos/videos, but not for shared albums? And, try as I might, I couldn’t find a way to “select all” and tell Photos.app to download everything like you can with your music stored in iTunes Match.

So, uh, Apple? How can I get the full-res versions of my five years worth of shared photos and videos? I obviously have the originals of the pictures that my wife and myself took. But there are hundreds if not thousands of images of my kids taken by their grandparents as well as all the photos of my sister’s kid, our friends’ kids, etc. that I would very much like to keep safe.

And, just to make sure I’m not absolutely crazy or mis-remembering something, I took a look at my historical Arq backups. I upgraded my iMac to Catalina on October 7th. On October 6th, that folder of shared albums was 112 GB…

By October 8th, it was gone…

And they certainly weren’t moved inside my Photos.app library because that hasn’t grown in an equivalent size.

Just 112 GB of memories.

Gone.

(Note for the HN crowd: Obviously, as this post talks about, I have backups. I haven’t technically lost anything. But this new caching strategy does my backups no good going forward as I won’t reliably have the full-res content automatically cached locally and available to be backed up.)

Followup…

Nick Heer writes about AirPods at Pixel Envy:

It’s also the category of tech products that, I think, comes closest to feeling futuristic today

I’ve never managed to phrase it like that in my head, but that’s exactly my feelings, too. For two reasons:

One: AirPods are a damn marvel of technology and miniaturization. They’re literally the same form factor as the EarPods we’ve had for almost a decade, and yet Apple somehow managed to fit inside a Bluetooth transmitter, optical sensor, accelerometer, and still had enough room left over for five hours of battery.

And two: Nothing better illustrates Apple’s incredible talent at designing, building, and shipping a product when they own the entire stack. I think AirPods are the best product – hardware or software – that Apple has released in the last five years if not longer. Better than Apple Watch, the insane year over year CPU jumps, and massive software and hardware camera improvements. The only thing that comes close to that level of integration and seamlessness in my mind is Face ID.

If you had shown my 2002 self a pair of AirPods as I was busy unboxing my first iPod and its white, corded earbuds, I simply wouldn’t have believed you. (And then I would have grabbed them from your hands and run away.)