My most popular Mac app - and also the first app I built and brought to market way back in 2007 - is VirtualHostX. And after twelve years and 50,000 customers, I'm incredibly lucky to have a passionate user base who are invested in seeing the app thrive.
But I should have more.
If you look at a chart of new customers per month from August 2007 through November 2019, you can see there was a huge sea change trending downwards after I moved away from working with Apple's built-in web server, to managing my own inside a virtual machine with VirtualBox.
(The technical reasons for making that change are explained in this blog post.)
In talking with customers and looking at the little in-app analytics data I do collect, I'm fairly confident in the reason. And that's my dependency on VirtualBox. Specifically, its requirement to install a kernel extension on users' machines. And with Apple's security decision to crackdown on 3rd party kernel extensions, it's made onboarding new customers - especially non-technical ones - a nightmare.
We can debate the merits of Apple's tightening down of macOS in recent years, but that's not the point of this blog post. I'm fine with them making 3rd party kexts an opt-in choice for users. My issue is how their (lack of) UI has destroyed my conversion rate.
Plenty of people discover VirtualHostX through search and word of mouth. Downloads of the app are great. Always have been. But when new users go from my Welcome screen, to the VirtualBox installer, and then to the main window, my conversions fall off a cliff because I'm ultimately at the mercy of the VirtualBox installer doing its job. And, to be fair to the hardworking developers on the project, they're at the mercy of what Apple allows.
So, here's what the flow for my first-time users looks like. If you've never installed VirtualHostX before (likely, most of you reading this), here's what is supposed to happen.
- VHX presents the Welcome Window on first launch and any subsequent launches where it detects VirtualBox (or any other helper components) aren't installed.
- It prompts the user to click a button which launches VirtualBox's own
.pkginstaller. - When my app detects that the installation has succeeded, it takes you to the main screen where you can begin using that app.
At least, that's what's supposed to happen. Watch this video of me running the VirtualBox installer a few times in a row.
Here's what I want to point out...
As expected, macOS blocks the installation of the new 3rd party kernel extension and instead prompts the user to either open Security Preferences or pick the default option, which is to do nothing.
Like many of my non-technical users - or folks who are just in a hurry or don't pay close enough attention - I chose to click "OK" in the video.
But no matter what choice I pick, before I can even make a choice, the installation fails because the kernel extension was initially blocked. To be clear: the installer will fail, 100% of the time, on a clean copy of macOS before the user has the opportunity to approve or deny the kext.
But here's where it gets insidious. The first time you run the installer, to macOS's credit, the system at least does display the prompt to open Security Preferences. If, however, you're a saavy enough user to think to try and run the installer a second, third, fourth, fifth time, it will continue to fail - but will never again prompt you to approve the extension. So if you miss the initial prompt and aren't enough of a techie to know what you need to do to proceed, well...
So that's what my app is dealing with when I try and onboard a new customer. I (speaking as a fairly well-experienced Mac developer) don't know enough about .pkg files or kernel extensions (now that they require a specifically approved Developer ID account to distribute), to know for sure if this blame is on VirtualBox for not installing the kext the correct way, or if this really is Apple's chosen flow for installing and approving extensions.
In any case, I get a boat load of support requests from new customers wondering why my app isn't working. And I'm scared to think about how many simply never email and give me the opportunity to guide them through the process.
So, over the years I've done the best I could by writing step by step, illustrated support documents as well as sending out Getting Started drip email campaigns to new customers with explicit instructions. But my sales continued to decline.
So with this week's release of VirtualHostX Pro being the biggest update in the app's twelve year history, I decided to revisit the problem. And I think I found a solution. To be clear: it's an incredibly dumb, hack of a solution, but it works.
And like nearly all of my best ideas, it came from talking things over with a friend...
I don't really have say over what VirtualBox is doing and what macOS allows, but what if I took control of the process and did everything myself by wrapping their .pkg in my own .pkg?
A few hours later, it worked! And I shipped it last night. Watch this...
The first thing you'll notice is that I'm presenting my own obnoxious UI within Installer.app to try and grab the user's attention and prevent them from just automatically clicking "Continue". I want them to actually read what I have to say and need them to do.
And on the second screen I even show an annotated screenshot of the two approval steps they'll need to take. (By the way, that screenshot looks like crap because I have to embed it inside an .rtfd document, which doesn't support @2x images in any way that I could find. If you know of a method, please let me know!)
Moving on, the installer proceeds to install the software. But, here's the trick. It's not actually installing anything. The .pkg doesn't actually contain any installable assets. It just goes through the motions and runs my post install shell script.
It's my post installer script that then installs VirtualBox's .pkg in the background silently for the user. My .pkg already requires and requested admin rights, which are passed along to the VirtualBox installer.
Next, after their installer finishes, I do a quick check to verify whether or not VirtualBox's four kernel extensions have been installed:
KEXTCOUNT=`/usr/sbin/kextstat | grep virtualbox | wc -l | awk '{print $1}'`
If they aren't installed, I launch System Preferences to the appropriate Security & Privacy tab and hope that the user remembers my previous instructions to click the "Allow" button.
Many apps do similar things. I'm thinking of all the Mac utilities that rely on Accessibility permissions or Full Disk access. They'll open System Preferences to the correct pane and hope for the best.
However, my shell script keeps going and runs the VirtualBox installer again, pauses, and then opens System Preferences again. And it will keep doing this until it finally times out after twenty attempts - which in my testing translates into a few minutes.
The thing I really, really like about this approach is the experience for the user.
- The Installer.app continues to show the "Running package scripts..." message the entire time. To them, it appears that the installer is still continuing to, well, do its thing and install. Normally, at this point, the native VirtualBox installer would fail. But the
whileloop in my shell script keeps VirtualHostX's installer in a permanent pending state. - Users (myself included) are not very observant. It's super easy for them to miss the one-time kernel extension warning message - or even that System Preferences opened for them at all. In many of my early tests, System Preferences would open in the background in the middle of the screen behind my installer window and not even be noticeable. But every 10 seconds or so my script will bring System Preferences back to the front - hopefully ensuring the user sees it.
- Once they do click "Allow", on the next iteration of the
whileloop, my installer will detect the new kernel extensions and show a success message to the user.
I was giggling with excitement the first time I got this flow to work end-to-end, and I must have run twenty installs in each macOS going back to Sierra to verify its behavior. (Thank you Parallels!)
I have no idea if this will solve my onboarding problem and thus improve my conversions, but I'm really optimistic after doing a few hallway tests with select customers.
Until Apple gives 3rd party developers a way to properly request the permissions we need to build the apps our customers want, hacks like these are going to be the norm.