Aftermath

So last week was incredibly stupid.

Broken is #1 on Hacker News

I’ve had one thing I’ve written reach the #1 spot on Hacker News before, plus an old website that reached the top of the trending del.icio.us charts, as well as a few moronic tweets that garnered a couple hundred likes, and one appearance on Slashdot back in the day.

But the last seven days were my first real taste of how awful it must be to actually be, you know, internet famous.

So, because all this was new to me and I’m a data geek, I’d thought I’d share some final numbers. And if you’re a HN visitor, make sure you stick around for a fun surprise.

But first…

Let me say this. I’m often highly critical of Apple. Because I believe in their vision of computing, and also because my livelihood depends on their ecosystem remaining healthy. I’ve written many times on this blog about dumb bugs I’ve encountered on macOS and iOS. Even going so far as to refer to Photos.app as “fuckery”. And I’m no stranger to writing pithy, negative, reactionary tweets, either.

But here’s the thing. My big post from last week? The one you’re probably aware of if you’re reading this site right now? It wasn’t fair.

Don’t get me wrong. I fully stand behind every criticism I leveled at Apple. From the specific bugs, to the broader statements about detecting a lack of focus on the Mac in recent years, to my final thesis about their lock-step, annual release cycle hurting the company’s ability to maintain software quality.

But the part that wasn’t fair. The parts that I regret are my direct insults at those in charge. I’m all for eating the rich and all that, but in a general, class-warfare sense. When I’ve written mean things about Apple previously, I kept my words pointed at the company as a whole. Last week I took aim at a specific, small group of people. I forgot that for many of them, the Mac has been their life’s work. I’ve attended WWDC eleven times and had the opportunity to talk with quite a few Apple executives. They were great. And I don’t think they were just gaslighting me either. So I shouldn’t attack them for putting so much effort into a product that I also love – even when I vehemently disagree with them at times. Some folks inside the company reminded me of that in the past few days and called me out on my shit. And so, I’m sorry. I’ll do better.

Here’s what you came here for…

I guess I’ll go through this in the order it happened.

After a ridiculously stupid first fifteen minutes on Catalina, I posted this dumb screenshot and tweet as I was walking out the door to dinner. I thought I was exceedingly clever with my phrasing. Turns out, the internet did, too.

By the time we got our drinks, my phone was buzzing every five to ten seconds with another like or retweet. A little while later it had ramped to a twitter notification (literally) every second. And by the time dinner was over, those notifications stopped saying “@SoAndSo liked your tweet”. Instead, they changed to “15 people liked your tweet”. Every. Second.

At first it was fun. But by the time I was sitting in my kids’ room that night waiting for them to fall asleep, I realized my phone was simply unusable because the interruptions never stopped. Even after I turned off alerts, it was impossible to wade through all the noise in my mentions to find legit comments from people I was interested in. I’ve never used Twitter’s “quality” feature before to filter my notifications, because I’m not popular enough. But now I realize that feature is absolutely necessary for high-profile folks to actually use the platform.

Here are the final engagements numbers, according to Twitter, seven days later:

Tweet Engagement Statistics

And here’s a fun Twitter visualization. It’s me scrolling as fast as I can through my activity timeline over the past four days – Twitter won’t let me go back the full seven since all this started.

Scrolling Twitter activity timeline

And when watching that, keep in mind that the vast majority of those table cells are notifications like “12 people retweeted you”, not just a single “@timapple liked your tweet”.

So that tweet blew up on Monday. On Wednesday night I got the bright idea to write a lengthy followup about the Catalina bugs I ran into and my thoughts on the state of the Mac in general. 2,500 words over an hour or so. Do a quick check for spelling mistakes. Publish. Go to sleep.

Around 11am the next day, I received a Twitter DM from an Apple engineering manager who (I thought) had somehow stumbled across my post. And they very kindly asked if I could jump on a call to go into more details about my post and the bugs I identified. They were super awesome and I’m looking forward to working with them more this week.

But I did find it odd that it hit their radar (no pun intended) so quickly and that they would be so immediately proactive about reaching out. And that’s when, just a few minutes later, Vigil texted me that my web server had fallen over.

Both this blog and my company website were inaccessible. And so were all the other sites I host on this machine. I couldn’t even SSH in. I logged into Linode to check the recovery console or maybe just reboot the machine, but happened to glance at the recent activity graph.

Here’s the last 30 days network averages for this server – with the first big spike corresponding to the morning after I published the post.

I’m not an expert, but I’m hardly a server-side dummy either. My $20/month VPS should have been able to handle this amount of traffic. Why didn’t it? Because after running WordPress since 2006, I switched to Ghost earlier this year. Not because I was dissatisfied with WP, but I just wanted to learn more about Node and I thought it would be a good way to get my feet wet. Well, that experiment ended literally last Sunday night when I decided to switch back to WordPress…and forgot to turn on caching. ????

While I did go through the trouble to make sure all of my static assets were coming from a CDN…I. Forgot. To. Turn. On. WordPress. Caching.

Luckily, my domain was already using Cloudflare for DNS. So I immediately logged in and turned on their caching layer (it had been off while I was testing my WordPress migration earlier that week), and within ten minutes my server naturally began responding again. Thanks, Cloudflare!

And what does Cloudflare have to say about my traffic?

Cloudflare Traffic Graph

513,286 requests. And keep in mind, because all of my static assets are served from another CDN (Hi, Bunny ?), the only request being served through Cloudflare was the HTML for my post.

And just for the sake of completeness, here’s what BunnyCDN‘s graphs look like for that time period…

BunnyCDN Traffic Graphs

So, those are the overall technical stats for the week. Before I get to the really interesting surprise at the end, let’s take a look at some more unique / esoteric observations I noticed.

First, beyond the obvious conversations around Catalina that my tweet and post brought up, what was the topic that people emailed, @’d, and DM’d me about the most?

If you look closely at the Catalina screenshot I posted, you’ll see my desktop’s wallpaper is a hi-res satellite image of North America. Across all the above communication channels, I had nearly fifty different people ask me where it came from and if I could send it to them.

I was sorry to reply that I don’t have access to the image. Because it’s not a picture (well, yes, it is technically). It’s an app called Downlink made by the fantastic Anthony Colangelo. It sets your Mac wallpaper to a live satellite image of the continent of your choice every twenty minutes. And it’s free! Go get it!

Because I’m not normally popular, I’ve mostly forgotten that WordPress spam is a thing. But holeee shit. I don’t have comments enabled on my blog because I would rather folks respond by writing on their own blogs and linking to mine. You know, like how the web used to work before Facebook and Twitter destroyed all the good in the world.

But as my post climbed higher and higher on Hacker News, I started getting emails from my blog’s WordPress install telling me about new pingbacks. A small percentage were legit posts on other websites discussing what I wrote, but for the most part – nearly 200! – were spam blogs that just regurgitated what another credible website wrote or even each other. And they were all shitty, awful, wordpress.com subdomains like the following. (I’m posting an image of their URLs because I refuse to send them any google juice.)

Wordpress Pingback Spam

I know spammers are relentless and will ultimately ruin everything, and I certainly don’t envy Facebook’s and Twitter’s bot problem, but is WordPress.com (with the help of Akismet) not capable of shutting this stuff down? I don’t know. It’s not my area of expertise.

Moving on. With so many people visiting my personal website, did any of them explore further and reach my company website? Did it affect my app sales at all?

Yes!

Here are visits to my company website over the past 30 days:

clickontyler.com visits graph

And, more excitingly, here are my app sales for that period:

App sales graph

You can see the spike in sales and then how quickly they fall off again back towards normal. That bump certainly won’t make me rich, but it was a nice surprise.

Given that this post – and most of my blog and company website – are about the Apple ecosystem, what did the device breakdown by brand look like?

Device breakdown by brand

Apple devices (both macOS and iOS) were 66.7%. Google (specifically Pixels, not Android as a whole) was 5.4%. The unknown segment was 12.2%

43% were from the US. San Francisco being the most popular city – not surprisingly.

Ok, I’m almost ready for the real fun statistic. But one more thing before I get there. What channel sent the most traffic my way?

Top referring channels

Hacker News sent by far the most traffic – nearly 3:1 above Twitter in second place.

But of those acquisition channels, what source provided the most engaged visitors? This is the question I’m most interested in and what I’ve been teasing throughout this entire summary post. What I want to know is average time spent by referring website. Some of these results make sense. And others I found incredibly surprising.

I’ve segmented these time spent numbers to only apply to visitors of my popular blog post – those who entered my site via the short, earlier in the week one that basically just reposted my viral tweet aren’t included.

  1. instapaper.com – 7m 12s People who took the time to bookmark my post for reading later, must have really intended to read it.
  2. clickontyler.com – 5m 36s This isn’t that surprising given that these are visitors coming from my company’s website and likely already have some passing interest in my content. (And I did make sure to filter out visits from my own IP address.)
  3. mjtsai.com 4m 52s For those of you not familiar. Michael Tsai has run one of the definitive Apple developer link blogs for years. Not only does he round-up the best sources of information, he arranges them in a narrative structure that reads like a news article with his own commentary interspersed.
  4. relay.fm 4m 49s The kind folks over at the Connected podcast listed me in their show notes. I’m kinda scared to hear what their commentary was.
  5. newsblur.com 3m 13s My preferred RSS web app.
  6. theregister.co.uk 2m 26s Their reporter emailed me for a quote. I declined. So they published my email declining their request for a quote. I’m super media savvy.
  7. pinboard.in 1m 57s My favorite bookmarking service.

And now I have to skip through a couple pages in my stats app to get to the good stuff. Twitter and Hacker News.

Twitter, by nature, is probably going to rank low on time spent. And it did. 46s.

Hacker News?

There were two stories on HN last week linking to my website. The first amassed 315 comments. The second has 536 currently. And HN is generally known for having a good, mostly thoughtful comments section.

So, 851 comments – many of them full paragraphs or longer – about a 2,500 word blog post that should probably take an average of ten minutes to read.

Forty. Two. Seconds.