July 2020 – tyler.io

I’ve been a paying customer of Vimeo since 2014 – specifically, their Pro plan.

(Please ignore that the above email receipt is for their Plus plan. Shortly after subscribing I realized I needed to upgrade to Pro, which supports commercial use.)

I needed a reliable way to host a bunch of screencasts and video tutorials I was producing for my software. I went with Vimeo because they’re the obvious choice in this space for two main reasons:

I didn’t want my content hosted on YouTube where it would be surrounded on all sides by an infinite expanse of flaming garbage. (Don’t take that criticism too harshly – I’m a happy YouTube Premium subscriber.)
Vimeo is really, really, really good at what they do. I mean, supremely top-notch when it comes to ease-of-use, controlling the end-user experience, speed, and quality.

But when my renewal email arrived in April, myself and other small developers were seeing sales slow down as the pandemic worsened.

Another $240/year was a tough sell for the small amount of video content I was hosting with them, and I wondered if there might be a cheaper alternative – either another service or by hosting videos myself.

If you’re posting commercial content – like I often do – Vimeo’s terms require you to be a Pro subscriber or higher. And to be clear, I still think Vimeo at that price (at almost any price, really) is a great value. It’s cheaper than a company with four employees paying for Slack.

But at my tiny scale and thinking about cutting costs, given the amount of video content I’m actually posting each month and hosting with them, and also being a huge nerd, it didn’t make sense any longer.

So, let’s do this ourselves.

We need three things:

Somewhere (cheaper than Vimeo) to serve video from.
A way to play the video on our website.
An easy way to convert raw video into an appropriate size and format for streaming.

So, where to actually host the video files? My requirements are somewhere reliable that won’t fall over if you get Slashdotted (ha, remember that?), can deliver quickly to visitors in any location, and won’t bankrupt you.

I have plenty of space available (and almost always enough bandwidth available) on my Linode (affiliate link – but Linode is wonderful) servers. But even my primary $20/month VPS with them was brought to its knees multiple times when traffic spiked. (Granted, I was running WordPress at the time.) And I don’t want to take the chance of my business website going down because one of my dumb blog posts on this website gets noticed.

A lifetime ago, I wrote about using S3 as a poor man’s content delivery network. That’s still an easy and reliable choice, but the cost for bandwidth doesn’t scale as a true CDN. Which is, of course, why Amazon offers CloudFront. But even then, at $0.085/GB, I’ve had multiple months in the past calendar year that would have been over $200.

Instead, let me introduce you to my secret weapon: BunnyCDN. (I get credit if you click that link. But I only use it because I completely love their service as you’ll see below.)

I switched to Bunny last October after using MaxCDN (now StackPath) since 2011 for all of my static assets. And I’ve been utterly flabbergasted at just how damn good they are. For a number of reasons:

The main metric that matters: they’re really fast. I’m seeing equivalent performance as what I was used to with MaxCDN and the larger CDNs I’ve used at my day job.
The two times I’ve needed to email them with a support question, a knowledgable, helpful human has replied quickly and without needing to go through dumb self-serve solutions.
Bunny’s control panel where you manage your account, choose bandwidth settings, etc. is super easy to use and incredibly powerful (at least for my needs). It’s a really nice change from what I had with MaxCDN and the AWS Console (powerful as hell, but obtuse).
Holeeeee crap their bandwidth is cheap.

For my static website assets (images, stylesheets, JavaScript, etc.) I use Bunny’s standard tier, which serves my content from all of their PoPs. That means visitors will connect to the server closest to them at $0.01/GB (for US and EU PoPs).

However, my video content uses their volume tier, which is distributed from 7 PoPs (instead of 41). While still being plenty fast enough, that reduces the cost further to $0.005/GB. That works out to a cool $5/TB when my traffic spikes.

I have zero understanding of how bandwidth is actually priced globally between providers or why some companies charge such a premium compared to others for the same bits over the wire. I assume you’re paying for reliability and/or speed? All I know is that I haven’t seen a blip of downtime or noticed any slowness in the nine months I’ve used BunnyCDN. I can’t explain how they can be so much less expensive. Maybe they won’t scale to “real” traffic? But for what I need, they’ve been a phenomenal choice.

(Ok, I’m done going on and on about them now. This really isn’t any sort of paid or sponsored content crap for Bunny. I’ve just been so impressed since switching, that I finally wanted to give them recognition.)

So that’s my answer to point #1 – where to host the video files.

The next thing we need is a way to show those videos inline on your website.

Gone are the days of embedded Flash video. Now, you can mostly get away with just using the native HTML5 video element.

However, if you want more control over the appearance of your video and how it behaves, take a look at Video.js. It’s a wonderful open source player from the kind folks at Brightcove that provides a similar level of control as the point and click customization options that Vimeo offers.

The last puzzle piece is getting your original videos into a web-friendly format and uploading them for distribution.

BunnyCDN offers their own online storage zones that you can simply upload to via FTP, but for historical reasons, I still store my large app downloads and videos on S3 and then have Bunny (and previously MaxCDN) pull from AWS as an origin server.

And as for the correct video format, I’m using Handbrake’s command line interface to convert my source videos files into mp4 using the Vimeo YouTube HQ 720p60 and Vimeo YouTube HQ 1080p60 presets.

But, because, hey, I’m a geek who doesn’t like doing extra work, I’ve wrapped up the whole process into a short shell script that

Converts the source video to 720p
Converts the source video to 1080p
Uses qlmanage to output a frame of the video as a png for previewing in the HTML player before the video starts
imagemagick to convert that png to a jpg
Uploads all three assets to S3 with s3cmd
Outputs the custom shortcode I use to embed videos in my blog posts

I just pass my video’s filename to that script, and a few minutes later I can copy and paste the output into my text editor and make it available on my website. Honestly, it’s an even faster process than using Vimeo.

Here’s the script. You’ll of course want to modify the paths to make it work with your own setup since it’s completely specific to my needs right now.

#!/bin/bash

bucket=$1

base=$(basename -- "$2")
extension="${base##*.}"
filename="${base%.*}"

p720="/Users/thall/Dropbox/Freelance/Videos/720p/$filename.mp4"
p1080="/Users/thall/Dropbox/Freelance/Videos/1080p/$filename.mp4"
png="/Users/thall/Dropbox/Freelance/Videos/jpg/$base.png"
jpg="/Users/thall/Dropbox/Freelance/Videos/jpg/$filename.jpg"

/usr/local/bin/HandBrakeCLI -Z "Vimeo YouTube HQ 720p60" -i "$2" -o $p720
/usr/local/bin/HandBrakeCLI -Z "Vimeo YouTube HQ 1080p60" -i "$2" -o $p1080

/usr/bin/qlmanage -t "$2" -s 1080 -o "/Users/thall/Dropbox/Freelance/Videos/jpg/"
/usr/local/bin/magick convert $png $jpg
rm $png

s3cmd put --acl-public --guess-mime-type $p720 "s3://$bucket/720p/$base"
s3cmd put --acl-public --guess-mime-type $p1080 "s3://$bucket/1080p/$base"
s3cmd put --acl-public --guess-mime-type $jpg "s3://$bucket/jpg/$filename.jpg"

echo "[tylervideo url=\"$base\" img=\"$filename.jpg\"]"

So that’s how I moved off Vimeo and started hosting my video content myself. On average, my bandwidth bill is about $11/month – and that includes videos, static assets, and ALSO binary downloads for all of my Mac apps. Previously, I was paying $20/month just for video hosting on top of the rest of my bandwidth.

It’s definitely a geekier solution that requires more work up front to setup, and I’m not sure I would recommend it for a “real” business, but for my needs it was a fun project and I’m happy to save $200 a year.

The fifteen days since my last post are the longest I’ve gone all year between writing. My resolution at the start of this calendar year was to write 50 posts in 2020. So, I’m not quite behind yet, but I am glad that my reasons for missing two weeks eventually led to the topic of tonight’s post.

The first, and most practical reason I’ve fallen behind my writing schedule is that June and July are the busiest months for my 9 to 5 job. (Well, lately, more like my 8 to midnight job.)

To keep myself sane while dealing with my work deadlines, I’ve found myself tinkering around with an idea I’ve wanted to try building for years now. Oddly enough, it’s not yet-another-app, but a website (web service, maybe?). And it’s actually something that’s designed to be self-hosted. I haven’t yet decided if it will (eventually) be open source, or if I might solicit feedback from friends (real and online) just in case it’s more useful than I think.

But I digress.

This topic of this post is about a pattern in how I work – my process – that I only recently noticed the other day. In hindsight, it’s how I’ve always built things – how I’ve approached new projects. But for some reason, I never clued into the fact that, yes, this is how I work. These are the same steps I take with each new “thing”. And this is the step I’m currently on.

It was that realization that prompted this tweet last week.

Step 1. Build it as fast as possible.

Step 2. Wait 48 hours.

Step 3. Try again.

That’s not exactly it, but close. The longer version is…

Sometimes the idea for something new comes in a flash of inspiration. And other times (as in the current case) it meanders around in the back of my head for years – just waiting for the right moment or combination of external factors.

For this project, it’s the result of the rebirth of the indie web movement, my long time interest in self-hosting and owning the tools and data I run my business with, and Apple’s WWDC announcements about Safari and their OS’s upcoming privacy improvements.

But whatever the idea or genesis, there’s always a tipping point where that initial spark really catches fire and I have to build it now. I don’t know how to describe the feeling I get in my head (my whole body really) other than electricity-at-the-thought-of-new-possibilities. It also feels incredibly similar to when the dumb part of my brain is trying to convince the smart part of my brain that the ridiculous impulse purchase I’m considering is actually a good and rational thing to buy.

And so, full steam ahead, I try and build a proof-of-concept as fast as I can. I give no regard to code quality, architecture, performance, look and feel, anything. My only goal is to see if I can make it work in some functional way in the simplest form possible.

Writing code in this state is an ultimate high for me. It’s a pure act of creativity. Sure, I enjoy alcohol and other substances, but no feeling I’ve ever gotten from those things can compare to the literal buzz I get when my hands on the keyboard are perfectly in-sync with the instructions pouring out of my head.

And while the word may be in my job title, I hate calling myself an “engineer”. Beyond the math, there’s almost no engineering involved in how I approach the code I write. That doesn’t mean I’m not careful, thoughtful, or that I don’t make well-reasoned decisions, but I’ve always viewed my development style more like art – sculpture, specifically.

I start with a raw block of material and massage and shape it towards a final form. Sometimes, especially at the start, I may have no idea what the end goal is. But over time as I add new pieces, move and rearrange others, it takes on more definition, structure, and patterns emerge. If you look closely, you’ll see influences from prior art and other developers, too.

That’s all to say that the things I build are rarely assembled from a blueprint. They’re organic, and I’ve often found myself at odds with other peers in the industry because of that. But that’s a topic for another post.

Once the prototype is done, once I’ve proven to myself that my idea is possible, I let it sit.

For a day. For a week. In one unique instance, seven years.

During this time I focus on other things. My real work, my family, baseball (2020 season?), whatever. Anything other than the new project. And as I go about my day, my brain will naturally start to surface ideas, features, goals, must-haves, and nice-to-haves. All the little details and tentpoles that would eventually define the full scope if I choose to move forward.

If I’m on my phone or in the car, I’ll jot down (or dictate) the idea into Drafts. If I’m on my Mac, the note will go straight into Trello, which is where all of my personal projects are tracked.

It’s also the naming phase. I almost never have an idea for what to call the project when I start building it. But during this ideation downtime, coming up with a good name for the new thing seems to be a natural byproduct of listing every possible feature.

And so, once I’ve given myself enough time to sit. To think. To let all that initial, throwaway code settle and be still in my mind, I’ll revisit it with fresh eyes and attempt to place it into one of four buckets.

This is crap / dumb / stupid / a nice distraction but not worth pursuing.
This is great – but only for me.
Other people might like this.
Other people might pay for this.

In the first case, I save everything to git if I haven’t already, and move on. (I never throw away code, though.)

Number two is often very likely. I have lots of one-off projects and tools that are only useful for me.

If I think it’s useful and worth sharing, and if I’m also willing to stomach the shit-storm of asshole internet trolls who find joy in complaining about the quality of code other people publish for free, I’ll post it to GitHub. I love doing this.

But if I decide it might really be useful, typically based on feedback from friends, I’ll begin thinking about how long it might take to reach an MVP if not even a 1.0. Is there a valid business model? Is it something I feel comfortable asking strangers to pay money for in exchange for the responsibility that places on me as the developer and support person?

And if all those stars align, I’ll officially add it to my Trello development calendar, and it’ll be done just as soon as I finish the other 437 projects ahead of it.

But once the project reaches this stage, I know enough about myself to understand that I do my best planning and thinking visually. And given that the prototype was built with almost no thought to UI / UX, and that I now (hopefully) have a long backlog of features in Trello, I’ll spend whatever amount of time it takes to settle on a “final” look and/or flow that I can move forward with. (I put “final” in quotes because, ha, this stuff is never really done and will be changing forever.)

It’s just my personality and a near like writer’s block in my head, but until I’m convinced I’ve nailed down the main window (if it’s a Mac app), the first few screens (iOS app), or HTML/CSS structure (web app), I can’t write any more code. It’s impossible for me to move beyond the working prototype stage until I have a basic shell or the chrome to put my ideas into.

It doesn’t matter if the UI I settle on will eventually be completely thrown out and replaced, but it has to reach some initial threshold where my inner critic says “Ok, that’ll do for now.” Only then can I continue.

And it’s that first, final design stage I find myself in with this current project. I’ve got a working prototype churning away on a DigitalOcean VPS this very moment that I’ve been using every day this past week. And I’ve got 200+ cards in my Trello backlog. And I’m aching with excitement to stretch some development muscles I haven’t exercised in years.

I love this part.

DIY Video Hosting

Process