Fixing Broken Backblaze B2 Scripts when Run From cron

Just a quick note for my future self and anyone else who might be running into this problem.

Last week I migrated all of my backups off of Amazon S3 and rsync.net to Backblaze B2. The cost savings are enormous – especially for a small business like myself. And the server-to-server transfer speeds using their b2 Python script, while not as fast as using a raw rsync connection, are quite a bit quicker than using S3.

Before committing to B2, I gave it a really thorough test by seeding it with 350,000 files totaling 450GB. The whole process took about eight hours coming from my primary Linode server in Atlanta. I was quite pleased.

Anyway, after testing all of my scripts, I put them into cron and ignored them for the next few days assuming they’d “just work”. But when I went back to check on them, I found every one had been failing silently.

At first I thought maybe the b2 command wasn’t found in $PATH when running via cron for some reason, but that wasn’t it. Next I double-checked that b2 was using the correct credentials I had previously authorized it with by hand. Nope.

Turns out, b2 was throwing this Python exception.

Creating a Pipfile for this project...
Creating a virtualenv for this project...
Traceback (most recent call last):
  File "/usr/local/bin/pew", line 7, in <module>
    from pew.pew import pew
  File "/usr/local/lib/python2.7/site-packages/pew/__init__.py", line 11, in <module>
    from . import pew
  File "/usr/local/lib/python2.7/site-packages/pew/pew.py", line 36, in <module>
    from pew._utils import (check_call, invoke, expandpath, own, env_bin_dir,
  File "/usr/local/lib/python2.7/site-packages/pew/_utils.py", line 22, in <module>
    encoding = locale.getlocale()[1] or 'ascii'
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 564, in getlocale
    return _parse_localename(localename)
  File "/usr/local/Cellar/python/2.7.13/Frameworks/Python.framework/Versions/2.7/lib/python2.7/locale.py", line 477, in _parse_localename
    raise ValueError, 'unknown locale: %s' % localename
ValueError: unknown locale: UTF-8

I’m hardly a Python expert, and I’ve traditionally had nothing but problems anytime I’ve had to do anything with pip, so this didn’t surprise me. What did surprise me was that this error was happening both locally on my Mac (10.14.4) and on my remote Ubuntu 18.04 box.

After some googling I found this bug in pipenv. The solution is to add the following to your b2 scripts that are run by cron:

export LC_ALL=en_US.UTF-8
export LANG=en_US.UTF-8

And that fixed it.

I know macOS ships with a mostly-broken installation of Python, but the latest Ubuntu LTS? Anyway, if this is common Python/pip knowledge, at least now I know, too.