Install Tarsnap On a DigitalOcean VPS or Any Ubuntu 14.04 LTS System

Online backups for the truly paranoid

Securely and remotely backup your server using Tarsnap

Tarsnap is a great service. It’s extremely affordable and secure. I mentioned it briefly in my previous post about MySQL backups.

Tarsnap is only $0.25 / GB-month for storage and $0.25 / GB for bandwidth, which is extremely affordable. You can add funds to your account whenever you like, which is also very nice. Your initial fund deposit must be at least $5. You can signup for a Tarsnap account here.

Installing it on a stock Ubuntu 14.04 LTS installation requires some additional steps to get everything working nicely.

1. Install dependencies

2. Install Tarsnap

Download Tarsnap, I do it like this with wget:

Now we need to extract, configure, and compile Tarsnap.

3. Configure Tarsnap

First, copy the example config to the live config:

If you receive an error with that command, like mkdir: cannot create directory , ignore it and continue on.

Next, we need to generate a tarsnap key for your machine. I like to keep my tarsnap key in my home directory, so I run something like this:

You’ll be prompted for your Tarsnap password when running tarsnap-keygen.

Now, edit the tarsnap.conf file:

Point the keyfile directive to the key file we created a couple steps ago. The top of your tarsnap.conf file should look similar to this now:

4. Use Tarsnap to Make a Backup

You’ll want get familiar with the Tarsnap manpages. To create your first archive, with a name of servername-20140805, do this:

That will backup the /home/youruser folder to Tarsnap! Depending on the size of the backup and speed of your connection, it could take quite some time to finish the backup.

Below is the output of tarsnap –help, if you’re interested.
tarsnap-help

You can read more about Tarsnap at their homepage, tarsnap.com. They also have a page describing their infrastructure setup, which is kinda neat. Tarsnap also runs a bug bounty program.

I’ve only tested this on DigitalOcean VPS’s running Ubuntu 14.04 x64, but it should work on most Ubuntu variants.

Did I miss something or get something totally wrong? If so, please let me know, the comments are open!

Backup and Compress a MySQL Database With One Command

backup_mysql

I’ll show you how to restore the backup, too!

I like to use simple bash scripts to do various tasks. Backing up MySQL is one of them.

I recently decided to start compressing my MySQL backups, as I started including all databases in one fell swoop. I use bzip2 to compress the .sql files produced by mysqldump. bzip2 is standard on pretty much every *nix operating system, so you likely won’t need to install it.

I’m also using Tarsnap for backups now, which is a great service, btw. So cutting the size down on the backups sent to Tarsnap will save me a bit of money. I’ll be doing an article later on that focuses entirely on Tarsnap. I’m pretty in love with it.

Online backups for the truly paranoid

A secure online backup service.

Anyway, here’s the command I use to backup and compress all databases on my MySQL server:

That will create a backup of all databases. The -cq9 piece in the bzip2 command uses stdin for input and tells bzip2 to be quiet. The number, 9, specifies the compression level that bzip2 should use.

The script embedded in the Gist below is what I use to all my databases to /home/tyler/mysql-backups/, and then that folder gets backed up to Tarsnap.

To restore the database, you’ll want to bunzip2 the .sql.bz2 file first:

That will leave you with a backupname.sql file. Then bring the resulting .sql file into MySQL like so:

The databases will need to either already exist, or there will need to be CREATE DATABASE statements in the .sql file. It’s up to you. I like to create my databases before hand, but it’s just personal preference.

That’s all there is to it. How do you take care of your backups? I’d love to hear how others are doing it. Comments are open.

There’s a thread going on at Hacker News, too.

Update: Made a slight modification to the code and gist suggested by sluggo.

Update 2: HackerNews user Nanzikambe suggested the method above will destroy disk I/O on your server. He suggests using ZFS snapshots instead. The example he posted is in the Gist below, and includes the ability to send the backup to a remote server. A good tutorial on backing up MySQL using ZFS can be found here.

Receive Alerts On SSH or SFTP Logins with Papertrail

papertrail

Frustration-free log management, plus a lot more

I’ve been a huge fan of Papertrail ever since I discovered it, probably about a year ago or so. I use it mostly to monitor server logs. I currently have two servers setup to send syslog messages to Papertrail.

The Papertrail Events dashboard can be a bit overwhelming at first, but the provided search is powerful and allows you to finely control which log messages you see and which you don’t.

You can even setup saved searches to fire when a specific event occurs. For example, I have a saved search that searches for the following:
Accepted publickey for tyler

When that message shows up in Papertrail, it means that I logged in, or that someone else has logged in using my SSH key. This can be quite handy, especially if you’re a one man shop like me and are usually the only person that has SSH or SFTP access to a server.

Getting a DigitalOcean VPS added to Papertrail, especially if it’s running Debian or Ubuntu, is super easy. It just requires that you modify /etc/rsyslog.conf and add a line to the end of the file that will send a copy of the system logs to Papertrail.

Papertrail can monitor application logs, too, such as Apache httpd logs and MySQL server logs, although that takes a bit more configuration to get working properly.

If nothing else, it’s just nice having system logs aggregated in one central place, where everything is easy to search through, making it easy to find exactly what you’re looking for. If you’re an admin for one server or hundreds of servers, Papertrail could turn out to be one of your favorite tools. It’s definitely one of my favorites.

I suggest you give Papertrail a try, can’t hurt, they even have a plan that’s free forever. It’s definitely a great service for monitoring server logs.

Copy.com Referrals Give Crazy Amounts of Free Cloud Storage Space

copy_bonuses

Lots of free cloud storage, 5GB for every referral :)

I use Copy a lot. I still maintain a Dropbox account for sharing files with friends who still use it, which is most of them honestly. But, I have such a crazy amount of free storage at Copy that I use it for absolutely everything.

All of my photos and video go there from my Nexus 4. On top of that, when my Nexus 4 gets full, I’ll just archive it to Copy, so I essentially have two of every photo and video in Copy. Dropbox holds just one copy of all videos and photos, I only have about 20GB of storage there, so that stuff eventually gets brought down to a local drive. Which I should probably just bring it local right off the bat and just skip Dropbox all together for that stuff. But I digress…

I’m currently using about 19GB out of 185GB available to me. It’s all due to this post I made back in August of 2013. It resulted in a bunch of referral signups, which gave me an additional 5GB each. I started out with 15GB of storage.

If you’re a blogger and want some decent amounts of free cloud storage, check out Copy. Drop your referral link in relevant posts and you’ll slowly start building up referrals and gaining additional storage.

Copy has clients for Android, iOS, Linux, Windows, MacOS, and probably others. Just visit their download page. I’ve been very impressed with Copy and have been using them for nearly a year. They’re run by Barracuda Networks.

I dig Copy’s Fair Storage concept, which they explain like this:

We believe in a simple concept of fairness. Everyone paying for the same data they are sharing doesn’t work for us. We think that’s like going to dinner together and everyone having to pay the entire bill. People sharing content should equally divide the amount of storage being used. With Copy, you can split the bill. So a 12 GB folder shared between 3 people only counts as 4 GB per person.

Unless I’m missing something, that sounds as lot better than what Dropbox does. With Dropbox, if someone shares a 4GB folder with me, that’s 4GB taken away from my available storage, plus the other persons. So both of us ending up getting the 4GB taken from our accounts. Fucked, but that’s a topic for another post.

I still think you should check out Copy. I seriously recommend it, and that’s after I’ve been using it constantly for nearly the last year. I’ve a couple minor gripes about the Linux client, but I can deal with them.

Affiliate links are rampant throughout this post. If you don’t want to help me out, here’s a non-affiliate link for Copy, and here’s a non-affiliate link for Dropbox. Pretty sure you get extra space if you sign up from an affiliate link though, so, your call. Here’s the affiliate Copy signup link and the affiliate Dropbox signup link. :)

How-To: Install lolcommits On Ubuntu

lolcommits

Lolcat-style photos as you commit

I’ve always had problems installing lolcommits on Xubuntu and other Ubuntu-based Linux distributions.

The installation instructions are very simple. Only requiring you to run two commands, sudo apt-get install mplayer imagemagick libmagickwand-dev and then sudo gem install lolcommits (need sudo for linux). Pretty simple.

The gem install lolcommits command is where things usually go bad for me. I typically see something like this:

To fix this, you need to install a newer ruby-dev package:

You can now try to install the lolcommits gem again. It’ll actually install this time:

A GitHub user documented this solution in issue #54. Another user suggests that the installation guide should be updated to make a note of this, but I haven’t seen it noted anywhere but in issue #54.

It’d sure save me a bit of time if it was noted somewhere, that’s partly why I’m writing this post.

After you’ve got lolocommits installed, see the README on GitHub for usage instructions and examples.

I usually run lolcommits --enable --delay=2 --fork when enabling lolcommits. That will capture a photo in a forked process, after a 2 second delay. I like this method because you’re not left waiting for the photo before being able to type into your terminal again.

Lolcommits is kinda cool, but not really useful in a practical sense. I do use it pretty much everywhere though, and have the default storage location linked to Copy. That way all my images are in the same place, no matter which machine I’m using at home.

If nothing else, it’s something kinda neat to be able to offer to your clients.

tyler-hat-lolcommits

I no longer smoke. ;)