Backing up the maps and configs isn't a simple task. If you figure that the entire server folder is just about 8GB in size currently, it takes a long time to just copy a file over to a backup. Now, figure every time you make a change to an area, or grass grows, or a mob lays an egg, or you drop an item...it changes the chunk file. It's to the point that if done incorrectly, the backup contains files that are actually 5-10 minutes apart from each other in age. This isn't good for anyone.
The solution is to only copy files that have been changed since the last backup, and linux provides some utilities to do that. Cnc wrote a pretty good script to take care of this incremental backup, and it's the system we still use. There's 2 interesting principles at use here:
Archive copying, instead of true copying. Basically, if I copy file 1 to file 2, it doesn't actually make a second copy, it creates a link. Then, if file 1 is deleted, the actual file isn't deleted but preserved with file 2 being the owner. To actually delete the data, you have to delete file 1 AND file 2.
Rsync bit-by-bit incremental backing up. Rsync looks at the file's hash to see if any data in the file is changed, and only makes a backup if the file is changed.
So, let's see what the script looks like:
#!/bin/bash
#
# Minecraft Incremental Backup Script
# Author: Randy (cncr04s) (cncr04s@gmail.com)
# Modified By: Joseph Gullo (surfrock66) (surfrock66@surfrock66.com)
#
# This script makes an incremental backup of the current set of maps, then
# pushes the old backups to an older folder. The result is incremental
# backups with 30 minute intervals, stored in folders with names
# indicating their age. The key to this script is that backups
# only store changed files from the most recent backup, so there’s less
# reads from the current map, reducing the chance of errors in frequently
# changed chunk files.
#
# This is the name of the screen holding the server process
NAME=”Bukkit”
# This is the description of the server, used in sending in-game notices
DESC=”teh3l3m3nts Minecraft Server”
# Commit the server to do a write-to-disc for all chunks
screen -dr “$NAME” -p 0 -X stuff “$(printf “save-all\r”)”
# Send message to the server indicating beginning of backup, mostly as
# a notice to players that there may be lag
screen -dr “$NAME” -p 0 -X stuff “$(printf “say Backing up world(s)\r”)”
sleep 3
# Move into the Backup directory; all following commands use relative
# direcory references
cd /Backup
# Create a new copy of the last backup, using –archive and -links
# To explain those flags, look at these examples:
# cp -al file1 file2
# To really delete the file, you have to delete file1 and file2.
# If applied to a directory, “cp -al” creates hard links (equivalent to “ln”)
# recursively for all files inside the directory.
# This means that if the “original directory” is deleted the
# “copied link” does not lose its target.
# Practically, this means that unchanged files only exist once.
cp -al backup.01/. backup.00
# This pushes changes from the server directory into the copy of
# the most recent backup. It does 2 things; ignores the dynmap
# tiles, and deletes any files it’s overwriting. Also uses the
# -a archive flag, with use similar to that above.
rsync -a –exclude ’tiles’ –delete /var/bukkit/ backup.00/
# Store the date and time in a variable; write that variable to a file
_now=$(date +”%Y_%m_%d_%k_%M”)
echo $_now > backup.00/$_now.txt
sleep 3
# Return the console back to the bukkit directory
cd /var/bukkit
So that's the deal, so far so good, and once we get the new server and throw in the 1.5TB hard drive, we'll be keeping backups FOR EVER. I may write a for loop for that folder moving scheme. Just for the record, our current 60 backups take up 51GB...imagine how many we'll be able to keep :O
This entry was posted on Monday, October 15th, 2012 at 6:38 pm by surfrock66 and is filed under Owner's Corner.
You can leave a response, or trackback from your own site.
One Response to “Owner’s Corner: Incremental Map and Plugin Backups”
One Response to “Owner’s Corner: Incremental Map and Plugin Backups”
Nice. I’ve got a 1.5 TB on my desktop, and it makes me drool every time I think about it.