Backup

From Things and Stuff Wiki
Revision as of 21:24, 18 December 2015 by Milk (talk | contribs) (→‎To sort)
Jump to navigation Jump to search


General

See also Cloud#Storage

Specifics

"Delta based incrementals make sense for tape drives. You run a full backup once, then incremental deltas for every day. When enough time has passed since the full backup, you do a new full backup, and then future incrementals are based on that. Repeat forever."

Rsync

See Rsync 

-a, --archive
   archive mode; equals -rlptgoD (no -H,-A,-X)


-l, --links                 copy symlinks as symlinks
-p, --perms                 preserve permissions
-t, --times                 preserve modification times
-o, --owner                 preserve owner (super-user only)
-g, --group                 preserve group
-D                          same as --devices --specials



-X, --xattrs                preserve extended attributes

Basic command

rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} /* /path/to/backup/folder

/run/media/milk/3tbackup/silver_june_2013

todo: update excludes; 

rsync -aAXv /* /run/media/milk/3tbackup/silver_june_2013 --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*"} \
--exclude={"/media/*","/lost+found"} \
--exclude={"/home/*/.gvfs","/home/*/.thumbnails","/home/*/.cache","/home/*/.macromedia","/var/lib/mpd/*"}

Basic script

#!/bin/sh

if [ $# -lt 1 ]; then 
    echo "No destination defined. Usage: $0 destination" >&2
    exit 1
elif [ $# -gt 1 ]; then
    echo "Too many arguments. Usage: $0 destination" >&2
    exit 1
fi

START=$(date +%s)
rsync -aAXv /* $1 --exclude={/dev/*,/proc/*,/sys/*,/tmp/*,/run/*,/mnt/*,/media/*,/lost+found,/var/lib/pacman/sync/*}
FINISH=$(date +%s)
echo "total time: $(( ($FINISH-$START) / 60 )) minutes, $(( ($FINISH-$START) % 60 )) seconds"

touch $1/"Backup from $(date '+%A, %d %B %Y, %T')"

rsnapshot

  • rsnapshot - Local filesystem snapshots are handled with rsync. Secure remote connections are handled with rsync over ssh, while anonymous rsync connections simply use an rsync server. Both remote and local transfers depend on rsync. rsnapshot saves much more disk space than you might imagine. The amount of space required is roughly the size of one full backup, plus a copy of each additional file that is changed. rsnapshot makes extensive use of hard links, so if the file doesn't change, the next snapshot is simply a hard link to the exact same file.

Grsync

Arno's SmartBackup Script

luckybackup

unison

oldtime

syncbackup

rdiff-backup

  • rdiff-backup backs up one directory to another, possibly over a network. The target directory ends up a copy of the source directory, but extra reverse diffs are stored in a special subdirectory of that target directory, so you can still recover files lost some time ago. The idea is to combine the best features of a mirror and an incremental backup. rdiff-backup also preserves subdirectories, hard links, dev files, permissions, uid/gid ownership, modification times, extended attributes, acls, and resource forks. Also, rdiff-backup can operate in a bandwidth efficient manner over a pipe, like rsync. Thus you can use rdiff-backup and ssh to securely back a hard drive up to a remote location, and only the differences will be transmitted. Finally, rdiff-backup is easy to use and settings have sensical defaults.
  • rdiffWeb is a web interface for browsing and restoring from rdiff-backup repositories. It is written in Python and is distributed under the GPL license.

rbackup

Duplicity

  • Duplicity backs directories by producing encrypted tar-format volumes and uploading them to a remote or local file server. Because duplicity uses librsync, the incremental archives are space efficient and only record the parts of files that have changed since the last backup. Because duplicity uses GnuPG to encrypt and/or sign these archives, they will be safe from spying and/or modification by the server.

duply (simple duplicity)

DejaDup

  • Déjà Dup (day-ja-doop) is a simple backup tool. It hides the complexity of doing backups the Right Way (encrypted, off-site, and regular) and uses duplicity as the backend.

Duplicati

  • Duplicati is a free backup client that securely stores encrypted, incremental, compressed backups on cloud storage services and remote file servers. It works with Amazon S3, Windows Live SkyDrive, Google Drive (Google Docs), Rackspace Cloud Files or WebDAV, SSH, FTP (and many more). Duplicati has built-in AES-256 encryption and backups can be signed using GNU Privacy Guard. A built-in scheduler makes sure that backups are always up-to-date. Last but not least, Duplicati provides various options and tweaks like filters, deletion rules, transfer and bandwidth options to run backups for specific purposes.

Areca Backup

BackupPC

AMANDA

"the Amanda planner runs on the server to decide exactly how to go about backing things up. It, too, contacts each Amanda client and requests an estimate of the size of full and incremental dumps for each DLE. It then does some complex planning based on the history of each DLE, the estimated sizes, the available storage space, and a number of tweakable parameters to decide what to back up. This often confuses newcomers, who have control issues and want to tell Amanda when to do full backups and when to do incrementals. The planner is one of Amanda's strengths! Don't fight it!"

Bacula

Backup Ninja

  • Backupninja allows you to coordinate system backup by dropping a few simple configuration files into /etc/backup.d/. Most programs you might use for making backups don't have their own configuration file format. Backupninja provides a centralized way to configure and schedule many different backup utilities. It allows for secure, remote, incremental filesytem backup (via rdiff-backup), compressed incremental data, backup system and hardware info, encrypted remote backups (via duplicity), safe backup of MySQL/PostgreSQL databases, subversion or trac repositories, burn CD/DVDs or create ISOs, incremental rsync with hardlinking.

DAR

  • Disk ARchive is a shell command that backs up directory trees and files, taking care of hard links, Extended Attributes, sparse files, MacOS's file forks, any inode type (including Solaris Door inodes), etc.

backup2l

  • backup2l - low-maintenance backup/restore tool. backup2l is a lightweight command line tool for generating, maintaining and restoring backups on a mountable file system (e. g. hard disk). The main design goals are are low maintenance effort, efficiency, transparency and robustness. In a default installation, backups are created autonomously by a cron script. supports hierarchical differential backups with a user-specified number of levels and backups per level. With this scheme, the total number of archives that have to be stored only increases logarithmically with the number of differential backups since the last full backup. Hence, small incremental backups can be generated at short intervals while time- and space-consuming full backups are only sparsely needed.

Obnam

  • Obnam is an easy, secure backup program. Snapshot backups. Every generation looks like a complete snapshot, so you don't need to care about full versus incremental backups, or rotate real or virtual tapes. Data de-duplication, across files, and backup generations. If the backup repository already contains a particular chunk of data, it will be re-used, even if it was in another file in an older backup generation. This way, you don't need to worry about moving around large files, or modifying them. Encrypted backups, using GnuPG.

Box Backup

ZBackup

urbackup

File and image backups are made while the system is running without interrupting current processes. UrBackup also continously watches folders you want backed up, in oder to quickly find differences to previous backups. Thus incremental file backups are really fast. Your files can be restored through the web interface or the Windows Explorer while the backups of drive volumes can be restored with a bootable CD or USB-Stick (bare metal restore).

Bup

burp

  • Burp is a network backup and restore program. It uses librsync in order to save network traffic and to save on the amount of space that is used by each backup. It also uses VSS (Volume Shadow Copy Service) to make snapshots when backing up Windows computers.

ddar

ZPAQ

SNEBU

Attic

BorgBackup

  • BorgBackup (short: Borg) is a deduplicating backup program. Optionally, it supports compression and authenticated encryption. The main goal of Borg is to provide an efficient and secure way to backup data. The data deduplication technique used makes Borg suitable for daily backups since only changes are stored. The authenticated encryption technique makes it suitable for backups to not fully trusted targets.

To sort

Btrfs

See also *nix#Btrfs.

Using btrfs snapshots instead of cp -al has two major advantages. First of all creating a snapshot is much faster than using hardlinks. and the second advantage is, that meta-information about the file will be preserved (ownership, access and modification-time and also file-attributes). When using hardlinks this information will have the state of the most recent backup-process (also for older backups). Last but not least, if you use a new version of btrfs you can also lock the snapshots down to be read-only.

"Of course, the problem with this is that snapshots are, essentially, COW hard links; this means that if there's a corruption on the disk for a file, it'll affect all child snapshots."

"Rsync integration. Now that we have code to efficiently find newly updated files, we need to tie it into tools such as rsync and dirvish. (For bonus points, we can even tell rsync _which blocks_ inside a file have changed. Would need to work with the rsync developers on that one.)"

basic snapshot management

from+to btrfs

rsync based

Imaging

Other

Retrieved from "https://wiki.thingsandstuff.org/index.php?title=Backup&oldid=32448"