For what it's worth, I've just put the python script I use for keeping a
local ZFS filesystem in synch with a remote one by transferring snapshots
onto github here:
https://github.com/mafm/random-public-code
Basically, it looks for the most recent snapshot name the two filesystems
have in common and then incrementally transfers all snaphots since then
over, after destroying any local snapshot that doesn't appear on remote.
It seems to work fine for me in practice, though it could be improved. If
anyone wants to take it, and improve it, feel free.
The script gets called by the root crontab something like this:
*/5 * * * * export PYTHONPATH=/wherever; /usr/bin/python
$PYTHONPATH/replicate_zfs_snapshots.py sydney
tank-microserver-0-mirror-2tb/share/kapsia
tank/sydney-tank-replica/share/kapsia
Code below:
#!/usr/bin/env python
"""Usage: replicate_zfs_snapshots.py <remote-host> <remote-filesystem>
<local-filesystem> [-h | --help | -v | --verbose | -q | --quiet | -n |
--dry-run]
-n --dry-run
-h --help Show this
-v --verbose Log more than default
-q --quiet Log less than default
Example:
python replicate_zfs_snapshots.py sydney
tank-microserver-0-mirror-2tb/share/kapsia
tank/sydney-tank-replica/share/kapsia
This script synchronizes ZFS snapshots between filesystems on a local and
remote linux box.
All logging output generated by this script is written to syslog.
We assume that:
* passwordless ssh is set up between host running this script
and the remote host.
* the user the ssh connection logs in to on the remote host is allowed
password-less sudo on read-only commands (see /etc/sudoers.d/zfs).
* The user running this script is allowed to use destructive ZFS
commands: destroy, zfs receive, etc.
* That the local and remote filesystems have at least one initial
snapshot in common.
This script could be smarter and better:
* We could add command line arguments.
* We could add another script to check that the two filesystems were
actually synchronised successfully.
* We could make sure that all snapshots matched. Even if we have one
snapshot in common, we could make sure that *all* snapshots on
remote filesystem were present locally.
* We could compress zfs send output - or does ssh do that anyway?
* As long as local filesystem exists, and remote has snapshots, we
could synchronise the two filesystems by first transferring a
non-incremental initial snapshot, and then an incremental one,
instead of failing because we don't initially have a common snapshot
on both sides.
"""
from docopt import docopt
import subprocess
from kmds.lib import simple_syslog as logger
class ZfsReplicationNoLocalSnapshots(Exception):
pass
class ZfsReplicationNoRemoteSnapshots(Exception):
pass
class ZfsReplicationNoSnapshotsInCommon(Exception):
pass
def snapshots_in_creation_order(filesystem, host=None):
"Return list of snapshots on FILESYSTEM in order of creation."
result = []
if host:
cmd = "ssh {} sudo zfs list -r -t snapshot -s creation {} -o
name".format(host, filesystem)
else:
cmd = "sudo zfs list -r -t snapshot -s creation {} -o
name".format(filesystem)
lines = subprocess.check_output(cmd, stderr=subprocess.STDOUT,
shell=True).split('\n')
snapshot_prefix = filesystem + "@"
for line in lines:
if line.startswith(snapshot_prefix):
result.append(line)
return result
def strip_filesystem_name(snapshot_name):
"""Given the name of a snapshot, strip the filesystem part.
We require (and check) that the snapshot name contains a single
'@' separating filesystem name from the 'snapshot' part of the name.
"""
assert snapshot_name.count("@")==1
return snapshot_name.split("@")[1]
def execute_shell_command(cmd, dry_run=True):
if dry_run:
logger.info("would execute: {}".format(cmd))
else:
logger.info("executing: {}".format(cmd))
text = subprocess.check_output(cmd, stderr=subprocess.STDOUT,
shell=True).split('\n')
if not text:
logger.debug(" no output")
else:
logger.debug(" output:")
for line in text:
logger.debug(" {}".format(line))
def replicate_snapshots(remote_host, remote_filesystem, local_filesystem,
dry_run=True):
"""Synchronise ZFS snapshots from remote filesystem to a local
filesystem."""
logger.info("Started. remote host: {}, remote-fs: {}, local-filesystem:
{}, dry-run: {}".format(
remote_host, remote_filesystem, local_filesystem, dry_run))
local_snapshots = snapshots_in_creation_order(local_filesystem)
if not local_snapshots:
raise ZfsReplicationNoLocalSnapshots("No local snapshots",
local_filesystem)
remote_snapshots = snapshots_in_creation_order(remote_filesystem,
remote_host)
if not remote_snapshots:
raise ZfsReplicationNoRemoteSnapshots("No remote snapshots",
"host:
{}".format(remote_host),
"filesystem:
{}".format(remote_filesystem))
remote_set = set(map(strip_filesystem_name, remote_snapshots))
local_set = set(map(strip_filesystem_name, local_snapshots))
last_common_snapshot = next((s for s in reversed(remote_snapshots) if
strip_filesystem_name(s) in local_set), None)
snapshots_missing_in_remote = [s for s in local_snapshots if not
strip_filesystem_name(s) in remote_set]
last_remote_snapshot = remote_snapshots[-1]
logger.debug("Local snapshots:")
for snapshot in local_snapshots:
logger.debug(" {}".format(snapshot))
logger.debug("Remote snapshots:")
for snapshot in remote_snapshots:
logger.debug(" {}".format(snapshot))
logger.debug("Last common snapshot: {}".format(last_common_snapshot))
logger.debug("Last remote snapshot: {}".format(last_remote_snapshot))
if snapshots_missing_in_remote:
logger.debug("Present locally, but not in remote:")
for snapshot in snapshots_missing_in_remote:
logger.debug(" {}".format(snapshot))
if not last_common_snapshot:
raise ZfsReplicationNoRemoteSnapshots("No remote snapshots",
"host:
{}".format(remote_host),
"remote_filesystem:
{}".format(remote_filesystem),
"local_filesystem:
{}".format(local_filesystem))
if last_remote_snapshot == last_common_snapshot:
logger.info("No work to do. Last remote snapshot '{}' already on
local filesystem.".format(
strip_filesystem_name(last_remote_snapshot)))
return
for snapshot in snapshots_missing_in_remote:
execute_shell_command("sudo zfs destroy {}".format(snapshot),
dry_run)
execute_shell_command(("ssh {} ".format(remote_host)
+ "sudo zfs send -I {} {}
".format(last_common_snapshot, last_remote_snapshot)
+ "| sudo zfs receive -F
{}".format(local_filesystem)),
dry_run)
if __name__ == '__main__':
arguments=docopt(__doc__)
logger.init("Kapsia/ZfsReplicateSnapshots", logger.INFO)
if arguments['--verbose']:
logger.setLevel(logger.DEBUG)
logger.debug('Arguments: {}'.format(arguments))
if arguments['--quiet']:
logger.setLevel(logger.ERROR)
try:
replicate_snapshots(arguments['<remote-host>'],
arguments['<remote-filesystem>'], arguments['<local-filesystem>'],
arguments['--dry-run'])
except Exception as e:
logger.critical("Exception: {}: {}".format(type(e), e))
logger.info("Finished.")
Post by Michael KjörlingPost by k***@public.gmane.org# determine the latest remote backup daily snapshot and encrypt
# encryption key is contained in /root/streampass on remote host so as to
not have it show up in a process listing
ssh $REMOTE_HOST -p $SSH_PORT "zfs list -t snapshot | grep
openssl enc -aes-256-cbc -a -salt -pass file:/root/streampass" >
remote-new.asc
# decrypt the latest remote backup daily
# decryption key is contained in /root/streampass on backup server
openssl enc -d -aes-256-cbc -a -pass file:/root/streampass -in
./remote-new.asc > remote-new
REMOTE_NEW=`cat remote-new`
That's an interesting script. Out of curiosity, though, is there any
particular reason why you're shipping double-encrypted data? Assuming
that the ssh connection is encrypted (seems like a safe assumption to
* encrypt it using OpenSSL
* encrypt it as part of SSH communications
* ship the package across the Internet
* decrypt it as part of SSH communications
* decrypt it using OpenSSL
* magic
The AES key also becomes an additional shared secret between the
systems.
It seems to me like double-encrypting the _snapshot list_ is rather
overkill. What was your reason for doing it that way?
--
People who think they know everything really annoy
those of us who know we dont. (Bjarne Stroustrup)
To unsubscribe from this group and stop receiving emails from it, send an
To unsubscribe from this group and stop receiving emails from it, send an email to zfs-discuss+unsubscribe-VKpPRiiRko7s4Z89Ie/***@public.gmane.org