feat: add zstd in-transfer compression support

* clean up trailing whitespace
* update README.md and add a standalone Credits section
This commit is contained in:
surtur 2021-05-03 14:57:57 +02:00
parent 6d3c3f04b5
commit e4eed494f1
Signed by: wanderer
GPG Key ID: 19CE1EC1D9E0486D
2 changed files with 26 additions and 8 deletions

View File

@ -1,8 +1,10 @@
# btrfs-sync
Smart and easy sync of BTRFS snapshots, locally or through SSH.
> **Note**: This fork simply adds in-transfer `zstd` compression support
![Example](resources/btrfs-sync.gif)
Smart and easy sync of BTRFS snapshots, locally or through SSH.
See example [gif](resources/btrfs-sync.gif).
Can be used in conjunction with [btrfs-snp](https://ownyourbits.com/2017/12/27/schedule-btrfs-snapshots-with-btrfs-snp/)
@ -31,8 +33,8 @@ Delete subvolume (no-commit): '/media/USBdrive/home-snapshots/hourly_2018-03-08_
## Features
- Simple syntax
- Progress indication
- Support for _xz_ or _pbzip2_ compression in order to save bandwidth.
- Optional progress indication (using `pv`)
- Support for `zstd`, `xz` or `pbzip2` compression in order to save bandwidth.
- Retention policy
- Automatic incremental synchronization
- Cron friendly
@ -49,6 +51,7 @@ Usage:
-d|--delete delete snapshots in <dst> that don't exist in <src>
-z|--xz use xz compression. Saves bandwidth, but uses one CPU
-Z|--pbzip2 use pbzip2 compression. Saves bandwidth, but uses all CPUs
-s|--zstd use zstd compression.
-q|--quiet don't display progress
-v|--verbose display more information
-h|--help show usage
@ -57,7 +60,7 @@ Usage:
<user> requires privileged permissions at <host> for the 'btrfs' command
```
## Examples
## Examples
### Manual
@ -85,7 +88,7 @@ Synchronize only monthly snapshots of _home_ to a USB drive in another machine
# btrfs-sync /home/user/.snapshots/monthly_* user@server:/media/USBdrive/home-snapshots
```
### Cron
### Cron
Daily synchronization over the internet, keep only last 50
@ -107,4 +110,8 @@ EOF
chmod +x /etc/cron.daily/btrfs-sync
```
More at [ownyourbits.com](https://ownyourbits.com/2018/03/09/easy-sync-of-btrfs-snapshots-with-btrfs-sync/)
### Credits
Original work by *nachoparker* of [ownyourbits.com](https://ownyourbits.com/2018/03/09/easy-sync-of-btrfs-snapshots-with-btrfs-sync/)
zstd compression support added by wanderer of [dotya.ml](https://git.dotya.ml/wanderer)

13
btrfs-sync Normal file → Executable file
View File

@ -43,6 +43,7 @@ print_usage() {
-d|--delete delete snapshots in <dst> that don't exist in <src>
-z|--xz use xz compression. Saves bandwidth, but uses one CPU
-Z|--pbzip2 use pbzip2 compression. Saves bandwidth, but uses all CPUs
-s|--zstd use zstd compression.
-p|--port SSH port. Default 22
-q|--quiet don't display progress
-v|--verbose display more information
@ -76,7 +77,7 @@ PORT=22
ZIP=cat PIZ=cat
SILENT=">/dev/null"
OPTS=$( getopt -o hqzZk:p:dv -l quiet -l help -l xz -l pbzip2 -l keep: -l port: -l delete -l verbose -- "$@" 2>/dev/null )
OPTS=$( getopt -o hqzZsk:p:dv -l quiet -l help -l xz -l pbzip2 -l zstd -l keep: -l port: -l delete -l verbose -- "$@" 2>/dev/null )
[[ $? -ne 0 ]] && { echo "error parsing arguments"; exit 1; }
eval set -- "$OPTS"
@ -89,6 +90,7 @@ while true; do
-p|--port ) PORT=$2 ; shift 2 ;;
-z|--xz ) ZIP=xz PIZ=( xz -d ); shift 1 ;;
-Z|--pbzip2 ) ZIP=pbzip2 PIZ=( pbzip2 -d ); shift 1 ;;
-s|--zstd ) ZIP=zstd PIZ=( zstd -d ); shift 1 ;;
-v|--verbose) SILENT="" VERBOSE=1 ; shift 1 ;;
--) shift; break ;;
esac
@ -160,6 +162,15 @@ while read entry; do SRCS+=( "$entry" ); done < <(
}
}
## check zstd
[[ "$ZIP" == "zstd" ]] && {
"${SRC_CMD[@]}" type zstd &>/dev/null && \
"${DST_CMD[@]}" type zstd &>/dev/null || {
echo "INFO: 'zstd' not installed on both ends, fallback to 'xz'"
ZIP=xz PIZ=unxz
}
}
## use 'pv' command if available
PV=( pv -F"time elapsed [%t] | rate %r | total size [%b]" )
[[ "$QUIET" == "1" ]] && PV=( cat ) || type pv &>/dev/null || {