marco/backup.py
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Start Python reimplementation. Added make_backup method

Browse Source
This commit is contained in:
Marco Cetica
2026-01-20 12:22:00 +01:00
parent e07fa5e7c3
commit 6e97943cb7
11 changed files with 490 additions and 1317 deletions
Download Patch File Download Diff File Expand all files Collapse all files

24
.gitea/workflows/backup.sh.yml
View File

@@ -1,24 +0,0 @@
name: backup.sh
on:
push:
branches: [master]
workflow_dispatch:
jobs:
bash:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Install dependencies
run: |
apt update && apt -y install rsync tar openssl shellcheck
- name: Run ShellCheck
run: |
shellcheck backup.sh
shellcheck tests.sh
- name: Install using Makefile
run: |
make install
- name: Run unit tests
run: |
./tests.sh I_HAVE_READ_THE_HELPER

15
Makefile
View File

@@ -1,15 +0,0 @@
all:
install
install:
mkdir -p /usr/local/bin /usr/local/etc
cp -R backup.sh /usr/local/bin/backup.sh
cp -R sources.bk /usr/local/etc/sources.bk
cp -R backup.sh.1 /usr/share/man/man1/backup.sh.1
chmod 755 /usr/local/bin/backup.sh
chmod 644 /usr/local/etc/sources.bk
uninstall:
rm -rf /usr/local/bin/backup.sh
rm -ff /usr/local/etc/sources.bk
rm -rf /usr/share/man/man1/backup.sh.1

244
README.md
View File

@@ -1,243 +1,3 @@
# backup.sh ![](https://git.marcocetica.com/marco/backup.sh/actions/workflows/backup.sh.yml/badge.svg) # backup.py
`backup.sh` is a POSIX compliant, modular and lightweight backup utility to save and encrypt your files. Work in progress
This tool is intended to be used on small scale UNIX environments such as VPS, personal servers and
workstations. `backup.sh` uses [rsync](https://linux.die.net/man/1/rsync), [tar](https://linux.die.net/man/1/tar),
[gpg](https://linux.die.net/man/1/gpg) and [sha256sum](https://linux.die.net/man/1/sha256sum)
to copy, compress, encrypt the backup and verify the backup.
## Installation
`backup.sh` is a single source file, to install it you can copy the script wherever you want. Alternatively, if you
are running a DEB/RPM distribution, you can install it with the following command:
```sh
$> sudo apt install ./bin/backup.sh-1.0.0.x86_64.deb # Debian
$> sudo dnf install ./bin/backup.sh-1.0.0-2.x86_64.rpm # RHEL
```
For any other UNIX system, you can use the following command:
```sh
$> sudo make install
```
This will copy `backup.sh` into `/usr/local/bin/backup.sh`, `sources.bk` into `/usr/local/etc/sources.bk` and
`backup.sh.1` into `/usr/share/man/man1/backup.sh.1`. To uninstall the program along with the sample _sources file_ and the manual page, you can issue `sudo make uninstall`.
At this point you still need to install the following dependencies:
- `Bash(v>=4)`
- `rsync`
- `tar`
- `gpg`
## Usage
To show the available options, you can run `backup.sh --help`, which will print out the following message:
```text
backup.sh v1.0.0 - POSIX compliant, modular and lightweight backup utility.
Syntax: ./backup.sh [-b|-e|-c|-V|-h]
options:
-b|--backup SOURCES DEST PASS Backup folders from SOURCES file.
-e|--extract ARCHIVE PASS Extract ARCHIVE using PASS.
-c|--checksum Generate/check SHA256 of a backup.
-V|--verbose Enable verbose mode.
-h|--help Show this helper.
General help with the software: https://github.com/ceticamarco/backup.sh
Report bugs to: Marco Cetica(<email@marcocetica.com>)
```
As you can see, `backup.sh` supports three options: **backup creation**, **backup extraction** and **checksum** to verify the
integrity of a backup. The first option requires
root permissions, while the second one does not. The checksum option must be used in combination of one of the previous options.
### Backup creation
To specify the directories to back up, `backup.sh` uses an associative array
defined in a text file(called _sources file_) with the following syntax:
```text
<LABEL>=<PATH>
```
Where `<LABEL>` is the name of the backup and `<PATH>` is its path. For example,
if you want to back up `/etc/nginx` and `/etc/ssh`, add the following entries to the _sources file_:
```text
nginx=/etc/nginx/
ssh=/etc/ssh/
```
`backup.sh` will create two folders inside the backup archive with the following syntax:
```text
backup-<LABEL>-<YYYYMMDD>
```
In the previous example, this would be:
```text
backup-nginx-<YYYYMMDD>
backup-ssh-<YYYYMMDD>
```
You can add as many entries as you want, just be sure to use the proper syntax. In particular,
the _sources file_, **should not** include:
- Spaces between the label and the equal sign;
- Empty lines;
- Comments.
You can find a sample _sources file_ at `sources.bk`(or at `/usr/local/etc/sources.bk`).
After having defined the _sources file_, you can invoke `backup.sh` using the following syntax:
```sh
$> sudo ./backup.sh --backup <SOURCES_FILE> <DEST> <ENCRYPTION_PASSWORD>
```
Where `<SOURCES_FILE>` is the _sources file_, `<DEST>` is the absolute path of the output of the backup
**without trailing slashes** and `<ENCRYPTION_PASSWORD>` is the password to encrypt the compressed archive.
In the previous example, this would be:
```sh
$> sudo ./backup.sh --backup sources.bk /home/john badpw1234
```
You can also tell `backup.sh` to generate a SHA256 file containing the hash of each file using the `-c` option.
In the previous example, this would be:
```sh
$> sudo ./backup.sh --checksum --backup sources.bk /home/john badpw1234
```
The backup utility will begin to copy the files defined in the _sources file_:
```text
Copying nginx(1/2)
Copying ssh(2/2)
Compressing backup...
Encrypting backup...
File name: /home/john/backup-<HOSTNAME>-<YYYYMMDD>.tar.gz.enc
Checksum file: /home/john/backup-<HOSTNAME>-<YYYYMMDD>.sha256
File size: 7336400696(6.9G)
Elapsed time: 259 seconds.
```
After that, you will find the backup archive and the checksum file in
`/home/john/backup-<HOSTNAME>-<YYYYMMDD>.tar.gz.enc` and `/home/john/backup-<HOSTNAME>-<YYYYMMDD>.sha256`, respectively.
You can also use `backup.sh` from a crontab rule:
```sh
$> sudo crontab -e
30 03 * * 6 EKEY=$(cat /home/john/.ekey) bash -c '/usr/local/bin/backup.sh -b /usr/local/etc/sources.bk /home/john $EKEY' > /dev/null 2>&1
```
This will automatically run `backup.sh` every Saturday morning at 03:30 AM. In the example above, the encryption
key is stored in a local file(with fixed permissions) to avoid password leaking in crontab logs. You can also
adopt this practice while using the `--extract` option to avoid password leaking in shell history.
By default `backup.sh` is very quiet, to add some verbosity to the output, be sure to use the `-V`(`--verbose`) option.
### Backup extraction
`backup.sh` can also be used to extract and to verify the encrypted backup.
To do so, use the following commands:
```sh
$> ./backup.sh --extract <ENCRYPTED_ARCHIVE> <ARCHIVE_PASSWORD>
```
Where `<ENCRYPTED_ARCHIVE>` is the encrypted backup and `<ARCHIVE_PASSWORD>` is the backup password.
For instance:
```sh
$> ./backup.sh --extract backup-<hostname>-<YYYYMMDD>.tar.gz.enc badpw1234
```
This will create a new folder called `backup.sh.tmp` in your local directory with the following content:
```text
backup-nginx-<YYYYMMDD>
backup-ssh-<YYYYMMDD>
```
**note**: be sure to rename any directory with that name to avoid collisions.
If you also want to verify the integrity of the backup data, use the following commands:
```sh
$> ./backup.sh --checksum --extract <ENCRYPTED_ARCHIVE> <ARCHIVE_PASSWORD> <CHECKSUM_ABSOLUTE_PATH>
```
For instance:
```sh
$> ./backup.sh --checksum --extract backup-<hostname>-<YYYYMMDD>.tar.gz.enc badpw1234 backup-<hostname>-<YYYYMMDD>.sha256
```
## How does backup.sh work?
**backup.sh** uses _rsync_ to copy the files, _tar_ to compress the backup, _gpg_ to encrypt it and
_sha256sum_ to verify it.
By default, rsync is being used with the following parameters:
```
$> rsync -aPhrq --delete
```
That is:
- a: archive mode: rsync copies files recursively while preserving as much metadata as possible;
- P: progress/partial: allows rsync to resume interrupted transfers and to shows progress information;
- h: human readable output, rsync shows output numbers in a more readable way;
- r: recursive mode: forces rsync to copy directories and their content;
- q: quiet mode: reduces the amount of information rsync produces;
- delete: delete mode: forces rsync to delete any extraneous files at the destination dir.
If specified(`--checksum` option), `backup.sh` can also generate the checksum of each file of the backup.
To do so, it uses `sha256sum(1)` to compute the hash of every single file using the SHA256 hashing algorithm.
The checksum file contains nothing but the checksums of the files, no other information about the files stored
on the backup archive is exposed on the unencrypted checksum file. This may be an issue if you want plausible
deniability(see privacy section for more information).
After that the backup folder is being encrypted using gpg. By default, it is used with the following parameters:
```
$> gpg -a \
--symmetric \
--cipher-algo=AES256 \
--no-symkey-cache \
--pinentry-mode=loopback \
--batch --passphrase "$PASSWORD" \
--output "$OUTPUT" \
"$INPUT"
```
This command encrypts the backup using the AES-256 symmetric encryption algorithm with a 256bit key. Here is what each flag do:
- `--symmetric`: Use symmetric encryption;
- `--cipher-algo=AES256`: Use AES256 algorithm;
- `--no-symkey-cache`: Do not save password on GPG's cache;
- `--pinentry-mode=loopback --batch`: Do not prompt the user;
- `--passphrase-fd 3 3<< "$PASSWORD"`: Read password without revealing it on `ps`;
- `--output`: Specify output file;
- `$INPUT`: Specify input file.
## Plausible Deniability
While `backup.sh` provide some pretty strong security against bruteforce attack(assuming a strong passphrase is being used)
it should by no means considered a viable tool against a cryptanalysis investigation. Many of the copying, compressing and
encrypting operations made by `backup.sh` during the backup process can be used to invalidate plausible deniability.
In particular, you should pay attention to the following details:
1. The `--checksum` option generates an **UNENCRYPTED** checksum file containing the _digests_ of **EVERY**
file in your backup archive. If your files are known to your adversary(e.g., a banned book), they may use a rainbow table attack to
determine whether you own a given file, voiding your plausible deniability;
2. Since `backup.sh` is essentially a set of shell commands, an eavesdropper could monitor the whole backup process to extract
the name of the files or the encryption password.
## Unit tests
`backup.sh` provides some unit tests inside the `tests.sh` script. This script generates some dummy files inside the following
directories:
- /var/log
- /var/www
- /etc/nginx
- /etc/ssh
For this reason, this script should **NOT** be used in non-testing environments. To run all tests, issue the following command:
```sh
$> sudo ./tests.sh I_HAVE_READ_THE_HELPER
```
## License
This software is released under GPLv3, you can obtain a copy of this license by cloning this repository or by visiting
[this page](https://choosealicense.com/licenses/gpl-3.0/).

488
backup.py Executable file
View File

@@ -0,0 +1,488 @@
#!/usr/bin/env python3
# backup.py: modular and lightweight backup utility
# Developed by Marco Cetica (c) 2018, 2023, 2024, 2026
#
import argparse
import shutil
import sys
import os
import time
import tarfile
import subprocess
import hashlib
import getpass
from pathlib import Path
from datetime import datetime
from dataclasses import dataclass
from typing import Generic, TypeVar, Union, Literal, List
T = TypeVar("T")
@dataclass(frozen=True)
class Ok(Generic[T]):
"""Sum type to represent results"""
value: T
success: Literal[True] = True
@dataclass(frozen=True)
class Err:
error: str
success: Literal[False] = False
Result = Union[Ok[T], Err]
@dataclass
class BackupSource:
"""Struct to represent a mapping between a label and a path"""
label: str
path: Path
@dataclass
class BackupState:
"""Struct to represent a backup state"""
sources: List[BackupSource]
output_path: Path
password: str
checksum: bool
verbose: bool
class BackupProgress:
"""Progress indicator for backup operations"""
def __init__(self, total: int, operation: str):
self.total = total
self.current = 0
self.operation = operation
def update(self, message: str = ""):
"""Update progress"""
self.current += 1
percentage = (self.current / self.total) * 100 if self.total > 0 else 0
status = f"\r{self.operation}: [{self.current}/{self.total}] {percentage: .1f}% - {message}"
print(f"\r\033[K{status}", end='', flush=True)
def finish(self):
"""Print new line"""
print()
class Backup:
@staticmethod
def check_deps() -> Result[None]:
"""Check whether dependencies are installed"""
missing_deps = []
for dep in ["gpg", "tar"]:
if not shutil.which(dep):
missing_deps.append(dep)
if missing_deps:
return Err(f"Missing dependencies: {', '.join(missing_deps)}")
return Ok(None)
@staticmethod
def prettify_size(byte_size: int) -> str:
"""Convert byte_size in powers of 1024"""
units = ["B", "KiB", "MiB", "GiB", "TiB", "PiB", "EiB"]
idx = 0
size = float(byte_size)
while size >= 1024.0 and idx < (len(units) - 1):
size /= 1024.0
idx += 1
if size.is_integer():
return f"{int(size)} {units[idx]}"
return f"{size:.2f} {units[idx]}"
@staticmethod
def parse_sources_file(sources_file: Path) -> Result[List[BackupSource]]:
"""Parse the sources file returning a list of BackupSource elements"""
if not sources_file.exists():
return Err("Sources file does not exist")
sources: List[BackupSource] = []
try:
with open(sources_file, 'r') as f:
for pos, line in enumerate(f, 1):
line = line.strip()
if not line or line.startswith('#'):
continue
if '=' not in line:
print(f"Warning: invalid format at line {pos}: {line}")
label, path_str = line.split('=', 1)
path = Path(path_str.strip())
if not path.exists():
return Err(f"Path does not exist: {path}")
sources.append(BackupSource(label.strip(), path))
except IOError as err:
return Err(f"Failed to read sources file: {err}")
if not sources:
return Err(f"No valid sources found in file")
return Ok(sources)
@staticmethod
def should_ignore_file(path: Path) -> bool:
"""Check whether a file should be ignored"""
try:
# Skip UNIX sockets
if path.is_socket():
return True
# Skip broken symlinks
if path.is_symlink() and not path.exists():
return True
# Skip named pipes (FIFOs)
if path.stat().st_mode & 0o170000 == 0o010000:
return True
return False
except (OSError, IOError):
# Skip files that can't be checked
return True
@staticmethod
def ignore_special_files(directory: str, contents: List[str]) -> List[str]:
"""Return a list of files to ignore"""
ignored_files: List[str] = []
dir_path = Path(directory)
for item in contents:
item_path = dir_path / item
if Backup.should_ignore_file(item_path):
ignored_files.append(item)
return ignored_files
@staticmethod
def copy_files(source: Path, destination: Path, verbose: bool) -> Result[None]:
"""Copy files and directories preserving their metadata"""
try:
# Handle single file
if source.is_file():
# Parent directory might not exists, so we try to create it first
destination.parent.mkdir(parents=True, exist_ok=True)
if verbose:
print(f"Copying file {source} -> {destination}")
# Copy file and its metadata
shutil.copy2(source, destination)
return Ok(None)
# Handle directory
if source.is_dir():
# If destination directory exists, we remove it
# This approach mimics rsync's --delete option
if destination.exists():
shutil.rmtree(destination)
if verbose:
print(f"Copying directory {source} -> {destination}")
# Copy directory and its metadata.
# We also ignore special files and we preserves links instead
# of following them.
shutil.copytree(
source,
destination,
symlinks=True, # True = preserve symlinks
copy_function=shutil.copy2,
ignore=Backup.ignore_special_files,
dirs_exist_ok=False
)
return Ok(None)
return Err(f"The following source element is neither a file nor a directory: {source}")
except (IOError, OSError, shutil.Error) as err:
return Err(f"Copy failed: {err}")
@staticmethod
def cleanup_files(*paths: Path) -> None:
"""Clean up temporary files and directories"""
for path in paths:
if not path.exists():
continue
if path.is_dir():
shutil.rmtree(path, ignore_errors=True)
else:
path.unlink(missing_ok=True)
@staticmethod
def collect_files(directory: Path) -> List[Path]:
"""Collect all files in a directory (recursively)"""
files = []
for item in directory.rglob('*'):
if item.is_file() and not item.is_symlink():
files.append(item)
return files
@staticmethod
def compute_file_hash(file_path: Path) -> Result[str]:
"""Compute SHA256 hash of a given file"""
try:
with open(file_path, "rb") as f:
for byte_block in iter(lambda: f.read(4096), b""):
hashlib.sha256().update(byte_block)
return Ok(hashlib.sha256().hexdigest())
except IOError as e:
return Err(f"Failed to read file {file_path}: {e}")
@staticmethod
def create_tarball(source_dir: Path, output_file: Path, verbose: bool) -> Result[None]:
"""Create a compressed tar archive of the backup directory"""
if verbose:
print("Compressing backup...")
cmd = [
"tar",
"-czf",
str(output_file),
"-C",
str(source_dir.parent),
source_dir.name
]
if verbose:
cmd[1] += "-v"
# capture here means suppress it/holding it
result = subprocess.run(cmd, capture_output=not verbose, text=None)
if result.returncode != 0:
error_msg = f"tar failed: {result.stderr if result.stderr else 'Unknown error code'}"
return Err(error_msg)
return Ok(None)
@staticmethod
def encrypt_file(input_file: Path, output_file: Path, password: str, verbose: bool) -> Result[None]:
"""Encrypt a file with GPG in symmetric mode (using AES256)"""
if verbose:
print("Encrypting backup...")
cmd = [
"gpg", "-a",
"--symmetric",
"--cipher-algo=AES256",
"--no-symkey-cache",
"--pinentry-mode=loopback",
"--batch",
"--passphrase-fd", "0",
"--output", str(output_file),
str(input_file)
]
result = subprocess.run(
cmd,
input=password.encode(),
capture_output=not verbose
)
if result.returncode != 0:
return Err(f"Encryption failed: {result.stderr.decode()}")
return Ok(None)
def make_backup(self, config: BackupState) -> Result[None]:
"""Create an encrypted backup from specified sources file"""
# Check root permissions
if os.geteuid() != 0:
return Err("Run this program as root!")
start_time = time.time()
date_str = datetime.now().strftime("%Y%m%d")
hostname = os.uname().nodename
# Create working directory
work_dir = config.output_path / "backup.sh.tmp"
if not work_dir.exists():
work_dir.mkdir(parents=True, exist_ok=True)
# Format output files
backup_archive = config.output_path / f"backup-{hostname}-{date_str}.tar.gz.enc"
checksum_file = config.output_path / f"backup-{hostname}-{date_str}.sha256"
temp_tarball = config.output_path / "backup.sh.tar.gz"
# Backup each source
sources_count = len(config.sources)
for idx, source in enumerate(config.sources, 1):
print(f"Copying {source.label} ({idx}/{sources_count})")
# Create source subdirectory
source_dir = work_dir / f"backup-{source.label}-{date_str}"
if not source_dir.exists():
source_dir.mkdir(parents=True, exist_ok=True)
# Copy files
copy_res = self.copy_files(source.path, work_dir, config.verbose)
match copy_res:
case Err():
self.cleanup_files(work_dir, temp_tarball)
return copy_res
case Ok(): pass
# Compute checksum when requested
if config.checksum:
files = self.collect_files(source_dir)
backup_progress: BackupProgress | None = None
if config.verbose:
backup_progress = BackupProgress(len(files), "Computing checksums")
checksum_fd = open(checksum_file, 'a')
for file in files:
hash_result = self.compute_file_hash(file)
match hash_result:
case Err():
checksum_fd.close()
self.cleanup_files(work_dir, temp_tarball)
return hash_result
case Ok(value=v):
checksum_fd.write(f"{v}\n")
if config.verbose and backup_progress is not None:
backup_progress.update(str(file.name))
checksum_fd.close()
if config.verbose and backup_progress is not None:
backup_progress.finish()
# Create compressed archive
archive_res = self.create_tarball(work_dir, temp_tarball, config.verbose)
match archive_res:
case Err():
self.cleanup_files(work_dir, temp_tarball)
return archive_res
case Ok(): pass
# Encrypt the archive
encrypt_res = self.encrypt_file(temp_tarball, backup_archive, config.password, config.verbose)
match encrypt_res:
case Err():
self.cleanup_files(work_dir, temp_tarball)
return encrypt_res
case Ok(): pass
# Cleanup temporary files
self.cleanup_files(work_dir, temp_tarball)
# Compute file size
if not backup_archive.exists():
return Err("Unable to create backup archive")
elapsed_time = time.time() - start_time
file_size = backup_archive.stat().st_size
file_size_hr = Backup.prettify_size(file_size)
print(f"\nBackup complete")
print(f"File name: {backup_archive}")
if config.checksum:
print(f"Checksum file: {checksum_file}")
print(f"File size: {file_size} bytes ({file_size_hr})")
print(f"Elapsed time: {elapsed_time:.2f} seconds")
return Ok(None)
def main():
parser = argparse.ArgumentParser(
description="backup.py - modular and lightweight backup utility",
formatter_class=argparse.RawDescriptionHelpFormatter,
epilog="""
Usage:
> Create a backup
sudo ./backup.py --backup sources.bk /home/user $PASS
> Create backup with checksum
sudo ./backup.py --checksum --backup sources.bk /home/user $PASS
> Extract and verify checksum
./backup.py --checksum --extract backup.tar.gz.enc $PASS hashes.sha256
For more information visit: https://git.marcocetica.com/marco/backup.py
or issue `man backup.py` on your terminal.
"""
)
parser.add_argument(
"-b", "--backup",
nargs=3,
metavar=("SOURCES", "DEST", "PASS"),
help="Backup files from SOURCES path to DEST directory with password PASS"
)
parser.add_argument(
"-e", "--extract",
nargs="+",
metavar="ARCHIVE",
help="Extract ARCHIVE (optionally with PASS and SHA256 file)"
)
parser.add_argument(
"-c", "--checksum",
action="store_true",
help="Generate or check SHA256 checksums"
)
parser.add_argument(
"-V", "--verbose",
action="store_true",
help="Enable verbose mode"
)
args = parser.parse_args()
# Check whether dependencies are installed
deps_res = Backup.check_deps()
match deps_res:
case Err(error=e): print(f"Error: {e}", file=sys.stderr)
case Ok(): pass
backup = Backup()
if args.backup:
sources_file, output_path, password = args.backup
sources_path = Path(sources_file)
output_dir = Path(output_path)
# Create output directory if it doesn't exist
if not output_dir.exists():
output_dir.mkdir(parents=True, exist_ok=True)
# Parse sources file
sources_res = Backup.parse_sources_file(sources_path)
config: BackupState
match sources_res:
case Err(error=e):
print(f"Error: {e}", file=sys.stderr)
sys.exit(1)
case Ok(value=v):
# Create a backup state
config = BackupState(
sources=v,
output_path=output_dir,
password=password,
checksum=args.checksum,
verbose=args.verbose
)
if __name__ == "__main__":
main()

329
backup.sh
View File

@@ -1,329 +0,0 @@
#!/usr/bin/env bash
# backup.sh is a POSIX compliant, modular and lightweight
# backup utility to save and encrypt your files.
#
# To specify the source directories to backup,
# create a text file with the following syntax:
#
# <LABEL>=<PATH>
#
# for example(filename: 'sources.bk'):
# nginx=/etc/nginx/
# ssh=/etc/ssh/
# logs=/var/log/
#
# After that you can launch the script with(sample usage):
# sudo ./backup.sh --checksum --backup sources.bk /home/john badpw1234
#
# This will create an encrypted tar archive(password: 'badpw1234')
# in '/home/john/backup-<hostname>-<YYYYMMDD>.tar.gz.enc' containing
# the following three directories:
# backup-nginx-<YYYYMMDD>
# backup-ssh-<YYYYMMDD>
# backup-logs-<YYYYMMDD>
#
# as well as a SHA256 file('/home/john/backup-<hostname>-<YYYYMMDD>.sha256')
# containing the file hashes of the backup.
#
# You can then decrypt it using:
# ./backup.sh --checksum --extract backup-<hostname>-<YYYYMMDD>.tar.gz.enc badpw1234 $PWD/backup-<hostname>-<YYYYMMDD>.sha256
# which will also check the integrity of the backup(optional feature).
#
# You can read the full guide on https://github.com/ceticamarco/backup.sh
# or on the manual page.
# Copyright (c) 2018,2023,2024 Marco Cetica <email@marcocetica.com>
#
set -e
checkdeps() {
if [ "${BASH_VERSINFO[0]}" -lt 4 ]; then
echo "This version of Bash is not supported."
exit 1
fi
# Check if dependencies are installed
missing_dep=0
deps="rsync tar gpg"
for dep in $deps; do
if ! command -v "$dep" > /dev/null 2>&1; then
printf "Cannot find '%s', please install it.\n" "$dep"
missing_dep=1
fi
done
if [ $missing_dep -ne 0 ]; then
exit 1
fi
}
# $1: filename
gethash() {
if [ "$(uname)" = "Linux" ]; then
sha256sum "$1" | awk '{print $1}'
else
sha256 -q "$1"
fi
}
# $1: sources.bk file
# $2: output path
# $3: password
# $4: compute sha256(0,1)
# $5: verbosity flag(0,1)
make_backup() {
BACKUP_SH_SOURCES_PATH="$1"
BACKUP_SH_OUTPATH="$2"
BACKUP_SH_PASS="$3"
BACKUP_SH_SHA256="$4"
BACKUP_SH_VERBOSE="$5"
if [ "$BACKUP_SH_VERBOSE" -eq 1 ]; then
BACKUP_SH_COMMAND="rsync -aPhr --delete"
else
BACKUP_SH_COMMAND="rsync -aPhrq --delete"
fi
BACKUP_SH_DATE="$(date +'%Y%m%d')"
BACKUP_SH_FOLDER="backup.sh.tmp"
BACKUP_SH_OUTPUT="$BACKUP_SH_OUTPATH/$BACKUP_SH_FOLDER"
BACKUP_SH_START_TIME="$(date +%s)"
BACKUP_SH_FILENAME="$BACKUP_SH_OUTPATH/backup-$(uname -n)-$BACKUP_SH_DATE.tar.gz.enc"
BACKUP_SH_CHECKSUM_FILE="$BACKUP_SH_OUTPATH/backup-$(uname -n)-$BACKUP_SH_DATE.sha256"
# Check for root permissions
[ "$(id -u)" -ne 0 ] && { echo "Run this tool as root!"; exit 1; }
# Create temporary directory
mkdir -p "$BACKUP_SH_OUTPUT"
# For each item in the array, make a backup
BACKUP_SH_TOTAL=$(wc -l < "$BACKUP_SH_SOURCES_PATH")
BACKUP_SH_PROGRESS=1
while IFS='=' read -r label path; do
# Check if entry path exists
[ -e "$path" ] || { echo "$path does not exist"; exit 1; }
# Define a subdir for each backup entry
BACKUP_SH_SUBDIR="$BACKUP_SH_OUTPUT/backup-$label-$BACKUP_SH_DATE"
mkdir -p "$BACKUP_SH_SUBDIR"
printf "Copying %s(%s/%s)\n" "$label" "$BACKUP_SH_PROGRESS" "$BACKUP_SH_TOTAL"
# Copy files
$BACKUP_SH_COMMAND "$path" "$BACKUP_SH_SUBDIR"
# Compute SHA256
if [ "$BACKUP_SH_SHA256" -eq 1 ]; then
shopt -s globstar dotglob
for file in "$BACKUP_SH_SUBDIR"/**/*; do
# Skip symbolic links and directories
[ -d "$file" ] || [ -L "$file" ] && continue
if [ "$BACKUP_SH_VERBOSE" -eq 1 ]; then
printf "Computing checksum for '%s'...\n" "$file"
fi
gethash "$file" >> "$BACKUP_SH_CHECKSUM_FILE"
done
shopt -u globstar dotglob
fi
BACKUP_SH_PROGRESS=$((BACKUP_SH_PROGRESS+1))
done < "$BACKUP_SH_SOURCES_PATH"
# Compress backup directory
echo "Compressing backup..."
if [ "$BACKUP_SH_VERBOSE" -eq 1 ]; then
tar -cvzf "$BACKUP_SH_OUTPATH/backup.sh.tar.gz" \
-C "$BACKUP_SH_OUTPATH" "$BACKUP_SH_FOLDER"
else
tar -czf "$BACKUP_SH_OUTPATH/backup.sh.tar.gz" \
-C "$BACKUP_SH_OUTPATH" "$BACKUP_SH_FOLDER" > /dev/null 2>&1
fi
# Encrypt backup directory
echo "Encrypting backup..."
gpg -a \
--symmetric \
--cipher-algo=AES256 \
--no-symkey-cache \
--pinentry-mode=loopback \
--batch --passphrase-fd 3 3<<< "$BACKUP_SH_PASS" \
--output "$BACKUP_SH_FILENAME" \
"$BACKUP_SH_OUTPATH/backup.sh.tar.gz" > /dev/null 2>&1
# Remove temporary files
rm -rf "$BACKUP_SH_OUTPUT"
rm -rf "$BACKUP_SH_OUTPATH/backup.sh.tar.gz"
# Print file name, file size, file hash and elapsed time,
BACKUP_SH_END_TIME="$(date +%s)"
BACKUP_SH_FILE_SIZE="$(find "$BACKUP_SH_FILENAME" -exec ls -l {} \; | awk '{print $5}')"
BACKUP_SH_FILE_SIZE_H="$(find "$BACKUP_SH_FILENAME" -exec ls -lh {} \; | awk '{print $5}')"
echo "File name: $BACKUP_SH_FILENAME"
[ "$BACKUP_SH_SHA256" -eq 1 ] && { echo "Checksum file: $BACKUP_SH_CHECKSUM_FILE"; }
echo "File size: $BACKUP_SH_FILE_SIZE($BACKUP_SH_FILE_SIZE_H)"
printf "Elapsed time: %s seconds.\n" "$((BACKUP_SH_END_TIME - BACKUP_SH_START_TIME))"
}
# $1: archive file
# $2: archive password
# $3: sha256 file(optional)
# $4: verbosity flag(0,1)
extract_backup() {
BACKUP_SH_ARCHIVE_PATH="$1"
BACKUP_SH_ARCHIVE_PW="$2"
BACKUP_SH_SHA256_FILE="$3"
BACKUP_SH_VERBOSE="$4"
# Decrypt the archive
echo "Decrypting backup..."
gpg -a \
--quiet \
--decrypt \
--no-symkey-cache \
--pinentry-mode=loopback \
--batch --passphrase-fd 3 3<<< "$BACKUP_SH_ARCHIVE_PW" \
--output backup.sh.tar.gz \
"$BACKUP_SH_ARCHIVE_PATH"
# Extract archive
if [ "$BACKUP_SH_VERBOSE" -eq 1 ]; then
tar -xzvf backup.sh.tar.gz
else
tar -xzf backup.sh.tar.gz > /dev/null 2>&1
fi
# If specified, use SHA256 file to compute checksum of files
if [ -n "$BACKUP_SH_SHA256_FILE" ]; then
shopt -s globstar dotglob
for file in "backup.sh.tmp"/**/*; do
# Skip symbolic links and directories
[ -d "$file" ] || [ -L "$file" ] && continue;
# Compute sha256 for current file
SHA256="$(gethash "$file")"
# Check if checksum file contains hash
if ! grep -wq "$SHA256" "$BACKUP_SH_SHA256_FILE"; then
rm -rf backup.sh.tar.gz backup.sh.tmp
printf "[FATAL] - integrity error for '%s'.\n" "$file"
exit 1
fi
if [ "$BACKUP_SH_VERBOSE" -eq 1 ]; then
printf "[OK] - integrity check for '%s' passed.\n" "$file"
fi
done
shopt -u globstar dotglob
fi
rm -rf backup.sh.tar.gz
}
helper() {
CLI_NAME="$1"
cat <<EOF
backup.sh v1.0.0 - POSIX compliant, modular and lightweight backup utility.
Syntax: $CLI_NAME [-b|-e|-c|-V|-h]
options:
-b|--backup SOURCES DEST PASS Backup folders from SOURCES file.
-e|--extract ARCHIVE PASS Extract ARCHIVE using PASS.
-c|--checksum Generate/check SHA256 of a backup.
-V|--verbose Enable verbose mode.
-h|--help Show this helper.
General help with the software: https://github.com/ceticamarco/backup.sh
Report bugs to: Marco Cetica(<email@marcocetica.com>)
EOF
}
main() {
# Check whether dependecies are installed
checkdeps
if [ $# -eq 0 ]; then
echo "Please, specify an argument."
echo "For more information, try --help."
exit 1
fi
CHECKSUM_FLAG=0
VERBOSE_FLAG=0
# Parse CLI arguments
while [ $# -gt 0 ]; do
case $1 in
-b|--backup)
BACKUP_SH_SOURCES_PATH="$2"
BACKUP_SH_OUTPATH="$3"
BACKUP_SH_PASSWORD="$4"
if [ -z "$BACKUP_SH_SOURCES_PATH" ] || [ -z "$BACKUP_SH_OUTPATH" ] || [ -z "$BACKUP_SH_PASSWORD" ]; then
echo "Please, specify a source file, an output path and a password."
echo "For more informatio, try --help"
exit 1
fi
[ -f "$BACKUP_SH_SOURCES_PATH" ] || { echo "Sources file does not exist"; exit 1; }
make_backup "$BACKUP_SH_SOURCES_PATH" "$BACKUP_SH_OUTPATH" "$BACKUP_SH_PASSWORD" "$CHECKSUM_FLAG" "$VERBOSE_FLAG"
exit 0
;;
-e|--extract)
BACKUP_SH_ARCHIVE_FILE="$2"
BACKUP_SH_ARCHIVE_PW="$3"
BACKUP_SH_SHA256_FILE="$4"
if [ "$CHECKSUM_FLAG" -eq 1 ]; then
if [ -z "$BACKUP_SH_ARCHIVE_FILE" ] || [ -z "$BACKUP_SH_ARCHIVE_PW" ] || [ -z "$BACKUP_SH_SHA256_FILE" ]; then
echo "Please, specify an encrypted archive, a password and a SHA256 file."
echo "For more informatio, try --help"
exit 1
fi
else
if [ -z "$BACKUP_SH_ARCHIVE_FILE" ] || [ -z "$BACKUP_SH_ARCHIVE_PW" ]; then
echo "Please, specify an encrypted archive and a password."
echo "For more information, try --help"
exit 1
fi
fi
# Check whether backup file exists or not
[ -f "$BACKUP_SH_ARCHIVE_FILE" ] || { echo "Backup file does not exist"; exit 1; }
if [ "$CHECKSUM_FLAG" -eq 1 ]; then
[ -f "$BACKUP_SH_SHA256_FILE" ] || { echo "Checksum file does not exist"; exit 1; }
extract_backup "$BACKUP_SH_ARCHIVE_FILE" "$BACKUP_SH_ARCHIVE_PW" "$BACKUP_SH_SHA256_FILE" "$VERBOSE_FLAG"
else
extract_backup "$BACKUP_SH_ARCHIVE_FILE" "$BACKUP_SH_ARCHIVE_PW" "" "$VERBOSE_FLAG"
fi
exit 0
;;
-c|--checksum)
[ $# -eq 1 ] && { echo "Use '--checksum' with '--backup' or '--extract'"; exit 1; }
CHECKSUM_FLAG=1
shift 1
;;
-V|--verbose)
[ $# -eq 1 ] && { echo "Use '--verbose' with '--backup' or '--extract'"; exit 1; }
VERBOSE_FLAG=1
shift 1
;;
-h|--help)
helper "$0"
exit 0
;;
*)
echo "Unknown option $1."
echo "For more information, try --help"
exit 1
;;
esac
done
}
main "$@"
# vim: ts=4 sw=4 softtabstop=4 expandtab:

346
backup.sh.1
View File

@@ -1,346 +0,0 @@
.\" Automatically generated by Pandoc 3.5
.\"
.TH "backup.sh" "1" "October 21, 2024" "Marco Cetica" "General Commands Manual"
.SH NAME
\f[B]backup.sh\f[R] \- POSIX compliant, modular and lightweight backup
utility.
.SH SYNOPSIS
.IP
.EX
Syntax: ./backup.sh [\-b|\-e|\-c|\-V|\-h]
options:
\-b|\-\-backup SOURCES DEST PASS Backup folders from SOURCES file.
\-e|\-\-extract ARCHIVE PASS Extract ARCHIVE using PASS.
\-c|\-\-checksum Generate/check SHA256 of a backup.
\-V|\-\-verbose Enable verbose mode.
\-h|\-\-help Show this helper.
General help with the software: https://github.com/ceticamarco/backup.sh
Report bugs to: Marco Cetica(<email\[at]marcocetica.com>)
.EE
.SH DESCRIPTION
\f[B]backup.sh\f[R] is a POSIX compliant, modular and lightweight backup
utility to save and encrypt your files.
This tool is intended to be used on small scale UNIX environment such as
VPS, small servers and workstations.
\f[B]backup.sh\f[R] uses \f[I]rsync\f[R], \f[I]tar\f[R],
\f[I]sha256sum\f[R] and \f[I]gpg\f[R] to copy, compress, verify and
encrypt the backup.
.SH OPTIONS
\f[B]backup.sh\f[R] supports three options: \f[B]backup creation\f[R],
\f[B]backup extraction\f[R] and \f[B]checksum\f[R] to verify the
integrity of a backup.
The first option requires root permissions, while the second one does
not.
The checksum option must be used in combination of one of the previous
options.
.SS Backup creation
To specify the directories to back up, \f[CR]backup.sh\f[R] uses an
associative array defined in a text file(called \f[I]sources file\f[R])
with the following syntax:
.IP
.EX
<LABEL>=<PATH>
.EE
.PP
Where \f[CR]<LABEL>\f[R] is the name of the backup and \f[CR]<PATH>\f[R]
is its path.
For example, if you want to back up \f[CR]/etc/nginx\f[R] and
\f[CR]/etc/ssh\f[R], add the following entries to the \f[I]sources
file\f[R]:
.IP
.EX
nginx=/etc/nginx/
ssh=/etc/ssh/
.EE
.PP
\f[CR]backup.sh\f[R] will create two folders inside the backup archive
with the following syntax:
.IP
.EX
backup\-<LABEL>\-<YYYYMMDD>
.EE
.PP
In the previous example, this would be:
.IP
.EX
backup\-nginx\-<YYYYMMDD>
backup\-ssh\-<YYYYMMDD>
.EE
.PP
You can add as many entries as you want, just be sure to use the proper
syntax.
In particular, the \f[I]sources file\f[R], \f[B]should not\f[R] include:
\- Spaces between the label and the equal sign;
.PD 0
.P
.PD
\- Empty lines;
.PD 0
.P
.PD
\- Comments.
.PP
You can find a sample \f[I]sources file\f[R] at \f[CR]sources.bk\f[R](or
at \f[CR]/usr/local/etc/sources.bk\f[R]).
.PP
After having defined the \f[I]sources file\f[R], you can invoke
\f[CR]backup.sh\f[R] using the following syntax:
.IP
.EX
$> sudo ./backup.sh \-\-backup <SOURCES_FILE> <DEST> <ENCRYPTION_PASSWORD>
.EE
.PP
Where \f[CR]<SOURCES_FILE>\f[R] is the \f[I]sources file\f[R],
\f[CR]<DEST>\f[R] is the absolute path of the output of the backup
\f[B]without trailing slashes\f[R] and \f[CR]<ENCRYPTION_PASSWORD>\f[R]
is the password to encrypt the compressed archive.
.PP
In the previous example, this would be:
.IP
.EX
$> sudo ./backup.sh \-\-backup sources.bk /home/john badpw1234
.EE
.PP
You can also tell \f[CR]backup.sh\f[R] to generate a SHA256 file
containing the hash of each file using the \f[CR]\-c\f[R] option.
In the previous example, this would be:
.IP
.EX
$> sudo ./backup.sh \-\-checksum \-\-backup sources.bk /home/john badpw1234
.EE
.PP
The backup utility will begin to copy the files defined in the
\f[I]sources file\f[R]:
.IP
.EX
Copying nginx(1/2)
Copying ssh(2/2)
Compressing backup...
Encrypting backup...
File name: /home/john/backup\-<HOSTNAME>\-<YYYYMMDD>.tar.gz.enc
Checksum file: /home/john/backup\-<HOSTNAME>\-<YYYYMMDD>.sha256
File size: 7336400696(6.9G)
Elapsed time: 259 seconds.
.EE
.PP
After that, you will find the backup archive and the checksum file in
\f[CR]/home/john/backup\-<HOSTNAME>\-<YYYYMMDD>.tar.gz.enc\f[R] and
\f[CR]/home/john/backup\-<HOSTNAME>\-<YYYYMMDD>.sha256\f[R],
respectively.
.PP
You can also use \f[CR]backup.sh\f[R] from a crontab rule:
.IP
.EX
$> sudo crontab \-e
30 03 * * 6 EKEY=$(cat /home/john/.ekey) bash \-c \[aq]/usr/local/bin/backup.sh \-b /usr/local/etc/sources.bk /home/john $EKEY\[aq] > /dev/null 2>&1
.EE
.PP
This will automatically run \f[CR]backup.sh\f[R] every Saturday morning
at 03:30 AM.
In the example above, the encryption key is stored in a local file(with
fixed permissions) to avoid password leaking in crontab logs.
You can also adopt this practice while using the \f[CR]\-\-extract\f[R]
option to avoid password leaking in shell history.
.PP
By default \f[CR]backup.sh\f[R] is very quiet, to add some verbosity to
the output, be sure to use the \f[CR]\-V\f[R](\f[CR]\-\-verbose\f[R])
option.
.SS Backup extraction
\f[B]backup.sh\f[R] can also be used to extract and to verify the
encrypted backup.
To do so, use the following commands:
.IP
.EX
$> ./backup.sh \-\-extract <ENCRYPTED_ARCHIVE> <ARCHIVE_PASSWORD>
.EE
.PP
Where \f[CR]<ENCRYPTED_ARCHIVE>\f[R] is the encrypted backup and
\f[CR]<ARCHIVE_PASSWORD>\f[R] is the backup password.
.PP
For instance:
.IP
.EX
$> ./backup.sh \-\-extract backup\-<hostname>\-<YYYYMMDD>.tar.gz.enc badpw1234
.EE
.PP
This will create a new folder called \f[CR]backup.sh.tmp\f[R] in your
local directory with the following content:
.IP
.EX
backup\-nginx\-<YYYYMMDD>
backup\-ssh\-<YYYYMMDD>
.EE
.PP
\f[B]note\f[R]: be sure to rename any directory with that name to avoid
collisions.
.PP
If you also want to verify the integrity of the backup data, use the
following commands:
.IP
.EX
$> ./backup.sh \-\-checksum \-\-extract <ENCRYPTED_ARCHIVE> <ARCHIVE_PASSWORD> <CHECKSUM_ABSOLUTE_PATH>
.EE
.PP
For instance:
.IP
.EX
$> ./backup.sh \-\-checksum \-\-extract backup\-<hostname>\-<YYYYMMDD>.tar.gz.enc badpw1234 backup\-<hostname>\-<YYYYMMDD>.sha256
.EE
.SS How does backup.sh work?
\f[B]backup.sh\f[R] uses \f[I]rsync\f[R] to copy the files,
\f[I]tar\f[R] to compress the backup, \f[I]gpg\f[R] to encrypt it and
\f[I]sha256sum\f[R] to verify it.
By default, rsync is being used with the following parameters:
.IP
.EX
$> rsync \-aPhrq \-\-delete
.EE
.PP
That is:
.IP
.EX
\- a: archive mode: rsync copies files recursively while preserving as much metadata as possible;
\- P: progress/partial: allows rsync to resume interrupted transfers and to shows progress information;
\- h: human readable output, rsync shows output numbers in a more readable way;
\- r: recursive mode: forces rsync to copy directories and their content;
\- q: quiet mode: reduces the amount of information rsync produces;
\- delete: delete mode: forces rsync to delete any extraneous files at the destination dir.
.EE
.PP
If specified(\f[CR]\-\-checksum\f[R] option), \f[CR]backup.sh\f[R] can
also generate the checksum of each file of the backup.
To do so, it uses \f[CR]sha256sum(1)\f[R] to compute the hash of every
single file using the SHA256 hashing algorithm.
The checksum file contains nothing but the checksums of the files, no
other information about the files stored on the backup archive is
exposed on the unencrypted checksum file.
This may be an issue if you want plausible deniability(see privacy
section for more information).
.PP
After that the backup folder is being encrypted using gpg.
By default, it is used with the following parameters:
.IP
.EX
$> gpg \-a \[rs]
\-\-symmetric \[rs]
\-\-cipher\-algo=AES256 \[rs]
\-\-no\-symkey\-cache \[rs]
\-\-pinentry\-mode=loopback \[rs]
\-\-batch \-\-passphrase \[dq]$PASSWORD\[dq] \[rs]
\-\-output \[dq]$OUTPUT\[dq] \[rs]
\[dq]$INPUT\[dq]
.EE
.PP
This command encrypts the backup using the AES\-256 symmetric encryption
algorithm with a 256bit key.
Here is what each flag do: \- \f[CR]\-\-symmetric\f[R]: Use symmetric
encryption;
.PD 0
.P
.PD
\- \f[CR]\-\-cipher\-algo=AES256\f[R]: Use AES256 algorithm;
.PD 0
.P
.PD
\- \f[CR]\-\-no\-symkey\-cache\f[R]: Do not save password on GPG\[cq]s
cache;
.PD 0
.P
.PD
\- \f[CR]\-\-pinentry\-mode=loopback \-\-batch\f[R]: Do not prompt the
user;
.PD 0
.P
.PD
\- \f[CR]\-\-passphrase\-fd 3 3<< \[dq]$PASSWORD\[dq]\f[R]: Read
password without revealing it on \f[CR]ps\f[R];
.PD 0
.P
.PD
\- \f[CR]\-\-output\f[R]: Specify output file;
.PD 0
.P
.PD
\- \f[CR]$INPUT\f[R]: Specify input file.
.SS Plausible Deniability
While \f[CR]backup.sh\f[R] provide some pretty strong security against
bruteforce attack(assuming a strong passphrase is being used) it should
by no means considered a viable tool against a cryptanalysis
investigation.
Many of the copying, compressing and encrypting operations made by
\f[CR]backup.sh\f[R] during the backup process can be used to invalidate
plausible deniability.
In particular, you should pay attention to the following details:
.IP "1." 3
The \f[CR]\-\-checksum\f[R] option generates an \f[B]UNENCRYPTED\f[R]
checksum file containing the \f[I]digests\f[R] of \f[B]EVERY\f[R] file
in your backup archive.
If your files are known to your adversary(e.g., a banned book), they may
use a rainbow table attack to determine whether you own a given file,
voiding your plausible deniability;
.PD 0
.P
.PD
.IP "2." 3
Since \f[CR]backup.sh\f[R] is essentially a set of shell commands, an
eavesdropper could monitor the whole backup process to extract the name
of the files or the encryption password.
.SH EXAMPLES
Below there are some examples that demonstrate \f[B]backup.sh\f[R]\[cq]s
usage.
.IP "1." 3
Create a backup of \f[CR]/etc/ssh\f[R], \f[CR]/var/www\f[R] and
\f[CR]/var/log\f[R] inside the \f[CR]/tmp\f[R] directory using a
password stored in \f[CR]/home/op1/.backup_pw\f[R]
.PP
The first thing to do is to define the source paths inside a
\f[I]sources file\f[R]:
.IP
.EX
$> cat sources.bk
ssh=/etc/ssh/
web_root=/var/www/
singleFile=/home/john/file.txt
logs=/var/log/
.EE
.PP
After that we can load our encryption key from the specified file inside
an environment variable:
.IP
.EX
$> ENC_KEY=$(cat /home/op1/.backup_pw)
.EE
.PP
Finally, we can start the backup process with:
.IP
.EX
$> sudo backup.sh \-\-backup sources.bk /tmp $ENC_KEY
.EE
.IP "2." 3
Extract the content of a backup made on 2023\-03\-14 with the password
`Ax98f!'
.PP
To do this, we can simply issue the following command:
.IP
.EX
$> backup.sh \-\-extract backup\-af9a8e6bfe15\-20230314.tar.gz.enc \[dq]Ax98f!\[dq]
.EE
.IP "3." 3
Extract the content of a backup made on 2018\-04\-25 using the password
in \f[CR]/home/john/.pw\f[R]
.PP
This example is very similar to the previous one, we just need to read
the password from the text file:
.IP
.EX
$> backup.sh \-\-extract backup\-af9a8e6bfe15\-20180425.tar.gz.enc \[dq]$(cat /home/john/.pw)\[dq]
.EE
.SH AUTHORS
\f[B]backup.sh\f[R] is being developed by Marco Cetica since late 2018.
.SH BUGS
Submit bug reports at: \c
.MT email@marcocetica.com
.ME \c
\ or open an issue on the issue tracker of the GitHub page of this
project: https://github.com/ice\-bit/backup.sh

BIN
bin/backup.sh-1.0.0-2.x86_64.rpm
View File

Binary file not shown.

BIN
bin/backup.sh-1.0.0.x86_64.deb
View File

Binary file not shown.

266
man.md
View File

@@ -1,266 +0,0 @@
---
title: backup.sh
section: 1
header: General Commands Manual
footer: Marco Cetica
date: October 21, 2024
---
# NAME
**backup.sh** - POSIX compliant, modular and lightweight backup utility.
# SYNOPSIS
```
Syntax: ./backup.sh [-b|-e|-c|-V|-h]
options:
-b|--backup SOURCES DEST PASS Backup folders from SOURCES file.
-e|--extract ARCHIVE PASS Extract ARCHIVE using PASS.
-c|--checksum Generate/check SHA256 of a backup.
-V|--verbose Enable verbose mode.
-h|--help Show this helper.
General help with the software: https://github.com/ceticamarco/backup.sh
Report bugs to: Marco Cetica(<email@marcocetica.com>)
```
# DESCRIPTION
**backup.sh** is a POSIX compliant, modular and lightweight backup utility to save and encrypt your files.
This tool is intended to be used on small scale UNIX environment such as VPS, small servers and workstations.
**backup.sh** uses _rsync_, _tar_, _sha256sum_ and _gpg_ to copy, compress, verify and encrypt the backup.
# OPTIONS
**backup.sh** supports three options: **backup creation**, **backup extraction** and **checksum** to verify the
integrity of a backup. The first option requires
root permissions, while the second one does not. The checksum option must be used in combination of one of the previous options.
## Backup creation
To specify the directories to back up, `backup.sh` uses an associative array
defined in a text file(called _sources file_) with the following syntax:
```
<LABEL>=<PATH>
```
Where `<LABEL>` is the name of the backup and `<PATH>` is its path. For example,
if you want to back up `/etc/nginx` and `/etc/ssh`, add the following entries to the _sources file_:
```
nginx=/etc/nginx/
ssh=/etc/ssh/
```
`backup.sh` will create two folders inside the backup archive with the following syntax:
```
backup-<LABEL>-<YYYYMMDD>
```
In the previous example, this would be:
```
backup-nginx-<YYYYMMDD>
backup-ssh-<YYYYMMDD>
```
You can add as many entries as you want, just be sure to use the proper syntax. In particular,
the _sources file_, **should not** include:
- Spaces between the label and the equal sign;
- Empty lines;
- Comments.
You can find a sample _sources file_ at `sources.bk`(or at `/usr/local/etc/sources.bk`).
After having defined the _sources file_, you can invoke `backup.sh` using the following syntax:
```
$> sudo ./backup.sh --backup <SOURCES_FILE> <DEST> <ENCRYPTION_PASSWORD>
```
Where `<SOURCES_FILE>` is the _sources file_, `<DEST>` is the absolute path of the output of the backup
**without trailing slashes** and `<ENCRYPTION_PASSWORD>` is the password to encrypt the compressed archive.
In the previous example, this would be:
```
$> sudo ./backup.sh --backup sources.bk /home/john badpw1234
```
You can also tell `backup.sh` to generate a SHA256 file containing the hash of each file using the `-c` option.
In the previous example, this would be:
```
$> sudo ./backup.sh --checksum --backup sources.bk /home/john badpw1234
```
The backup utility will begin to copy the files defined in the _sources file_:
```
Copying nginx(1/2)
Copying ssh(2/2)
Compressing backup...
Encrypting backup...
File name: /home/john/backup-<HOSTNAME>-<YYYYMMDD>.tar.gz.enc
Checksum file: /home/john/backup-<HOSTNAME>-<YYYYMMDD>.sha256
File size: 7336400696(6.9G)
Elapsed time: 259 seconds.
```
After that, you will find the backup archive and the checksum file in
`/home/john/backup-<HOSTNAME>-<YYYYMMDD>.tar.gz.enc` and `/home/john/backup-<HOSTNAME>-<YYYYMMDD>.sha256`, respectively.
You can also use `backup.sh` from a crontab rule:
```
$> sudo crontab -e
30 03 * * 6 EKEY=$(cat /home/john/.ekey) bash -c '/usr/local/bin/backup.sh -b /usr/local/etc/sources.bk /home/john $EKEY' > /dev/null 2>&1
```
This will automatically run `backup.sh` every Saturday morning at 03:30 AM. In the example above, the encryption
key is stored in a local file(with fixed permissions) to avoid password leaking in crontab logs. You can also
adopt this practice while using the `--extract` option to avoid password leaking in shell history.
By default `backup.sh` is very quiet, to add some verbosity to the output, be sure to use the `-V`(`--verbose`) option.
## Backup extraction
**backup.sh** can also be used to extract and to verify the encrypted backup.
To do so, use the following commands:
```
$> ./backup.sh --extract <ENCRYPTED_ARCHIVE> <ARCHIVE_PASSWORD>
```
Where `<ENCRYPTED_ARCHIVE>` is the encrypted backup and `<ARCHIVE_PASSWORD>` is the backup password.
For instance:
```
$> ./backup.sh --extract backup-<hostname>-<YYYYMMDD>.tar.gz.enc badpw1234
```
This will create a new folder called `backup.sh.tmp` in your local directory with the following content:
```
backup-nginx-<YYYYMMDD>
backup-ssh-<YYYYMMDD>
```
**note**: be sure to rename any directory with that name to avoid collisions.
If you also want to verify the integrity of the backup data, use the following commands:
```
$> ./backup.sh --checksum --extract <ENCRYPTED_ARCHIVE> <ARCHIVE_PASSWORD> <CHECKSUM_ABSOLUTE_PATH>
```
For instance:
```
$> ./backup.sh --checksum --extract backup-<hostname>-<YYYYMMDD>.tar.gz.enc badpw1234 backup-<hostname>-<YYYYMMDD>.sha256
```
## How does backup.sh work?
**backup.sh** uses _rsync_ to copy the files, _tar_ to compress the backup, _gpg_ to encrypt it and
_sha256sum_ to verify it.
By default, rsync is being used with the following parameters:
```
$> rsync -aPhrq --delete
```
That is:
- a: archive mode: rsync copies files recursively while preserving as much metadata as possible;
- P: progress/partial: allows rsync to resume interrupted transfers and to shows progress information;
- h: human readable output, rsync shows output numbers in a more readable way;
- r: recursive mode: forces rsync to copy directories and their content;
- q: quiet mode: reduces the amount of information rsync produces;
- delete: delete mode: forces rsync to delete any extraneous files at the destination dir.
If specified(`--checksum` option), `backup.sh` can also generate the checksum of each file of the backup.
To do so, it uses `sha256sum(1)` to compute the hash of every single file using the SHA256 hashing algorithm.
The checksum file contains nothing but the checksums of the files, no other information about the files stored
on the backup archive is exposed on the unencrypted checksum file. This may be an issue if you want plausible
deniability(see privacy section for more information).
After that the backup folder is being encrypted using gpg. By default, it is used with the following parameters:
```
$> gpg -a \
--symmetric \
--cipher-algo=AES256 \
--no-symkey-cache \
--pinentry-mode=loopback \
--batch --passphrase "$PASSWORD" \
--output "$OUTPUT" \
"$INPUT"
```
This command encrypts the backup using the AES-256 symmetric encryption algorithm with a 256bit key. Here is what each flag do:
- `--symmetric`: Use symmetric encryption;
- `--cipher-algo=AES256`: Use AES256 algorithm;
- `--no-symkey-cache`: Do not save password on GPG's cache;
- `--pinentry-mode=loopback --batch`: Do not prompt the user;
- `--passphrase-fd 3 3<< "$PASSWORD"`: Read password without revealing it on `ps`;
- `--output`: Specify output file;
- `$INPUT`: Specify input file.
## Plausible Deniability
While `backup.sh` provide some pretty strong security against bruteforce attack(assuming a strong passphrase is being used)
it should by no means considered a viable tool against a cryptanalysis investigation. Many of the copying, compressing and
encrypting operations made by `backup.sh` during the backup process can be used to invalidate plausible deniability.
In particular, you should pay attention to the following details:
1. The `--checksum` option generates an **UNENCRYPTED** checksum file containing the _digests_ of **EVERY**
file in your backup archive. If your files are known to your adversary(e.g., a banned book), they may use a rainbow table attack to
determine whether you own a given file, voiding your plausible deniability;
2. Since `backup.sh` is essentially a set of shell commands, an eavesdropper could monitor the whole backup process to extract
the name of the files or the encryption password.
# EXAMPLES
Below there are some examples that demonstrate **backup.sh**'s usage.
1. Create a backup of `/etc/ssh`, `/var/www` and `/var/log` inside the `/tmp` directory using a password
stored in `/home/op1/.backup_pw`
The first thing to do is to define the source paths inside a _sources file_:
```
$> cat sources.bk
ssh=/etc/ssh/
web_root=/var/www/
singleFile=/home/john/file.txt
logs=/var/log/
```
After that we can load our encryption key from the specified file inside an environment variable:
```
$> ENC_KEY=$(cat /home/op1/.backup_pw)
```
Finally, we can start the backup process with:
```
$> sudo backup.sh --backup sources.bk /tmp $ENC_KEY
```
2. Extract the content of a backup made on 2023-03-14 with the password 'Ax98f!'
To do this, we can simply issue the following command:
```
$> backup.sh --extract backup-af9a8e6bfe15-20230314.tar.gz.enc "Ax98f!"
```
3. Extract the content of a backup made on 2018-04-25 using the password in `/home/john/.pw`
This example is very similar to the previous one, we just need to read the password from the text file:
```
$> backup.sh --extract backup-af9a8e6bfe15-20180425.tar.gz.enc "$(cat /home/john/.pw)"
```
# AUTHORS
**backup.sh** is being developed by Marco Cetica since late 2018.
# BUGS
Submit bug reports at: <email@marcocetica.com> or open an issue
on the issue tracker of the GitHub page of this project: https://github.com/ice-bit/backup.sh

4
sources.bk
View File

@@ -1,4 +0,0 @@
ssh=/etc/ssh/
nginx=/etc/nginx/
logs=/var/log/
web_root=/var/www/

91
tests.sh
View File

@@ -1,91 +0,0 @@
#!/usr/bin/env bash
# Unit tests for backup.sh
# This tool is NOT intended to be used outside
# of a testing environment, please use at your own risk.
# By Marco Cetica 2023 (<email@marcocetica.com>)
#
set -e
helper() {
cat <<EOF
backup.sh unit testing suite.
Do **NOT** use this tool outside of a testing environment.
This tool creates a lot of dummy files to emulate a real production
environment. Any file inside the following directories will be overwritten:
- /var/log
- /var/www
- /etc/nginx
- /etc/ssh
Please, use at your own risk.
To acknowledge that, run again this tool with 'I_HAVE_READ_THE_HELPER' as a parameter.
EOF
}
create_files() {
mkdir -p /etc/{ssh,nginx}
mkdir -p /var/{www,log}
touch /etc/ssh/{ssh_config,sshd_config,moduli,ssh_host_dsa_key}
touch /etc/nginx/{nginx.conf,fastcgi.conf,mime.types}
touch /var/www/{index.html,style.css,logic.js}
touch /var/log/{access.log,error.log,lastlog,messages}
for file in ssh_config sshd_config moduli ssh_host_dsa_key ; do
head -c 1M </dev/random > /etc/ssh/$file
done
for file in index.html style.css logic.js ; do
head -c 1M </dev/random > /var/www/$file
done
for file in nginx.conf fastcgi.conf mime.types ; do
head -c 1M </dev/random > /etc/nginx/$file
done
for file in access.log error.log lastlog messages ; do
head -c 1M </dev/random > /var/log/$file
done
}
execute_backup() {
./backup.sh -V -c -b sources.bk "$PWD" badpw
}
extract_backup() {
host="$(uname -n)"
date="$(date +'%Y%m%d')"
./backup.sh -V -c -e "$PWD"/backup-"$host"-"$date".tar.gz.enc badpw "$PWD"/backup-"$host"-"$date".sha256
}
test_backup() {
for dir in "$PWD/backup.sh.tmp/"backup-*-* ; do
if [ ! -d "$dir" ]; then
echo "Can't find '$dir' backup!"
exit 1
else
echo "Found '$dir'"
fi
done
}
if [ $# -eq 0 ]; then
helper
exit 1
fi
if [ "$1" = "I_HAVE_READ_THE_HELPER" ]; then
if [ "$(id -u)" -ne 0 ]; then
echo "Run this tool as root"
exit 1
fi
create_files
execute_backup
extract_backup
test_backup
fi