Backing Up Proxmox ZFS to AWS S3 Glacier with Rclone

Managing reliable, cost-effective backups for your Proxmox VE environment can be challenging, especially if you’re dealing with terabytes of critical VM and container data. If you’re using ZFS on Proxmox, you already have powerful snapshot and replication tools - but storing long-term archives offsite is just as crucial.

In this article, I’ll walk you through how to use AWS S3 Glacier as a low-cost long-term backup destination using rclone, a versatile command-line tool for managing cloud storage.

We’ll also go through how to set up an AWS IAM user specifically for this task with the right permissions. Let’s get into it.

---

🧠 Understanding the Tech Stack

Before we dive into the setup, let’s quickly understand the moving parts.

✅ Proxmox VE + ZFS

Proxmox VE is a popular open-source virtualization platform. ZFS, known for its robustness and built-in snapshot capabilities, is commonly used for storage in Proxmox environments. With zfs send and zfs snapshot, you can generate point-in-time representations of your datasets-perfect for incremental backups.

☁️ AWS S3 Glacier

S3 Glacier is Amazon’s archival storage class, optimized for long-term storage at low cost. Retrieval can take several minutes to hours, so it’s not suitable for active-use data, but perfect for disaster recovery backups.

🔄 Rclone

Rclone is a command-line program for syncing files and directories to and from many cloud storage providers, including AWS S3. It’s ideal for scripting and automation in headless or server environments.

---

🔧 Setting Up AWS for Rclone Access

You’ll need an AWS account and a dedicated IAM user with an access key.

1️⃣ Create an AWS Account

Go to https://aws.amazon.com and sign up. The free tier gives limited access, but Glacier storage costs are already very low.

2️⃣ Create an IAM User for Rclone

1. Open the IAM Console.
2. Click Users → Add users.
3. Username: rclone-backup
4. Select Attach policies directly and choose AmazonS3FullAccess.

Once the IAM user is created, create an access key and save the Access Key ID and Secret Access Key. You’ll use them to configure Rclone.

---

⚙️ Configuring Rclone to Use AWS S3 Glacier

First, install rclone on your Proxmox node:

sudo apt install rclone

Then, configure the remote:

rclone config

Follow the prompts:

1. New remote → aws-glacier
2. Storage → s3
3. Provider → AWS
4. Access Key ID → your IAM access key
5. Secret Access Key → your IAM secret key
6. Region → e.g., eu-central-1
7. Endpoint → leave blank
8. Location constraint → same as region
9. Storage class → GLACIER or GLACIER_IR (for faster restore)

If you encounter have a problem choosing the location constraint (only EU was available for me), then edit the rclone config at /root/.config/rclone/rclone.conf and add the same value for the location constraint as you have chosen for the region.

Verify the config:

rclone ls aws-glacier:

And create the bucket:

rclone mkdir aws-glacier:proxmox-zfs-backup

Your bucket name needs to be globally unique - I’d suggest you to add some unique part to the bucket name.

---

🛠️ Creating a ZFS Snapshot and Upload Script

Now you can use ZFS to create a snapshot, send it to a file, and push that file to Glacier using Rclone.

Here’s a simple backup script:

#!/bin/bash
set -euo pipefail

POOL="rpool"
DATASET="data/vm-100-disk-0"
SNAP_NAME="backup-$(date +%F)"
SNAP_FULL="${POOL}/${DATASET}@${SNAP_NAME}"
DUMP_FILE="/tmp/${DATASET//\//_}_${SNAP_NAME}.zfs"

# Create ZFS snapshot
zfs snapshot "${SNAP_FULL}"

# Dump snapshot to file
zfs send "${SNAP_FULL}" > "${DUMP_FILE}"

# Upload to AWS Glacier
rclone copy "${DUMP_FILE}" aws-glacier:proxmox-zfs-backup/

# Clean up local file
rm "${DUMP_FILE}"

Make it executable:

chmod +x /usr/local/bin/zfs-glacier-backup.sh

And run via cron or systemd timer for automation.

---

💾 Uploading a Single Folder

Alternatively you could just sync parts of the ZFS file system into the bucket by using the following script:

rclone -vv sync --fast-list --checksum /mnt/ZFS/data aws-glacier:proxmox-zfs-backup/

Using this command, rclone will sync all files/subfolders of the given folder into the bucket. By using the param --fast-list fetching the existing file list from the bucket uses as less as possible calls and --checksum will also build checksums of all uploaded files. You can also use this command in a cron job to periodically sync a folder into your S3 glacier.

---

🔁 Restore Process (⚠️ Slow!)

Restoring from Glacier takes time.

1. Use rclone copy to pull the backup file:

rclone copy aws-glacier:proxmox-zfs-backup/<file>.zfs /tmp/

1. Then receive it into ZFS:

zfs receive -F rpool/data/restore-vm

Be aware: Glacier retrievals can take hours unless you use Glacier Instant Retrieval (GLACIER_IR) or Expedited retrieval (which incurs higher costs).

---

✅ Tips for Efficient Backups

• Use zfs send -i for incremental backups.
• Keep a local cache for recent snapshots, and upload only periodically.
• Tag snapshots with metadata (e.g., backup type, date).
• Monitor S3 costs regularly using AWS Cost Explorer.

---

🔐 Security Best Practices

• Never embed AWS credentials in public scripts.
• Rotate keys regularly.
• Consider setting bucket lifecycle policies to auto-delete after X days.
• Use bucket encryption (SSE-S3 or SSE-KMS) for sensitive data.

---

📦 Example Rclone Configuration Block

Here’s what your ~/.config/rclone/rclone.conf might look like:

[aws-glacier]
type = s3
provider = AWS
env_auth = false
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
region = eu-central-1
location_constraint = eu-central-1 
acl = private
storage_class = GLACIER_IR

---

🧩 Integrating with Proxmox Backup Job

While Proxmox doesn’t directly support custom post-backup hooks yet, you can schedule this ZFS+Rclone method with your own cron jobs or systemd services, independent from Proxmox’s backup GUI.

---

🧭 Final Thoughts

Storing ZFS snapshots from Proxmox to AWS S3 Glacier using rclone provides a powerful, low-cost, and flexible backup solution. While Glacier isn’t instant-access storage, its affordability makes it ideal for archival and disaster recovery strategies.

By combining native ZFS features with robust cloud tooling like Rclone and AWS IAM, you’re in full control of how and where your data is stored - securely, scalably, and scriptably. 🛡️

Let me know in the comments if you’d like a follow-up post on incremental backups, bucket lifecycle policies, or restore benchmarks!