Backing Up Proxmox ZFS to AWS S3 Glacier with Rclone

Managing reliable, cost-effective backups for your Proxmox VE environment can be challenging, especially if you’re dealing with terabytes of critical VM and container data. If you’re using ZFS on Proxmox, you already have powerful snapshot and replication tools - but storing long-term archives offsite is just as crucial.

In this article, I’ll walk you through how to use AWS S3 Glacier as a low-cost long-term backup destination using rclone, a versatile command-line tool for managing cloud storage.

We’ll also go through how to set up an AWS IAM user specifically for this task with the right permissions. Let’s get into it.

---

🧠 Understanding the Tech Stack

Before we dive into the setup, let’s quickly understand the moving parts.

Proxmox VE + ZFS

Proxmox VE is a popular open-source virtualization platform. ZFS, known for its robustness and built-in snapshot capabilities, is commonly used for storage in Proxmox environments. With zfs send and zfs snapshot, you can generate point-in-time representations of your datasets-perfect for incremental backups.

AWS S3 Glacier

S3 Glacier is Amazon’s archival storage class, optimized for long-term storage at low cost. Retrieval can take several minutes to hours, so it’s not suitable for active-use data, but perfect for disaster recovery backups.

Rclone

Rclone is a command-line program for syncing files and directories to and from many cloud storage providers, including AWS S3. It’s ideal for scripting and automation in headless or server environments.

---

🔧 Setting Up AWS for Rclone Access

You’ll need an AWS account and a dedicated IAM user with an access key.

1 Create an AWS Account

Go to https://aws.amazon.com and sign up. The free tier gives limited access, but Glacier storage costs are already very low.

2 Create an IAM User for Rclone

1. Open the IAM Console.
2. Click Users → Add users.
3. Username: rclone-backup
4. Select Attach policies directly and choose AmazonS3FullAccess.

Once the IAM user is created, create an access key and save the Access Key ID and Secret Access Key. You’ll use them to configure Rclone.

---

⚙️ Configuring Rclone to Use AWS S3 Glacier

First, install rclone on your Proxmox node:

sudo apt install rclone

Then, configure the remote:

rclone config

Follow the prompts:

1. New remote → aws-glacier
2. Storage → s3
3. Provider → AWS
4. Access Key ID → your IAM access key
5. Secret Access Key → your IAM secret key
6. Region → e.g., eu-central-1
7. Endpoint → leave blank
8. Location constraint → same as region
9. Storage class → GLACIER or GLACIER_IR (for faster restore)

If you encounter have a problem choosing the location constraint (only EU was available for me), then edit the rclone config at /root/.config/rclone/rclone.conf and add the same value for the location constraint as you have chosen for the region.

Verify the config:

rclone ls aws-glacier:

And create the bucket:

rclone mkdir aws-glacier:proxmox-zfs-backup

Your bucket name needs to be globally unique - I’d suggest you to add some unique part to the bucket name.

---

🛠️ Creating a ZFS Snapshot and Upload Script

Now you can use ZFS to create a snapshot, send it to a file, and push that file to Glacier using Rclone.

Here’s a simple backup script:

#!/bin/bash
set -euo pipefail

POOL="rpool"
DATASET="data/vm-100-disk-0"
SNAP_NAME="backup-$(date +%F)"
SNAP_FULL="${POOL}/${DATASET}@${SNAP_NAME}"
DUMP_FILE="/tmp/${DATASET//\//_}_${SNAP_NAME}.zfs"

# Create ZFS snapshot
zfs snapshot "${SNAP_FULL}"

# Dump snapshot to file
zfs send "${SNAP_FULL}" > "${DUMP_FILE}"

# Upload to AWS Glacier
rclone copy "${DUMP_FILE}" aws-glacier:proxmox-zfs-backup/

# Clean up local file
rm "${DUMP_FILE}"

Make it executable:

chmod +x /usr/local/bin/zfs-glacier-backup.sh

And run via cron or systemd timer for automation.

---

💾 Uploading a Single Folder

Alternatively you could just sync parts of the ZFS file system into the bucket by using the following script:

rclone -vv sync --fast-list --checksum /mnt/ZFS/data aws-glacier:proxmox-zfs-backup/

Using this command, rclone will sync all files/subfolders of the given folder into the bucket. By using the param --fast-list fetching the existing file list from the bucket uses as less as possible calls and --checksum will also build checksums of all uploaded files. You can also use this command in a cron job to periodically sync a folder into your S3 glacier.

---

🔁 Restore Process (⚠️ Slow!)

Restoring from Glacier takes time.

1. Use rclone copy to pull the backup file:

rclone copy aws-glacier:proxmox-zfs-backup/<file>.zfs /tmp/

1. Then receive it into ZFS:

zfs receive -F rpool/data/restore-vm

Be aware: Glacier retrievals can take hours unless you use Glacier Instant Retrieval (GLACIER_IR) or Expedited retrieval (which incurs higher costs).

---

✅ Tips for Efficient Backups

• Use zfs send -i for incremental backups.
• Keep a local cache for recent snapshots, and upload only periodically.
• Tag snapshots with metadata (e.g., backup type, date).
• Monitor S3 costs regularly using AWS Cost Explorer.

---

🔐 Security Best Practices

• Never embed AWS credentials in public scripts.
• Rotate keys regularly.
• Consider setting bucket lifecycle policies to auto-delete after X days.
• Use bucket encryption (SSE-S3 or SSE-KMS) for sensitive data.

---

📦 Example Rclone Configuration Block

Here’s what your ~/.config/rclone/rclone.conf might look like:

[aws-glacier]
type = s3
provider = AWS
env_auth = false
access_key_id = YOUR_ACCESS_KEY
secret_access_key = YOUR_SECRET_KEY
region = eu-central-1
location_constraint = eu-central-1 
acl = private
storage_class = GLACIER_IR

---

🧩 Integrating with Proxmox Backup Job

While Proxmox doesn’t directly support custom post-backup hooks yet, you can schedule this ZFS+Rclone method with your own cron jobs or systemd services, independent from Proxmox’s backup GUI.

---

⚠️ UPDATE: Costs and S3/Glacier behaviour to watch out for

After publishing this guide I ran into a practical cost pitfall that I want to highlight up front: storing many small files directly in S3/Glacier can blow up your AWS bill - quickly.

I experienced this first-hand when I tried to back up an Immich photo library by uploading each photo (plus its metadata sidecar) file-by-file. The result was hundreds of thousands of objects (each photo plus a metadata file), and the combination of storage, request, and lifecycle/early-delete charges produced a much larger bill than I expected.

Key points to understand:

• Small objects are cheap per-GB, but each object still creates per-object overhead and triggers API requests for PUT/GET/HEAD/LIST. When you have hundreds of thousands of objects, request costs alone add up fast.
• Metadata/sidecar files effectively double the object count in setups like Immich (one photo + one small metadata file). That doubles request counts and increases cost and listing time.
• Existence checks (HEAD or LIST operations) count as requests and are billed. Tools that probe the remote to decide what to upload (like rclone in some modes) can generate a large number of these requests.
• Glacier storage classes (and many archival classes) have minimum storage durations and early-delete fees. Deleting or moving very recently uploaded objects out of Glacier can incur additional charges.
• Small objects are still billed by size and by request - there is no free “tiny object” tier. In other words, a 1 KB sidecar still costs a request and a little storage; at scale that matters.

Recommendations to avoid the cost trap:

• Prefer bundling many small files into larger archives (tar, tar+gz, or zstd) before uploading. One large file produces far fewer requests and listings than many tiny files.
• Use zfs send to generate single snapshot streams (as shown earlier) and upload those snapshot files rather than syncing millions of small files. This also plays nicely with incremental sends.
• Tune rclone to reduce API calls: use --fast-list to reduce per-object list requests, avoid aggressive --checksum or modes that force extra HEAD/GETs unless you need them, and test dry-runs to measure request counts.
• Use S3 lifecycle policies and understand minimum retention windows for your chosen storage class (Glacier Flexible/Deep/Instant Retrieval have different retention and cost characteristics). Set lifecycle rules to transition objects after a few days only once you understand the retention/early-delete costs.
• For large-scale object management (deleting lots of short-lived objects, or inventorying), consider S3 Inventory + S3 Batch Operations or AWS CLI scripts that operate on manifests to avoid per-object API churn.
• Monitor costs with AWS Cost Explorer and enable billing alerts before running large uploads.

Real-world example (what happened to me): I uploaded an Immich export where each photo had an associated small metadata file. I avoided snapshots intentionally (to preserve file-level deltas) and uploaded file-by-file. Rclone performed many existence checks and uploads; AWS counted PUT requests, LIST/HEAD requests, and then charged for lifecycle transitions and the Glacier minimum durations - the bill spiked. The fix was to stop per-file uploads, switch to larger archives and adopt lifecycle rules after a short trial period.

Dry-runs, estimating request counts and cost pointers

Before you hit “go” on a large upload, do a few dry-runs and inventory checks to estimate how many objects and requests you’ll generate. Here are practical commands and short notes to help you measure and estimate:

Quick object count and total size (fast):

# Shows total number of objects and total bytes (may paginate internally)
rclone size aws-glacier:proxmox-zfs-backup --fast-list

# Or count objects with a listing (fast-list reduces API chatter):
rclone ls aws-glacier:proxmox-zfs-backup --fast-list | wc -l

Do a dry-run sync to see planned operations (no PUTs performed):

rclone sync --dry-run --fast-list --transfers=4 --checkers=8 /mnt/ZFS/data \
  aws-glacier:proxmox-zfs-backup/ -vv --stats=1s

# Read the summary from rclone after the dry-run: it shows number of files to transfer and total bytes.

Additional tips:

• Use rclone copy --dry-run if you prefer a copy test instead of sync. Dry-runs still require listing the remote and will therefore generate LIST/HEAD requests during the simulation - but they avoid PUTs.
• Each object you upload will at minimum generate a PUT request (billed), and many tools also do HEAD or LIST checks per object (also billed).
• Use rclone size + rclone ls | wc -l on a representative subset to extrapolate for the whole dataset before executing a full run.

Sample workflow to test and estimate in three steps:

1. Pick a representative subset (e.g., 1000 files). Run a real upload and record the number of PUT requests seen in CloudWatch or Cost Explorer.
2. Run a dry-run rclone sync --dry-run for the full dataset and note the reported file count.
3. Extrapolate request counts and check the AWS S3/Glacier pricing pages below to estimate cost.

Helpful AWS documentation links (check pricing and retention rules before large runs):

• S3 pricing overview
• Amazon S3 Glacier pricing
• S3 Inventory (useful to create manifests instead of listing live)
• S3 Batch Operations (apply actions to large inventories without per-object API churn)
• AWS Cost Explorer and Budgets

Armed with a dry-run and a small real test, you’ll have far better visibility into how many requests and how much data you’ll actually push - and can adjust strategy (archive vs per-file) before incurring large bills.

---

🧠 Final Thoughts

Storing ZFS snapshots from Proxmox to AWS S3 Glacier using rclone provides a powerful, low-cost, and flexible backup solution. While Glacier isn’t instant-access storage, its affordability makes it ideal for archival and disaster recovery strategies.

By combining native ZFS features with robust cloud tooling like Rclone and AWS IAM, you’re in full control of how and where your data is stored - securely, scalably, and scriptably. 🛡️

If you try this at scale, plan uploads carefully (archive when possible), measure request counts during a dry run, and enable cost alerts - it will save you unpleasant surprises.

Want to help fuel more posts? You know what to do: