Tuesday 25 January 2022

Proxmox Backup Server Evaluation

 I'm already running Proxmox Virtualization Environments (3 x PVE clusters) at $WORK. Currently these just dump backups to NFS, but I am evaluating more sophisticated options in advance of replacing the existing Simplivity clusters with Proxmox. Proxmox Backup Server has (not surprisingly) very good integration with PVE and hence an obvious choice.

The first (actually second - the first was a prototype and got deleted) PVE cluster provides our Dev and Test environments, so for the evaluation I just created a VM there. When I get around to implementing the service properly it will be on separate hardware.

Performance

Currently the daily backups on the dev/test cluster are getting close to overrunning their out-of-hours time slot - so a faster backup is critical for the next environment. My test cases were a couple of recent LXC containers and a very old Centos 5 VM (on the basis that if it works with that it should work for anything!).
There were little differences in time for the backups comparing NFS with LZO compression, NFS with ZSTD compression and PBS (although more than the graph suggests - that's a log scale) until I ran a second backup of the VM using PBS. Backup time for 55Gb VM dropped from 15 minutes to 3 seconds! For that reason alone, PBS looks the winner here. Sadly, the LXCs did not exhibit the same speed up - apparently this an architecture thing due to a VMs ability to track dirty pages.

Data size

In terms of the data footprint, there was again, not much difference between the methods until I got to the second round of backups via PBS. The de-duplication offers a huge gain for VMs and for LXCs. I expect this reduction in data volumes to carry over to the replication when I look at offsite replication.

I am somewhat wary of de-duplication having fallen foul of its implementation in Simplivity a couple of times, however on Simplivity, the virtual disk is included in the same de-duplication dataset as the backups - so one bad block not only takes out your VM but also all your backups too. With the PVE + PBS model, the virtual disk is separate from the backups.

Utility

The ability to mount a PBS backup and extract a single file or directory tree is really useful.

What's not to like...

I've not fully grokked the backup pruning but the facility is an elegant solution to the problem of maintaining older backups. 

Although I've done 2 rounds of backups on PBS, I can currently only see one version in the user interfaces of both PVE and PBS. 

Like local PVE backups, the naming is based on the VM/T ID rather than its given name. If you run an environment like mine that means you need to maintain a lookup table (I've already got this scripted against the API).

The lack of a speed up for LXCs is doubly disappointing - unlike VMs, the LXCs need to be offline for a large part of the backup, but I'm happy with the trade-off of using mostly VMs in the new environment. In my research, I also came across a post by Stefano Marinelli who is using Borg backup for his LXCs with some success. However I'd prefer not to start down the road of providing different solutions to what should be the same problem - particularly when its about provisioning something I hope not to depend upon!
 

Update

Although the first backup was visible in the PBS after it was run, on subsequent runs it was not: after the third run, two versions were visible, after the fourth, three....

Bit of a strange one, but effectively resolved.