Blog

VM-level backup with block-level CBT (QEMU dirty bitmaps), continuous DR replication, cross-restore to Proxmox/Hyper-V/VMware, and streaming format conversion — capabilities the Bacula Enterprise CloudStack plugin does not cover.

Companion technical document to the PodHeitor CloudStack plugin page.

1. The problem: the Bacula Enterprise CloudStack plugin is API-only

The Bacula Enterprise CloudStack plugin operates strictly over the CloudStack API: snapshots a volume, downloads via API, restores via API. That has three hard limits:

• No real CBT. BEE incrementals are API-level snapshot diffs — volume granularity, not block granularity. For a 200 GB VM with 500 MB modified, BEE still transfers the full snapshot delta from CloudStack, not just the dirty blocks.
• No continuous replication. BEE has no DR push mode. Secondary-site replication requires a second tool (CloudStack Snapshot Schedule + manual copy).
• No cross-restore. BEE only restores CloudStack backups to CloudStack. Migration to Proxmox/Hyper-V/VMware requires manual qemu-img conversion plus config rebuild.

The PodHeitor CloudStack Plugin removes all three bottlenecks with direct KVM/libvirt access via NBD, native QEMU dirty bitmap, and a streaming conversion pipeline.

2. Architecture — 100% Rust cdylib

v2.0.0+ is a 100% Rust architecture. The FD-side .so is a pure Rust cdylib (built from ../PodHeitor Rust cdylib/crates/plugin-cloudstack/) that dlopens into bacula-fd. No Bacula AGPLv3 source is statically linked. The legacy C++ shim was removed — every responsibility (plugin identity, parameter validation, PTCOMM driver, virtual-file buffering, restore replay) is reimplemented in the Rust metaplugin-rs framework.

┌──────────────────────────────────────────────────────────┐
│  Bacula File Daemon (FD)                                 │
│  ┌────────────────────────────┐                          │
│  │  podheitor-cloudstack-fd.so│──PTCOMM──▶┌────────────┐ │
│  │  (Rust cdylib)             │◀─────────│  Rust      │ │
│  │                            │          │  Backend   │ │
│  └────────────────────────────┘          └─────┬──────┘ │
│                                                │        │
│  ┌──────────────────────────────────────────────┴─────┐ │
│  │   CloudStack API (HMAC-SHA1) + SSH/NBD data plane  │ │
│  └────────────────────────────────────────────────────┘ │
└──────────────────────────────────────────────────────────┘

The backend (~12,000 LOC Rust) is spawned as a subprocess per job — preserving the PTCOMM crash-isolation boundary. The cdylib is minimal: plugin identity + PTCOMM driver + virtual-file buffering. All backup, restore, replication and conversion logic lives in the backend.

3. Three-Layer Change Detection (CBT)

Real PodHeitor incrementals aren’t snapshot diffs — they are a three-layer pipeline that covers the happy path and the pathological cases:

Typical result: backup window reduction > 90% vs Full on successive incrementals.

4. Operating modes

5. Cross-restore — streaming conversion

The plugin restores a CloudStack VM into a different hypervisor than the source, selected by the cross_restore parameter:

Conversion is streaming: no intermediate file is materialised. The pipeline is NBD read → qemu-img convert (stdin) → SSH/SMB push (stdout). For a 100 GB VM going CloudStack→Hyper-V, that avoids 100 GB of temporary I/O on the FD host.

6. Data plane — three NBD modes

The nbd_access_mode parameter selects the byte-read path from the KVM hypervisor:

Typical arithmetic: direct delivers 3–5 GB/s on 10 GbE LAN; ssh_tunnel ~1.5–2.5 GB/s (crypto overhead bound); api_only rarely exceeds 200 MB/s.

7. Continuous DR replication (bitmap-push)

bitmap_push mode turns the plugin into an out-of-band replication agent: cycles every cycle_interval seconds (default 300) read the current dirty bitmap, compute delta vs last cycle, send to dr_host:dr_port over AES-256-GCM transport with derived PSK.

• PSK ≥ 32 chars required.
• Exponential backoff retries: retry_count=3, retry_delay_ms=5000, jitter retry_jitter_ms=1000.
• Resilience: if the receiver dies, the sender keeps the accumulated bitmap; reconnect resumes from the last confirmed cycle.
• Bandwidth saving: 95%+ vs full copy (delta-only push).

Failover automates promotion: failover_pre drains in-flight, failover_exec presents the replica disks to a new CloudStack VM at the DR site. failback_pre/failback_exec + reprotect close the loop.

8. Performance

9. Memory-mapped Hash DB

The PodHeitor hash DB is a flat memory-mapped array indexed by xxHash64. Unlike traditional DBs (LMDB, RocksDB), there is no SSD latency on the hot path — the kernel page cache keeps the working set resident. Result:

• O(1) lookup with no syscall (mmap-only).
• SSD-latency-independent: run on HDD if you want.
• Working set scales with unique blocks, not with total volume.

10. Documented anti-patterns

• Don’t disable quiesce on DB workloads. Without qemu-guest-agent freeze/thaw, DB snapshots are crash-consistent — valid but not application-consistent. For Postgres/MySQL/MSSQL keep quiesce=true.
• Don’t use verify_ssl=false in production. Import the CloudStack management CA into the FD host trust store. verify_ssl=false exists for PoC/lab.
• Don’t run bitmap_push over WAN without a cap. Default 300s cycle + delta can saturate 100 Mbps on write-heavy VMs. Configure bandwidth_limit or raise cycle_interval.

11. License posture

The plugin ships under LicenseRef-PodHeitor-Proprietary. No Bacula AGPLv3 source is statically linked. Binding is exclusively via independent extern "C" declarations.

Ready to evaluate?

Free 30-day trial for qualified CloudStack workloads (KVM, validated on CloudStack 4.16/4.19/4.20). We guarantee at least 50% discount vs Bacula Enterprise, Veeam, Commvault or NetBackup, with more features included — and cross-restore that none of those three offer.

Heitor Faria — Founder, PodHeitor International
✉ [email protected]
☎ +1 (789) 726-1749 · +55 (61) 98268-4220 (WhatsApp)
🔗 PodHeitor CloudStack plugin page

Disponível em: Português (Portuguese (Brazil))EnglishEspañol (Spanish)