Best Practices

Bacula Enterprise Only

This solution is only available for Bacula Enterprise. For subscription inquiries, please reach out to sales@baculasystems.com.

Openstack Instances and Snapshot Consistency

Openstack Nova usually relies on QEMU to manage virtual machines. QEMU provides a mechanism to communicate with an agent installed on the instance, allowing the instance to take specific actions before a snapshot is created.

This communication happens via a virtual device added to the instance when OpenStack Nova detects that the image being used contains the property hw_qemu_guest_agent=yes.

Verifying that hw_qemu_guest_agent is activated:

In Horizon: the Custom Properties of the image should display hw_qemu_guest_agent yes.
On CLI the command: openstack image show -f value -c properties <name or uuid> should display 'hw_qemu_guest_agent': 'yes'
Check that the instance is set up for communication by verifying if the file /dev/virtio-ports/org.qemu.guest_agent.0 exists.

Note

It is possible to update the properties of an already existing image.

In Horizon: Go to Images, select the relevant image, and choose Update Image Metadata. Under libvirt Driver Options for Images, set hw_qemu_guest_agent to yes.
On CLI: openstack image set --property hw_qemu_guest_agent=yes <image name or uuid>

Warning

While it is possible to update the properties of an already existing image, it is important that these properties are present at a server creation time to be relevant.

To ensure that snapshots are consistent, you can create a custom hook file for freeze/thaw actions at /etc/qemu-ga/fsfreeze-hook.d/test_hook.sh

After taking a snapshot, verify that the hook is executed for both freeze and thaw events.

Here is an example of such a hook script:

Test hook

#!/bin/bash

case $1 in
freeze)
echo "I'm frozen" > /tmp/freeze
;;
thaw)
echo "I'm thawed" >> /tmp/freeze
;;
*)
exit 1
;;
esac

It is then easy to check /tmp/freeze and confirm that the instance has been frozen/thawed.

Another way to verify this is by examining the QEMU guest agent logs, as shown in the example below.

tail /var/log/qemu-ga/qga-fsfreeze-hook.log
Mon Jan 01 12:00:00 PM EST 2025: Executing /etc/qemu-ga/fsfreeze-hook.d/test-hook.sh freeze
Mon Jan 01 12:00:01 PM EST 2025: /etc/qemu-ga/fsfreeze-hook.sh finished successfully
Mon Jan 01 12:00:02 PM EST 2025: Executing /etc/qemu-ga/fsfreeze-hook.d/test-hook.sh thaw
Mon Jan 01 12:00:03 PM EST 2025: /etc/qemu-ga/fsfreeze-hook.sh finished successfully

It is possible to force OpenStack to check the quiescing of an instance by adding os_require_quiesce=yes property on an image.

Warning

As for hw_qemu_guest_agent property, os_require_quiesce must be set on the image before the creation of the server.

If the property is set, and OpenStack cannot communicate with the QEMU guest agent at backup time, the snapshot is aborted, an error is raised, and the error is reported by the plugin.

Warning

Manually checking that an OpenStack server snapshot triggers freeze/thaw is the only way to be sure that snapshots are application-consistent.
Without setting os_require_quiesce=yes, there is no way to remotely check if the QEMU guest agent is installed and active on a server.
If os_require_quiesce=yes is not set, the status of backup (application or crash consistent) is unknown. The snapshot proceeds regardless of the QEMU guest agent status, so the backup may be either application-consistent or crash-consistent.

Backup of the .bmp, .sha Files and the /opt/bacula/etc/openstack.conf File

As the .bmp and .sha files generated in the /opt/bacula/working/openstack directory are very important for the Incremental backups, it is recommended to have a specific backup job to backup these files regularly, even if they are already included in the current OpenStack backups, so they can be easily restored in the case of a disaster with the Bacula proxy server.

Additionally, the /opt/bacula/etc/openstack.conf file, which is also important for the OpenStack backups, can be also included in the same backup job.

This backup job should be configured to run daily using Full level.

For example, below are the Job and Fileset configurations recommended to use for the backup of the .bmp and .sha files in the /opt/bacula/working/openstack directory, along with the /opt/bacula/etc/openstack.conf file:

Job {
    Name = "openstack_working_and_conf_files-job"
    Type = "Backup"
    Client = "bacula-proxy-vm-rhel9-fd"
    Fileset = "openstack_working_and_conf_files-fileset"
    JobDefs = "BackupsToDisk"
    Messages = "Default"
    Pool = "DiskBackup365d"
    Schedule = "DailyFull"
    Storage = "DiskAutochanger"
}

Fileset {
        Name = "openstack_working_and_conf_files-fileset"
        Include {
              Options {
          Compression = LZO
                      Signature = "Md5"
              }
            File = "/opt/bacula/working/openstack"
            File = "/opt/bacula/etc/openstack.conf"
    }
}

In the case of a partial or total disaster of the Bacula proxy server, it is possible to create a new Bacula proxy server, install the Bacula File Daemon and the OpenStack Plugin, and recover the .bmp and .sha files, as well as the /opt/bacula/etc/openstack.conf file from the latest successful Full backup.