Mastering Last Mile Configuration Automation — Part 2: Let’s Bake a Packer Image
In the first part of this series, we explored the solution blueprint for automating the setup of a Virtual Machine that has a Blobfuse mount backed by an Azure Blob Storage container. Now, we’ll move on to the next critical stage: creating a Packer templatethat lays the groundwork for Blobfuse to function seamlessly.
This Packer template serves as the definition of what we want our Virtual Machine to be setup like. It would be ideal if it were flexible, configurable, and ready for the “last mile” adjustments handled by Terraform. By focusing on staging configuration files and installing Blobfuse executables, we’ll ensure our image is primed for efficient integration with Azure Storage.
Packer’s Role
Packer enables us to preconfigure a virtual machine (VM) image with all the necessary components for Blobfuse, but without locking in environment-specific details. This is key because our Terraform configuration, which provisions the Azure environment, determines variables like the storage account name and container details. By keeping these specifics as placeholders in the Packer template, we ensure that the image remains reusable and adaptable across environments.
Packer’s role is to initialize the VM, acting more like a blueprint than a fully configured machine. This means we can set up sane defaults, such as a standard configuration path (/opt/blobfuse2) and a mount directory (/mnt/blobfuse-mount), while leaving the environment-specific details for Terraform to handle later.
Staging Blobfuse Configuration Files
The first task in building our Packer image is to stage the configuration files that Blobfuse requires. These files are essential for mounting Azure Blob Storage as a filesystem on the VM. However, since the storage account name and container details are unknown at this stage, we use placeholders in the configuration file.
YAML Configuration Files
We provide two versions of the Blobfuse configuration YAML, depending on the storage account’s setup:
- Managed Identity: For environments that leverage Azure’s Managed Identity to authenticate access.
- Access Key: For scenarios where an access key is used for authentication.
blobfuse_config_msi = "blobfuse-msi-config.yaml"
blobfuse_config_key = "blobfuse-key-config.yaml"
These YAML files are staged in the /opt/blobfuse2 directory. They include placeholders for critical variables such as the storage account name and container name. For example the one for Managed Identity looks like this:
# Refer ./setup/baseConfig.yaml for full set of config parameters
allow-other: true
logging:
type: syslog
level: log_debug
components:
- libfuse
- file_cache
- attr_cache
- azstorage
libfuse:
attribute-expiration-sec: 120
entry-expiration-sec: 120
negative-entry-expiration-sec: 240
file_cache:
path: /tmp/ansible-inventory
timeout-sec: 120
max-size-mb: 4096
cleanup-on-start: false
attr_cache:
timeout-sec: 7200
azstorage:
type: block
account-name: AZURE_STORAGE_ACCOUNT_NAME
endpoint: https://AZURE_STORAGE_ACCOUNT_NAME.blob.core.windows.net
mode: msi
appid: AZURE_CLIENT_ID
container: AZURE_STORAGE_CONTAINER_NAME
The one for Azure Storage Account Keys looks pretty similar but with one key difference:
# Refer ./setup/baseConfig.yaml for full set of config parameters
allow-other: true
logging:
type: syslog
level: log_debug
components:
- libfuse
- file_cache
- attr_cache
- azstorage
libfuse:
attribute-expiration-sec: 120
entry-expiration-sec: 120
negative-entry-expiration-sec: 240
file_cache:
path: /tmp/blobfuse-mount
timeout-sec: 120
max-size-mb: 4096
cleanup-on-start: false
attr_cache:
timeout-sec: 7200
azstorage:
type: block
account-name: AZURE_STORAGE_ACCOUNT_NAME
account-key: AZURE_STORAGE_ACCOUNT_KEY
endpoint: https://AZURE_STORAGE_ACCOUNT_NAME.blob.core.windows.net
mode: key
container: AZURE_STORAGE_CONTAINER_NAME
Using Packer file provisioners, these configuration files are copied to the VM during the image creation process. The placeholders can later be replaced using simple commands, like sed, to inject the actual values after Terraform provisions the environment. This approach ensures the Packer image remains generic while allowing for customization during deployment.
Installing Blobfuse Executables
With the configuration files staged, the next step is to ensure the VM has the necessary Blobfuse libraries and executables installed. This is where Packer’s shell provisioners come into play.
Installing Blobfuse
Using shell scripts, we install the Blobfuse package from the official Azure repositories. This process includes:
- Adding the Microsoft repository key and package source.
- Installing the Blobfuse library using a package manager like apt (for Ubuntu-based distributions).
Here’s an example of the shell script used in the Packer template:
provisioner "shell" {
execute_command = local.execute_command
inline = [
"wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb",
"dpkg -i packages-microsoft-prod.deb",
"apt-get update",
"apt-get -y install libfuse3-dev fuse3",
"apt-get -y install blobfuse2"
]
}
This ensures that Blobfuse is available on the VM as soon as it is provisioned.
Preparing Directories for Blobfuse
With Blobfuse installed, we create the directories required for its operation. By default, these include:
- /opt/blobfuse2: For configuration files.
- /mnt/blobfuse-mount: For the mount point where Blobfuse will attach the Azure Blob Storage container.
These directories are created using a simple shell provisioner in the Packer template:
provisioner "shell" {
execute_command = local.execute_command
inline = ["mkdir -p ${var.blobfuse_mount_dir}"]
}
provisioner "shell" {
execute_command = local.execute_command
inline = ["mkdir -p ${var.blobfuse_config_path}"]
}
These defaults provide a robust starting point for any Blobfuse setup, while still allowing flexibility for additional mounts or customization as needed.
Bringing It All Together
The Packer template we’ve created combines file provisioners for staging configuration files and shell provisioners for installing Blobfuse and preparing directories. This dual approach ensures that the image is both functional and adaptable. The placeholders in the configuration files enable Terraform to apply “last mile” customization during deployment, injecting the environment-specific details that Blobfuse requires to connect to Azure Storage.
Conclusion
By baking a Packer image with staged configuration files and installed Blobfuse executables, we’ve created a versatile foundation for Azure Storage integration. This approach keeps the image generic, allowing for maximum reusability across environments. With this setup, the heavy lifting is done upfront, leaving Terraform to handle the final configuration steps. In the next part of this series, we’ll explore how to complete the “last mile” configuration using Terraform to replace placeholders and mount Azure Blob Storage.