In the first part of this series, we explored the solution blueprint for automating the setup of a Virtual Machine that has a Blobfuse mount backed by an Azure Blob Storage container. Now, we’ll move on to the next critical stage: creating a Packer templatethat lays the groundwork for Blobfuse to function seamlessly.

This Packer template serves as the definition of what we want our Virtual Machine to be setup like. It would be ideal if it were flexible, configurable, and ready for the “last mile” adjustments handled by Terraform. By focusing on staging configuration files and installing Blobfuse executables, we’ll ensure our image is primed for efficient integration with Azure Storage.

Packer’s Role

Packer enables us to preconfigure a virtual machine (VM) image with all the necessary components for Blobfuse, but without locking in environment-specific details. This is key because our Terraform configuration, which provisions the Azure environment, determines variables like the storage account name and container details. By keeping these specifics as placeholders in the Packer template, we ensure that the image remains reusable and adaptable across environments.

Packer’s role is to initialize the VM, acting more like a blueprint than a fully configured machine. This means we can set up sane defaults, such as a standard configuration path (/opt/blobfuse2) and a mount directory (/mnt/blobfuse-mount), while leaving the environment-specific details for Terraform to handle later.

Staging Blobfuse Configuration Files

The first task in building our Packer image is to stage the configuration files that Blobfuse requires. These files are essential for mounting Azure Blob Storage as a filesystem on the VM. However, since the storage account name and container details are unknown at this stage, we use placeholders in the configuration file.

YAML Configuration Files

We provide two versions of the Blobfuse configuration YAML, depending on the storage account’s setup:

  1. Managed Identity: For environments that leverage Azure’s Managed Identity to authenticate access.
  2. Access Key: For scenarios where an access key is used for authentication.
blobfuse_config_msi = "blobfuse-msi-config.yaml"
blobfuse_config_key = "blobfuse-key-config.yaml"

These YAML files are staged in the /opt/blobfuse2 directory. They include placeholders for critical variables such as the storage account name and container name. For example the one for Managed Identity looks like this:

# Refer ./setup/baseConfig.yaml for full set of config parameters

allow-other: true

logging:
  type: syslog
  level: log_debug

components:
  - libfuse
  - file_cache
  - attr_cache
  - azstorage

libfuse:
  attribute-expiration-sec: 120
  entry-expiration-sec: 120
  negative-entry-expiration-sec: 240

file_cache:
  path: /tmp/ansible-inventory
  timeout-sec: 120
  max-size-mb: 4096
  cleanup-on-start: false

attr_cache:
  timeout-sec: 7200

azstorage:
  type: block
  account-name: AZURE_STORAGE_ACCOUNT_NAME
  endpoint: https://AZURE_STORAGE_ACCOUNT_NAME.blob.core.windows.net
  mode: msi
  appid: AZURE_CLIENT_ID
  container: AZURE_STORAGE_CONTAINER_NAME

The one for Azure Storage Account Keys looks pretty similar but with one key difference:

# Refer ./setup/baseConfig.yaml for full set of config parameters

allow-other: true

logging:
  type: syslog
  level: log_debug

components:
  - libfuse
  - file_cache
  - attr_cache
  - azstorage

libfuse:
  attribute-expiration-sec: 120
  entry-expiration-sec: 120
  negative-entry-expiration-sec: 240

file_cache:
  path: /tmp/blobfuse-mount
  timeout-sec: 120
  max-size-mb: 4096
  cleanup-on-start: false

attr_cache:
  timeout-sec: 7200

azstorage:
  type: block
  account-name: AZURE_STORAGE_ACCOUNT_NAME
  account-key: AZURE_STORAGE_ACCOUNT_KEY
  endpoint: https://AZURE_STORAGE_ACCOUNT_NAME.blob.core.windows.net
  mode: key
  container: AZURE_STORAGE_CONTAINER_NAME

Using Packer file provisioners, these configuration files are copied to the VM during the image creation process. The placeholders can later be replaced using simple commands, like sed, to inject the actual values after Terraform provisions the environment. This approach ensures the Packer image remains generic while allowing for customization during deployment.

Installing Blobfuse Executables

With the configuration files staged, the next step is to ensure the VM has the necessary Blobfuse libraries and executables installed. This is where Packer’s shell provisioners come into play.

Installing Blobfuse

Using shell scripts, we install the Blobfuse package from the official Azure repositories. This process includes:

  1. Adding the Microsoft repository key and package source.
  2. Installing the Blobfuse library using a package manager like apt (for Ubuntu-based distributions).

Here’s an example of the shell script used in the Packer template:

provisioner "shell" {
  execute_command = local.execute_command
  inline = [
    "wget https://packages.microsoft.com/config/ubuntu/18.04/packages-microsoft-prod.deb", 
    "dpkg -i packages-microsoft-prod.deb",
    "apt-get update",
    "apt-get -y install libfuse3-dev fuse3",
    "apt-get -y install blobfuse2"
  ]
}

This ensures that Blobfuse is available on the VM as soon as it is provisioned.

Preparing Directories for Blobfuse

With Blobfuse installed, we create the directories required for its operation. By default, these include:

  1. /opt/blobfuse2: For configuration files.
  2. /mnt/blobfuse-mount: For the mount point where Blobfuse will attach the Azure Blob Storage container.

These directories are created using a simple shell provisioner in the Packer template:

provisioner "shell" {
  execute_command = local.execute_command
  inline = ["mkdir -p ${var.blobfuse_mount_dir}"]
}
provisioner "shell" {
  execute_command = local.execute_command
  inline = ["mkdir -p ${var.blobfuse_config_path}"]
}

These defaults provide a robust starting point for any Blobfuse setup, while still allowing flexibility for additional mounts or customization as needed.

Bringing It All Together

The Packer template we’ve created combines file provisioners for staging configuration files and shell provisioners for installing Blobfuse and preparing directories. This dual approach ensures that the image is both functional and adaptable. The placeholders in the configuration files enable Terraform to apply “last mile” customization during deployment, injecting the environment-specific details that Blobfuse requires to connect to Azure Storage.

Conclusion

By baking a Packer image with staged configuration files and installed Blobfuse executables, we’ve created a versatile foundation for Azure Storage integration. This approach keeps the image generic, allowing for maximum reusability across environments. With this setup, the heavy lifting is done upfront, leaving Terraform to handle the final configuration steps. In the next part of this series, we’ll explore how to complete the “last mile” configuration using Terraform to replace placeholders and mount Azure Blob Storage.