Introduction

Maintaining consistency across environments is a cornerstone of robust cloud operations. In Azure, separating Compute Galleries for development (Dev) and production (Prod) environments ensures a clean boundary between experimental and mission-critical workloads. This separation allows for innovation and iteration in Dev while protecting Prod from untested or faulty changes. But how do you handle machine images, especially when leveraging dynamic versioning and automated pipelines in Dev? More importantly, how do you ensure the exact image tested in Dev makes its way to Prod without introducing variability or risks? The answer lies in promoting images from Dev to Prod, rather than rebuilding them separately. This article outlines how to implement this strategy using Azure Compute Galleries, the Azure CLI, and Azure DevOps.

Step 1: Dynamically Versioning Images in DEV

The journey begins in the Dev environment, where new machine images are dynamically created using a Packer build pipeline. Each time a Packer template is modified, the pipeline generates a new version of the image in the Dev Azure Compute Gallery. This approach allows teams to iterate quickly, testing configurations and updates in an isolated environment. However, this dynamism also brings challenges. While Dev is designed for experimentation, the images it produces must be carefully evaluated before moving to Prod. After testing, the focus shifts to promoting the image.

A tempting but flawed solution would be to rebuild the image in Prod using the same Packer template. Although this might seem straightforward, it risks introducing subtle variations. Commands like apt-get update or apt-get upgrade may pull newer package versions, leading to inconsistencies. These small differences can become significant when running mission-critical workloads. To avoid these risks, it’s critical to maintain a single source of truth by promoting the exact image from Dev to Prod. This process guarantees that the version approved in Dev is identical to what runs in Prod, eliminating unexpected discrepancies.

Step 2: Setting Up the Azure CLI for Image Promotion

To promote images, you can use the Azure CLI. First, authenticate with Azure using a service principal and set the target subscription. You’ll need to make sure that the identity you use has some key role assignments set to ensure it has the necessary access to both Compute Galleries.

DEV Compute Gallery (Source):

  • Role Needed: Reader or Compute Gallery Image Version Reader role.
  • Purpose: Allows the pipeline to read details about the image definitions and versions in the Dev Azure Compute Gallery. This ensures the pipeline can fetch and reference the specific image version to be promoted.

PROD Compute Gallery (Destination):

  • Role Needed: Contributor or Compute Gallery Image Version Contributor role.
  • Purpose: Allows the pipeline to create image versions in the Prod Azure Compute Gallery. The role must grant the ability to write new image versions to the gallery while ensuring the pipeline cannot inadvertently modify other resources in the gallery.

Here is the full script to perform the promotion of a Virtual Machine Image from DEV to PROD Azure Compute Galleries:

echo 'Azure CLI authN'

az login --service-principal -u $ARM_CLIENT_ID -p $ARM_CLIENT_SECRET --tenant $ARM_TENANT_ID

az account set --subscription $ARM_SUBSCRIPTION_ID

# Input arguments
imageName="$1"
imageVersion="$2"
sourceSharedComputeGallery="$3"
sourceResourceGroup="$4"
destinationSharedComputeImageGallery="$5"
destinationResourceGroup="$6"
replicationLocations="$7"

# Get source image and version details
sourceImage=$(az sig image-definition show --gallery-name "$sourceSharedComputeGallery" --resource-group "$sourceResourceGroup" -i "$imageName")
sourceImageVersion=$(az sig image-version show --gallery-name "$sourceSharedComputeGallery" --resource-group "$sourceResourceGroup" -i "$imageName" --gallery-image-version "$imageVersion")

# Create image definition in the destination gallery if it doesn't exist
destinationImage=$(az sig image-definition show --gallery-name "$destinationSharedComputeImageGallery" --resource-group "$destinationResourceGroup" -i "$imageName" 2>/dev/null)

if [[ -z "$destinationImage" ]]; then
    echo "Error, the destination Image does not exist: $destinationSharedComputeImageGallery/$imageName"
    exit 1
fi

# Get replication target regions and prepare replication input
targetRegions=$(echo "$replicationLocations" | tr ',' '\n')
replicationInput=""
for region in $targetRegions; do
    replicationInput+="$region "
done
replicationInput=${replicationInput%?} # Remove trailing comma

# Copy image version from source to destination gallery
az sig image-version create \
    --gallery-name "$destinationSharedComputeImageGallery" \
    --resource-group "$destinationResourceGroup" \
    --gallery-image-definition "$imageName" \
    --gallery-image-version "$imageVersion" \
    --image-version "$(echo $sourceImageVersion | jq -r '.id')" \
    --replica-count 1 \
    --storage-account-type Standard_LRS \
    --target-regions $replicationInput

This script retrieves the image and version details from the DEV gallery then confirms that the corresponding image definition exists in the PROD gallery. If the image definition doesn’t exist, you’ll need to create it manually (or in Terraform) before proceeding. Once validated, the script uses the az sig image-version create command to copy the image version from the Dev gallery to the Prod gallery, specifying target replication regions to ensure global availability.

Step 3: Automating Promotion with Azure DevOps

Integrating the promotion process into an Azure DevOps pipeline streamlines the workflow. The pipeline accepts the image name and version as parameters and runs the promotion script within a dedicated stage. Proper access control is crucial here: the pipeline identity needs read access to the Dev gallery and write access to the Prod gallery. By adhering to the principle of least privilege, you can limit potential risks while enabling efficient promotion. trigger: none

parameters:
- name: image_name
  displayName: "Image to promote, must exist in the 'Azure Compute Gallery', galtpashareddevopsdev"
  type: string
- name: image_version
  displayName: "Version to promoted, must exist in the 'Azure Compute Gallery', galtpashareddevopsdev"
  type: string

stages:
- stage: stage_promote
  displayName: "Promote To Production Stage"
  pool: 
    vmImage: 'ubuntu-latest'
  jobs:
  - job: job_promote
    timeoutInMinutes: 0
    displayName: "Promote image $:$ to Production"
    steps:
    - task: Bash@3
      displayName: "AZ CLI - Packer Promote Script"
      inputs:
        filePath: .azdo-pipelines/scripts/packer-promote.sh
        workingDirectory: .azdo-pipelines/
        arguments: $ $ galtpashareddevopsdev rg-tpa-shared-devops-dev-gallery galtpashareddevopsprod rg-tpa-shared-devops-prod-gallery westus3,eastus,eastus2,westus2,centralus
      env:
        ARM_CLIENT_ID: $(ARM_CLIENT_ID)
        ARM_CLIENT_SECRET: $(ARM_CLIENT_SECRET)
        ARM_SUBSCRIPTION_ID: $(ARM_SUBSCRIPTION_ID)
        ARM_TENANT_ID: $(ARM_TENANT_ID)

One of the key advantages of this approach is maintaining a clean version history in the Prod gallery. While the Dev gallery can retain a long history of versions for testing and experimentation, the Prod gallery should include only well-tested and approved versions. This separation ensures that production workloads operate on stable, trusted images, free from the clutter of experimental or untested changes.

Conclusion

Promoting machine images from DEV to PROD rather than rebuilding them is a simple yet powerful strategy to ensure consistency, reliability, and safety in your Azure environments. By baking images once in DEV, testing them thoroughly, and using Azure CLI and DevOps pipelines to promote them to PROD, you eliminate risks associated with variability while maintaining a clean separation between environments. This approach empowers teams to innovate in DEV without compromising the stability of production workloads. With the right setup, you can ensure that every image running in production is exactly what you intended — stable, tested, and reliable.