As someone who frequently uses GitHub Actions for managing Terraform workflows, I’ve always been keen to streamline processes and avoid redundancy. My traditional approach to GitHub Actions workflows has been effective, but over time, I’ve noticed it’s also become quite verbose and repetitive. This realization has led me on a journey to explore how to make my workflows more modular, reusable, and efficient. This article, the first in a series, focuses on how I currently set up my workflows. While future parts will delve into making GitHub Actions reusable, here, I’ll walk you through my setup, the challenges I face, and the thought process behind the conventions I use.

The Problem: Verbose Workflows and Lack of Modularity

The current setup works, but it’s not as streamlined as I’d like. For instance:

I have a single YAML file where jobs are hardcoded, leading to repetitive tasks and reduced maintainability. While GitHub environments offer a convenient way to organize configuration attributes, parameterizing around them when using reusable workflows has proven challenging.

I want the same workflow to handle multiple triggers — manual dispatch, pull requests, and merges — without duplicating code. This leads to verbose workflows that are hard to maintain.

The Conventions I Follow

To manage complexity, I rely on four types of variables, each serving a distinct purpose:

Environment Context (Green)

  • Includes variables like APPLICATION_NAME and ENVIRONMENT_NAME.
  • These are dynamically set based on the environment and passed down the pipeline.

Terraform Runtime Configuration (Gray)

  • Variables like TERRAFORM_WORKING_DIRECTORY and TERRAFORM_VERSION are passed into Terraform as input variables.

Authentication Context (Blue)

  • Variables required for authentication, such as ARM_OID_REQUEST_TOKEN.

Backend Context (Yellow)

  • Includes backend configuration for Terraform state, dynamically set using GitHub environment variables.

Alt

All of these “contextualizers” are split between either GitHub Repository or GitHub Environment variables / secrets. Anything that can change from environment to environment, you guessed it, goes into a GitHub Environment. Usually, that means that the APPLICATION_NAME is the lonely sole holding the fort at the Repository level.

A Deep Dive into My Setup

My typical Terraform workflow for GitHub Actions consists of one primary job, terraform-plan, which executes four key steps.

Alt

Whatever Trigger I’m using, in this case I’m demonstrating the workflow_dispatch trigger because it is really the only trigger that demonstrates, in my opinion, the appropriate use of GitHub Actions. Jobs and Steps are called out in dark blue. Contextualizers are called out with their respective colors. Orange is used to denote actual Bash script commands and Magenta is used to highlight those few places where GitHub Actions steps in and helps me out by passing state around to the Jobs and Steps that I define.

1. Checkout Source Code

This is the obligatory first step where the workflow checks out the repository’s codebase. Every workflow needs this step to ensure it’s working with the latest source code.

2. Authenticate with Azure

I use Azure OpenID Connect (OIDC) for authentication. This step performs an Azure login and generates a token stored in a hidden variable called ACTIONS_ID_TOKEN_REQUEST_TOKEN.

While this is not how GitHub output variables typically work, I can reference this variable directly in a subsequent step simply by treating it as an environment variable in Bash. This feels unusual, but it works, likely due to the following permissions set at the top of the workflow:

permissions:
  id-token: write
  contents: read

3. Exporting Variables for Terraform

In this step, I map GitHub environment variables to Terraform-specific environment variables. This includes: Authentication Context: Passing the OIDC token (ACTIONS_ID_TOKEN_REQUEST_TOKEN) to ARM_OID_REQUEST_TOKEN, which is required by the azurerm Terraform provider.

Backend Context: Using GitHub environment variables for backend configuration, such as the resource group, storage account, and container names.

Environment Context: Dynamically setting variables like APPLICATION_NAME and ENVIRONMENT_NAME for clarity and consistency across workflows. Here’s a snippet of how I pass these variables:

    - id: plan
      name: Terraform Plan
      env:
        ARM_SUBSCRIPTION_ID: $
        ARM_TENANT_ID: $
        ARM_CLIENT_ID: $
        ARM_USE_OIDC: true
        TF_VAR_application_name: $
        TF_VAR_environment_name: $
      working-directory: $
      run: |
        export ARM_OID_REQUEST_TOKEN=$ACTIONS_ID_TOKEN_REQUEST_TOKEN
        
        terraform init \
          -backend-config="resource_group_name=$" \
          -backend-config="storage_account_name=$" \
          -backend-config="container_name=$" \
          -backend-config="key=$-$"
          
        terraform plan

The Authentication Context gets dumped into well known environment variables that are picked up by the “azurerm” Terraform provider. Because I am using OIDC, the most important Authentication Context attribute is pulled in from the ACTIONS_ID_TOKEN_REQUEST_TOKEN and stored into a well known environment variable ARM_OID_REQUEST_TOKEN.

The Backend Context are referenced inline within the bash script to modify terraform init to use the backend-config that I want to specify.

The Environment Context is used in two places. First to parameterize the Terraform Root Module by passing in application_name and environment_name as input variables. No logic is performed based on these names, so you won’t see rediculous things like:

foo = var.environment_name == "dev" ? "this" : "that"

These names are used to create a standard naming convention of resources and consistent tagging. That’s it. Any other logic should be based on other input variables that are more type-safe and deterministic. For example:

resource "something_special" {

  count = var.something_enabled ? 1 : 0

}

If something_special needs to be turned on in the dev environment, then my TFVARS configuration for dev will have the input variable something_enabled set to true. This is the way.

4. Running Terraform Commands

Finally, I execute terraform init and terraform plan. My setup uses a convention-based approach for backend configuration, dynamically setting the key based on APPLICATION_NAME and ENVIRONMENT_NAME. This ensures consistency and works seamlessly across most scenarios:

terraform init \
  -backend-config="resource_group_name=${BACKEND_RESOURCE_GROUP_NAME}" \
  -backend-config="storage_account_name=${BACKEND_STORAGE_ACCOUNT_NAME}" \
  -backend-config="container_name=${BACKEND_STORAGE_CONTAINER_NAME}" \
  -backend-config="key=${APPLICATION_NAME}-${ENVIRONMENT_NAME}"

terraform plan

This four-step structure works well for managing Terraform workflows, but it’s verbose when duplicated across multiple triggers and workflows.

Challenges with GitHub Environments

One of the main reasons I haven’t fully transitioned to reusable workflows is the difficulty of parameterizing around GitHub environments. While GitHub allows environments to be passed as parameters in workflow_dispatch, referencing these dynamically in reusable workflows often feels unintuitive and inconsistent.

For example, I can select an environment from a dropdown when triggering a workflow manually, but passing the environment’s name as a string to a reusable workflow for dynamic behavior introduces complexities. This limitation has been a key blocker in making my workflows fully modular.

The Odd Behavior of Hidden Variables

Another curious aspect of my setup is how the ACTIONS_ID_TOKEN_REQUEST_TOKEN variable works. Unlike GitHub’s traditional output variables, this token behaves more like a system environment variable. It seems to bypass the usual scoping rules and can be referenced directly in subsequent steps, which is not immediately obvious in the documentation.

This behavior, while effective, adds to the complexity and makes it harder to adopt a truly reusable approach without thorough testing and experimentation.

Conclusion

This setup has served me well for managing Terraform workflows in GitHub Actions, but it’s not without its pain points. The verbosity and challenges with parameterizing GitHub environments are key areas I’m working to address. While this article focused on my current approach, future parts in this series will explore potential solutions for making GitHub Actions reusable and more DRY.

If you’ve faced similar challenges or have insights into making GitHub Actions reusable with GitHub environments, I’d love to hear your thoughts. Stay tuned for the next part, where I’ll dive deeper into reusable workflows and share my findings.