Managing complex GitHub Actions workflows can feel like untangling a knot when dynamic inputs, reusable workflows, and environment variables collide. Recently, I encountered a specific issue while trying to streamline the execution of Terraform plan commands using GitHub Actions. My goal was to create a flexible, reusable setup while preserving environment-specific configurations across different workflows. Here’s the problem, the journey, and the solution.

The Problem: Environment Context Challenges in Reusable Workflows

I designed my GitHub Actions setup with a two-layer approach. At the top, an entry point workflow (atat-manual-trigger-terraform-plan-with-params.yaml) serves as the trigger, allowing a variety of GitHub event types to launch the same reusable Terraform workflow. The reusable workflow (atat-terraform-core-workflow-plan-with-params.yaml) encapsulates all the Terraform logic, isolating the business logic from the triggering mechanism. This design avoids duplicating steps across multiple workflows.

The entry point workflow uses workflow_dispatch as its trigger, passing parameters such as the environment. The reusable workflow uses workflow_call to allow seamless invocation by other workflows. These workflows rely heavily on GitHub environment variables, grouped into distinct contexts:

Environment Context (Green)

  • Includes variables like APPLICATION_NAME and ENVIRONMENT_NAME.
  • These are dynamically set based on the environment and passed down the pipeline.

Terraform Runtime Configuration (Gray)

  • Variables like TERRAFORM_WORKING_DIRECTORY and TERRAFORM_VERSION are passed into Terraform as input variables.

Authentication Context (Blue)

  • Variables required for authentication, such as ARM_TENANT_ID.

Backend Context (Yellow)

  • Includes backend configuration for Terraform state, dynamically set using GitHub environment variables.

Alt

My thinking is to allow myself to use a wide variety of triggers to launch the same GitHub Actions workflow — in this case, runs Terraform Plan. The actual Terraform Plan takes place within a reusable GitHub Actions workflow and is encapsulated in the file atat-terraform-core-workflow-plan-with-params.yaml. If I don’t have this encapsulation then all of the steps within this workflow will need to be copy pasta’d into different YAML files that really only change the trigger.

Alt

Despite careful planning, I ran into issues setting the environment attribute, which should allow GitHub environment variables to propagate to jobs and, ideally, my reusable workflow. Errors popped up no matter where I tried to place the environment declaration.

The Attempts and Errors

Initially, I added the environment key at the job level within the entry point workflow:

name: AT-AT-Manual-Trigger-Terraform-Plan-With-Params

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to run against'
        type: environment
        required: true

permissions:
  id-token: write
  contents: read

jobs:

  terraform-plan:
    environment:
      name: $
    uses: ./.github/workflows/atat-terraform-core-workflow-plan-with-params.yaml
    with:
      application-name: $
      environment-name: $
      client-id: $
      tenant-id: $
      subscription-id: $
      terraform-version: $
      terraform-directory: $
      backend-resource-group: $
      backend-storage-account: $
      backend-storage-container: $
I am referencing the Reusable Workflow in this GitHub repository using a relative path ./.github/workflows/atat-terraform-core-workflow-plan-with-params.yaml.
When I run this using the workflow_dispatch trigger I get the following error:
Invalid workflow file: .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml#L20
The workflow is not valid. .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml (Line: 20, Col: 5): Unexpected value ‘uses’ .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml (Line: 21, Col: 5): Unexpected value ‘with’
jobs:

  terraform-plan:
    environment: 
      name:  $
    uses: ./.github/workflows/atat-terraform-core-workflow-plan-with-params.yaml
    with:
      ...

The problem seems to stem from the environment attribute I am setting on the job.

If I remove that then the Reusable Workflow executes but I have no way of referencing all the GitHub Environment variables using vars. Hence, the Reusable Workflow fails with the first time it tries to do something where it needs GitHub Environment variable values. In this case, when we attempt to authenticate with Azure using azure/login.

terraform-plan / terraform-plan Login failed with Error: Using auth-type: SERVICE_PRINCIPAL. Not all values are present. Ensure ‘client-id’ and ‘tenant-id’ are supplied.. Double check if the ‘auth-type’ is correct. Refer to https://github.com/Azure/login#readme for more information.

jobs:

  terraform-plan:
    uses: ./.github/workflows/atat-terraform-core-workflow-plan-with-params.yaml
    with:
      ...

Clearly, without the environment being explicitly set, GitHub environment variables were inaccessible within the reusable workflow. I attempted to set the environment at the workflow level instead:

name: AT-AT-Manual-Trigger-Terraform-Plan-With-Params

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to run against'
        type: environment
        required: true

permissions:
  id-token: write
  contents: read

environment: 
  name:  $

jobs:

  terraform-plan:
    uses: ./.github/workflows/atat-terraform-core-workflow-plan-with-params.yaml
    with:
      ...

This approach also failed with a clear error:

Invalid workflow file: .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml#L15 The workflow is not valid. .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml (Line: 15, Col: 1): Unexpected value ‘environment’

GitHub doesn’t support setting the environment globally for an entire workflow; it must be defined per job.

This seems broken — or maybe I am missing a better place to set the environment at the Job-level. However, the error message seems pretty clear, I can’t set the environment at the Workflow level. It must be set per-job. However, when using a reusable workflow, it doesn’t like the environment attribute set.

I then tried moving the environment block beneath the with block:

name: AT-AT-Manual-Trigger-Terraform-Plan-With-Params

on:
  workflow_dispatch:
    inputs:
      environment:
        description: 'Environment to run against'
        type: environment
        required: true

permissions:
  id-token: write
  contents: read

jobs:

  terraform-plan:
    uses: ./.github/workflows/atat-terraform-core-workflow-plan-with-params.yaml
    with:
      ...
    environment: 
      name:  $

However, this returned the same error, suggesting that GitHub Actions does not allow the environment key with reusable workflows at all.

Invalid workflow file: .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml#L30 The workflow is not valid. .github/workflows/atat-manual-trigger-terraform-plan-with-params.yaml (Line: 30, Col: 5): Unexpected value ‘environment’

The Current State: Searching for Solutions

At this point, the inability to dynamically set the environment at the job level has left me stuck. The environment key must be set per job, but it cannot be used alongside the uses key for reusable workflows. This effectively blocks access to the vars. variables in reusable workflows.

Potential workarounds I’m exploring include:

  • Passing GitHub Environment Variables Explicitly via a Dummy Job: Introducing a preparatory job in the workflow that accesses vars. variables and exposes them as outputs. These outputs are then passed as inputs to the reusable workflow job. While this approach enables dynamic access to environment variables, it adds an extra job and requires maintenance of output-variable mappings.
  • GitHub Feature Request: Raising an issue with GitHub to support dynamic environments in reusable workflows or enhance the behavior of the environment key.

Conclusion: A Roadblock in Workflow Modularity

Reusable workflows are a powerful feature, but the lack of support for dynamically setting environment limits their flexibility in real-world scenarios. Without a clear path to access GitHub Environment variables, workflows that rely on context-sensitive configurations face significant hurdles. Until GitHub addresses this limitation, I’ll continue exploring creative workarounds to achieve the modularity and simplicity I envisioned.