Testing Terraform modules can often be challenging, especially when the resources rely on external providers with strict validation requirements. In my case, I encountered issues while trying to test the logic of my GitHub Federated Identity Credential module using Terraform’s new mocking feature. Below, I’ll walk through the use case, setup, challenges, and my observations regarding the behavior of mock providers.

The Use Case

The module I’m testing creates an Entra ID (formerly Azure AD) Application, Service Principal, and Federated Identity Credential. These resources enable GitHub Actions to authenticate with Azure without requiring a secret.

To validate the module, I set up examples that showcase its usage. These examples serve a dual purpose:

  • Help users understand how to use the module.
  • Act as test cases for automated Terraform tests.

Here’s an example configuration that sets up a Federated Identity Credential for a fictional GitHub repository bar in the organization foo:

module "credential" {

  source = "../../"

  github_organization = "foo"
  repository_name     = "bar"
  entity_type         = "environment"
  environment_name    = "fizz"
  owners              = []

}

The entity_type input controls the structure of the Subject property in the Federated Identity Credential. The logic to determine the subject is encapsulated in the following local block:

locals {
  subject_prefix = "repo:${var.github_organization}/${var.repository_name}"
  subjects = {
    "environment" = try("${local.subject_prefix}:environment:${var.environment_name}")
    "branch"      = try("${local.subject_prefix}:ref:refs/heads/${var.branch_name}")
    "pr"          = try("${local.subject_prefix}:pull_request")
    "tag"         = try("${local.subject_prefix}:ref:refs/tags/${var.tag_name}")
  }
}

I aim to add validation to ensure the correct optional arguments are supplied based on the entity_type. Since this is primarily a logic test, I decided to use Terraform’s mock providers to simplify the process.

Setting Up Mock Providers

Terraform introduced the ability to mock providers in version 1.7.0, enabling module testing without actual API calls. Here’s how I configured the mock provider for Entra ID (azuread):

mock_provider "azuread" {}

The mock provider is then linked in the test setup (environment.ftest.hcl):

run "environment" {
  module {
    source = "./examples/environment"
  }

  providers = {
    azuread = azuread
  }

  assert {
    condition     = length(module.credential.service_principal.object_id) > 0
    error_message = "Must do something"
  }
}

This setup is designed to validate the logic for generating the Subject based on entity_type.

The Problem

When running the test, Terraform encounters errors related to invalid values for resources dependent on the mocked azuread provider:

Errors

Invalid UUID for client_id:

Error: expected "client_id" to be a valid UUID, got fu7jn5ma

Invalid application_id:

Error: parsing "l72bsz17": parsing the Application ID: the number of segments didn't match

These errors indicate that the mocked azuread_application and azuread_service_principal resources are returning random string values for properties like client_id and application_id. Since the azuread_application_federated_identity_credential resource performs client-side validation, the test fails because these values do not conform to expected formats.

tests\environment.tftest.hcl… in progress
 run "setup"… pass
 run "environment"… fail
╷
│ Error: expected "client_id" to be a valid UUID, got fu7jn5ma
│ 
│ with module.credential.azuread_service_principal.main,
│ on main.tf line 9, in resource "azuread_service_principal" "main":
│ 9: client_id = azuread_application.main.client_id
│ 
╵
╷
│ Error: parsing "l72bsz17": parsing the Application ID: the number of segments didn't match
│ 
│ Expected a Application ID that matched (containing 2 segments):
│ 
│ > /applications/applicationId
│ 
│ However this value was provided (which was parsed into 0 segments):
│ 
│ > l72bsz17
│
│ The following Segments are expected:
│
│ * Segment 0 - this should be the literal value "applications"
│ * Segment 1 - this should be the user specified value for this applicationId [for example "applicationId"]
│
│ The following Segments were parsed:
│
│ * Segment 0 - not found
│ * Segment 1 - not found
│
│
│ with module.credential.azuread_application_federated_identity_credential.main,
│ on main.tf line 26, in resource "azuread_application_federated_identity_credential" "main":
│ 26: application_id = azuread_application.main.id
│
╵
tests\environment.tftest.hcl… tearing down
Terraform encountered an error destroying resources created while executing tests\environment.tftest.hcl/environment.
╷
│ Error: expected "client_id" to be a valid UUID, got fu7jn5ma
│
│ with module.credential.azuread_service_principal.main,
│ on main.tf line 9, in resource "azuread_service_principal" "main":
│ 9: client_id = azuread_application.main.client_id
│
╵
╷
│ Error: parsing "l72bsz17": parsing the Application ID: the number of segments didn't match
│
│ Expected a Application ID that matched (containing 2 segments):
│
│ > /applications/applicationId
│
│ However this value was provided (which was parsed into 0 segments):
│
│ > l72bsz17
│
│ The following Segments are expected:
│
│ * Segment 0 - this should be the literal value "applications"
│ * Segment 1 - this should be the user specified value for this applicationId [for example "applicationId"]
│
│ The following Segments were parsed:
│
│ * Segment 0 - not found
│ * Segment 1 - not found
│
│
│ with module.credential.azuread_application_federated_identity_credential.main,
│ on main.tf line 26, in resource "azuread_application_federated_identity_credential" "main":
│ 26: application_id = azuread_application.main.id
│
╵

Analyzing the Mock Provider Behavior

Mock providers are supposed to simulate the behavior of a real provider, but in this case, they seem to return arbitrary strings for resource attributes. This introduces the following issues:

  1. Client-Side Validation Failures:

Resources like azuread_application_federated_identity_credential perform client-side validation on the client_id and application_id. The random strings returned by the mock provider fail these checks.

  1. Dependencies Between Resources:

In my module, the azuread_application_federated_identity_credential depends on outputs from azuread_application and azuread_service_principal. When these outputs are invalid, the dependent resource cannot be created.

Key Observations and Open Questions

  • Mock Outputs and Validation:

It appears that mock providers do not handle validation requirements for dependent resources, which can lead to failures. This raises the question: should mock providers generate outputs that satisfy format expectations for such scenarios?

  • Community Experience:

Is this behavior a common pain point for others using mock providers in Terraform Test? If so, how are they addressing these limitations?

  • Potential Workarounds:

Would it be possible to override specific outputs in the mock provider to produce valid values for testing? For example, specifying UUIDs or formatted strings as outputs for certain attributes.

Conclusion

Testing Terraform modules with mock providers can be a powerful way to isolate and validate logic. However, when resources rely on strict validation of mocked attributes, it introduces challenges that can hinder the effectiveness of tests. In my case, the issues with the azuread provider mock have led to a deeper exploration of whether these limitations are inherent to Terraform’s current mocking implementation or if they can be addressed through configuration.

If you’ve faced similar issues or have insights into improving mock provider behavior, I’d love to hear your thoughts!