Challenges using Terraform with Azure Serverless Architecture
We’ve been exercising the AzureRM and AzureAD Terraform providers with a healthcare client who wants to go serverless with a new product they are building. We’ve been able to automate a significant portion of the infrastructure with relative ease, however, there are limitations to both the Terraform providers and the operational models of Azure Functions that are causing some pain.
The objectives of our architecture are:
Store all secrets in KeyVault and apply least privledge access control around secrets Codify as much of the environment configuration in Terraform to minimize additional post-deployment manual effort Employ a Microservices Architecture utilizing Azure Serverless & PaaS Offerings (Azure Functions, EventGrid and CosmosDB) to minimize development time, run cost and operational overhead.
Terraform ‘AzureRM’ Provider Issues
The AzureRM provider for Terraform boasts a large number of resources, unfortunately, we’ve found that many of these are incomplete or lack basic documentation required to quickly get up and running that it’s older and more actively developed, peer, the AWS provider, benefits from.
Event Subscriptions for EventGrid Triggered Azure Functions must be created manually
Creating a simple EventGrid trigger will produce some very simplistic code. The missing piece of this puzzle is establishing an EventGrid subscription for this particular Azure Function. This is done by using a WebHook.
You’ll need to create an Event Grid Subscription. From the portal we can gain some insight into this process. The Azure Functions portal interface automatically detects if your Function is using EventGridTrigger and will create this hand shortcut to kick off the process.
Notice this ‘Endpoint Details’ section down at the bottom.
Taking a closer look at the webhook this is generating we can see that the WebHook URL is embedding a code that EventGrid will use to authenticate itself with the Azure Function.
When you look at the ‘azurerm_eventgrid_event_subscription’ resource and see the ‘webhook_endpoint’ property you might feel a rush of excitement! However, alas, the problem is not with the Event Subscription resource but with the Azure Function resource ‘azurerm_function_app’. The ‘code’ parameter in the above WebHook Endpoint URL is the same for all functions in this Function App. However, the Function App resource does not output the master key. There is also an open issue / feature request (#699) but it doesn’t appear to be on the ‘short term roadmap’. There is a shim / work around using a ‘azurerm_template_deployment’ but building JSON text in a Terraform file feels very hacky.
CosmosDB Database and Collections must be created manually.
This is because the ‘azurerm_cosmosdb_sql_database’ resource does not support setting ‘shared throughput’. Normally, when you go and create a Cosmos DB SQL Database you are able to specify whether the Throughput Units are shared across the collections in the database or not.
If you don’t select this option then every collection you create will have its own dedicated Throughput Units (RUs) which will cost you a staggering amount of money! In a Dev/Test environment, this is clearly not needed and should be avoided.
There is an open issue in GitHub (#3623) that has been open since June 2019. There appears to be a pull request (#4616) that promises a fix so hopefully our pain is temporary.