Important
While this page describes and summarizes important aspects of the composition of AVM modules, it may not reference All of the shared and language specific requirements.
Therefore, this guide MUST be used in conjunction with the Terraform specifications. ALL AVM modules (Resource and Pattern modules) MUST meet the respective requirements described in these specifications!
Important
Before jumping on implementing your contribution, please review the AVM Module specifications, in particular the Terraform specification pages, to make sure your contribution complies with the AVM module’s design and principles.
Repositories
Each Terraform AVM module will have its own GitHub Repository in the Azure
GitHub Organization as per SNFR19.
This repo will be created by the Module Owners and the AVM Core team collaboratively, including the configuration of permissions as per SNFR9
Directory and File Structure
Below is the directory and file structure expected for each AVM Terraform repository/module.
See template repo here.
tests/
- (for unit tests and additional tests if required - e.g. tflint etc.)unit/
- (optional, may use further sub-directories if required)
modules/
- (for sub-modules only if used)examples/
- (all examples must deploy without successfully without requiring input - these are customer facing)<at least one folder>
- (at least one example that uses the variable defaults minimum/required parameters/variables only)<other folders for examples as required>
/...
- (Module files that live in the root of module directory)_header.md
- (required for documentation generation)_footer.md
- (required for documentation generation)main.tf
locals.tf
variables.tf
outputs.tf
terraform.tf
README.md
(autogenerated)main.resource1.tf
(If a larger module you may chose to use dot notation for each resource)locals.resource1.tf
Directory and File Structure
/ root
│
├───.github/
│ ├───actions/
│ │ ├───avmfix/
│ │ │ └───action.yml
│ │ ├───docs-check/
│ │ │ └───action.yml
│ │ ├───e2e-getexamples/
│ │ │ └───action.yml
│ │ ├───e2e-testexamples/
│ │ │ └───action.yml
│ │ ├───linting/
│ │ │ └───action.yml
│ │ └───version-check/
│ │ └───action.yml
│ ├───policies/
│ │ ├───avmrequiredfiles.yml
│ │ └───branchprotection.yml
│ ├───workflows/
│ │ ├───e2e.yml
│ │ ├───linting.yml
│ │ └───version-check.yml
│ ├───CODEOWNERS
│ └───dependabot.yml
├───.vscode/
│ ├───extensions.json
│ └───settings.json
├───examples/
│ ├───<example_folder>/
│ │ ├───README.md
│ │ ├───_footer.md
│ │ ├───_header.md
│ │ ├───main.tf
│ │ └───variables.tf
│ ├───.terraform-docs.yml
│ └───README.md
├───modules/
│ └───README.md
├───tests/
│ └───README.md
├───.gitignore
├───.terraform-docs.yml
├───CODE_OF_CONDUCT.md
├───LICENSE
├───Makefile
├───README.md
├───SECURITY.md
├───SUPPORT.md
├───_footer.md
├───_header.md
├───avm
├───avm.bat
├───locals.tf
├───main.privateendpoint.tf
├───main.telemetry.tf
├───main.tf
├───outputs.tf
├───terraform.tf
└───variables.tf
Code Styling
This section points to conventions to be followed when developing a module.
Casing
Use snake_casing
as per TFNFR3.
Make sure to review all specifications of Category: Inputs/Outputs
within the Terraform specification pages.
Tip
See examples in specifications SNFR14 and TFFR14.
Resources
Resources are primarily leveraged by resource modules to declare the primary resource of the main resource type deployed by the AVM module.
Make sure to review all specifications covering resource properties and usage.
Tip
See examples in specifications SFR1 and RMFR1.
Outputs
Make sure to review all specifications of Category: Inputs/Outputs
within the Terraform specification pages.
Tip
See examples in specification RMFR7 and TFFR2.
Interfaces
Note
This section is only relevant for contributions to resource modules.
To meet RMFR4 and RMFR5 AVM resource modules must leverage consistent interfaces for all the optional features/extension resources supported by the AVM module primary resource.
Please refer to the Terraform Interfaces page.
Telemetry
To meet SFR3 & SFR4.
We use a telemetry provider modtm.
This is a lightweight telemetry provider that sends telemetry data to Azure Application Insights via a HTTP POST front end service.
The telemetry provider is included in the module by default and is enabled by default.
You do not need to change the configuration included in the template repo.
You must make sure to have the modtm
provider in your required_providers
.
The linter will check this for you.
However, inside your terraform.tf
file please make sure you have this entry:
terraform {
required_providers {
# .. other required providers as needed
modtm = {
source = "Azure/modtm"
version = "~> 0.3"
}
}
}
Eventual Consistency
When creating modules, it is important to understand that the Azure Resource Manager (ARM) API is sometimes eventually consistent.
This means that when you create a resource, it may not be available immediately.
A good example of this is data plane role assignments.
When you create such a role assignment, it may take some time for the role assignment to be available.
We can use an optional time_sleep
resource to wait for the role assignment to be available before creating resources that depend on it.
# In variables.tf...
variable "wait_for_rbac_before_foo_operations" {
type: object({
create = optional(string, "30s")
destroy = optional(string, "0s")
})
default = {}
description = <<DESCRIPTION
This variable controls the amount of time to wait before performing foo operations.
It only applies when `var.role_assignments` and `var.foo` are both set.
This is useful when you are creating role assignments on the bar resource and immediately creating foo resources in it.
The default is 30 seconds for create and 0 seconds for destroy.
DESCRIPTION
}
# In main.tf...
resource "time_sleep" "wait_for_rbac_before_foo_operations" {
count = length(var.role_assignments) > 0 && length(var.foo) > 0 ? 1 : 0
depends_on = [
azurerm_role_assignment.this
]
create_duration = var.wait_for_rbac_before_foo_operations.create
destroy_duration = var.wait_for_rbac_before_foo_operations.destroy
# This ensures that the sleep is re-created when the role assignments change.
triggers = {
role_assignments = jsonencode(var.role_assignments)
}
}
resource "azurerm_foo" "this" {
for_each = var.foo
depends_on = [
time_sleep.wait_for_rbac_before_foo_operations
]
# ...
}
High-level contribution flow
---
config:
nodeSpacing: 20
rankSpacing: 20
diagramPadding: 50
padding: 5
flowchart:
wrappingWidth: 300
padding: 5
layout: elk
elk:
mergeEdges: true
nodePlacementStrategy: LINEAR_SEGMENTS
---
flowchart TD
A(1 - Fork the module source repository)
click A "/AVM-Test/contributing/terraform/terraform-contribution-flow/#1-fork-the-module-source-repository"
B(2 - Setup your Azure test environment)
click B "/AVM-Test/contributing/terraform/terraform-contribution-flow/#2-prepare-your-azure-test-environment"
C(3 - Implement your contribution)
click C "/AVM-Test/contributing/terraform/terraform-contribution-flow/#3-implement-your-contribution"
D{4 - Pre-commit<br>checks successful?}
click D "/AVM-Test/contributing/terraform/terraform-contribution-flow/#4-run-pre-commit-checks"
E(5 - Create a pull request to the upstream repository)
click E "/AVM-Test/contributing/terraform/terraform-contribution-flow/#5-create-a-pull-request-to-the-upstream-repository"
A --> B
B --> C
C --> D
D -->|yes|E
D -->|no|C
GitFlow for contributors
The GitFlow process outlined here depicts and suggests a way of working with Git and GitHub. It serves to synchronize the forked repository with the original upstream repository. It is not a strict requirement to follow this process, but it is highly recommended to do so.
---
config:
logLevel: debug
gitGraph:
rotateCommitLabel: false
---
gitGraph LR:
commit id:"fork"
branch fork/main
checkout fork/main
commit id:"checkout feature" type: HIGHLIGHT
branch feature
checkout feature
commit id:"checkout fix"
branch fix
checkout main
merge feature id: "Pull Request 'Feature'" type: HIGHLIGHT
checkout fix
commit id:"Patch 1"
commit id:"Patch 2"
checkout main
merge fix id: "Pull Request 'Fix'" type: HIGHLIGHT
Tip
When implementing the GitFlow process as described, it is advisable to configure the local clone of your forked repository with an additional remote for the upstream repository. This will allow you to easily synchronize your locally forked repository with the upstream repository. Remember, there is a difference between the forked repository on GitHub and the clone of the forked repository on your local machine.

Note
Each time in the following sections we refer to ‘your xyz’, it is an indicator that you have to change something in your own environment.
Prepare your developer environment
1. Fork the module source repository
Important
Each Terraform AVM module will have its own GitHub repository in the Azure
GitHub Organization as per SNFR19.
This repository will be created by the Module owners and the AVM Core team collaboratively, including the configuration of permissions as per SNFR9
Module contributors are expected to fork the corresponding repository and work on a branch from within their fork, before then creating a Pull Request (PR) back into the source repository’s main
branch.
To do so, simply navigate to your desired repository, select the 'Fork'
button to the top right of the UI, select where the fork should be created (i.e., the owning organization) and finally click ‘Create fork’.
Note
If the module repository you want to contribute to is not yet available, please get in touch with the respective module owner which can be tracked in the Terraform Resource Modules index see PrimaryModuleOwnerGHHandle
column.
Optional: The usage of local source branches
For consistent contributors but also Azure-org members in general it is possible to get invited as collaborator of the module repository which enables you to work on branches instead of forks. To get invited get in touch with the module owner since it’s the module owner’s decision who gets invited as collaborator.
2. Prepare your Azure test environment
AVM performs end-to-end (e2e) test deployments of all modules in Azure for validation. We recommend you to perform a local e2e test deployment of your module before you create a PR to the upstream repository. Especially because the e2e test deployment will be triggered automatically once you create a PR to the upstream repository.
Have/create an Azure Active Directory Service Principal with at least Contributor
& User Access Administrator
permissions on the Management-Group/Subscription you want to test the modules in. You might find the following links useful:
# Linux/MacOs
export ARM_SUBSCRIPTION_ID=$(az account show --query id --output tsv) # or set <subscription_id>
export ARM_TENANT_ID=$(az account show --query tenantId --output tsv) # or set <tenant_id>
export ARM_CLIENT_ID=<client_id>
export ARM_CLIENT_SECRET=<service_principal_password>
# Windows/Powershell
$env:ARM_SUBSCRIPTION_ID = $(az account show --query id --output tsv) # or set <subscription_id>
$env:ARM_TENANT_ID = $(az account show --query tenantId --output tsv) # or set <tenant_id>
$env:ARM_CLIENT_ID = "<client_id>"
$env:ARM_CLIENT_SECRET = "<service_principal_password>"
Change to the root of your module repository and run ./avm docscheck
(Linux/MacOs) / avm.bat docscheck
(Windows) to verify the container image is working as expected or needs to be pulled first. You will need this later.

3. Implement your contribution
To implement your contribution, we kindly ask you to first review the Terraform specifications and composition guidelines in particular to make sure your contribution complies with the repository’s design and principles.
Tip
To get a head start on developing your module, consider using the tooling recommended per spec TFNFR37. For example you can use the newres tool to help with creating variables.tf
and main.tf
if you’re developing a module using Azurerm provider.
4. Run Pre-commit Checks
Important
Make sure you have Docker installed and running on your machine.
Note
To simplify and help with the execution of commands like pre-commit
, pr-check
, docscheck
, fmt
, test-example
, etc. there is now a simplified avm script available distributed to all repositories via terraform-azurerm-avm-template
which combines all scripts from the avm_scripts folder in the tfmod-scaffold repository using avmmakefile.
The avm script also makes sure to pull the latest mcr.microsoft.com/azterraform:latest
container image before executing any command.
4.1. Run pre-commit and pr-check
The following commands will run all pre-commit checks and the pr-check.
# Running all pre-commit checks
# `pre-commit` runs depsensure fmt fumpt autofix docs
# `pr-check` runs fmtcheck tfvalidatecheck lint unit-test
## Linux/MacOs
./avm pre-commit
./avm pr-check
## Windows
avm.bat pre-commit
avm.bat pr-check
4.2 Run e2e tests
Currently you have two options to run e2e tests:
Note
With the help of the avm script and the commands ./avm test-example
(Linux/MacOs) / avm.bat test-example
(Windows) you will be able to run it in a more simplified way. Currently the test-example
command is not completely ready yet and will be released soon. Therefore please use the below docker command for now.
Run e2e tests with the help of the azterraform docker container image.
# Linux/MacOs
docker run --rm -v $(pwd):/src -w /src -v $HOME/.azure:/root/.azure -e TF_IN_AUTOMATION -e AVM_MOD_PATH=/src -e AVM_EXAMPLE=<example_folder> -e ARM_SUBSCRIPTION_ID -e ARM_TENANT_ID -e ARM_CLIENT_ID -e ARM_CLIENT_SECRET mcr.microsoft.com/azterraform:latest make test-example
# Powershell
docker run --rm -v ${pwd}:/src -w /src -v $HOME/.azure:/root/.azure -e TF_IN_AUTOMATION -e AVM_MOD_PATH=/src -e AVM_EXAMPLE=<example_folder> -e ARM_SUBSCRIPTION_ID -e ARM_TENANT_ID -e ARM_CLIENT_ID -e ARM_CLIENT_SECRET mcr.microsoft.com/azterraform:latest make test-example
Make sure to replace <client_id>
and <service_principal_password>
with the values of your service principal as well as <example_folder>
(e.g. default
) with the name of the example folder you want to run e2e tests for.
Run e2e tests with the help of terraform init/plan/apply
.
Simply run terraform init
and terraform apply
in the example
folder you want to run e2e tests for. Make sure to set the environment variables ARM_SUBSCRIPTION_ID
, ARM_TENANT_ID
, ARM_CLIENT_ID
and ARM_CLIENT_SECRET
before you run terraform init
and terraform apply
or make sure you have a valid Azure CLI session and are logged in with az login
.
5. Create a pull request to the upstream repository
Once you are satisfied with your contribution and validated it, submit a pull request to the upstream repository and work with the module owner to get the module reviewed by the AVM Core team, by following the initial module review process for Terraform Modules, described here. This is a prerequisite for publishing the module. Once the review process is complete and your PR is approved, merge it into the upstream repository and the Module owner will publish the module to the HashiCorp Terraform Registry.
5.1 Create the Pull Request [Contributor]
These steps are performed by the contributor:
- Navigate to the upstream repository and click on the
Pull requests
tab. - Click on the
New pull request
button. - Ensure the
base repository
is set to the upstream AVM repo. - Ensure the
base
branch is set to main
. - Ensure your
head repository
and compare
branch are set to your fork and the branch you are working on. - Click on the
Create pull request
button.
5.2 Review the Pull Request [Owner]
- IMPORTANT: The module owner must first check for any malicious code or changes to workflow files. If they are found, the owner should close the PR and report the contributor.
- Review the changes made by the contributor and determine whether end to end tests need to be run.
- If end to end tests do not need to be run (e.g. doc changes, small changes, etc) then so long as the static analysis passes, the PR can be merged to main.
- If end to end tests do need to be run, then follow the steps in 5.3.
5.3 Release Branch and Run End to End Tests [Owner]
- IMPORTANT: The module owner must first check for any malicious code or changes to workflow files. If they are found, the owner should close the PR and report the contributor.
- Create a release branch from
main
. Suggested naminmg convention is release/<description-of-change>
. - Open the PR created by the contributor and click
Edit
at the top right of the PR. - Change the
base
branch to the release branch you just created. - Wait for the PR checks to run, validate the code looks good and then merge the PR into the release branch.
- Create a new PR from the release branch to the
main
branch of the AVM module. - The end to end tests should trigger and you can approve the run.
- Once the end to end tests have passed, merge the PR into the
main
branch. - If the end to end tests fail, investigate the failure. You have two options:
- Work with the contributor to resolve the issue and ask them to submit a new PR from their fork branch to the release branch.
- Re-run the tests and merge to
main
. Repeat the loop as required.
- If the issue is a simple fix, resolve it directly in the release branch, re-run the tests and merge to
main
.
Common mistakes to avoid and recommendations to follow
- If you contribute to a new module then search and update
TODOs
(which are coming with the terraform-azurerm-avm-template) within the code and remove the TODO
comments once complete terraform.lock.hcl
shouldn’t be in the repository as per the .gitignore
file- Update the
support.md
file \_header.md
needs to be updatedsupport.md
needs to be updated- Exclude
terraform.tfvars
file from the repository
Subsections of Contribution Flow
This section describes the contribution flow for module owners who are responsible for creating and maintaining Terraform Module repositories.
Important
This contribution flow is for Module owners only.
As a Terraform Module Owner you need to be aware of the AVM contribution process overview & Terraform specifications (including Interfaces) as as these need to be considered during pull request reviews for the modules you own.
Info
Make sure module authors/contributors tested their module in their environment before raising a PR. The PR uses e2e checks with 1ES agents in the 1ES subscriptions. At the moment their is no read access to the 1ES subscription. Also if more than two subscriptions are required for testing, that’s currently not supported.
1. Owner Activities and Responsibilities
Familiarise yourself with the responsibilities as Module Owner outlined in Team Definitions & RACI and in the TF Issue Triage.
- Watch Pull Request (PR) and issue (questions/feedback) activity for your module(s) in your repository and ensure that PRs are reviewed and merged in a timely manner as outlined in SNFR11.
Info
Make sure module authors/contributors tested their module in their environment before raising a PR. Also because once a PR is raised a e2e GitHib workflow pipeline is required to be run successfully before the PR can be merged. This is to ensure that the module is working as expected and is compliant with the AVM specifications.
2. GitHub repository creation and configuration
Familiarise yourself with the AVM Resource Module Naming in the module index csv’s.
- Example:
terraform-<provider>-avm-res-<rp>-<ARM resource type>
Important
Make sure you have access to the Azure organisation see GitHub Account Link and Access.
Create the module repostory using terraform-azuremrm-avm-template in the Azure
organisation with the following details (internal only). You will then have to complete the configuration of your repository and start an internal business review.
Create GitHub teams as outlined in SNFR20 and add respective parent teams:
Segments:
avm-res-<RP>-<modulename>-module-owners-tf
avm-res-<RP>-<modulename>-module-contributors-tf
Examples:
avm-res-compute-virtualmachine-module-owners-tf
avm-res-compute-virtualmachine-module-contributors-tf
If a secondary owner is required, add the secondary owner to the avm-res-<RP>-<modulename>-module-owners-tf
team.
Add these teams with the following permissions directly to the repository:
- Admin:
avm-core-team-technical-terraform
= AVM Core Team (Terraform Technical) - Admin:
terraform-avm
= Terraform PG - Admin:
avm-res-<RP>-<modulename>-module-owners-tf
= AVM Terraform Module Owners - Write:
avm-res-<RP>-<modulename>-module-contributors-tf
= AVM Terraform Module Contributors
Make sure the branch protection rules for the main
branch are inherited from the Azure/terraform-azurerm-avm-template
repository:
- Require a pull request before merging
- Dismiss stale pull request approvals when new commits are pushed
- Require review from Code Owners
- Require linear history
- Do not allow bypassing the above settings
The respoitory environment test
will be automatically created within 4 hours, it will have approvals and secrets applied to it ready to run end to end tests. You should not create this environment manually.
- If you wish to use your own tenant and subscription for end to end tests, you can override the secrets by setting
ARM_TENANT_ID_OVERRIDE
, ARM_SUBSCRIPTION_ID_OVERRIDE
, and ARM_CLIENT_ID_OVERRIDE
secrets. - If you need to supply additional secrets or variables for your end to end tests, you can add them to the
test
environment. They must be prefixed with TF_VAR_
, otherwise they will be ignored.
3. GitHub Repository Labels
As per SNFR23 the repositories created by module owners MUST have and use the pre-defined GitHub labels. To apply these labels to the repository review the PowerShell script Set-AvmGitHubLabels.ps1
that is provided in SNFR23.
Set-AvmGitHubLabels.ps1 -RepositoryName "Azure/MyGitHubRepo" -CreateCsvLabelExports $false -NoUserPrompts $true
4. Module Handover Activities
- Add new owner as maintainer in your
avm-res-<RP>-<modulename>-module-owners-tf
team and remove any other individual including yourself. - In case primary owner leaves, switches roles or abandons the repo and the corresponding team then the parent team (if assigned) doesn’t have the permissions to gain back access and a ticket with GitHub support needs to be created (but the team can still be removed from the repo since the team
avm-core-team
has permissions on it).
5. Grept
Grept is a linting tool for repositories, ensures predefined standards, maintains codebase consistency, and quality.
It’s using the grept configuration files from the Azure-Verified-Modules-Grept repository.
You can see here which files are synced from the terraform-azurerm-avm-template
repository.
- Set environment variables
# Linux/MacOS
export GITHUB_REPOSITORY_OWNER=Azure
export GITHUB_REPOSITORY=Azure/terraform-azurerm-avm-res-<RP>-<modulename>"
# Windows
$env:GITHUB_REPOSITORY_OWNER="Azure"
$env:GITHUB_REPOSITORY="Azure/terraform-azurerm-avm-res-<RP>-<modulename>"
- Run grept
# Linux/MacOS
./avm grept-apply
# Windows
avm.bat grept-apply
6. Review the module
Once the development of the module has been completed, get the module reviewed from the AVM Core team by following the AVM Review of Terraform Modules process here which is a pre-requisite for the next step.
7. Publish the module
Once a module has been reviewed and is ready to be published, follow the below steps to publish the module to the HashiCorp Registry.
Ensure your module is ready for publishing:
- Create a tag for the module version you want to publish.
Create tag: git tag -a 0.1.0 -m "0.1.0"
Push tag: git push
Create a release on Github based on the tag you just created. Make sure to generate the release notes using the Generate release notes
button.
Optional: Instead of creating the tag via git cli, you can also create both the tag and release via Github UI. Just go to the releases tab and click on Draft a new release
. Make sure to create the tag from the main
branch.

- Elevate your respository access using the Open Source Management Portal (aka.ms/opensource/portal).
- Sign in to the HashiCorp Registry using GitHub.
- Publish a module by selecting the
Publish
button in the top right corner, then Module
- Select the repository and accept the terms.
Info
Once a module gets updated and becomes a new version/release it will be automatically published with the latest published release version to the HashiCorp Registry.
Important
When an AVM Module is published to the HashiCorp Registry, it MUST follow the below requirements:
- Resource Module:
terraform-<provider>-avm-res-<rp>-<ARM resource type>
as per RMNFR1 - Pattern Module:
terraform-<provider>-avm-ptn-<patternmodulename>
as per PMNFR1