SNFR2 - E2E Testing
ID: SNFR2 - Category: Testing - E2E Testing
Modules MUST implement end-to-end (deployment) testing that create actual resources to validate that module deployments work. In Bicep tests are sourced from the directories in /tests/e2e
. In Terraform, these are in /examples
.
Each test MUST run and complete without user inputs successfully, for automation purposes.
Each test MUST also destroy/clean-up its resources and test dependencies following a run.
Tip
Resources/Dependencies Required for E2E Tests
It is likely that to complete E2E tests, a number of resources will be required as dependencies to enable the tests to pass successfully. Some examples:
- When testing the Diagnostic Settings interface for a Resource Module, you will need an existing Log Analytics Workspace to be able to send the logs to as a destination.
- When testing the Private Endpoints interface for a Resource Module, you will need an existing Virtual Network, Subnet and Private DNS Zone to be able to complete the Private Endpoint deployment and configuration.
Module owners MUST:
- Create the required resources that their module depends upon in the test file/directory
- They MUST either use:
- Simple/native resource declarations/definitions in their respective IaC language,
OR - Another already published AVM Module that MUST be pinned to a specific published version.
- They MUST NOT use any local directory path references or local copies of AVM modules in their own modules test directory.
- Simple/native resource declarations/definitions in their respective IaC language,
- They MUST either use:
Skipping Deployments (SHOULD NOT)
Deployment tests are an important part of a module’s validation and a staple of AVM’s CI environment. However, there are situations where certain e2e-test-deployments cannot be performed against AVM’s test environment (e.g., if a special configuration/registration (such as certain AI models) is required). For these cases, the CI offers the possibility to ‘skip’ specific test cases by placing a file named .e2eignore
in their test folder.
Note
A skipped test case is still added to the ‘Usage Examples’ section of the module’s readme and should be manually validated in regular intervals.
You MUST add a note to the tests metadata description, which explains the excemption.
If you require that a test is skipped and add an “.e2eignore” file (e.g. \<module\>/tests/e2e/\<testname\>/.e2eignore
) to a pull request, a member of the AVM Core Technical Bicep Team must approve set pull request. The content of the file is logged the module’s workflow runs and transparently communicates why the test case is skipped during the deployment validation stage. It iss hence important to specify the reason for skipping the deployment in this file.
Sample filecontent:
The test is skipped, as only one instance of this service can be deployed to a subscription.
Note
For resource modules, the ‘defaults’ and ‘waf-aligned’ tests can’t be skipped.
The deployment of a test can be skipped by adding a .e2eignore
file into a test folder (e.g. /examples/<testname>).