Review the generated resource

After running the generator, source code for the new resource you added will be in the v2/apis directory.

Have a look through the files in the directory named after the group and version of the resource that was added. Source code for the custom resource will be in this folder, with supporting code in subpackages called arm, storage, and webhook.

In our Workspaces example, the best place to start is v2/api/synapse/v1api20210601/workspace_types_gen.go There may be other resources that already exist in that same directory - that’s expected if ASO already supported some resources from that provider and API version.

Look for the the struct declaration representing the resource you just added. It should be near the top of the appropriate file, and look something like this:

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:name="Ready",type="string",JSONPath=".status.conditions[?(@.type=='Ready')].status"
// +kubebuilder:printcolumn:name="Severity",type="string",JSONPath=".status.conditions[?(@.type=='Ready')].severity"
// +kubebuilder:printcolumn:name="Reason",type="string",JSONPath=".status.conditions[?(@.type=='Ready')].reason"
// +kubebuilder:printcolumn:name="Message",type="string",JSONPath=".status.conditions[?(@.type=='Ready')].message"
// Generator information:
// - Generated from: /synapse/resource-manager/Microsoft.Synapse/stable/2021-06-01/workspace.json
// - ARM URI: /subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Synapse/workspaces/{workspaceName}
type Workspace struct {
    metav1.TypeMeta   `json:",inline"`
    metav1.ObjectMeta `json:"metadata,omitempty"`
    Spec              Workspace_Spec   `json:"spec,omitempty"`
    Status            Workspace_STATUS `json:"status,omitempty"`
}

Look over the Spec and Status types and their properties (and the properties of their properties and so-on). For a different perspective on the same structure, review the structure.txt file which shows the entire structure of the resource as a tree (this can often be easier to review for complex resources).

The Azure REST API specs from which these types are generated are not perfect, so we often have to make some adjustments.

Check: properties that should have been marked read-only but weren’t

On any resource, there are properties that represent information provided by Azure that you don’t get to specify when creating the resource. Sometimes these properties are not correctly marked as read-only, and our code generator includes them on the resource spec.

This is a problem because it means that the user can try to set these properties, and the operator will try to send them to Azure, which will fail.

We have a section in the config where we can modify the shape of resource to remove these as needed. Look for “Deal with properties that should have been marked readOnly but weren’t”.

Here’s an example, where DocumentDB missed setting the provisioningState property to read-only. We override it in the config:

  - group: documentdb
    name: Location  # This type is subsequently flattened into NamespacesTopics_Spec
    property: ProvisioningState
    remove: true
    because: This property should have been marked readonly but wasn't.

Check: secrets generated by Azure

Some resources, such as microsoft.storage/accounts and microsoft.documentdb/databaseAccounts, have keys and endpoints generated by Azure. Unfortunately, there is no good way to automatically detect these in the specs read by the code generator.

You must manually identify if the resource you are adding has keys (or endpoints) which users will want exported to a secret store.

If the resource in question does have Azure generated secrets or endpoints, add those endpoints to the azure-arm.yaml configuration file by specifying $azureGeneratedSecrets.

Our example resource above does not have any Azure generated secrets. As mentioned above, microsoft.documentdb/databaseAccounts has Azure generated secrets. Here is the snippet from the configuration file showing how they were configured:

  documentdb:
    2021-05-15:
      DatabaseAccount:
        $export: true
        $azureGeneratedSecrets:
          - PrimaryMasterKey
          - SecondaryMasterKey
          - PrimaryReadonlyMasterKey
          - SecondaryReadonlyMasterKey
          - DocumentEndpoint

Since these properties are manually configured, they must also be retrieved from Azure manually. This is done using the extension framework supported by the operator. Use the documentdb secrets extension as a template for authoring an extension for your resource. Most extensions go into the customizations package found at the group level.

Check: consumption of secrets

Some resources take secrets as input from the user, for example, when creating a virtual machine the user may supply a password. This is a string field but because it represents a secret value we don’t want to just stick it on the Spec as a string. It needs to be a genruntime.SecretReference (a reference to a secret) instead, so that users can safely supply their secret to the operator without it being accessible via plain-text.

In most cases, the Swagger/OpenAPI document will annotate these properties with an extension x-ms-secret, which indicates that value is a secret. The ASO code-generator uses that annotation to automatically transform the annotated field to a genruntime.SecretReference.

In some cases, the resources Swagger specification is missing this annotation. If that happens, use an override in our azure-arm.yaml to force a particular property to be treated as a secret. For example, under synapse:

WorkspaceProperties:
  SqlAdministratorLoginPassword:
    $isSecret: true

The name WorkspaceProperties comes from the API as defined in azure-rest-api-specs repository. You can usually make a guess at the name, and then review any errors generated by the code generator to find the correct name (these will often read Did you mean ...? with a suggested correct.)

Check: configurable properties

On some resources, we find properties that may need to be dynamically configured at runtime instead of hard coded specified directly in the Yaml file containing the resource.

Typical examples of this are the PrincipalID and TenantID properties, which typically need to be set to a GUID that’s generated by Azure when a different resource is created.

Configure these properties using the $importConfigMapMode declaration. Possible values are Optional, for properties that are sometimes read from a configmap, and Required for properties that must always be read from a configmap.

For example, a Synapse DataLake Storage Account has an AccountUrl that can need to be set at runtime if the related Storage Account is also created by ASO.

  synapse:
    2021-06-01:
      DataLakeStorageAccountDetails:
        AccountUrl:
          $importConfigMapMode: optional

With the resource code generated and all the adjustments made, we have an opportunity to customize the behaviour of the resource.