Core to Azure Service Operator (ASO) v2 is our code generator. This consumes ARM JSON Schema and Swagger specifications and generates code for each desired resource that works with our generic operator reconciler.
Key packages used to structure the code of the generator are as follows:
||Short for Abstract Syntax Tree Model, this package contains the core data types for defining the Go functions, data types, and methods we generate.|
||Support for generation of individual functions, based on the interface `astmodel.Function|
||Support for generation of interface implementations, based on the
||Support for generation of test cases (used to verify our generated code works as expected), based on the
||Intention revealing utility methods for creating the underlying Go abstract syntax tree we serialize as the last step of generation.|
||The core processing pipeline of the code generator|
||Individual pipeline stages that are composed to form the code generator itself.|
||Support methods to make writing tests easier|
Directory structure overview
In this diagram is shown the full directory structure of the ASO code generator, including all the packages named above.
The size of each dot reflects the size of the file; the legend in the corner shows the meaning of colour.
At the core, the code generator works with a rich semantic object model that captures the structure of the resources (and related types) we are generating.
Underpinning this object model is the
astmodel.Type represents a specific data type. The most widely used implementations of
Type fall into a few separate groups:
This list is not exhaustive; other implementations of
Type are used within limited scopes. For example,
OneOfType represent JSON Schema techniques for creating object definitions via composition.
Usefully, there is also
TypeName which is both a type in itself and an indirect reference to a type defined elsewhere.
Type is given a
TypeName, it becomes a
TypeDefinition and can be emitted as the source code for a Go type definition. A set of many
TypeDefinition, each with a unique name is a
ObjectType act as containers, each implementing
FunctionContainer, and embedding
InterfaceImplementer. These do pretty much what you’d expect from the names, though the implementations differ between
ObjectType. For example, where an
PropertyContainer by providing support for an arbitrary set of properties,
ResourceType has only
Most implementations of
astmodel.Function are found in the functions package. New function implementations should go here; existing implementations are slowly being relocated.
All implementations of
astmodel.TestCase are found in the testcases package. New test case implementations should go here.
Resources, Objects and other types
Each distinct resource is represented by a
Spec of each resource is an
ObjectType containing a collection of
PropertyDefinition values, along with implementations of the
TestCase interfaces. The
Status of a resource is a different
Some properties capture primitive values (strings, integers, and so on), while others are themselves complex
ObjectType definitions, forming a hierarchy of information.
MetaType wrappers (including
ValidatedType) are used to add semantic information to both properties and to type definitions.
The code generator itself, found in the package
codegen, is structured as a pipeline, composed of a series of stages that transform our object model incrementally. All the pipeline stages are found in the subpackage
One reason for this is to allow the creation of multiple pipelines (currently we have separate definitions targeting Azure, Crossplane, and for testing) each sharing the majority of their implementation. Another reason is to allow individual pipeline stages to be tested in isolation, though not all existing pipeline stages have tests in this form. New stage implementations should have isolated tests where possible. The helper method
RunTestPipeline() is useful here.
Pipeline stages are instances of
pipeline.Stage. Each has a factory method that returns a
pipeline.Stage instance. In operation, each accepts a
pipeline.State containing the current object model and transforms it into a new state, that is passed to the next stage in turn. If a stage returns an error, the pipeline run is aborted. Each
State instance is immutable, allowing comparison between states when debugging.
New stages should use the
NewStage() function. You’ll see some older stages that predate a structural change use the deprecated
NewLegacyStage() factory instead; these older stages are slowly being migrated and
NewLegacyStage() will be deleted when this is complete.
Code is generated as a Go abstract syntax tree fragments which are then serialized to files as compilable Go code, forming a part of the operator itself.
To make generation easier, our
astbuilder package contains a wide variety of helper methods that allow declarative construction of the required tree. We are using the
dst package instead of the standard Go core
ast package, as it provides better control of comments and whitespace in the final output.