autorest

Writing Tests

Build Prerequisites

To test AutoRest in each language, you must set up your machine according to the requirements on the Building Code page.

Architecture

Tests are split into unit and acceptance tests. Unit tests validate the AutoRest application itself and how it interprets Swagger documents. Acceptance tests validate the generated code in each language and are written in those languages.

Unit tests

Unit tests need to be updated when core parts of AutoRest change, not when the language-specific generator code changes. Unit tests are located in:

\src\core\AutoRest.Core.Tests
These need to be updated when the command-line AutoRest application itself changes
\src\modeler\AutoRest.Swagger.Tests
\src\modeler\AutoRest.Swagger.Composite.Tests
These need to be updated when there are changes to how AutoRest processes Swagger files
\src\generator\AutoRest.CSharp.Unit.Tests
These need to be updated when there are changes to the C# generator, but are also useful if a change reflect all the way to generated code

How to add unit tests

In the unit test project (e.g. AutoRest.CSharp.Unit.Tests), there are two things to add:

  1. The Swagger file used for this test. Place it in Resource\<TestName>\<TestName>.[yaml/json], e.g. Resource\Bug1560\Bug1560.yaml for a test concerning Github issue 1560.
  2. The unit test file <TestName>.cs, e.g. Bug1560.cs. This file should have the following layout:
     // Copyright (c) Microsoft Corporation. All rights reserved.
     // Licensed under the MIT License. See License.txt in the project root for license information.
     // 
    
     using ...
    
     namespace AutoRest.CSharp.Unit.Tests
     {
         public class TestName : BugTest
         {
             public TestName(ITestOutputHelper output) : base(output)
             {
             }
    
             /// <summary>
             /// Test description.
             /// </summary>
             [Fact]
             public async Task SomeTestCase()
             {
                 using (var fileSystem = GenerateCodeForTestFromSpec())
                 {
                     /* inspect generated code */
                 }
             }
    
             /* further tests */
         }
     }
    

    GenerateCodeForTestFromSpec will automatically load and process the Swagger file if named as described previously. Refer to existing unit tests for further assistance, including compilation of the generated source and inspection of the resulting assembly using reflection.

Acceptance tests (and test server)

Acceptance tests are run against a Node.js test server (which uses Express framework). The code for the test server is checked in to the \src\dev\TestServer folder in the repository.

There are two main components to the test server: the Swagger definitions that describe the server and the code that handles requests to the server and responds with the appropriate status code, payload, etc. if the request is constructed correctly.

How to add acceptance tests for scenarios

  1. Add your scenarios to the Swagger files that describe the test server (located in the \src\dev\TestServer\swagger folder).
  2. Update the test server
    • Update the routes to generate appropriate responses for your scenarios at paths specified in the Swagger files in step 1. This code is located in the \src\dev\TestServer\server\routes\*.js files.
    • For each scenario, the coverage dictionary needs to be incremented for the name of your scenario. This name will be used in the test report coverage.
    • Update the coverage dictionary in \src\dev\TestServer\server\app.js to include the names of your new scenarios. This lets the final test report include your scenarios when reporting on the coverage for each language.
  3. Regenerate the expected code using gulp regenerate:expected (this will use the Swagger files to generate client libraries for the test server).
  4. In each language, write tests that cover your scenarios (for example, in C#, you must update \src\generator\AutoRest.CSharp.Tests\AcceptanceTests.cs or \src\generator\AutoRest.CSharp.Azure.Tests\AcceptanceTests.cs). You will make calls to the test server using the generated code from step 3.
  5. Run the tests

Running Tests

When you run tests, the test server is automatically started and the code that is generated for the test server Swagger files will correctly target this new instance.

Command Line

Tests can be run with gulp test. You can run tests for each language individually with gulp:test:[language name]. Use gulp -T to find the correct names.

Visual Studio

In Visual Studio, you can run tests for all languages using Task Runner Explorer. C# tests can also be run and debugged in Test Explorer.

Debugging the test server

When updating the test server code to return the appropriate responses for your scenarios, it can be useful to debug the code to make sure the test code calls the paths that you are expecting.

Visual Studio

  1. Install Node.js Tools for Visual Studio solution.
  2. Open the \src\dev\TestServer\server\SwaggerBATServer.sln.
  3. Run the SwaggerBATServer project.
  4. Make sure that the port that the test server is using matches the port that is used by the tests when you run them.