Automated testing in the early stages of an ‘API first’ approach to API development

Estafet consultants occasionally produce short, practical tech notes designed to help the broader software development community and colleagues. 

If you would like to have a more detailed discussion about any of these areas and/or how Estafet can help your organisation and teams with best practices in improving your SDLC, you are very welcome to contact us at enquiries@estafet.com 

Introduction

API development has seen a paradigm shift with the “API First” methodology and OpenAPI specifications. When executed correctly, the API-first approach can be ‘extremely’ efficient, allowing stakeholders to understand and contribute to the design of the API in its early stages. In this document, we will cover the problems that can arise and one of the ‘correct’ ways to solve them.  By the end of this document we will have demonstrated how to:

  1. Easily create an OpenAPI Spec, as descriptively as possible
  2. Validate the OpenAPI specification’s structure
  3. Create BDD style requirements for the API endpoints, understood by all the stakeholders
  4. Create component tests that are executed against the most recent version of the OpenAPI specification, proving that it follows the business requirements

The problem

While developing APIs without an API-first approach might seem quicker initially, it can lead to extended development cycles, increased technical debt, and a product that doesn’t entirely meet stakeholder requirements or expectations, part of the reason is that all the stakeholders might not fully understand the system in its early stages.

Let’s say we decided to adopt the API-first approach in our next project. More often than not, APIs tend to require modifications in the early stages of their development due to stakeholder misalignment or evolving specifications. We need a clear way to describe the system to all stakeholders so we start by developing an OpenAPI specification YAML/JSON. While it provides some structure and clarity to the reader of the document, it still does not describe the API’s behaviour in a readable and concise way. 

  • On top of that, how are we sure that the contents of the spec are semantically correct?
  • Who  is validating the structure of the document?
  • Who is responsible for aligning the business objects(schemas) to the business domain requirements?

A solution

The purpose of this document is to demonstrate a component testing technique that can catch issues with the API specification as early as the initial design of the OpenAPI spec itself, and a way to validate your OpenAPI specification as part of CI, ensuring that you cannot pass a malformed spec to a tool/client/third party.

The technique also allows all stakeholders to understand what the system must do before API development even starts.

We will use the following project to demonstrate a proof of concept: https://github.com/stdNullPtr/API-first-books-api-spec/

Foundational Concepts

Before diving deep into the nuances of API-first development, it’s crucial to understand the basics:

API

At its core, an API is a set of rules and protocols that allows one software application to interact with another. It defines the methods and structures developers can use to request and exchange information.

API-First Approach

Traditional software development often involves designing the API after building the main application. In contrast, the API-first approach emphasises designing the API first, before any coding begins. This ensures a clear contract between the frontend and backend teams and can result in faster, more reliable development cycles.

OpenAPI

An open standard for defining and describing APIs. OpenAPI Specifications (formerly known as Swagger specifications) are simple yet powerful descriptions of RESTful APIs, written in JSON or YAML format.

(OpenAPI Initiative)

Cucumber

A software tool specialised for behaviour-driven development (BDD). It enables the non-technical stakeholders to understand and contribute to the application’s development by allowing requirements to be written in plain English. Cucumber tests are written in a language called Gherkin, which can then be executed and verified against the application.

(Cucumber)

OpenAPI Generator

An open-source tool that automates the generation of API client libraries, server stubs, documentation, and other essential code pieces, using OpenAPI Specifications. It helps streamline the development process by creating boilerplate code, ensuring consistency and saving time.

(OpenAPI Generator)

Understanding the API first approach

In its essence, the API-first approach is a paradigm shift in how we think about software development. Instead of viewing the API as an afterthought or merely a byproduct, it’s viewed as a first-class citizen. Here’s why this perspective is advantageous:

Clear Contract

By defining the API upfront, there’s a clear agreement on how the software should behave. This minimises misunderstandings between teams and streamlines the development process.

Parallel Development

With a well-defined API, frontend and backend teams can work simultaneously. The frontend team can mock the API responses, while the backend team focuses on implementing the business logic.

Consistency and Reusability

An API designed first often results in more consistent endpoints, making it easier for developers to understand and consume. Furthermore, a well-crafted API can be reused across multiple projects or platforms.

Writing an OpenAPI Specification

A “must” read: https://learn.openapis.org/

An OpenAPI Specification is a detailed, structured description of an API, written using the OpenAPI format. It encompasses all the various facets of an API, such as the endpoints available, the HTTP methods they support, expected input and output formats, and even authentication methods. The goal is to provide a complete blueprint of the API so that both humans and machines can understand its capabilities and interactions without ambiguity.

Once written, the specification can serve multiple purposes: It can be a reference document for developers, act as the foundation for generating code, or even be rendered into interactive API documentation tools.

In our case for the PoC, the OpenAPI specification is our source of truth and as such it must be defined as descriptive and detailed as possible, with as many examples and documentation inside it, as possible. The specification can be found here

Notice how the specification refers to schemas outside the file. This is crucial in the cases of large APIs and multiple people working together on them, the schema definitions should be as modular as possible, without impacting readability. In the provided example we can see a balance, externalising some object schemas, and leaving the rest in the main YAML, allowing us to reuse schemas.

Enhancing Swagger Documentation with Examples

The Power of Examples in Documentation

While a well-structured specification offers a clear view of the API’s capabilities, incorporating examples into the documentation further amplifies its value. A good starting point in understanding what the examples are: https://learn.openapis.org/specification/docs.html

Examples offer:

  • Clarification: An abstract API endpoint definition can sometimes be hard to grasp. Seeing a concrete example can make it immediately clear what the API call is expected to look like and what it will return.
  • Quick Onboarding: For developers unfamiliar with the API, examples can serve as a quick reference guide, helping them to get started faster.
  • Reduced Ambiguity: While a specification describes the possible shapes of requests and responses, examples provide explicit instances, leaving little room for misunderstandings.
  • Enhanced Testing: Examples can be used as test cases or even as mock responses when setting up test environments or stub servers.

Integration with Tools

When you embed examples directly into the Swagger specification, several tools can utilise them beyond mere documentation. For instance:

  • API Mock Servers: Tools that generate mock servers from OpenAPI specs can use the provided examples as responses, enabling a more realistic simulation of the API.
  • Client SDK Generation: When generating client libraries, some tools can incorporate the examples into the generated documentation or even the code itself, aiding developers in understanding how to use the generated library.
  • Testing Tools: Automated testing tools can leverage examples as test cases, ensuring that the actual API implementation aligns with the documented behaviour.

In our case, the integration we have with the OpenAPI generator tool allows it to use the examples provided in the spec to generate Swagger documentation and to set up example responses from the generated API (which we will cover later). This is identical to the API Mock Servers case mentioned above.

In conclusion, when following API First development approach, the OpenAPI specification MUST be as detailed and as clear as possible since it is our only source of truth. By nature, the spec should be understandable by all the stakeholders, allowing them to agree upon it.

Understanding component testing

Component testing, often referred to as module or unit testing (though there are subtle differences), focuses on verifying individual components of the software in isolation. A ‘component’ can be a single function, method, procedure, module, or object in a software application. The primary objective of component testing is to ensure that each individual component functions correctly in isolation from the rest of the application.

For example, in the context of an API, a single endpoint or a specific set of related endpoints can be seen as a ‘component’. Testing this in isolation would mean ensuring that given a specific request, the endpoint behaves correctly, irrespective of the larger application’s state.

Component Testing and Cucumber BDD

Behaviour-Driven Development (BDD) is an approach that bridges the gap between technical and non-technical stakeholders in a software project. It emphasises collaboration and ‘shared understanding‘, advocating for the description of software behaviour without detailing how that functionality is implemented.

Cucumber is a widely used tool that facilitates BDD. It allows for writing tests in a natural, human-readable language, making the test scenarios comprehensible to non-technical stakeholders as well. When you combine component testing and Cucumber BDD, you get:

  • Clear Descriptions: Each component’s functionality is described in plain language, making it easier for all stakeholders, from developers to product owners, to understand the expected behaviour.
  • Collaborative Approach: As the test scenarios are comprehensible to a wider audience, it fosters collaboration. Product managers, QA engineers, and developers can collectively define and refine the expected behaviour.
  • Automatable Verification: While the scenarios are written in plain language, tools like Cucumber can automate their execution. This means that the written scenarios aren’t just passive documentation but can actively verify the software’s behaviour.
  • Rapid Feedback: With automated component tests, developers receive quick feedback on their changes. If a modification breaks an existing component, the tests will immediately signal it.

In the journey of API development, component testing using Cucumber BDD ensures that each individual piece of the API behaves as expected. Moreover, the use of plain language in test scenarios ensures transparency and mutual understanding among all project stakeholders. This alignment is vital for maintaining the quality and correctness of the API as it evolves.

BDD Example with our PoC


Let’s say we have an API endpoint that we need to define. The relevant stakeholder can describe the behaviour he needs from the system, using Gherkin syntax:

If this looks too technical, we can change it to something easier to understand, like:

This is the power of Gherkin and BDD, we define in plain English what we expect from the system, and the developer/testers will implement the functional steps behind the simple text, which looks like this (snippet, referring to the first example provided above):

This allows us to shape our system in a way that everyone can understand, while also having it tested!

You can find the related .feature file, in our example project here.

A perfect example of how you can describe a BDD scenario in Jira

The Role of OpenAPI Generator in setting up the tests

In the previous example (BDD Example with our PoC) we looked at scenarios that interact with a backend “books api” service. This service was generated entirely during build time, by OpenAPI generator plugin.

The way this works is:

  1. Create a Spring Boot Application: https://github.com/stdNullPtr/API-first-books-api-spec/tree/master
  2. We create an OpenAPI spec: https://github.com/stdNullPtr/API-first-books-api-spec/blob/master/src/main/resources/openapi/openapi.yaml
  3. Following the documentation provided by the OpenAPI generator team (https://openapi-generator.tech/docs/plugins) we add a maven dependency for the generator plugin and populate some key fields: https://github.com/stdNullPtr/API-first-books-api-spec/blob/master/pom.xml#L92C4-L92C4
  4. Running mvn compile will now generate models and Spring Boot Rest controllers, which are derived from the OpenAPI spec
  5. We can now configure Cucumber to use the generated service for the component tests: https://github.com/stdNullPtr/API-first-books-api-spec/blob/master/src/test/java/com/estafet/booksapi/component/CucumberConfig.java#L10

From now on the component tests will start the generated service and execute the steps against it.

This technique brings us a few advantages:

  1. The OpenAPI spec is being semantically verified during the code generation, proving that our source of truth is correctly structured.
  2. The examples added in the specification are now the responses of the generated endpoints.
  3. We can use the generated models in the tests, allowing us to catch issues whenever the specification is modified in an incompatible way, for example, we can use a field to verify its value and if for some reason that field is removed from the spec, the CI pipeline/build will fail.

Running Component Tests

Test it for yourself: 

1. git clone https://github.com/stdNullPtr/API-first-books-api-spec.git

2. Open the project in IntelliJ preferably

3. Navigate to src/test/java/com/estafet/booksapi/component/CucumberTestRunner.java

4. Run the test runner

5. Observe the component tests being executed against code entirely generated from the OpenAPI spec

    Conclusion

    When following the API-First approach, we are now able to:

    1. Easily create an OpenAPI Spec, as descriptively as possible
    2. Validate the OpenAPI specification’s structure
    3. Create BDD style requirements for the API endpoints, understood by all the stakeholders
    4. Create component tests that are executed against the most recent version of the OpenAPI specification, proving that it follows the business requirements

    By Antonio Lyubchev, Consultant at Estafet

    Stay Informed with Our Newsletter!

    Get the latest news, exclusive articles, and updates delivered to your inbox.