IIoT (Industrial IoT) architectures are typically distributed and asynchronous, with communication being event-driven, such as the publication (and corresponding subscription) of messages. These asynchronous architectures enhance scalability and tolerance to changes, but raise interoperability issues as the explicit knowledge of the internal structure of the messages and their categorization (topics) is diluted among the elements of the architecture.

In fact, this was also a problem for REST APIs, until the industry came together and proposed a standard way to define the structure and schema of synchronous APIs: OpenAPI (derived from Swagger). For asynchronous architectures, and inspired by OpenAPI, the AsyncAPI appeared to solve this problem:

AsyncAPI provides a specification that allows you to define Message-Driven APIs in a machine-readable format. It’s protocol-agnostic, so you can use it for APIs that work over Kafka, MQTT, AMQP, WebSockets, STOMP, etc. The spec is very similar to OpenAPI/Swagger so, if you’re familiar with it, AsyncAPI should be easy for you.

In AsyncAPI, the specifications of an API can be defined in YAML or JSON, which allows specifying, for example, the message brokers, the topics of interest, or the different message formats associated with each one of the topics, among other aspects. AsyncAPI is, however, in the early stages of development, and the AsyncAPI tool market is underdeveloped, mainly limited to the generation of documentation to be consumed by humans.

Similarly to what we’ve done for OpenAPI (see our API Composer or our API Discoverer), we believe a model-based approach would facilitate the modeling of AsyncAPI specifications and the development of Message-Driven APIs from them. 

Our initial contribution was the approach presented in the featured image above. This work is presented in the video below where Abel presents our work as part of our participation in the 1st AsyncAPI conference and the Models’20 conference. The title of the Models paper is A model-based approach for developing event-driven architectures with AsyncAPI and provides additional details and motivation around our application of model-driven engineering techniques to the world of event-driven architectures and, in particular, to AsyncAPI.

We have now extended this initial framework as depicted in the following figure. You can see how based on the AsyncAPI Specification we have developed an AsyncAPI JSON grammar in Xtext that validates message-driven API definitions conforming to the AsyncAPI Specification. Likewise, from this grammar, Xtext automatically generates the corresponding AsyncAPI metamodel and all the tooling—editor with content assist, parser, etc.—to easily create and transform AsyncAPI JSON definitions into AsyncAPI models conforming to the AsyncAPI metamodel.

Given the availability of both a metamodel for AsyncAPI and the specification of an API as a conformant model, the workflow can continue by executing a M2T transformation that generates an internal DSL. At this moment, the AsyncAPI Toolkit supports the Java language and generates a library that assists developers in the creation, publication and reception of well-formed messages by providing a fluent API. The AsyncAPI toolkit is available on GitHub, make sure you star/watch it to follow its evolution!

It is noteworthy that, since these architectures are message-based, data modeling plays a crucial role. For this reason, we have completed the above workflow with an alternative (graphical) concrete syntax focused on the modeling of the messages to be exchanged. This can be used to  bootstrap an AsyncAPI JSON definition that can be later manually refined.

This full approach has been published in the Sosym Journal under open access license so it’s freely available to everyone. In the following sections we describe a little bit more some of the key components but the paper includes all the nitty-gritty details.

Importing / Modeling an AsyncAPI specification

First, based on the AsyncAPI specification, we created an Xtext grammar. From this grammar, an Ecore metamodel is automatically derived, together with a set of editors and Eclipse-based tools. These editors allow creating JSON-based specifications of message-driven APIs using AsyncAPI. Specifications created using these editors are automatically parsed and reified as instances of the AsyncAPI metamodel.

Generating code to easily process messages from an AsyncAPI specification

Additionally, the prototype is able to generate Java code supporting the creation and serialization of JSON-based message payloads according to the modeled AsyncAPI, including nested JSON objects. No support for arrays is provided yet at this point, however. The excerpt below shows an example of an AsyncAPI specification supported by the prototype:

{
  "asyncapi": "1.2.0",
  "info": {
    "title": "Sample AsyncAPI specification",
    "version": "0.1.0",
  },
  "servers": [
    {
      "url": "broker.url:{port}",
      "scheme": "mqtt",
      "description": "This is an example description",
      "variables": {
        "port": {
          "default": "1883",
          "enum": [ "1883", "8883" ]
        }
      }
    }
  ],
  "topics": {
    "messages/device2controller": {
      "publish": { "$ref" : "#/components/messages/request“ }
    }
  }
},
"components": {
  "schemas": {
    "protocol_version": {
      "title": "Protocol version",
      "type": "integer",
      "default": 2,
      "x-friendly-name": "ProtocolVersion"
    },
    "id": {
      "title": "ID",
      "type": "string",
      "format": "XXXXXX YY ZZZZZZ W"
    },
    "status": {
      "title": "Status",
      "type": "string",
      "enum": ["OK", "ERROR"],
      "x-friendly-name" : "Status"
    },
    "environment": {
      "title": "Environment",
      "type": "string",
      "enum": ["DEV", "STAG","PROD" ],
      "x-friendly-name" : "Environment"
    }
  },
  "messages" : {
    "request" : {
      "summary" : "Request connectivity.",
      "description": "Request connectivity when status changes",
      "payload": {
        "type": "object",
        "properties": {
          "P": { "$ref": "#/components/schemas/protocol_version" },
          "ID": { "$ref": "#/components/schemas/id" },
          "E": { "$ref": "#/components/schemas/environment" },
          "M": {
            "x-friendly-name" : "Message",
            "properties": {
              "S": { "$ref": "#/components/schemas/status" },
              "C": {
                "title": "Content",
                "type": "string",
                "x-friendly-name": "Content"
              }
            }
          }
        }
      }
    }
  }
}

A specification like the above, allows generating messages as follows:

package tests;
import messages.device2controller.Request;
import messages.device2controller.Request.Payload.Environment;
import messages.device2controller.Request.Payload.Message;
import messages.device2controller.Request.Payload.PayloadBuilder;
import messages.device2controller.Request.Payload.Message.Status;

public class Test {

  public static void main(String[] args) {

    PayloadBuilder builder = Request.payloadBuilder();
 
    Request.Payload payload = builder
      .withProtocolVersion(2)
      .withEnvironment(Environment.DEV)
      .withID("id")
      .withMessage(
        Message.newBuilder()
          .withStatus(Status.OK)
          .withContent("Content")
          .build()
      ).build();
 
    System.out.println(payload.toJson(true));
    System.out.println(Request.Payload.fromJson(payload.toJson()).toJson(true));
  }
}

The code generated by our toolkit also allows to easily publish the messages built as explained above, and to subscribe to them using the servers configured in the AsyncAPI specification. Check our online documentation for an example!

Generating a new AsyncAPI from an Ecore model

Until now, we assumed that either you already had an AsyncAPI file to import or you would be using our AsyncAPI editor to create one. In fact, there is a third alternative: take an existing Ecore model you already have available and generate an skeleton AsyncAPI specification from it.

The generator will create a reusable JSON Schema for each domain class. Channels will be created out of annotated EClasses. Moreover, hosts information can also be specified via EAnnotations.

Besides its limitations, obtaining a JSON-based representation of an Ecore model poses several advantages:

• allows developers and architects to create a working AsyncAPI definition without requiring deep knowledge of the specification while
• keeps the modeling environment simple and manageable; and
• allows staying compliant with the AsyncAPI Specification for those unfamiliar with modeling (iv) also enabling experienced developers and architects to refine and complete details of the architecture that cannot be captured using Ecore in an easy way

In order to integrate data models in the proposed development workflow, we have defined both the Ecore to AsyncAPI model-to-model (M2M) and the AsyncAPI to JSON M2T transformations.