View on GitHub

Microservice DSL (MDSL)

A Domain-Specific Language (DSL) to specify (micro-)service contracts, their data representations and API endpoints.

Tools Overview, OpenAPI, GraphQL, Jolie, Java, Freemarker templating, AsyncAPI

Protocol Buffers Generator

The MDSL Eclipse plugin and the CLI allow API designers to generate Protocol Buffer specifications out of MDSL.

Usage

You can generate the specifications out of an MDSL model by using the Eclipse plugin or our CLI.

In Eclipse you find the generator in the MDSL context menu:

Protocol Buffers Specification Generator Context Menu in Eclipse

The following command generates a specification in case you work with the CLI:

./mdsl -i model.mdsl -g proto

Hint: Both tools generate the output into the src-gen folder which is located in the projects root directory (Eclipse) or the directory from which the mdsl command has been called (CLI). Both tools create the directory automatically in case it does not already exist.

Generator Output / Mapping

The generator maps the MDSL concepts to *.proto files as follows:

Example

The following example illustrates what the generator produces for an exemplary MDSL contract.

You find the complete sources (incl. generated *.proto file) of this example here.

We use the following MDSL model which was an outcome of this blogpost to illustrate our generator outputs:

API description ReferenceManagementServiceAPI

data type PaperItemDTO { "title":D<string>, "authors":D<string>, "venue":D<string>, "paperItemId":PaperItemKey }
data type PaperItemKey { "doi":D<string> }
data type createPaperItemParameter { "who":D<string>, "what":D<string>, "where":D<string> }

endpoint type PaperArchiveFacade
  serves as INFORMATION_HOLDER_RESOURCE
  exposes
    operation createPaperItem
      with responsibility STATE_CREATION_OPERATION
      expecting
        payload createPaperItemParameter
      delivering
        payload PaperItemDTO
    operation lookupPapersFromAuthor
      with responsibility RETRIEVAL_OPERATION
      expecting
        payload D<string>
      delivering
        payload PaperItemDTO*
    operation convertToMarkdownForWebsite
      expecting
        payload PaperItemKey
      delivering
        payload D<string>

For the MDSL contract above the generator produces the following *.proto file:

syntax = "proto3";

package ReferenceManagementServiceAPI;

message PaperItemDTO {
  string title = 1;
  string authors = 2;
  string venue = 3;
  PaperItemKey paperItemId = 4;
}

message PaperItemKey {
  string doi = 1;
}

message createPaperItemParameter {
  string who = 1;
  string what = 2;
  string where = 3;
}

message lookupPapersFromAuthorRequestMessage {
  string anonymous1 = 1;
}

message PaperItemDTOList {
  repeated PaperItemDTO entries = 1;
}

message ConvertToMarkdownForWebsiteResponseMessage {
  string anonymous2 = 1;
}

service PaperArchiveFacade {
  rpc lookupPapersFromAuthor(lookupPapersFromAuthorRequestMessage) returns (PaperItemDTOList);
  rpc createPaperItem(createPaperItemParameter) returns (PaperItemDTO);
  rpc convertToMarkdownForWebsite(PaperItemKey) returns (ConvertToMarkdownForWebsiteResponseMessage);
}

You can use the generated *.proto files to implement a gRPC interface.

Java Client/Server Sample

We explain how you can start implementing server and client code in Java with the following steps.

First, setup your project. We use Gradle as our build tool. In the build.gradle file we have to add the following dependencies:

implementation 'io.grpc:grpc-netty-shaded:1.33.1'
implementation 'io.grpc:grpc-protobuf:1.33.1'
implementation 'io.grpc:grpc-stub:1.33.1'
compileOnly 'org.apache.tomcat:annotations-api:6.0.53' // necessary for Java 9+

In addition, since we want to generate code from *.proto files, we have to add protobuf-gradle-plugin as follows:

plugins {
    id 'com.google.protobuf' version '0.8.14'
}

protobuf {
  protoc {
    artifact = "com.google.protobuf:protoc:3.12.0"
  }
  plugins {
    grpc {
      artifact = 'io.grpc:protoc-gen-grpc-java:1.33.1'
    }
  }
  generateProtoTasks {
    all()*.plugins {
      grpc {}
    }
  }
}

Now we are already ready to integrate our *.proto file into the project. We copy the file that we have generated above into the src/main/proto directory.

Once we copied the *.proto file we can run ./gradlew clean build and the sources will be generated into build/generated/source/proto/main/grpc and build/generated/source/proto/main/java. To let your IDE know that the generated source are there, you have to adjust the build.gradle file a bit. Just add the following code block:

sourceSets {
    main {
        java {
            srcDirs 'build/generated/source/proto/main/grpc'
            srcDirs 'build/generated/source/proto/main/java'
        }
    }
}

Lets start by implementing a server… Note that this code is based on the various gRPC examples that can be found here. We create a new class called ReferenceManagementServer. In the generated code you will find a class with the name *Grpc that has a subclass *ImplBase for each of your services defined in the *.proto file. In our case it is PaperArchiveFacadeGrpc.PaperArchiveFacadeImplBase. We want to implement the method createPaperItem now. For that, we just add a static subclass that overrides this *.ImplBase and the corresponding method:

public class ReferenceManagementServer {

    static class PaperFacade extends PaperArchiveFacadeGrpc.PaperArchiveFacadeImplBase {
        @Override
        public void createPaperItem(ReferenceManagementAPI.createPaperItemParameter request, StreamObserver<ReferenceManagementAPI.PaperItemDTO> responseObserver) {
            // TODO implement method
        }
    }
}

As you can see in the generated *.proto file above, this RPC shall respond with a PaperItemDTO. The simplest implementation for this little tutorial just constructs such an object and returns it:

public class ReferenceManagementServer {

    static class PaperFacade extends PaperArchiveFacadeGrpc.PaperArchiveFacadeImplBase {
        @Override
        public void createPaperItem(ReferenceManagementAPI.createPaperItemParameter request, StreamObserver<ReferenceManagementAPI.PaperItemDTO> responseObserver) {
            PaperItemDTO paperItemDTO = PaperItemDTO.newBuilder()
                    .setTitle(request.getWhat())
                    .setAuthors(request.getWho())
                    .setVenue(request.getWhere())
                    .build();
            responseObserver.onNext(paperItemDTO);
            responseObserver.onCompleted();
        }
    }
}

As you can see, we fill the title, authors and venue attributes of the object with the values we receive from the request object.

In addition to that we need some code to start and stop the server (you can also find this in the gRPC samples); which leads us to the following exemplary server code:

public class ReferenceManagementServer {

    private static final Logger logger = Logger.getLogger(ReferenceManagementServer.class.getName());

    private Server server;

    private void start() throws IOException {
        /* The port on which the server should run */
        int port = 50051;
        server = ServerBuilder.forPort(port)
                .addService(new PaperFacade())
                .build()
                .start();
        logger.info("Server started, listening on " + port);
        Runtime.getRuntime().addShutdownHook(new Thread() {
            @Override
            public void run() {
                // Use stderr here since the logger may have been reset by its JVM shutdown hook.
                System.err.println("*** shutting down gRPC server since JVM is shutting down");
                try {
                    ReferenceManagementServer.this.stop();
                } catch (InterruptedException e) {
                    e.printStackTrace(System.err);
                }
                System.err.println("*** server shut down");
            }
        });
    }

    private void stop() throws InterruptedException {
        if (server != null) {
            server.shutdown().awaitTermination(30, TimeUnit.SECONDS);
        }
    }

    /**
     * Await termination on the main thread since the grpc library uses daemon threads.
     */
    private void blockUntilShutdown() throws InterruptedException {
        if (server != null) {
            server.awaitTermination();
        }
    }

    /**
     * Main launches the server from the command line.
     */
    public static void main(String[] args) throws IOException, InterruptedException {
        final ReferenceManagementServer server = new ReferenceManagementServer();
        server.start();
        server.blockUntilShutdown();
    }

    static class PaperFacade extends PaperArchiveFacadeGrpc.PaperArchiveFacadeImplBase {
        @Override
        public void createPaperItem(ReferenceManagementAPI.createPaperItemParameter request, StreamObserver<ReferenceManagementAPI.PaperItemDTO> responseObserver) {
            PaperItemDTO paperItemDTO = PaperItemDTO.newBuilder()
                    .setTitle(request.getWhat())
                    .setAuthors(request.getWho())
                    .setVenue(request.getWhere())
                    .build();
            responseObserver.onNext(paperItemDTO);
            responseObserver.onCompleted();
        }
    }
}

We can now simply start the server by running the main method:

Nov 26, 2020 12:28:29 PM io.mdsl.samples.grpc.reference_management.ReferenceManagementServer start
INFO: Server started, listening on 50051

To test if our server is working properly, lets also implement a client in the class ReferenceManagementClient. According to the gRPC examples, we first implement a method to send our createPaperItem request and print out the response we get:

public class ReferenceManagementClient {

    private static final Logger logger = Logger.getLogger(ReferenceManagementClient.class.getName());

    private final PaperArchiveFacadeGrpc.PaperArchiveFacadeBlockingStub blockingStub;

    public ReferenceManagementClient(Channel channel) {
        blockingStub = PaperArchiveFacadeGrpc.newBlockingStub(channel);
    }

    public void createPaperItem(String what, String who, String where) {
        logger.info("We will try to create a paper item with the values what='" + what + "', who='" + who + "', and where='" + where + "'.");
        createPaperItemParameter request = createPaperItemParameter.newBuilder()
                .setWhat(what)
                .setWho(who)
                .setWhere(where)
                .build();

        PaperItemDTO response;
        try {
            response = blockingStub.createPaperItem(request);
            logger.info("Received response: title=" + response.getTitle() + ", authors=" + response.getAuthors() + ", venue=" + response.getVenue());
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return;
        }
    }
}

As you can see, we take the input (what, who, where) as strings, construct the parameter object with them, and try to send it to the server using the generated class PaperArchiveFacadeGrpc.PaperArchiveFacadeBlockingStub. In this case we use the blocking stub, since we want to wait for the answer (blocking) and print it.

Now, we add a main method as well to call the method above:

public class ReferenceManagementClient {

    private static final Logger logger = Logger.getLogger(ReferenceManagementClient.class.getName());

    private final PaperArchiveFacadeGrpc.PaperArchiveFacadeBlockingStub blockingStub;

    public ReferenceManagementClient(Channel channel) {
        blockingStub = PaperArchiveFacadeGrpc.newBlockingStub(channel);
    }

    public void createPaperItem(String what, String who, String where) {
        logger.info("We will try to create a paper item with the values what='" + what + "', who='" + who + "', and where='" + where + "'.");
        createPaperItemParameter request = createPaperItemParameter.newBuilder()
                .setWhat(what)
                .setWho(who)
                .setWhere(where)
                .build();

        PaperItemDTO response;
        try {
            response = blockingStub.createPaperItem(request);
            logger.info("Received response: title=" + response.getTitle() + ", authors=" + response.getAuthors() + ", venue=" + response.getVenue());
        } catch (StatusRuntimeException e) {
            logger.log(Level.WARNING, "RPC failed: {0}", e.getStatus());
            return;
        }
    }

    public static void main(String[] args) throws Exception {
        String target = "localhost:50051";
        ManagedChannel channel = ManagedChannelBuilder.forTarget(target)
                .usePlaintext()
                .build();
        try {
            ReferenceManagementClient client = new ReferenceManagementClient(channel);
            client.createPaperItem("Domain-driven Service Design", "Stefan Kapferer und Olaf Zimmermann", "Ostschweizer Fachhochschule - OST");
        } finally {
            channel.shutdownNow().awaitTermination(5, TimeUnit.SECONDS);
        }
    }
}

That’s it. We can now run the servers and the clients main methods and get the following output on the clients terminal:

Nov 26, 2020 2:01:49 PM io.mdsl.samples.grpc.reference_management.ReferenceManagementClient createPaperItem
INFO: We will try to create a paper item with the values what='Domain-driven Service Design', who='Stefan Kapferer und Olaf Zimmermann', and where='Ostschweizer Fachhochschule - OST'.
Nov 26, 2020 2:01:49 PM io.mdsl.samples.grpc.reference_management.ReferenceManagementClient createPaperItem
INFO: Received response: title=Domain-driven Service Design, authors=Stefan Kapferer and Olaf Zimmermann, venue=Ostschweizer Fachhochschule

Seems to work. For more and other examples we refer to the gRPC examples repository.

You find the complete sources (incl. generated *.proto file, server and client code) of this example here.

Other Generators

Also checkout our other generators:

Site Navigation

Copyright: Stefan Kapferer and Olaf Zimmermann, 2020-2021. All rights reserved. See license information.