gRPC in the real world: The Kubernetes Container Runtime Interface

A .proto file is a description of a gRPC API written in the Protocol Buffers language specification . Protocol Buffers is a binary format. Protocol Buffers is a binary format. It is NOT a self-describing language. Thus, there needs to be a common "dictionary" used by both client and server to encode and decode text and numbers into the Protocol Buffers binary format. This common dictionary is the .proto file. (See the figure below)

The following is an example of a simple .proto file named coolcar.proto This .proto file defines a single data structure, car along with a service, CoolCarService that publishes a procedure GetCar(). The procedure retrieves a car by vehicle identification number (VIN).

syntax = "proto3";
package coolcars;
message Car {
‎ ‎ ‎ ‎ string make = 1;
‎ ‎ ‎ string mode = 2;
‎ ‎ ‎ int32 year = 3;
‎ ‎ ‎ string VIN = 4;
}
message CarRequest {
‎ ‎ ‎ string VIN = 1;
}
service CoolCarService {
‎ ‎ ‎ rpc GetCar (CarRequest) returns (Car) {}
}

And, if that were not enough, if you want to use the API, you have to write a gRPC client that supports accessing the server via HTTP/2 (the newest version of the Web's foundational protocol that very few people know how to work with yet). That client needs to be able to do the Protocol Buffers encoding and decoding as specified in the same .proto file used by the server.

It's a lot of work. Fortunately, it's work you don't have to do. The Cloud Native Computing Foundation (CNCF) , the organization that oversees gRPC development (of which Google is a platinum member), publishes a number of projects that do most of the work required to create a fully functional gRPC server and client. These projects provide a framework that does all the programming required to create an HTTP/2 server and execute the routing to each procedure defined in the API. Also, the projects do the work of encoding and decoding data into and from the Protocol Buffers binary format. In most cases, all the programmer needs to do is supply the internal logic for each procedure defined in the .proto file. The framework takes care of the rest.

This is not to say using a project is a walk in the park. Each of the implementations has its own way of working with gRPC. For example, the .NET implementation publishes a number of classes in a special namespace named Gprc that cover all aspects of gRPC operations. Each of these classes have a lot of details that need to be understood. The same is true of the Java implementation where these classes are in io.grpc.* packages . Again, lots of classes, lots of details. These are but only two of the thirteen implementations of gRPC listed on grpc.io. Go, which is not an object-oriented language, has its own approach to implementing a gRPC API, as does Python. So, no matter which road you plan to take, you can expect to experience a significant learning curve absorbing the ins and outs of the given implementation.

For the purposes of this article, we at ProgrammableWeb have decided to focus on a detailed examination of the Node.js implementation of gRPC. The approach we've taken is to define a demonstration gRPC API in the required .proto file and then implement the API using the Node.js libraries published here .

The name of the demonstration gRPC API is SimpleService. You can find the source code for SimpleService in the ProgrammableWeb GitHub repository here .

The SimpleService project that accompanies this installment ships with a gRPC server and a CLI tool that allows users to directly interact with the API running in the gRPC server. Figure 1 below shows an animation of the CLI tool calling the procedure named Blabber() on the associated gRPC server. Blabber() is implemented using a bidirectional stream.

Before we go into the details about how SimpleService implements each of its remote procedures, let's take a look at the work required to get the gRPC server up and running. The Node.js libraries used to implement SimpleService offer two ways to implement the API. One way is called static. The other is called dynamic.

The static implementation uses JavaScript classes that are created using a special version of the protoc code auto-generation tool. This tool is called grpc_tools_node_protoc. A developer will run grpc_tools_node_protoc against the API's .proto file to create JavaScript objects that correspond to the messages and procedures defined in the .proto file. The developer then programs the server and client using these classes. To learn more about how auto-generation works, you can read the associated article on ProgrammableWeb.

ProgrammableWeb publishes an interactive tutorial that demonstrates how to use the protoc tool to auto-generate gRPC code for use on Node.js.

The dynamic implementation, on the other hand, uses the Node.js Protobuf loader library (@grpc/proto-loader ) to load the .proto file into memory at runtime and to generate the JavaScript message and procedure objects behind the scenes. No manual code auto-generation is required. The library does it all.

The SimpleService demonstration gRPC API implements dynamic implementation. In fact, you'll see the mechanics of dynamic loading in the examination of the Node.js implementation of the gRPC server that we'll cover next.

Listing 1, below shows the file, server.js. The file contains the Node.js-based logic that initializes and starts the SimpleService gRPC server.

The server initialization code is pretty straightforward. Lines 1 and 2, declare the required gRPC packages that will do the heavy lifting behind the scenes. These packages contain the HTTP/2 server intelligence as well as the routing, serialization and deserialization mechanisms that will be used under the covers to deliver and process data coming from a calling client.

Line 5 defines the location and name of the Protocol Buffers .proto file that the packages use for routing, serialization and deserialization purposes. (Remember, the .proto file is the authoritative definition of the API. The .proto file provides the definitions of the Protocol Buffer messages and remote procedures.)

Line 6 defines the port on which the server will listen for incoming traffic. This port can be custom-defined using the operating system's environment variable PORT. If PORT is not defined in an environment variable, the default port 8080 will be used.

Line 8 uses the protoLoader object provided by the Protobuf Loader library. The protoLoader, as described above, is the library that does the work of converting the messages and procedures defined by the API's .proto file into useful JavasScript

The packageDefinition is used to create the custom-named simple_proto object at Line 16. The simple_proto object is the representation of the gRPC service defined in the .proto file.

Lines 22 - 68 are functions that correspond to the API procedures defined in the .proto file. These functions contain the actual business logic behind each procedure.

Line 71 in Listing 1 above is where the gRPC is created in the function main(). Notice that Line 72 creates an object named implementations. The properties add, subtract, multiply, divide, chatter, blabber, and ping are added to implementations at Lines 73 -79. Also, notice that each property is assigned the name of one of the functions defined previously in Lines 22 - 68. For example, implementations.add is assigned the function add() at Line 73.

The assignment of a function to a property of the object implementations is how route-to-procedure binding is implemented using the grpc package.

A new gRPC server object is created at Line 81. Then at line 82 a service is added to the server using the method server.addService(). Notice that the simple_proto object that represents the definitions in the .proto file and the implementations object is passed to server.addService() as parameters. The internals of server.addService() will do all the work of routing client calls to the relevant remote procedure. Also, server.addService() handles all the work of passing data to and from the given remote procedure. This includes not only data passed request/response calls, but also data passed in unidirectional and bidirectional streams.

As you can see, this is a lot of work. Imagine having to do all of it from scratch. As said, previously, it's a herculean task. Fortunately, the libraries do all the work for you. All that the programmer needs to do is provide the business logic for the functions that are defined in the .proto file and routed by the grpc package accordingly. So, let's take a look at how the implementation of the remote procedures is facilitated using the Node.js grpc package. Let's start off by examining request/response communications.

The demonstration API has four math functions and a ping function that support request/response communication between the client and server. Listing 2 below shows an excerpt from the API .proto file in which these functions are defined according to the Protocol Buffers specification.

rpc Add (Request) returns (Response) { }
rpc Subtract (Request) returns (Response) { }
rpc Multiply (Request) returns (Response) { }
rpc Divide (Request) returns (Response) { }
rpc Ping (PingRequest) returns (PingResponse) { }

Listing 2: The demonstration API procedures that communicate using request/response

We're going to take a close look at the implementation of the Add() procedure because its structure is representative of the way any request/response call is implemented using the Node.js grpc package.

Listing 3 below shows the Node.js code for the procedure named add().

The add() function is bound to the Add() procedure defined in the .proto file is using the following statement as shown above in Listing 1, Line 73:

implementations.ping = ping;

Be advised, this binding technique is special to the Node.js gRPC package. Libraries and frameworks in other languages such as Java and C# will have a different way of binding procedures defined in a .proto file to actual implementation code.

As you can see above in Listing 3, Line 1, the function add() has two parameters, call and callback. The internals of the Node.js grpc package provides values for these parameters each time the add() procedure is called by a client.

The call parameter represents an object that is the logical call made by the client to the function. The call object has the property, request, that represents the actual HTTP/2 request made by the calling client. The callback parameter represents a callback function that's provided by the Node.js grpc package at runtime. The callback function is a wrapper for the HTTP/2 response that the Node.js grpc code sends back to the calling client.

The callback function has two implicit parameters err and data structured like so callback(err, data). Should the add() function throw an error, the error message or object that represents the error will be passed as the err parameter. If all goes well, the result of the function will be passed as the data parameter. These err and data parameters along with their ordering are standard Node.js convention for callback functions.

Let's move onto examining the implementation logic of the add() function. The following statement is from Listing 3 Line 2 above. The statement assigns the array of numbers passed by the calling client to a local variable named input.

const input = call.request.numbers;

The code above assigns the value of the property call.request.numbers to the variable input. But, where does call.request.numbers come from? Let's look at the way the Add() procedure is defined in the .proto file:

rpc Add (Request) returns (Response) { }

Notice that the Add() procedure takes the custom Request message as a parameter and that the custom Request message is defined like so in the .proto file:

message Request {
repeated double numbers = 1;
}

Notice in the Protocol Buffers message above that the custom Request message defines a field named numbers. This Node.js grpc package translates this numbers field onto the call object passed to the add() procedure like so:

call.request.numbers;

Thus, call.request.numbers is the array of numeric values that the add() function needs to process.

The SimpleService gRPC API defines messages named Request and Response. These messages are custom to SimpleService. However, even though these message names represent custom messages in SimpleService, they might be confused with the standard HTTP communication terms Request and Response.

The SimpleService gRPC messages Request and Response have no intrinsic relation to an HTTP Request or an HTTP Response. That they share the same name is arbitrary. Thus, the term, request in the term

call.request.numbers;

represents an HTTP Request. It does not represent the custom message, Request.

Adding up the numbers is a straightforward undertaking. The statement shown below is from Listing 3 Line 3:

const result = input.reduce((a, b) => a + b, 0);

The code uses the reduce() function, which is a method implicit to any JavaScript array to traverse the elements in the array of numbers passed to the method and sum them up. Then, the result is passed onto the callback function like so:

callback(null, {result});

Notice that the result of input.reduce() is returned as a JavaScript object like so {result}. The Node.js library for grpc expects results to be passed back as a JSON object.

The implementation of the Ping() function works in a manner nearly identical to Add(). The purpose of Ping() is to echo back the data sent to it. Thus, it simply recycles the string sent by the caller like so:

callback(null, {result: call.request.data} );

Again, where does the value data come from? Take a look at the way Ping() procedure is defined in the .proto file:

rpc Ping (PingRequest) returns (PingResponse) { }

And then take a look at how the .proto file defines the PingRequest message.

message PingRequest {
string data = 1;
}

As you can see, data is the name of the single field in the PingRequest message. Thus, it gets represented by the Node.js grpc package as call.request.data.

Now that we've covered how request/response communication is implemented in the demonstration gRPC API, let's move onto streaming.

As described earlier, gRPC is designed to support unidirectional and bidirectional streams. The SimpleService API provides examples of each type. Unidirectional streaming is implemented in the Chatter() procedure. Bidirectional streaming is implemented in the Blabber() procedure.

The intent of the Chatter() procedure is to return a string defined by the chatItem field back in a stream from the server to client continuously. The number of times the chatItem value will be returned is indicated by the value defined in the limit field. Both chatItem and limit are defined in the message ChatterRequest as shown below:

message ChatterRequest {
string chatItem = 1;
int32 limit = 2;
}

The Chatter() procedure defined in the .proto file as follows...

rpc Chatter (ChatterRequest) returns (stream ChatterResponse) {}

… is implemented in the JavaScript chatter() function shown below in Listing 4.

The format defined by the Node.js grpc package for handling a stream is different from the request/response format shown in the add() and ping() functions shown above. Whereas a function supporting a gRPC request/response interaction uses a callback function to send data back in a response, sending data over a stream from server to client involves using the write method that is part of the call object. As with the add() and ping() functions shown earlier, the call method is provided by the Node.js grpc package to the chatter() function as a runtime parameter.

The way that chatter works is that call.write is invoked in a JavaScript for loop, as shown above in Listing 4 at Line 2. The number of iterations through the for loop is defined by the value assigned to call.request.limit.

WHERE

• call is the call object provided at runtime as a parameter by the Node.js grpc package
• request an object that represents the client request
• limit represents the limit fields specified in the ChatRequest message defined in the .proto file

Each iteration through the for loop will send a message back to the client in a stream. The stream is established by the internals of the Node.js grpc package. The format of the information sent back to the client in the stream conforms to the structure of the specified return message ChatterResponse like so:

message ChatterResponse {
string chatItem = 1;
int32 index = 2;
}

The message will be passed to call.write() in JSON format. The internals of the Node.js grpc package will serialize the JSON into the Protocol Buffers binary that gets sent in the stream. An example of JSON that gets sent to call.write() is as follows:

{chatItem: "hi there", index: 3}

In the case shown above the SimpleService server will send the string "hi there" back to the client 3 times in a stream.

When the for loop is finished, the call.end() method is invoked at Line 6 of Listing 4 and shuts down the stream.

The server-side code for supporting a bidirectional stream is similar to the code for supporting a unidirectional stream from client to server. The difference is that the server-side code has a call.on() event handler that captures messages coming in from the client stream.

Listing 5 below shows the blabber() function which is an excerpt of the server.js code listed above.

Notice that there is a JavaScript event handler, call.on() that captures the data event at Line 3 of Listing 5. The data event reflects a message coming in from the calling client to the server over an HTTP/2 stream established by the client. The call.write() method at Line 4 in Listing 5 returns a JSON object to the client. The JSON object has two properties, blab and index. The string attached to the blab field of the gRPC message is converted to uppercase and is assigned to the blab property of the JSON object. The index property is assigned the value of variable i. The value of i is a continuously incrementing value. Each time call.write() is invoked, the value of i increases by one, as shown at Line 5 in Listing 5 above.

The blabber method is intended to run until the HTTP/2 stream is broken. Thus, there is NO invocation of call.end() as is done for the unidirectionally streaming method chatter.

Now that we've covered the basics of creating a gRPC server under Node.js, let's move onto the client.

In order to work with the server we created above, we need a client that can directly access the .proto file that defines the SimpleService gRPC API. Also, the client needs to be able to access the gRPC server over HTTP/2. We could use a third-party tool such as BloomRPC to act as the client. (See Figure 2, below.)

For instructional purposes, we've taken the time to implement a gRPC client designed specifically for the SimpleService API. The client is an executable that you invoke from the command line. It's named sscli. You can view the complete code for it here .

The sscli command-line client for the SimpleService API is part of the ProgrammableWeb tutorial published on the Katacoda interactive learning environment.

Once installed, you can use sscli to interact with the SimpleService gRPC API. (Figure 1, at the start of this article, is an animated GIF that illustrates using the sscli command-line tool to invoke the blabber procedure under the SimpleService gRPC API.)

Let's take a look at the details.

The sscli command-line client for the SimpleService API is part of the ProgrammableWeb tutorial published on the Katacoda interactive learning environment.

Once installed, you can use sscli to interact with the SimpleService gRPC API. (Figure 1, at the start of this article, is an animated GIF that illustrates using the sscli command-line tool to invoke the blabber procedure under the SimpleService gRPC API.)

Let's take a look at the details.

The logic for the custom gRPC client is encapsulated into the JavaScript class SimpleServiceClient as shown below in Listing 6. The SimpleServiceClient class publishes public methods that correspond to the procedures published in the SimpleService API.

The sscli command line tool uses the class to execute client behavior against the gRPC server.

For example, when the sscli tool wants to call the Add() procedure on the SimpleService, it will call SimpleServiceClient.add() as shown below in Listing 6:

const SERVER_URL = 'localhost:8080';
const PROTO_PATH = __dirname + '/proto/simple.proto';
const client = new SimpleServiceClient(SERVER_URL, PROTO_PATH);
const numbers = [1,1,1,3];
client.add(numbers, callback); //callback is the name of a function defined previously

Listing 6: An example of creating and using the JavaScript class, SimpleServiceClient

To call the Divide procedure, sscli calls the class SimpleServiceClient.divide(), to call the Multiply procedure sscli calls SimpleServiceClient.multiply() and so on. We're not going to go into the particulars of each client call. Instead, we'll look at how the class SimpleServiceClient calls the Blabber() procedure on the SimpleService API. We choose Blabber because it's a good example of how to implement bidirectional streaming between a gRPC server to a gRPC client. We'll get to the particulars of Blabber() in a moment, but first let's look at how the actual client that's part of the Node.js grpc package is created in the class SimpleServiceClient.


SimpleServiceClient publishes two parameters in the class's constructor. One parameter is serverUrl which is the URL of the gRPC server on the network. The other parameter, protoPath, is the exact path to the gRPC API .proto file either on disk or on the network. (Line 2, Listing 7 below.) The parameter protoPath can point to the location of a copy of a .proto file used by the server, but the content of both files must be identical. (There are situations in which the .proto files can differ in content, but describing such details is beyond the scope of this article.) In the case of the demonstration project, a custom-created shell script copies a .proto file from a single location into the directories of both the server and client where the code auto-generation will take place.

The constructor of SimpleServiceClient uses the protoLoader object to create the underlying messages and procedures that will be used by the client. This is done at Lines 3 -10 in Listing 6, below. (Be advised that Listing 7 show excerpts from the SimpleServiceClient class, not its entirety.)

The gRPC client is created at Line15 of Listing 7.

As mentioned above, SimpleServiceClient is the JavaScript class that allows the sscli command-line tool to interact with an underlying gRPC client via the class's methods. Also, as mentioned above, the method we're going to examine is Blabber() because it's a good example of implementing a client call that's bound to a bidirectional stream between a client and server.

The code for the method SimpleServiceClient.blabber({message, count}, callback) is displayed at Lines 23 - 58 in Listing 7 above.

The purpose of SimpleServiceClient.blabber({message, count}, callback) is to call the blabber() procedure on the SimpleService gRPC API server. When the client makes the call to blabber(), the server will return the string submitted in the message property of the first parameter's JSON object repeatedly. The number of times the string will be repeated is indicated by the count property of the JSON object.

For example, the client call...

SimpleServiceClient.blabber({"I like cheese", 5}, callback)

… will send the string, "I like cheese" five times to the server over HTTP/2 stream. The server will return each string sent from the client back to the client in an HTTP/2 stream.

The important thing to understand is that unlike the math and ping methods defined in SimpleServiceClient which are standard request/response methods, the Blabber() method is bidirectional. This means that a client streams to the server and the server likewise streams back to the client. It's as if there is a continuous connection between them. While this is just a very simple sample application for demonstration purposes, it is up to IT architects and developers to understand when such bidirectional streaming is a business requirement for a particular application.

The work of creating the bidirectional stream to the client is done at Line 28 in Listing 7 above:

const stream = this.client.Blabber();

Once the stream is established, the SimpleServiceClient.blabber() function bound to the stream's 'data' event like so:

stream.on('data', function (response) {
callback(null,JSON.stringify(response));
});

The 'data' event indicates that data is coming in from the server. The actual data is represented in the response parameter passed to the function defined in the stream.on() event handler. The event handler function in turn invokes the callback function which is passed as a parameter to SimpleServiceClient.blabber({message, count}, callback). This takes care of data coming in from the stream on the server-side.

Sending data to the server from the client requires some programming ingenuity. This is done at Lines 41 - 57 in Listing 7 above and which we replicated below in Listing 8:

Let's focus on the function interval() shown above at Line 48 of Listing 8.

The function interval() is provided by a third-party NPM package named interval-promise . The purpose of interval-promise is to execute programming statements repeatedly as a Node.js promise according to a predefined interval. The predefined interval is provided by the programmer as a parameter to the function invocation.

In the code you can see that interval() will execute the function blab() at line 49 Then at Line 50, it will check to see that the count parameter passed to the class' blabber() function is not null and that the current value of the iteration of the interval call is greater than the count parameter. (The value of iteration is injected automatically into the interval() function at runtime. If count exists and the iteration is greater than the value of the count value, the client stream is canceled, effectively bringing SimpleServiceClient.blabber() to a halt. (Line 55) Otherwise, interval() keeps executing. In this case interval() is configured to execute every 1000 milliseconds as part of the interval() function's declaration at Line 58.

The blab() function at Line 41 is nothing more than a wrapper to the method stream.write() which in turn sends a JSON object to the server over the pre-established HTTP/2 stream. This JSON object contains a value for the message property of the object. The stream.write() index property is set to the current value of the variable cnt at line Line 43. The variable cnt is an internal incremental counter with the blabber() function,

The result is a JSON object sent out on the client stream that might look like this:

{message: "I like cheese", index: 4}

Essentially, the client is sending data to the server over HTTP/2 and the server is responding back with data via HTTP/2. This is the basic behavior of an gRPC bidirectional streaming exchange.

There's a lot of benefit to using libraries and frameworks to implement gRPC, both from the server-side and client-side. For many companies, the time-saving benefit is justification enough. Yet, using a pre-existing framework or a set of third-party libraries such as the gRPC for Node.js libraries we demonstrated in this article still requires work. There's a lot to master.

As we mentioned at the beginning of this piece, each language-specific gRPC framework has its own way of implementing the gRPC specification. There are a lot of nuanced details that need to be understood. The time to adopt a framework will vary depending on the quality of the documentation available as well as the breadth and quality of support you'll experience among the variety of developer communities that have sprouted up in the gRPC space.

Getting the SimpleService demonstration project up and running took a good deal of time, much more than an afternoon's worth of research and coding. Some of the coding was done following the tutorials published on grpc.io . Other efforts were a matter of trial and error. Sometimes, we needed expert help from a live human being. Fortunately, we were able to get guidance from experts on the gRPC Gitter channel . Michael Lumish was particularly helpful when it came to providing insights and guidance to help better understand the intricacies of the Node.js implementation. His patience is admirable.

As you can see, just from the sheer volume of information we've provided in this article, to head-out on your own and build a comprehensive gRPC implementation from scratch would be an enormous undertaking. There's a good case to be made that it's beyond the capability of a single developer, no matter how extraordinary the technical acumen. The work might be fun, but you'd be hard-pressed to justify the benefit of taking a roll-your-own approach given the enormous expense. It's better to identify an implementation of gRPC that meets the needs at hand and then bite the bullet to absorb the significant learning curve. It might take time, but nowhere near the amount of time you'd have to invest to start from scratch.

gRPC is an important technology. Adoption continues to grow. Companies are accepting the complexity that goes into making the technology a useful, productive addition to their tech stacks. Hopefully, the information provided in this article will provide a direction as well as an inspiration for adopting a gRPC implementation that works for you.