Thursday, 25 August, 2022 UTC


Summary

A Google remote procedure call (gRPC) is Google’s open source version of the remote procedure call (RPC) framework. It’s a communication protocol leveraging HTTP/2 and protocol buffer (protobuf) technologies. gRPC enables a remote client or server to communicate with another server by simply calling the receiving server’s function as if it were local. This makes communicating and transferring large data sets between client and server much easier in distributed systems.
Like other RPC systems, gRPC defines a service. It specifies its methods and return types using protobuf — a Google serialization and deserialization protocol — to enable the easy definition of services and auto-generation of client libraries. gRPC uses this protocol, currently on version 3, as its interface definition language and serialization toolset.
For most modern applications, gRPC is an excellent choice for its outstanding support of all data types. It’s best suited for heavy data, like streaming data, and could be overkill for simple applications where large data transfers are of little concern.
This article will demonstrate how to use gRPC via a client and server-like communication between two Node.js applications. We’ll also highlight some safety measures when using gRPC as the communication mechanism in your services.

Tutorial prerequisites

The tutorial requires you to have OpenSSL and Node.js (version 4.0 or later) installed on your PC. Having a basic understanding of Node.js and JavaScript is essential. You will also need to ensure that your working environment has administrative privileges.
Setting up the Node.js project
First, to set the application’s folder structure, create a folder called event-app-node-grpc and initialize a Node.js project using npm by typing the following commands: 
#bash
$ mkdir event-app-node-grpc
$ cd event-app-node-grpc
$ npm init -y
Having initiated your application, build out the following folder structure for the application. You can access the complete working code used in this tutorial on GitHub:
Event-app-node-grpc
client
app.js
index.js
server
index.js
scripts
generate-certs.sh
events.proto
README.md
Installing packages
From the terminal, navigate to the root directory of your application. Install the following packages using the npm install command, as shown in the following code snippet:
#bash
$ npm install express uuid grpc @grpc/proto-loader
Let’s go over the packages you just installed in the above code snippet:
  • Express is your application HTTP server.
  • Uuid is a valuable package for creating random hash numbers. We use it for event ID generation.
  • grpc is a gRPC library for Nodejs. It enables us to create a gRPC service in the Node.js runtime.
  • @grpc/proto-loader is a package needed to load protobuf files for use with gRPC. It uses the version 3 package of protobuf.js.
After installing the packages above, open the package.json file and add the following extra configurations to the scripts tags as shown in the code snippet below:
#package.json

"scripts": {
   "start": "node server/index.js",
   "generate:certs": "./scripts/generate-certs.sh"
 },
These additional configurations shown in the code snippet above are for the application’s runtime configuration and SSL certificate generation. When these configurations have been added, your updated package.json file will look similar to the code snippet below: 
#package.json(updated)

{
 "name": "event-app-node-grpc",
 "version": "1.0.0",
 "description": "A CRUD application to demonstrate the use of gRPC with NodeJS",
 "main": "server/index.js",
 "scripts": {
   "start": "node server/index.js",
   "generate:certs": "./scripts/generate-certs.sh"
 },
 "author": "(author name here)",
 "license": "ISC",
 "dependencies": {
   "@grpc/proto-loader": "^0.6.12",
   "express": "^4.18.1",
   "grpc": "^1.24.11",
   "uuid": "^8.3.2"
 }
}
The code snippet above shows the updated package.json file after the addition of the application runtime configuration and SSL certificate generation commands to the scripts tag.
Defining the protocol buffer
This tutorial illustrates how to use gRPC in a simple event tracking application. This demo application takes event details and saves them in an in-memory database while allowing the event data to be updated, fetched, and deleted.
In gRPC applications, the service interface and required payloads are in a protobuf file to enable communication between different applications. protobuf files have a .proto extension, as illustrated in our project setup schema.
Now, in the root directory of your application, create an events.proto file and add the following code to it. You can look at the project structure schema we defined earlier for reference.
#events.proto

syntax = "proto3";

service EventService {
   rpc GetAllEvents (Empty) returns (EventList) {}
   rpc GetEvent (EventId) returns (Event) {}
   rpc CreateEvent (Event) returns (Event) {}
   rpc UpdateEvent (Event) returns (Event) {}
   rpc DeleteEvent (EventId) returns (Empty) {}
}

message Empty {}

message Event {
   string id = 1;
   string name = 2;
   string description = 3;
   string location = 4;
   string duration = 5;
   int32 lucky_number = 6;
   string status = 7;
}

message EventList {
   repeated Event events = 1;
}

message EventId {
   string id = 1;
}
In the above proto definition code snippets, the following happened, we first specified the protocol buffer version using the syntax = "proto3" definition followed by the protocol service definition.
Next, in the protocol event service description, we created a service named EventService. We then created rpc functions within this service alongside their required parameters and expected return values. You can define as many services as your application needs, but, for simplicity, we only define one.
We also defined the data types for the rpc function in the EventService definition and the return values from the gRPC unique field numbering system. This describes the number of bytes used during encoding. You can see more details in protobuf’s official documentation.
Creating the gRPC server
Following our folder structure above, create a server folder in the root directory of your application, then create an index.js file in this server folder. Paste the following code snippet into your newly created server/index.js file:
#server/index.js

const PROTO_PATH = "./events.proto";

let grpc = require("grpc");
let protoLoader = require("@grpc/proto-loader");

let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
   keepCase: true,
   longs: String,
   enums: String,
   arrays: true
});

let eventsProto = grpc.loadPackageDefinition(packageDefinition);
In the above code snippet, we imported the events.proto file we previously defined as the variable PROTO_PATH, and loaded it using the protoLoader library loadSync method. We then saved the proto definitions in the eventsProto variable, which stores all the proto definitions.
Next, add the following code snippet right after the eventsProto variable in the server/index.js file defined earlier. 
const { v4: uuidv4 } = require("uuid");

const events = [
   {
       id: "34415c7c-f82d-4e44-88ca-ae2a1aaa92b7",
       name: "Birthday Party",
       description: "27th Birthday in Paris",
       location: "Paris France",
       duration: "All Day",
       lucky_number: 27,
       status: "Pending"
   },
];

const server = new grpc.Server();
In the above code snippet, we required the uuid package, which is used for generating random unique strings for our event IDs. Since we are using an in-memory database for this tutorial, we’ll define it as an array to store our list of events, and then set up our server instance by calling a new grpc.Server method. 
Next, we will register the application services. To do this, add the following code snippet. Place it right after the server variable in the code snippet above:
server.addService(eventsProto.EventService.service, {

   getAllEvents: (_, callback) => {
       callback(null, { events });
   },

   getEvent: (call, callback) => {
       let event = events.find(n => n.id == call.request.id);

       if (event) {
           callback(null, event);
       } else {
           callback({
               code: grpc.status.NOT_FOUND,
               details: "Event Not found"
           });
       }
   },

   createEvent: (call, callback) => {
       let event = call.request;

       event.id = uuidv4();
       events.push(event);
       callback(null, event);
   },

   updateEvent: (call, callback) => {
       let existingEvent = events.find(n => n.id == call.request.id);

       if (existingEvent) {
           existingEvent.name = call.request.name;
           existingEvent.description = call.request.description;
           existingEvent.location = call.request.location;
           existingEvent.duration = call.request.duration;
           existingEvent.lucky_number = call.request.lucky_number;
           existingEvent.status = call.request.status;
           callback(null, existingEvent);
       } else {
           callback({
               code: grpc.status.NOT_FOUND,
               details: "Event Not found"
           });
       }
   },

   deleteEvent: (call, callback) => {
       let existingEventIndex = events.findIndex(
           n => n.id == call.request.id
       );

       if (existingEventIndex != -1) {
           events.splice(existingEventIndex, 1);
           callback(null, {});
       } else {
           callback({
               code: grpc.status.NOT_FOUND,
               details: "Event Not found"
           });
       }
   }
});
In the code snippet above, we called the addService method on the gRPC server instance to register the application services, which was basically a create, read, and update operation on the events.
To allow the application server to start, paste the following code snippet right after the addService method from the above code snippet.
server.bind("127.0.0.1:50051", grpc.ServerCredentials.createInsecure());
console.log("Server listening at http://127.0.0.1:50051");
server.start();
Creating the gRPC client
Following our folder structure above, create a client folder in the root directory of your application, then create two files in the client folder you just created: an index.js file and an app.js file. Paste the following code snippet into the client/app.js file.
#client/app.js

const PROTO_PATH = "../events.proto";
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader");

let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
   keepCase: true,
   longs: String,
   enums: String,
   arrays: true
});

const EventService = grpc.loadPackageDefinition(packageDefinition).EventService;
const client = new EventService("127.0.0.1:50051", grpc.credentials.createInsecure());
module.exports = client;
In the code snippet above, we imported the proto definitions we made earlier, loaded it with protoLoader, hooked up the grpc client to the server application’s IP address, and exported the event service using the variable name client. We also attached an SSL certificate to the client for authorization and encryption of client-server communications.
Then, in the client/index.js file, paste the following code snippet:
#client/index.js

const client = require("./app");

const express = require("express");
const app = express();
app.use(express.json());
app.use(express.urlencoded());

app.get("/", (req, res) => {
   client.getAllEvents(null, (err, data) => {
       if (!err) {
           res.status(200).send({
               data
           });
       }
   });
});

app.post("/createEvent", (req, res) => {

   let newEvent = {
       name: req.body.name,
       description: req.body.age,
       location: req.body.address,
       duration: req.body.address,
       lucky_number: req.body.address,
       status: req.body.status
   };

   client.insert(newEvent, (err, data) => {
       if (err) throw err;
       res.status(200).send({
           data,
           message: 'Event created successfully'
       });
   });
});

app.post("/updateEvent", (req, res) => {
   let updateEvent = {
       name: req.body.name,
       description: req.body.age,
       location: req.body.address,
       duration: req.body.address,
       lucky_number: req.body.address,
       status: req.body.status
   };

   client.update(updateEvent, (err, data) => {
       if (err) throw err;

       res.status(200).send({
           data,
           message: 'Event updated successfully'
       });
   });
});

app.delete("/deleteEvent", (req, res) => {
   client.remove({ id: req.body.eventId }, (err, _) => {
       if (err) throw err;

       res.status(200).send({
           message: 'Event deleted successfully'
       });
   });
});

const PORT = process.env.PORT || 50050;
app.listen(PORT, () => {
   console.log("Client Server listening to port %d", PORT);
});
Taking a look at the code snippet above, we imported event-service from the client/app.js file. Then we configured an express server with simple endpoints to manage the creation, update, fetch, and delete events by remotely calling the server application using gRPC techniques.
Test the server and client applications
At this point, we can test our work to make sure we’re on track.

The server

Navigate to the root directory of the project using the terminal and then run the following commands:
$bash
$ npm run start
The server application should be live on http://localhost:50051:
% npm run start
> [email protected] start
> node server/index.js

Server listening at https://127.0.0.1:50051

The client

Open a new terminal window, navigate to the client folder from the root directory of your application, and then run the following commands: 
$bash
$ node index
The application should be live on http://localhost:50050:
$ event-app-node-grpc  cd client
$ event-app-node-grpc/client node index.js

Client Server listening to port 50050
To test, navigate to localhost:50050 on your browser or use an API testing tool like Postman. You should see the default event we initially added to our events array. Your response should be the same as the screenshot below:
Authenticating and securing the gRPC API
The gRPC protocol supports various authentication mechanisms, making it easy to adapt to new and existing systems. We can implement authentication in gRPC client-server communications using recommended mechanisms like SSL and TLS with or without Google token-based authentication. We can also build custom authentication by merely extending the built-in authentication function in gRPC.
By default, gRPC comes bundled with the following authentication mechanisms:
  • SSL and TLS to authenticate the server and encrypt the data exchanged between the client and the server
  • ALTS (a mutual transport and authentication protocol engineered by Google) to secure RPC communications for applications running on the Google Cloud Platform (GCP)
  • Generic token-based authentication mechanism to attach metadata-based credentials to requests and responses
We’ll implement authentication using SSL for this tutorial, as earlier highlighted in the tutorial introduction, then we will modify our code in the client/app.js and server/index.js files to accommodate this new development.

Generate an SSL certificate with OpenSSL

First, let’s generate an SSL certificate using OpenSSL. This process will require that you have OpenSSL. It also requires that you have permission to execute bash scripts. These are essential inorder to avoid permission errors.
In our folder structure, create a scripts folder and then create a file called generate-certs.sh in it. Paste the following code snippet into that file:
#scripts/generate-certs.sh

echo "Creating certs folder ..."
mkdir certs && cd certs

echo "Generating certificates ..."

openssl genrsa -passout pass:1111 -des3 -out ca.key 4096

openssl req -passin pass:1111 -new -x509 -days 365 -key ca.key -out ca.crt -subj  "/C=CL/ST=RM/L=Santiago/O=Test/OU=Test/CN=localhost"

openssl genrsa -passout pass:1111 -des3 -out server.key 4096

openssl req -passin pass:1111 -new -key server.key -out server.csr -subj  "/C=CL/ST=RM/L=Santiago/O=Test/OU=Server/CN=localhost"

openssl x509 -req -passin pass:1111 -days 365 -in server.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out server.crt

openssl rsa -passin pass:1111 -in server.key -out server.key

openssl genrsa -passout pass:1111 -des3 -out client.key 4096

openssl req -passin pass:1111 -new -key client.key -out client.csr -subj  "/C=CL/ST=RM/L=Santiago/O=Test/OU=Client/CN=localhost"

openssl x509 -passin pass:1111 -req -days 365 -in client.csr -CA ca.crt -CAkey ca.key -set_serial 01 -out client.crt

openssl rsa -passin pass:1111 -in client.key -out client.key
The above code generates the SSL certificates needed to establish a secured and encrypted connection between the server and client application. When it executes, it creates a certs folder, generates the SSL certificates for the server and client using OpenSSL, and then saves them in the certs folder. To learn more about these configurations and what they do, check out the OpenSSL website.

Generate an application SSL certificate with npm

Now use the script to generate an SSL certificate for your application by running the following commands in the terminal in the application’s root directory.
$bash
$ npm run generate:certs
This creates a certs folder that contains the generated SSL certificates.
Note that some admin privilege is required. If you encounter a permissions error when trying to run the script, use the following commands to give the script the execute privilege, and then try again.
$bash
$ cd scripts
$ chmod u+r+x generate-certs.sh
$ ./generate-certs.sh
You should have the following output on your terminal:
> [email protected] generate:certs
> ./scripts/generate-certs.sh

Creating certs folder…
Generating certificates…
Generating RSA private key, 4096 bit long modulus
………………………………………………..++
……………..++

e is 65537 (0x10001)
Generating RSA private key, 4096 bit long modulus
…………………++
………………………………………….….….….….….………..…………..++
e is 65537 (0x10001)
Signature ok
subject=/C=CL/ST=RM/L=Santiago/0=Test/OU=Server/CN=localhost
Getting CA Private Key
writing RSA key
Generating RSA private key, 4096 bit long modulus
………………………………………………..….….….…………..++
.…………..++
e is 65537 (0x10001)
Signature ok
subject=/C=CL/ST=RM/L=Santiago/0=Test/OU=Client/ON=localhost
Getting CA Private Key
writing RSA key
Updating the client/app.js and server/index.js files
So far we have generated the SSL certificates needed to authenticate our gRPC APIs. Now we’ll proceed to modify the client/index.js and server/index.js files in order to work with these generated certificates.
In the updated client/app.js file shown below, we introduced the fs module to help us read the generated certificates, and we then used those certificates to create gRPC SSL credentials. Finally, we applied those credentials to the gRPC service.
#client/app.js(updated)

const PROTO_PATH = "../events.proto";
const fs = require('fs');
const grpc = require("grpc");
const protoLoader = require("@grpc/proto-loader");

let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
   keepCase: true,
   longs: String,
   enums: String,
   arrays: true
});

const credentials = grpc.credentials.createSsl(
   fs.readFileSync('../certs/ca.crt'),
   fs.readFileSync('../certs/client.key'),
   fs.readFileSync('../certs/client.crt')
);

const EventService = grpc.loadPackageDefinition(packageDefinition).EventService;
const client = new EventService("localhost:50051",credentials);
module.exports = client;
We also introduced the fs module In the updated server/index.js file shown below to help us read the generated certificates. We then used those certificates to create gRPC SSL credentials and applied those credentials to the server.
#server/index.js(updated)

const PROTO_PATH = "./events.proto";
const fs = require('fs');

let grpc = require("grpc");
let protoLoader = require("@grpc/proto-loader");

let packageDefinition = protoLoader.loadSync(PROTO_PATH, {
   keepCase: true,
   longs: String,
   enums: String,
   arrays: true
});

let eventsProto = grpc.loadPackageDefinition(packageDefinition);

const server = new grpc.Server();

let credentials = grpc.ServerCredentials.createSsl(
   fs.readFileSync('./certs/ca.crt'), [{
   cert_chain: fs.readFileSync('./certs/server.crt'),
   private_key: fs.readFileSync('./certs/server.key')
}], true);

----------

----------

server.bind("0.0.0.0:50051", credentials);
console.log("Server listening at http://0.0.0.0:50051");
server.start();
Running the server and client applications
We’ve successfully implemented an event management solution using gRPC specifications. To test the endpoints, start the application from the terminal by working through the steps below.

The server

Navigate to the root directory of the project using the terminal and run the following commands:
$bash
$ npm run start
After running that, the server application should be live on http://0.0.0.0:50051:
event-app-node-grpc % npm run start

> [email protected] start
> node server/index.js

Server listening at http://0.0.0.0:50051

The client

Open a new terminal window, navigate to the client folder from the root directory of your application, and then run the following commands: 
$bash
$ node index
After running that, the client application should be live on http://localhost:50050:
> event-app-node-grpc % cd client
> client % node index
Client Server listening to port 50050
To test the application, navigate to localhost:50050 on your browser or use an API testing tool like Postman. You should see the default event we initially added to our events array. Your response should be the same as the screenshot below:
You can proceed to test other endpoints added to the application to ensure everything works as expected.
You built a secure API with gRPC!
In this tutorial, we built a simple API in gRPC using Node.js, noting its operating concepts alongside its numerous advantages, like HTTP/2 and SSL/TLS for end-to-end authentication and encryption to improve API security. 
Despite these advantages, gRPC also has weaknesses, ranging from limited browser support, its non-human readable data format, steep learning curve, and poor edge caching support. But regardless of these limitations, gRPC is the best option for communication between internal microservices, thanks to its unmatched performance and multilingual nature. The gRPC protocol is impressive, and has gained a significant level of adoption in the industry since its initial release in August 2016. It’s bound to continue to grow. 
There are many other things you can do with gRPC. The example in this tutorial is just the tip of the iceberg of what gRPC offers. You can review the documentation to expand your domain knowledge on gRPC to enhance your application communication processes and strategies for maintaining gRPC security.

More API Resources:

  • Building a secure GraphQL API with Node.js
  • Containerizing Go applications with Docker