Thursday, 11 October, 2018 UTC
Summary
Today’s post is written by Thomas Ladd of StackPath as part of our JS Devs Zone blog series, which highlights technical tutorials and thought leadership on JS Foundation technologies and the greater JavaScript ecosystem written by outstanding members of the JavaScript community.
Introduction
At StackPath, we recently launched a new customer portal to allow customers to configure all of their services with us (CDN,WAF, DNS, and Monitoring) in one place. The project involved combining many different data sources, some existing systems and some brand new. When starting a new project, development speed is often a top priority. We wanted to be able to move quickly, but also make good investments early on that would lead to a stable product long-term, allowing us to continue adding products and features to a strong foundation. In the end, we chose to use Apollo GraphQL, gRPC, React, and TypeScript and have been very happy with the results. In this blog, we will cover why we chose these technologies and demonstrate using a simple example project.
GraphQL
When I heard we needed to “combine many different data sources” in this project, using GraphQL as an API gateway immediately came to mind as a good fit.
The main advantages we saw for using GraphQL, as opposed to hitting REST API’s directly from our React application were:
- Hiding the complexity of the backend topology from the frontend and allowing the services to remain more isolated.
- Reducing the number of network round-trips, since GraphQL queries can retrieve multiple resources with a single request.
- Providing a clear way to add additional services in the future. Add additional queries and mutations to our schema, and start consuming them in our app.
- Using apollo-client and react-apollo simplifies frontend code around caching and data management.
- Flexible querying sets us up well when building future mobile and internal applications.
- The GraphQL schema provides a holistic view of all the data available in our system.
(If you’d like to learn more about GraphQL, I recommend checking out the official tutorial.)
Our GraphQL server acts mostly as a passthrough. All of our resolvers follow the same pattern: They request some data from backend services and may do a small amount of transformation so that the resolvers return data that matches our schema. Almost no business logic exists within the resolvers.
As a result, static typing is very effective to ensure that our service responses and transformation logic matches up with our schema. Since the GraphQL schema itself is a collection of types, it is pretty easy to generate TypeScript types from that schema. We use graphql-code-generator to generate types from our schema, and then use those types when writing our resolvers.
Example GraphQL
Our example app will be the standard TODO MVC, supporting listing, creating, and deleting TODO items. Our GraphQL schema for these three operations is:
const typeDefs = gql`
type Query {
todos: [Todo!]
}
type Mutation {
createTodo(input: CreateTodoInput!): CreateTodoPayload
deleteTodo(input: DeleteTodoInput!): DeleteTodoPayload
}
type Todo {
id: String!
title: String
}
input CreateTodoInput {
title: String!
}
type CreateTodoPayload {
todo: Todo
}
input DeleteTodoInput {
id: ID!
}
type DeleteTodoPayload {
success: Boolean
}
`;
In our package.json, we have the following scripts defined:
"scripts": {
"start": "ts-node src/index.ts",
"genTypes": "graphql get-schema && gql-gen --schema schema.json --template graphql-codegen-typescript-template --out ./src/types.ts"
},
With that schema, we are able to generate our types using
yarn genTypes, which populates our types.ts file with types that we then use in our resolver implementation:const resolvers: { [key: string]: any } = {
Query: {
todos: (): Promise<Array<Todo>> => {
// Get Todos
}
},
Mutation: {
createTodo: (
_obj: object,
args: CreateTodoMutationArgs
): Promise<CreateTodoPayload> => {
// Create Todo
},
deleteTodo: (
_obj: object,
args: DeleteTodoMutationArgs
): Promise<DeleteTodoPayload> => {
// Delete Todo
}
}
};
The body of each resolver involves making requests to our backend services, which we’ll build in the next example section.
gRPC
Initially, we planned on integrating with our backend services using REST APIs. However, our backend team had already standardized on using gRPC as means of communicating between services.
(For those unfamiliar with it, gRPC is a remote procedure call (RPC) framework open-sourced by Google utilizing HTTP/2 and protocol buffers. In gRPC, .proto files describe all of a service’s methods and all of the data types for inputs and outputs of those methods. From that proto file, protoc (protocol buffer compiler) generates both client and server code. You can read more about what gRPC is here.)
We still had the option of using a REST API exposed via grpc-gateway, but we wanted to explore using gRPC to see what advantages it brought to the table. The main advantages we saw with gRPC were:
- Really easy to generate a fully typed client for all of our backend services. We use this plugin to generate the TypeScript definitions.
- Both the backend and frontend teams being familiar with and using gRPC allowed for easy knowledge sharing.
Working with a typed client is a joy. Every service has a client where every method is typed with its request and response message type. We do not have to consult a separate API doc because the types for the client have all the information we and our IDE needs, and we know it is correct because it was generated from the proto file.
Example gRPC
We’ll define the same three operations that our GraphQL schema has in a proto file:
syntax = "proto3";
service TodoManager {
rpc CreateTodo(CreateTodoRequest) returns (CreateTodoResponse) {}
rpc GetTodos(GetTodosRequest) returns (GetTodosResponse) {}
rpc DeleteTodo(DeleteTodoRequest) returns (DeleteTodoResponse) {}
}
message CreateTodoRequest {
string title = 1;
}
message CreateTodoResponse {
Todo todo = 1;
}
message GetTodosRequest {}
// GetStacksRequest returns stacks
message GetTodosResponse {
repeated Todo results = 2;
}
message DeleteTodoRequest {
string todo_id = 1;
}
message DeleteTodoResponse {
bool success = 1;
}
message Todo {
string id = 1;
string title = 2;
}
With this todo.proto file, we use protoc to generate both the client and a skeleton for the service that just requires filling in the implementation for each method.
#!/bin/sh
PROTOC_PLUGIN="node_modules/.bin/grpc_tools_node_protoc_plugin"
OUT_DIR="."
./node_modules/.bin/grpc_tools_node_protoc \
--js_out="import_style=commonjs,binary:${OUT_DIR}" \
--grpc_out="${OUT_DIR}" \
--plugin="protoc-gen-grpc=${PROTOC_PLUGIN}" \
src/todo.proto
protoc \
--plugin=protoc-gen-ts=./node_modules/.bin/protoc-gen-ts \
--ts_out="${OUT_DIR}" \
src/todo.proto
Running this script generates four files:
- todo_grpc_pb.d.ts
- todo_grpc_pb.js
- todo_pb.d.ts
- todo_pb.js
The todo_pb.js contains the message objects and todo_grpc_pb.js contains the service/client object. The .d.ts files are TypeScript definitions for each one. A simple in-memory implementation of our service looks like this:
import { TodoManagerService } from "./todo_grpc_pb";
import {
CreateTodoRequest,
CreateTodoResponse,
GetTodosRequest,
GetTodosResponse,
DeleteTodoRequest,
DeleteTodoResponse,
Todo
} from "./todo_pb";
import grpc, { ServerUnaryCall, sendUnaryData } from "grpc";
let id = 1;
let todos: Array<Todo> = [];
const createTodo = (
call: ServerUnaryCall<CreateTodoRequest>,
callback: sendUnaryData<CreateTodoResponse>
) => {
const title = call.request.getTitle();
const todo = new Todo();
todo.setId(String(id++));
todo.setTitle(title);
todos = [...todos, todo];
const response = new CreateTodoResponse();
response.setTodo(todo);
callback(null, response);
};
const getTodos = (
call: ServerUnaryCall<GetTodosRequest>,
callback: sendUnaryData<GetTodosResponse>
) => {
const response = new GetTodosResponse();
response.setResultsList(todos);
callback(null, response);
};
const deleteTodo = (
call: ServerUnaryCall<DeleteTodoRequest>,
callback: sendUnaryData<DeleteTodoResponse>
) => {
const id = call.request.getTodoId();
todos = todos.filter(item => item.getId() !== id);
const response = new DeleteTodoResponse();
response.setSuccess(true);
callback(null, response);
};
function main() {
const server = new grpc.Server();
server.addService(TodoManagerService, { createTodo, getTodos, deleteTodo });
server.bind("0.0.0.0:50051", grpc.ServerCredentials.createInsecure());
server.start();
}
main();
Returning to our GraphQL server, we can now import the gRPC client and fill in the resolver implementation.
import {
GetTodosRequest,
CreateTodoRequest,
DeleteTodoRequest
} from "../../grpc/src/todo_pb";
import { createClient } from "../../grpc/src/client";
const todoClient = createClient();
const resolvers: { [key: string]: any } = {
Query: {
todos: (): Promise<Array<Todo>> => {
const request = new GetTodosRequest();
return new Promise((resolve, reject) => {
todoClient.getTodos(request, (error, response) => {
if (error) {
reject(error);
}
return resolve(response.toObject().resultsList);
});
});
}
},
Mutation: {
createTodo: (
_obj: object,
args: CreateTodoMutationArgs
): Promise<CreateTodoPayload> => {
const request = new CreateTodoRequest();
request.setTitle(args.input.title);
return new Promise((resolve, reject) => {
todoClient.createTodo(request, (error, response) => {
if (error) {
reject(error);
}
return resolve(response.toObject());
});
});
},
deleteTodo: (
_obj: object,
args: DeleteTodoMutationArgs
): Promise<DeleteTodoPayload> => {
const request = new DeleteTodoRequest();
request.setTodoId(args.input.id);
return new Promise((resolve, reject) => {
todoClient.deleteTodo(request, (error, response) => {
if (error) {
reject(error);
}
return resolve(response.toObject());
});
});
}
}
};
We now have a fully functional GraphQL server that communicates to a backend service using gRPC.
React
We did not spend much time discussing this choice. Our team’s experience was heavily weighted towards React, and we did not have any compelling reasons to switch to anything else. In order to achieve type safety between the GraphQL server and the frontend, we use Apollo CLI’s codegen:generate command to generate types for all of our queries.
Example React
We will need these three queries for our app:
query GetTodos {
todos {
id
title
}
}
mutation CreateTodo($input: CreateTodoInput!) {
createTodo(input: $input) {
todo {
id
title
}
}
}
mutation DeleteTodo($input: DeleteTodoInput!) {
deleteTodo(input: $input) {
success
}
}
From these queries and our schema definition, we can generate types for these queries. We take this a step further and generate the react-apollo components for each operation using apollo-typed-components, which gives us GetTodosQuery, CreateTodoMutation, and DeleteTodoMutation components in ApolloComps.tsx.
Note that TypeScript uses .ts and .tsx file extensions rather than .js and .jsx extensions. Unlike .jsx/.js, however, you’re required to use .tsx when a file contains any JSX so TypeScript can disambiguate between JSX and other TypeScript language features. For instance, angle bracket assertions like:
const foo = <Foo>bar
are valid in a .ts file, but not a .tsx file. Instead, the
as operator can be used to rewrite the expression as:const foo = bar as Foo
Our generated Query/Mutation components in ApolloComps.tsx all look similar to this:
/* Generated using apollo-typed-components */
import * as React from "react";
import { Mutation } from "react-apollo";
import { CreateTodo } from "./queries.graphql"
import { CreateTodo as CreateTodoType, CreateTodoVariables } from "types";
type GetComponentProps<T> = T extends React.Component<infer P> ? P : never;
type Omit<T, K> = Pick<T, Exclude<keyof T, K>>;
class CreateTodoMutationClass extends Mutation<CreateTodoType, CreateTodoVariables> {};
export const CreateTodoMutation = (props: Omit<GetComponentProps<CreateTodoMutationClass>, "mutation">) => <CreateTodoMutationClass mutation={CreateTodo} {...props} />;
The CreateTodoType and the CreateTodoVariables are types generated from Apollo’s codegen:generate command. CreateTodoVariables is the type for the arguments the mutation expects, and CreateTodoType is the type for the value the mutation returns. CreateTodoMutationClass is declared by simply extending react-apollo’s Mutation component with these two types, giving us a component to use whose
variables and data props are typed. The component we actually export is CreateTodoMutation, which is a wrapper around CreateTodoMutationClass that passes in the CreateTodo query defined above in queries.graphql. In order to property type CreateTodoMutation, we use two utility types: GetComponentProps and Omit. GetComponentProps takes a React component T and returns a type for the props that component T expects. The Omit utility type takes an object of type T and a key of type K, and returns a type identical to T, but with the K key removed. Combining these two utility types, we express CreateTodoMutation’s props type as the type of CreateTodoMutationClass’s props type with the mutation prop omitted, since it is being automatically supplied in the wrapper.Between the mutation definition, the types, and the components, there are admittedly quite a few moving pieces in ApolloComps.tsx. Thankfully, generating these files removes the onus from the developer to combine all of these pieces correctly for each operation and instead leaves us with a single, consumable component for each operation.
Utilizing all three of our generated components, our frontend code is then:
import React, { Component } from "react";
import { GetTodos } from "./queries.graphql";
import { GetTodos_todos } from "./types";
import {
GetTodosQuery,
CreateTodoMutation,
DeleteTodoMutation
} from "./ApolloComps";
type TodosListProps = {
onDelete: (id: string) => {};
todos: Array<GetTodos_todos>;
};
const TodosList = ({ onDelete, todos }: TodosListProps) => {
return (
<section className="main">
<ul className="todo-list">
{todos.map(todo => (
<li key={todo.id}>
<div className="view">
<label>{todo.title}</label>
<button className="destroy" onClick={() => onDelete(todo.id)} />
</div>
</li>
))}
</ul>
</section>
);
};
type AppProps = {
onCreate: (title: string) => {};
onDelete: (id: string) => {};
todos: Array<GetTodos_todos>;
};
type AppState = {
newTodo: string;
};
class App extends Component<AppProps, AppState> {
state = {
newTodo: ""
};
render() {
const { onCreate, onDelete, todos } = this.props;
return (
<section className="todoapp">
<header className="header">
<h1>{"todos"}</h1>
<input
className="new-todo"
placeholder="What needs to be done?"
value={this.state.newTodo}
onKeyDown={(e: React.KeyboardEvent) => {
if (e.keyCode !== 13) {
return;
}
e.preventDefault();
var val = this.state.newTodo.trim();
if (val) {
onCreate(val);
this.setState({ newTodo: "" });
}
}}
onChange={(e: React.ChangeEvent<HTMLInputElement>) =>
this.setState({ newTodo: e.target.value })
}
autoFocus={true}
/>
</header>
{todos.length ? <TodosList onDelete={onDelete} todos={todos} /> : null}
</section>
);
}
}
const ApolloApp = () => {
return (
<GetTodosQuery>
{({ data }) => (
<DeleteTodoMutation>
{deleteTodo => (
<CreateTodoMutation>
{createTodo => {
if (!data || !data.todos) {
return null;
}
const todos = data.todos;
return (
<App
onCreate={(title: string) =>
createTodo({
variables: {
input: {
title
}
},
update: (cache, response) => {
const createdTodo =
response.data &&
response.data.createTodo &&
response.data.createTodo.todo;
if (createdTodo) {
cache.writeQuery({
query: GetTodos,
data: {
todos: [...todos, createdTodo]
}
});
}
}
})
}
onDelete={(id: string) =>
deleteTodo({
variables: {
input: {
id
}
},
update: cache => {
cache.writeQuery({
query: GetTodos,
data: {
todos: todos.filter(item => item.id !== id)
}
});
}
})
}
todos={todos}
/>
);
}}
</CreateTodoMutation>
)}
</DeleteTodoMutation>
)}
</GetTodosQuery>
);
};
export default ApolloApp;By using Apollo GraphQL, gRPC, React, and TypeScript, we have a lot of flexibility when querying our data but are still able to keep our services fairly isolated from one another. Additionally, by achieving end-to-end type coverage, it is very difficult to accidentally misuse data or introduce a breaking change. If we do need to introduce a breaking change, it is easy to determine which parts of our system need to be fixed before making that change.
One of our main takeaways is how powerful enforceable data descriptions are (in this case, the GraphQL schema and the gRPC proto files). Being able to generate types and enforce that your implementation matches the spec ensures you don’t lose safety across the network in different parts of the system. Regardless of the specific technologies at play, type safety between services and clients really boosts confidence in overall system stability.
Example code at https://github.com/TLadd/todo-graphql-grpc
Example code at https://github.com/TLadd/todo-graphql-grpc
The post Graphql, gRPC, and End-to-End Type Coverage appeared first on JS Foundation.