Thursday, 2 March, 2017 UTC
Summary
“Agrippa Solutions” (www.agrippa.no) company from Norway is based in their solutions on node.js platform. Everyday node.js applications serve tens of thousands of operations and transactions per second. Node.js easily copes with such a load. But even the best platform with a badly written application can contribute to a real defeat. In this article I’d like to present performance, reliability and stability best practices for node.js applications deployed to production.
Below there is list best practices to improve your application performance built on node.js:
- Use validation parameters
- Use bundle and gzip compression
- Don’t use synchronous functions
- Handle logging correctly
- Handle exceptions properly
- Optimize SQL queries and Stored Procedures
Application wrote on node.js without handle of exceptions can be crashed in case some invalid input parameters e.g. invalid type conversion, too long string, etc. This means a break in the action of our application in time until be restarted, repeating the same operations by the customers many times, what leads to growth the number of requests function rest api (which each time are finished fail). Even if they pass through the api functions with node.js to database that can cause exceptions and error messages (not even a mention about the values saved to not appropriate column). This is the first step to defeat of company and the first important link in our chain to stabilize the operation of our server.
Function rest api at the beginning has to check correctness input parameters before sending them to Data Base or to another functions.
The list of available modules to validate you can find a lot of ready-made packages that can be used for this purpose. One of them jst express-validator (https://www.npmjs.com/package/express-validator) and is ideal for production. By this module we can validate body, queries and headers data
Below I present sample How use it:
1. First You have to place module to man file of server node.js e.g. index.js or server.js
var util = require('util'),
bodyParser = require('body-parser'),
express = require('express'),
expressValidator = require('express-validator'),
app = express();
app.use(bodyParser.json());
// this line must be immediately after any of the bodyParser middlewares!
app.use(expressValidator({
errorFormatter: function(param, msg, value) {
var namespace = param.split('.')
,root = namespace.shift()
,formParam = root;
while(namespace.length) {
formParam += '[' + namespace.shift() + ']';
}
return {
param : formParam,
msg : msg,
value : value
};
}
}));
app.listen(8888);
2. Declare in one file e.g. validator.js your own module with below source code. It will be used to validation in many files of our application
module.exports = function(request,response,schema)
{
request.checkBody(schema);
var errors = request.validationErrors();
if (errors) {
console.error({Error: errors});
response.status(422).send({Error: {Validator: errors}});
return true;
}
else return false;
}
3. Declare in schema to validation input parameters in separate file.
module.exports = {
'sensor_device_id': {
isInt:{
errorMessage: 'sensor_device_id - Integer expected'
},
errorMessage: 'sensor_device_id is required'
},
'active': {
notEmpty: true,
errorMessage: 'Active Validation error'
},
'organisation': {
isInt:{
errorMessage: 'Organisation - Integer expected'
},
errorMessage: 'Organisation is required'
}
}
3. Place instruction to every action where schema You should place correct name of file with defined schema
if(validator(request,response,schema)) return next();For instance
const validator = require('../schema/validator.js');
const schema_sensorActive = require('../schema/sensorActive.js');
app.post('/sensor', function (request, response, next) {
if(validator(request,response,schema_sensorActive)) return next();
const mssql = request.service.mssql;
const params = request.body;
const {
organisation=null,
sensor_device_id=null,
active=1
} = params;
const sqlQuery = 'EXEC [AppCenter].[Sensor_Active_UPDATE] ?, ?, ?, ?';
const sqlParams = [request.auth.UserID, organisation, sensor_device_id, active];
mssql.query(sqlQuery, sqlParams, {
success(res) {
try{
return response.status(200).send(res);
} catch(err) { return response.status(201).send({Error: res}); }
},
error: handleSqlError(response, sqlQuery, sqlParams)
});
}
Every time when Validator detect errors then information about parameter name and defined by us communication error will be sent to fronted client app.
You can also use validation for output parameter e.g after performed data from db, from device etc
Use bundle and gzip compression
In case of static files (e.g. html files) or json answers gzip compressing can greatly decrease the size of the response body and hence increase the speed of a web app.
For applications built on angular, angular2, react very important is creating bundle and compressed files – we have a lot of tools for this operation like webpack.
But we can’t forget about huge structures of data sending in json format. Pagination doesn’t replace edit operation where we fetch sometimes huge structure of document e.g. to edition. We can replace this mechanism by multiple fetching various pieces of data – but what if we have the possibility of compression which is a automatically served by browsers and mobile devices.
For applications built on angular, angular2, react very important is creating bundle and compressed files – we have a lot of tools for this operation like webpack.
But we can’t forget about huge structures of data sending in json format. Pagination doesn’t replace edit operation where we fetch sometimes huge structure of document e.g. to edition. We can replace this mechanism by multiple fetching various pieces of data – but what if we have the possibility of compression which is a automatically served by browsers and mobile devices.
const express = require('express')
const compression = require('compression')
const app = express()
app.use(compression())Here JSON as compressed result by our node.js in browser:
Don’t use synchronous functions
Node.js is based on an asynchronous architecture. Node.js is an open-source, cross-platform runtime environment for developing highly scalable server-side web applications, especially web servers written in JavaScript. It allows you to create applications that useasynchronous event-driven entry-exit system. Node.js provides an event-driven architecture and a non-blocking I/O API designed to optimize an application’s throughput and scalability for real-time web applications. Synchronous functions and methods tie up the executing process until they return.
More details:
Node.js vs PHP
Node.js Server Architecture in practice (IBM, Microsoft, Yahoo!, Walmart, Groupon, SAP, LinkedIn, Rakuten, PayPal, Voxer and GoDaddy)
Node.js – Features and Application
app.get('/sensor', function (req, res, next) {
if(validator(request,response,schema)) return next();
mssql()
.then(function (data) {
return performData(data)
})
.then(function (csv) {
try{
return response.status(200).send(res);
} catch(err) { return response.status(201).send({Error: res}); }
})
.catch(handleSqlError(response, sqlQuery, sqlParams))
})
In the previous point we learned that in Node.js we should use asynchronous functions. Remember that applying functions as console.log() or console.error() to print log messages to the terminal are synchronous. On production when the destination is a terminal or a file, they are not suitable. If you’re going logging app activity (for example, to catch exceptions), instead of using
mentioned synchronous functions, use a logging modules like Winston or other asynchronous.Use Try-catch construct to catch exceptions not only in synchronous. For Instance if TSQL Stored procedure returns two results in below code what will cause exception – two times sending response – that’s means crash of node.js server
mssql()
.then(function (data) {
return performData(data)
})
.then(function (csv) {
try{
return response.status(200).send(res);
} catch(err) { return response.status(201).send({Error: res}); }
})
.catch(handleSqlError(response, sqlQuery, sqlParams))And last very important issue: even super-written code in node.js will work slowly when response from the DB server will take longer for a few seconds which translates into time response our Rest api. The second important thing: more data processing on the TSQL side in DB and less on node.js side. This particularly applies to data collected from multiple tables
JRB System (www.jrbsystem.com) cooperate with Agrippa Solutions in case of Node.js technologies