Monday, 16 October, 2023 UTC
Summary
With numerous languages spoken worldwide, many individuals who speak English as a second language prefer to use their native tongue in their daily interactions. Having a chat application that can seamlessly translate messages from the sender’s native language to the recipient’s preferred language would prove invaluable, for both social media users and businesses catering to international customers.
This article aims to equip you with the knowledge of leveraging the Twilio Programmable Messaging API, WhatsApp, and the Google Cloud Translation API to build your own language translation chatbot. Using this application, users can have natural conversations without language barriers, while the app does the heavy lifting by translating messages on the other end of the conversation. By following the guidelines presented, you'll gain the ability to facilitate multilingual communication through your chat application.
Prerequisites - Node.js version 14 and above.
- A Twilio account to send and receive WhatsApp messages. Go to the signup page to get started.
- A Google Cloud account to implement the translation feature.
Express, a powerful web framework for Node.js, will be used in this project for the creation and handling of web requests. It serves as an excellent tool for setting up the server and managing routing.
To get started with the project, create a folder called language-translating-chat-app, and initialize a package.json file with the command below:
mkdir language-translating-chat-app cd language-translating-chat-app npm init -y Install express as a dependency by running the command below:
npm install express --save Open up the newly created project directory on your preferred text editor and create a file called app.js in your project folder and add the following lines of code:
// Import Express const express = require('express'); // Create an instance of Express const app = express(); app.use(express.json()); // Define a port number for the server const port = process.env.PORT || 3000; The code above imports and then instantiates
express into the application. A port number is then specified for the server to listen on.Define a route for the app. A route is a way of telling
express how to handle different web requests based on the path and method you provide. For this project, only one route is needed: a POST request to the root path (/). This is where Twilio will send WhatsApp messages from users.Use the code below to define a route for a POST request.
// Define a route for handling WhatsApp messages app.post('/',express.urlencoded(), (req, res) => { // Log the request body to the console console.log(req.body.Body); // Send back an empty response res.send(); }); In the code above,
app.post() takes three arguments namely: Path, Middleware and Callback function.The callback function takes two parameters:
req (request) and res (response). You will use the req.body.Body property to access the body of the message sent by Twilio and use the res.send() method to send back a response to Twilio. For now, you will just log the request body to the console and send back an empty response.Finally, start the server and listen for incoming requests on the port number defined earlier using the
app.listen() method. Copy the code below to start the server.// Start the server and listen for incoming requests app.listen(port, () => { // Print a message to the console indicating that the server is running console.log(`Server is running on port ${port}`); }); Test your code by running
Configuring the Twilio webhook node app.js in the terminal. If everything works fine, you should see the message “Server is running on port 3000” in your terminal.Webhooks are automated messages sent between apps when something happens. This project leverages the power of webhooks to direct Twilio where to send data whenever a significant event occurs, such as receiving a WhatsApp message. To enable this functionality, you need to connect your server’s URL with Twilio. This allows WhatsApp messages to be sent as POST requests. To do this, you will use a tool called ngrok.
ngrok is a service that creates a secure tunnel between a local server and the internet. It provides a temporary URL that can be used to access an application from anywhere. To use ngrok, install and download it. Once installed, start up your Node.js application, then run the ngrok service in the terminal with the command below.
ngrok http 3000 This command will start ngrok and create a tunnel for the app running on
port 3000. It will also display the temporary URL that can be used to access your app. It should look like the image below.In your Twilio console, you will need the Forwarding URL that begins with https, for example, the URL https://ea61-105-113-16-175.eu.ngrok.io in the image above will be used for configuration purposes in the next section.
Setting up Twilio programmable messaging Twilio Programmable Messaging is a service that allows you to integrate messaging into your platform using other APIs like SMS, Whatsapp, Facebook Messenger, and others. For this project, you will use the Whatsapp sandbox on Twilio to enable messaging.
To get started, go to your Twilio console and navigate to the Programmable Messaging section. Click on Whatsapp and select Sandbox from the left menu. This will take you to the sandbox configuration page.
On this page, scroll down to the When a message comes in section and paste the ngrok URL you copied from the terminal in the text box designated for POST requests. It should look like the image below.
Click on Save at the bottom of the page. This will save the webhook configuration and allow Twilio to send WhatsApp messages.
To test the app, join the Twilio sandbox by sending a WhatsApp message with the word "join" followed by a two-word code that is displayed on the sandbox configuration page. For example “join ******”.
If all goes well, you should receive a confirmation message from Twilio saying that you have joined the sandbox. It will look like the image below.
You can send any message you like to the sandbox number. Keep an eye on the terminal to see if your message gets logged.
Connect to Google Cloud To connect to Google Cloud, you can create a new project or select an existing project on the Google Cloud console. Make sure to enable billing and Cloud Translation API for the project.
To create an API Key, go to the API and services section, and type “Translate” in the search box. Click on the Cloud Translate API link, as seen in the image below.
Click on the Enable button to enable the API for your project. Go back to the API and services section and select Credentials from the side menu, click on the Create button, and select the API key option.
An API key will be generated for you. Save the Google API key as an environment variable by creating a file named
.env in the root directory of your application and add the code below:GOOGLE_API_KEY= <YOUR_API_KEY> Ensure you replace
Implementing translation <YOUR_API_KEY> with your actual API key.To add the translation functionality to your code, you need to understand how the application works. Here are the steps that a user must follow to use the app:
- The user messages the Twilio sandbox number and receives a welcome message.
- When the user sends another message, they will receive an instruction on how to use the app. The instruction is as follows:
- The default language for all users is English. To change the current language, the user will enter
/lang <language>. - This command changes the language to the one specified by the
<language>parameter. The parameter must be in the ISO 639-1 format. For example, to switch to French, the user sends/lang fr. - To save a WhatsApp contact, the user will send a message in this format:
whatsapp:<receiver's number>. The user only needs to send their message in this format once, unless they want to change their receiver.
- The default language for all users is English. To change the current language, the user will enter
Now that you’re up to date with how the application works. Follow the steps below to get started with adding the translation API to the project.
Install the
Google Cloud Translate package using the command below.npm install @google-cloud/translate In your app.js file, add the code below to the top of the file (where you initialized the
express package) to import the Translate package and create an instance with your project ID.// Import Translate from the Google Cloud client library const { Translate } = require("@google-cloud/translate").v2; // Create an instance of Translate with your project ID const translate = new Translate({ projectId: "Your_project_id_goes_here", key: process.env.GOOGLE_API_KEY, }); Then, create a file called helperfunctions.js in the root directory of your application and copy the code below inside the file.
// Import fs module for file operations const fs = require("fs"); // Define a file name to store the sender-receiver pairs const pairFile = "pairs.json"; // Define a function to write the preferred Receiver of a user to the file const setReciever = (user, receiver) => { // Read the file content as a string const data = fs.readFileSync(pairFile, "utf8"); // Parse the string as a JSON object const pairs = JSON.parse(data); // Set the reciever pairs[user] = receiver; // Stringify the JSON object as a string const newData = JSON.stringify(pairs); // Write the string to the file fs.writeFileSync(pairFile, newData); }; // Define a function to get the receiver's number from the file const getReceiver = (sender) => { // Read the file content as a string const data = fs.readFileSync(pairFile, "utf8"); // Parse the string as a JSON object const pairs = JSON.parse(data); // Return the receiver's number or null if not found return pairs[sender] || null; }; module.exports= { getReceiver, setReciever }; In the code above, the
setReceiver function takes in a user and receiver parameter and saves their data as a key-value pair inside pairs.json, a JSON file yet to be created. In the second function, the getReceiver retrieves the receiver’s value from the JSON file.Next, create a pairs.json file in the root directory of your project and add an empty object to the file like so:
{ } The JSON file will serve as local storage for the user's data. For this tutorial, you’ll be storing the user's preferred language in a JSON file for the sake of simplicity. For production applications, you should use a database to store persistent data.
Update the main route’s (
/) callback function inside the app.js file with the following code:// Define a route for handling WhatsApp messages app.post('/',express.urlencoded(), (req, res) => { // Define the text to be translated const text = req.body.Body; // Define the sender's number const from = req.body.From; // define twilio's number const twilioNumber = req.body.To; // Split the text by space const parts = text.split(" "); // get the reciever's number by splitting by the string "whatsapp:+" const to = parts[0].startsWith("whatsapp:+") ? parts.shift() : getReceiver(from); }); In the code above, a check is performed to see if the first word in the WhatsApp messages starts with "whatsapp:+". If it does, it’s saved as the receiver’s number, which is the
to variable.If it does not pass the check, the baton is passed to the
Connecting Twilio to the application getReceiver function. This function checks if there is a designated receiver assigned to the sender's number. If no receiver is found, the getReceiver function will return a null value.Go to your Twilio Console, and copy your
accountSID and authToken values from the main page. Open the .env file you created earlier, and save these values as environmental variables like so:ACCOUNT_SID=<YOUR_SID> AUTH_TOKEN=<YOUR_TOKEN> Install the
dotenv package using the command below. This package will enable you to load environment variables from your .env file.npm install dotenv -save Copy the code below at the top of the helperfunctions.js file to import the Twilio library and instantiate the
TwilioClient package.// Import TwilioClient const twilio = require("twilio"); // load the .env file require("dotenv").config() // Define your Twilio account SID const accountSid = process.env.ACCOUNT_SID; // Define your Twilio auth token const authToken = process.env.AUTH_TOKEN; // Create an instance of TwilioClient const client = twilio(accountSid, authToken); Create a JSON file called languages.json and add an empty object inside like so:
{ } In the helperfunctons.js file, create a
getter and setter function to update the languages.json file when a user updates their preferred language:// Define a file name to store the preferred languages of each user const languageFile = "languages.json"; // Define a function to read the preferred language of a user from the file const getLanguage = (user) => { // Read the file content as a string const data = fs.readFileSync(languageFile, "utf8"); // Parse the string as a JSON object const languages = JSON.parse(data); // Return the preferred language of the user or "en" as the default return languages[user] || "en"; }; // Define a function to write the preferred language of a user to the file const setLanguage = (user, language) => { // Read the file content as a string const data = fs.readFileSync(languageFile, "utf8"); // Parse the string as a JSON object const languages = JSON.parse(data); // Set the preferred language of the user languages[user] = language; // Stringify the JSON object as a string const newData = JSON.stringify(languages); // Write the string to the file fs.writeFileSync(languageFile, newData); }; In the helperfunctons.js file, create a new function to handle sending messages using the code below.
// Define a function to send a message using Twilio API const sendMessage = (from, to, body) => { // Send back a message to WhatsApp with Twilio API client.messages.create({ from, to, body }).then((message) => { // Log the message SID to the console console.log(message.sid); }); }; The code above will send a request to the Twilio API and return a promise with a message object. The message object contains information about the message, such as its SID, status, date, etc. Update the
module.exports object in the helperfunctions.js file with the following to import the new functions you just created.module.exports= { getReceiver, setReciever, setLanguage, getLanguage, sendMessage, } Using the code below, import all your helper functions into the app.js file. Ensure you add this next to the other import/initialize statements at the top of the app.js file.
//import helper functions const { sendMessage, setLanguage, getLanguage, setReciever, getReceiver, } = require("./helperfunctions.js"); Write an IF block inside the route’s callback function to check if the user wants to switch languages. Add this code to the bottom of the route’s (
/) callback function:// Check if the text starts with the /lang command to switch languages if (text.startsWith("/lang")) { // Get the language code from the text const language = parts[1]; if (language.length > 2) { sendMessage( twilioNumber, from, `Please ensure that your language is in the ISO 639-1 format or your message will not be delivered. You can search for your language format here https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes` ); return; } // Set the preferred language of the sender to the file setLanguage(from, language); // Send back a confirmation message to WhatsApp with Twilio API sendMessage( twilioNumber, from, `Your preferred language is set to ${language}.` ); return; } The code above extracts the language from the body of the message and checks if the language’s length is greater than 2. This check is done because the ISO_639-1 format is always two letters.
The
setLanguage function is then called to save the language to the JSON file. At the end of the code, a confirmation message is sent to the user to confirm that their language has been successfully sent.To alert the user that someone has added their number to the application, add an IF block to detect when a new number is being saved. Copy the code below to the bottom of the route’s (
/) callback function: if (text.startsWith("whatsapp:+")) { setReciever(from, to); sendMessage( twilioNumber, from, "You have successfully saved a new number for messaging" ); sendMessage( twilioNumber, to, `The contact ${from} has saved your number for messaging` ); return; } Add another IF block that checks if the
to variable is null. If it’s null, a message is sent to the user. This message prompts them to provide a valid number. Copy the code below to the bottom of the route’s (/) callback function:// Check if the receiver's number is null or not if (to === null) { // Send back a message to the sender asking them to provide a valid receiver's number sendMessage( twilioNumber, from, "Welcome to LinguaLink. Please provide a valid receiver's number in the format whatsapp:+146786543. To change your default language type /lang <language code>. For example, /lang fr will set your preferred language to French. Please ensure your language is in the ISO 639-1 format or your message will not be delivered. You can search for your language format here https://en.wikipedia.org/wiki/List_of_ISO_639-1_codes" ); return; } Lastly, add an ELSE block to send the message to the receiver. Before it is finally sent to the receiver, it is passed into the
translate method. Copy the code below to the bottom of the route’s (/) callback function:else { // Get the message from either the rest of the parts or from the whole text const message = parts.length > 0 ? parts.join(" ") : text; // Get the preferred language of both sender and receiver from the file const sourceLanguage = getLanguage(from); const targetLanguage = getLanguage(to); if (sourceLanguage !== targetLanguage) { // Translate the message using the translate() method translate.translate(message, targetLanguage).then(([translation]) => { // Send back a translated message to WhatsApp with Twilio API sendMessage(twilioNumber, to, translation); // Log the translation to the console console.log(`Text: ${message}`); console.log(`Translation: ${translation}`); }); } else { // Send back an untranslated message to WhatsApp with Twilio API sendMessage(twilioNumber, to, message); } } In your app.js file, include a
try and catch code block inside the callback function in the POST request to handle edge cases that might occur. The catch block will notify the user whenever something goes wrong in the application. Use the code below to update your application.try { // Check if the text starts with the /lang command to switch languages // your if/else code block } catch (err) { sendMessage(twilioNumber, to, 'Sorry, something went wrong. Please try again later.'); console.log(err); } Congratulations! You have successfully integrated translation functionality into your WhatsApp application.
Testing the application To test this application you need two Whatsapp numbers to send and receive messages. To get started, run your application on your terminal with the following command:
node app.js Then, navigate to Whatsapp on your device and click on the Twilio contact number you saved previously. If you have not joined the Twilio sandbox, enter the “join *****-****” command to connect to your sandbox. Check the image below for reference.
Next, type in any message and send it. You will receive an instruction message on how to proceed with the application.
Set your language preference to any language of your choice. If you don’t select any language, the application will set your default language to English. In the image below, the language is set to French using the ISO-639-1 format.
Use the “whatsapp:number” command to set your receiver’s number (The second Whatsapp number you are using to test).
Repeat the steps above for your second number. The receiver will receive a message whenever their number is saved on the application.
In this scenario, the sender's language preference is set to French, while the receiver's default language is English. So, when the sender sends a message in French, the receiver will receive the message in English, as it will be automatically translated from French to English. Check out the images below for illustration.
Conclusion In this article, you have gained valuable insights into building a language-translating chat application using Twilio and WhatsApp. You leveraged the Twilio API and Google Cloud Translate API for sending and receiving translated WhatsApp messages. This simple application can be further improved by adding a dropdown menu to select a language and upgrading your Twilio account to add more Whatsapp numbers to the sandbox.
I trust you found this article enjoyable and obtained new and practical knowledge. Make sure to explore other Twilio APIs to further improve this application and put it to the test with your friends.
Happy coding!