Wednesday, 6 December, 2023 UTC


Summary

The SAP Cloud Application Programming Model (CAP) describes itself as a framework of languages, libraries, and tools for building enterprise-grade services and applications. Its primary capabilities are database modeling via Core Data Services and other enterprise-grade features for localization, data privacy, authorization, and messaging. The framework offers a high abstraction for these services to relieve the coders from writing as much boilerplate code as possible. CAP provides the flexibility to include arbitrary JavaScript libraries in its lifecycle for all other service integrations, e.g., to bring the user experience to the next level. This post will demonstrate how to send and receive emails with a CAP application using the SendGrid JavaScript client and webhooks (aka inbound parses).
What are we building today?
We won't reinvent the wheel in this post and stick with the well-known hello-world-scenario used in CAP projects: The bookshop. Don't worry if you haven't used this one before. Bookshop examples are small CRUD web applications that use entities from a bookshop such as books, authors, orders, etc. The project that you will build is straightforward and only has a single entity: Books. All records of this entity will be exposed via a read-only REST endpoint. So it’s more of a RU (read and update) web app, without create and delete operations. Furthermore, the final application will accept HTTP POST requests to listen for incoming orders, reducing the "stock" field of the ordered book. With the help of Twilio SendGrid, we'll ensure the shop managers get notified when the stock of a given book runs low and provide an easy way to order more books from the suppliers.
Requirements
In order to follow this tutorial, the requirements include:
  • Sign up for a free SendGrid account here. No credit card required.
  • A verified sender (either a Single Sender Verification or Domain Authentication)
  • Point the MX record of your domain to SendGrid
  • Node.js installed (version 18 or newer)
  • ngrok installed
  • Cloud Application Programming Model (CDS) Command Line Interface installed
  • SQLite binary installed (Unix users probably already have it)
  • Optional: Visual Studio Code with installed extensions such as CDS Language Support and REST Client
Initialize a bookshop-project
To start, use the cds init command line to bootstrap new projects, which will generate all required files.
cds init bookshop --add samples cd bookshop npm install 
Next, you'll start your local web server using a command with three arguments:
- --in-memory ensures that the database schema is deployed to SQLite,
- --to rest controls the web protocol used to expose the data services.
- --watch restarts the server automatically when file changes occur.
For the last option, you'll need to include a new development dependency first:
npm add -D @sap/cds-dk cds serve all --watch --in-memory --to rest 
Proceed to test the basic project by sending an HTTP GET request to http://localhost:4004/catalog/Books. For this, I recommend using the REST Client extension for Visual Studio Code and a new file called requests.http:
### Get book data from the catalog GET http://localhost:4004/rest/catalog/Books 
You should see a response after sending the request. If not, you can also use curl to send this request.
curl http://localhost:4004/rest/catalog/Books 
Next, an action that defines a POST endpoint in your web service will be added. Add the line provided below in the catalog definition of srv/cat-services.cds.
using my.bookshop as my from '../db/data-model'; service CatalogService { @readonly entity Books as projection on my.Books; action submitOrder(book : Books:ID, quantity : Integer); } 
The implementation of this action will parse the incoming request and utilize the parameters to reduce the stock field value. If this field turns negative, the request must fail. Create a new file called srv/cat-service.js:
const cds = require("@sap/cds"); class CatalogService extends cds.ApplicationService { init() { const { Books } = cds.entities("my.bookshop"); // Reduce stock of ordered books if available stock suffices this.on("submitOrder", async (req) => { const { book, quantity } = req.data; let { stock, title } = await SELECT`stock, title`.from(Books, book); const remaining = stock - quantity; if (remaining < 0) { return req.reject(409, `${quantity} exceeds stock for book #${book}`); } await UPDATE(Books, book).with({ stock: remaining }); return { ID: book, stock: remaining }; }); return super.init(); } } module.exports = { CatalogService }; 
After saving the file, the service will automatically restart thanks to the 'watch' option.
Now, add the following lines to the requests.http file to define the second request template:
### Get stock data from the catalog GET http://localhost:4004/rest/catalog/Books ### Submit an order POST http://localhost:4004/rest/catalog/submitOrder HTTP/1.1 Content-Type: application/json { "book": 1, "quantity": 95 } 
You can use curl to send this request via the terminal.
curl -X POST -d '{"book":1,"quantity":95}' \ -H 'Content-Type: application/json' \ http://localhost:4004/rest/catalog/submitOrder 
The first request will return an empty, but successful, response. Additional requests will fail because the stock has been too low.
Send an email when the stock decreases significantly
In the next section, you'll add basic email notification capabilities to your project. You will be using a Node.js SendGrid client for this end.
Hence, you need to add the SendGrid Node.js client as a dependency to the project.
npm add @sendgrid/mail 
You need to replace the login details with the API Key from the SendGrid account, an email address that matches your verified sender and any recipient address. Make sure to keep them stored securely in the environment variables default-env.json:
{ "SENDGRID_API_KEY": "<Replace with API Key>", "SENDGRID_SENDER": "<Replace with Sender's Email Address>", "SENDGRID_RECEIVER": "<Replace with Receiver's Email Address>" } 
Now, initialize the SendGrid client and send an email when the stock reaches the threshold. Add the highlighted lines to the service implementation srv/cat-service.js:
const cds = require("@sap/cds"); const sgMail = require("@sendgrid/mail"); sgMail.setApiKey(process.env.SENDGRID_API_KEY); class CatalogService extends cds.ApplicationService { init() { const { Books } = cds.entities("my.bookshop"); // Reduce stock of ordered books if available stock suffices this.on("submitOrder", async (req) => { const { book, quantity } = req.data; let { stock, title } = await SELECT`stock, title`.from(Books, book); const remaining = stock - quantity; if (remaining < 0) { return req.reject(409, `${quantity} exceeds stock for book #${book}`); } await UPDATE(Books, book).with({ stock: remaining }); if (remaining < 10) { const msg = { to: process.env.SENDGRID_RECEIVER, from: process.env.SENDGRID_SENDER, subject: 'Low Stock Alert', text: `A customer just ordered ${quantity}x "${title}" and there are only ${remaining} left in stock.`, html: `<strong>A customer just ordered ${quantity}x "${title}" and there are only ${remaining} left in stock.</strong>`, }; sgMail .send(msg) .then(() => console.log('Email sent')) .catch((error) => console.error(error)); } return { ID: book, stock: remaining }; }); return super.init(); } } module.exports = { CatalogService }; 
Submit an order by triggering the second HTTP request.
You should now receive an email indicating the current stock status.
Listen to inbound email for restocking
The previous section set up a one-way communication channel from your project to the email of the bookstore managers. This last section will transform it into a two-way communication channel that can read responses sent back by the managers.
In the SendGrid dashboard, you can control what happens when SendGrid receives an email addressed to one of your domains. SendGrid will then invoke a pre-registered webhook. Thus, we'll utilize a custom middleware to implement the webhook and handle the email.
As the application is currently running on localhost, you'll need to open a channel to route traffic to your machine from the SendGrid data center. For this, you'll use ngrok.
To begin, tell the bookshop manager how they can respond to the initial email. Therefore, change the following line in the service implementation srv/cat-service.js:
const msg = { to: process.env.SENDGRID_RECEIVER, from: process.env.SENDGRID_SENDER, subject: 'Low Stock Alert', text: `A customer just ordered ${quantity}x "${title}" and there are only ${remaining} left in stock. Please respond with "Yes" `+ `if you would like to restock now.`, html: `<strong>A customer just ordered ${quantity}x "${title}" and there are only ${remaining} left in stock. Please respond with "Yes" `+ `if you would like to restock now.</strong>`, }; 
Next, set up an inbound parse webhook in SendGrid, which allows your application to receive inbound email requests from bookshop managers. SendGrid will forward all HTTP requests of content type multipart/form-data. To parse these requests in your app, install multer.
npm add multer 
To create a custom middleware with CAP, you only need to create a file srv/server.js and listen to the bootstrap event before you can initialize the SendGrid client.
This snippet will print the content of the email to the console and return a successful response.
const cds = require("@sap/cds"); const multer = require('multer'); const sgMail = require("@sendgrid/mail"); sgMail.setApiKey(process.env.SENDGRID_API_KEY); const upload = multer({ dest: 'uploads/' }) cds.on("bootstrap", (app) => { app.post( "/sendgridInbound", upload.none(), async (req, res) => { console.log(`Received message ${req.body.text}.`) res.end("ok"); } ); }); 
Modify the code so that it parses the email content, extracts information such as the restocking quantity, and updates the bookshop's stock accordingly. Additionally, send email notifications using SendGrid to inform about successful restocking or failure, enhancing the bookshop's communication capabilities.
const cds = require("@sap/cds"); const multer = require("multer"); const sgMail = require("@sendgrid/mail"); sgMail.setApiKey(process.env.SENDGRID_API_KEY); const upload = multer({ dest: "uploads/" }); async function collectBookDetails(sender, message) { if (message.includes("Yes")) { const restockPattern = /Yes (\d+)/i; const lastOrderPattern = /(\d+)x/; const titlePattern = /"(.*?)"/; const restock = message.match(restockPattern) ? +message.match(restockPattern)[1] : undefined; const originalMessage = message.match(/>(.*)/g).join(""); try { const lastOrder = +originalMessage.match(lastOrderPattern)[1]; const title = originalMessage.match(titlePattern)[1]; const { Books } = cds.entities("my.bookshop"); const books = await cds.read(Books).where({ title }); return { valid: true, restock: restock || lastOrder, book: books[0], }; } catch (err) { //regex didn't find a last order or book title return { valid: false }; } } return { valid: false, }; } cds.on("bootstrap", (app) => { app.post("/sendgridInboud", upload.none(), async (req, res) => { const { from } = JSON.parse(req.body.envelope), to = req.body.to, parsed = await collectBookDetails(from, req.body.text); if ( from === process.env.SENDGRID_RECEIVER && to === process.env.SENDGRID_SENDER && parsed.valid ) { const newStock = parsed.book.stock + parsed.restock; const { Books } = cds.entities("my.bookshop"); await UPDATE(Books, parsed.book.ID).with({ stock: newStock }); sgMail.send({ to: process.env.SENDGRID_RECEIVER, from: process.env.SENDGRID_SENDER, subject: `RE: ${req.body.subject}`, text: `Successfully restocked "${parsed.book.title}". Current stock: ${newStock}`, html: `<strong>Successfully restocked "${parsed.book.title}". Current stock: ${newStock}</strong>`, }); } else { const msg = { to: process.env.SENDGRID_RECEIVER, from: process.env.SENDGRID_SENDER, subject: `RE: ${req.body.subject}`, text: `Failed to restock. Please reply with "Yes <additionalStock>"`, html: `<strong>Failed to restock. Please reply with "Yes <additionalStock>"</strong>`, }; sgMail.send(msg); } res.end("ok"); }); }); 
Now you can restock by replying to the email with "Yes <additionalStock>". After which, you should receive an email confirming the restocking. If the restocking fails, you will receive an email indicating the failure.
Use ngrok to open a tunnel from your local port 4004 to the internet.
ngrok http 4004 
In your SendGrid dashboard, navigate to Settings -> Inbound Parse. Choose your domain, and enter the ngrok URL of your application to handle incoming emails.
Let's give it a shot. Respond with "Yes 2000" to the message you received a few minutes ago. Now query the current stock info via the first HTTP request once more.
What's next
Congratulations! You have just integrated a two-way communication channel to a CAP bookshop application using SendGrid and improved the bookshop managers' user experience!
You can further extend this application by integrating other communication channels such as SMS, WhatsApp, Voice or Video. Furthermore, you can also improve your login mechanism to protect your web application from bots. For reference, you can find the complete source code on GitHub.
Feel free to get in touch if you have questions regarding this application or any related topic.