Wednesday, 16 June, 2021 UTC


Summary

The Apprentice is back on TV screens around Asia, and Twilio is the official technology partner. Throughout the series Twilio APIs have supported the show and candidates; delivering notification messages to the teams, powering their solutions in episode 11, and running a Watch & Win competition for the viewers.
The Watch & Win competition was implemented as a chatbot over Facebook Messenger. In this post we will look at how you can build your own competition bot using Twilio Autopilot, Twilio Functions and Airtable.
Defining the competition
To build a watch and win competition bot, we need a few parameters for how the bot will work:
  • The competition will run every week that the show is broadcasting, with a new question per episode
  • The question will include a phrase that was said during the week's episode and three options for who said it
  • Viewers can enter the competition by chatting with a bot on Facebook Messenger and answering the question
  • We also need to capture some details from the user, such as their name, their email address and whether they agree with the terms of the competition
  • If the viewer has entered before we don't need to ask them for the details again, so we should store that information
With those details in mind, let's get building.
What you need to build this bot
To build this bot we're going to use Twilio Autopilot, Twilio Functions and Airtable. Twilio Autopilot will allow us to collect users' answers in a conversational manner and Airtable will be used to store the answers. Check out this post on using Airtable and Twilio for an introduction on how this works. Twilio Functions will let us return the right questions to be asked and connect Autopilot to the Airtable API to store the answers.
To follow along with this tutorial you will need:
  • A Twilio Account (if you don't have one yet,sign up for a free Twilio account here and receive $10 credit when you upgrade)
  • Node.js installed
  • A Facebook account with which you can create a Page
  • An Airtable account
  • ngrok so that you canrespond to webhooks in your local development environment
With those accounts to hand, let's get started.
Setting up Airtable to store the data
Let's get our database in shape first. Open your Airtable account and create a new base from scratch. Your new base will have one table with a few columns. We don't want to use those columns, but we can edit them into shape.
Start by renaming the table to "Users". We'll make the primary column the user's Facebook Messenger ID. Change the first column, the primary field, from "Name" to "ID". We still want the user's name, so change the "Notes" column from a "Long text" field to a "Single line text" field and change the column name to "Name". We need the user's email address so that we can contact them if they win, so change the third column to an "Email" field called "Email". Finally we want to know that the user agreed to the terms, so change the fourth column to a "Single line text" field and change the name to "Terms". Now we can add slots for the user's answers, add a "Single line text" field and call it "week1" then add as many more fields as you want to ask questions.
Once you're done with that, open up the Airtable API documentation and choose your base. At the top of the documentation will be the ID of your base. Open up another browser tab and head to your Airtable account. Here you can generate an API key. We'll need both the ID and the API key later.
Building the bot
Next up, we're going to create and train our Autopilot bot to ask the competition questions. Log in to the Twilio console and navigate to the Autopilot section. There are plenty of bot templates, but we'll start this bot from scratch, so click on Build from scratch. Give your bot a unique name, like ‘watch-and-win-bot’, and click Create bot.
When the bot is created you will see a number of tasks already set up for you: a greeting and goodbye that can be used at the start and end of conversations, and two fallback tasks. We will keep all of these tasks and add a couple more to ask the questions for our competition.
Let's update the greeting task, we want it to greet users and ask them whether they want to enter the competition. Click on Program next to the greeting task. You will find a JSON object that looks like this:
{ "actions": [ { "say": "Hello, what can I help you with today?" }, { "listen": true } ] } 
Autopilot tasks are trigger actions which are defined in JSON like the above. You can either define your actions statically, within the Autopilot interface, or you can trigger a webhook to your own application and respond dynamically. We are going to use both types of response in this bot. You can read more about actions in Autopilot in the documentation.
Update the "say" action to ask the user whether they want to enter the competition. Since we are asking a yes or no question, we can also restrict the tasks that can be triggered by the answer. Update the "listen" action to listen for the tasks "goodbye" and "enter_competition". Your actions JSON should look something like this:
{ "actions": [ { "say": "Hello, would you like to enter the competition?" }, { "listen": { "tasks": [ "enter_competition", "goodbye" ] } } ] } 
We now need a task that will start the competition questions. We want to trigger this task by either responding positively to a welcome message or by asking to enter the competition.
Click Add a task, name the new task enter_competition and click Add. Click on Train for your new task. Here is where you enter the phrases you want to trigger this task. The phrases should be in response to the greeting question, so enter some sample phrases that you expect your users will say, from simple, like "Yes", to a bit more complicated, like "I want to enter the competition". The more sample phrases you can think of, the more accurately the bot will be able to determine your users' intentions.
We'll update what the task does when it is triggered later. Save the task and click on Build model. This takes all the work we've just done and trains the bot to respond to any potential incoming phrase. Now open the Simulator from the navigation, here you can test that your bot is working as expected so far.
We need to have the bot return real questions and save them to our Airtable base, so we'll tackle that next using Twilio Functions.
Dynamic responses using Twilio Functions
Autopilot allows you to respond to tasks with static JSON, as we've seen so far, or by sending a webhook to a URL. Twilio Functions is an easy way to create endpoints that can respond to webhooks, hosted within the Twilio platform. We can develop our Twilio Functions locally using the Twilio CLI and Twilio Serverless Toolkit.
Follow the instructions to install the Twilio CLI and log in to your account and then install the Serverless Toolkit with:
twilio plugins:install @twilio-labs/plugin-serverless 
Generate a new Twilio Functions project with the following command:
twilio serverless:init competition-bot --empty 
The --empty flag means you don't get a bunch of example files. Once the project is created, open it in your favourite editor.

Responding to the bot with Functions

Let's start our work on these functions by responding to the bot with a "say" action. Create a new file called competition.protected.js in the functions directory of the project. (A function with .protected in the file name will only respond to requests that have a valid X-Twilio-Signature header.) Open competition.protected.js and enter the following code:
exports.handler = async (context, event, callback) => { const response = { actions: [{ say: "Hello from your function." }], }; callback(null, response); }; 
So far this function is returning static JSON like before, but this is just the start. Change the current directory to the root of the project, and run the functions project with the terminal command:
twilio serverless:start 
This will start your functions. We also need to make the functions available from a publicly accessible URL so that they can receive webhooks from Autopilot. Open up a new terminal window and start up ngrok with the following command.
ngrok http 3000 
The terminal will show you a URL that looks like https://RANDOM_SUBDOMAIN.ngrok.io. Copy this URL and return to your bot in the Twilio console. Go to the tasks, click on Program next to the enter_competition task. At the top you can choose between using an ActionBin or Actions URL, check Actions URL and enter https://RANDOM_SUBDOMAIN.ngrok.io/competition. Leave ngrok running as you build and test the rest of this application.
Save the model and build it again. Once the model is built, open the simulator again and start a conversation with the bot. This time when you say "Yes" to entering the competition, Twilio will make a request to your Functions project and the bot will respond with "Hello from your function.". You can also inspect the request with ngrok by opening localhost:4040 in your browser. In this request inspector you can see all the parameters that Twilio sends.
Now our bot is connected to our function we can get down to the work of creating a dynamic response.

Loading question and user data

We'll store our questions as a private asset in our Functions project. Create a file in the assets directory called questions.private.js. Each question will belong to a week, have some text for the question, three options and a time in the future that it is live until. This time will allow us to pick the currently active question.
Add the questions to questions.private.js like this:
module.exports = [ { week: 1, question: "Who is the CEO of Twilio?", options: { A: "Phil Nash 😁", B: "Jeff Lawson", C: "Chatri Sityodtong" }, liveUntil: new Date("2021-06-17T12:45:00Z") }, { week: 2, // Enter your own questions } ]; 
We are going to load and save user data in Airtable, so install the Airtable npm module with:
npm install airtable 
You collected your Airtable API key and base ID earlier. Add those to the .env file:
AIRTABLE_API_KEY=YOUR_API_KEY AIRTABLE_BASE_ID=YOUR_BASE_ID 
In this function we are going to find out if the user is already in our database and has answered this week's competition question. If they have already answered then we turn them away until the next episode has aired. If they are in the database but haven't answered we can greet them by name and ask them this week's question. If they aren't in the database then we greet them and ask them the current question, their name, their email address and whether they accept the terms.
Let's first load the questions and select the current one in competition.protected.js. In the Twilio Function environment we can require private assets into our function by retrieving its path from the Runtime.getAssets() function and then requiring it as normal. Once we have loaded the questions, we find the latest question. If there are no questions left, we can exit the entry and tell the user the competition is over.
const questions = require(Runtime.getAssets()["/questions.js"].path); exports.handler = async (context, event, callback) => { const now = new Date(); const questionDetails = questions.find((question) => { return now < question.liveUntil; }); if (!questionDetails) { return callback(null, { actions: [ { say: "This competition is now over. Thank you for playing!", }, ], }); } } 
Next we need to try to load the user details from the Airtable base. In the Autopilot webhook request there is a Memory parameter. This parameter contains a JSON string of details about the user we are interacting with as well as their responses to questions we ask and other arbitrary data that we tell the bot to remember. The Memory has a twilio property that stores details about the user and when we interact with the bot using the simulator the twilio object has a chat property. When we interact with the bot via Facebook Messenger, as we will later, the twilio object will have a messaging.facebook-messenger property. Both the chat and messaging.facebook-messenger objects have a From property that identifies the user, so we will use that as the ID.
 if (!questionDetails) { return callback(null, { actions: [ { say: "This competition is now over. Thank you for playing!", }, ], }); } const memory = JSON.parse(event.Memory); const user = memory.twilio.chat ? memory.twilio.chat : memory.twilio["messaging.facebook-messenger"]; const userId = user.From; 
We need to load any data we have about this user from Airtable. Require the Airtable library and load the base and the table using the API Key and Base ID you stored in the environment earlier.
const Airtable = require("airtable"); exports.handler = async (context, event, callback) => { // question loading const memory = JSON.parse(event.Memory); const user = memory.twilio.chat ? memory.twilio.chat : memory.twilio["messaging.facebook-messenger"]; const userId = user.From; const airtable = new Airtable({ apiKey: context.AIRTABLE_API_KEY }); const base = airtable.base(context.AIRTABLE_BASE_ID); const table = base.table("Users"); let users; try { users = await table .select({ maxRecords: 1, filterByFormula: `{ID} = '${userId}'`, }) .firstPage(); } catch (err) { users = []; } }; 
You can find rows in the Airtable table by ID, but we don't know that ID. Instead, we filter the results by the ID column, limit it to one record and load the first page. Now we know if we have the user details already if there is a record in the users array. There are three scenarios now, the user hasn't answered any questions before, they have answered other questions, but not the current one, or they have answered the current question. Based on this, we need to build up our JSON response to send back to Autopilot.
Let's add a response object that has an actions property which starts as an empty array. At the end of the function, we'll return this response to the callback function.
 } catch (err) { users = []; } const response = { actions: [], }; callback(null, response); }; 
Let's start by taking the case where we don't already have a user in our database. We want to welcome the new user and then ask them all the questions (this week's question, their name, email address and whether they accept the terms). We use the Remember action to remember the question week we are working with. We ask the questions using Autopilot's Collect action, which includes validations for some answers and uses built-in field types to recognise things like names, email addresses and yes/no answers. Note also that we name the questions the same as our Airtable column names, which will make it easy for us to insert them later.
 const response = { actions: [], }; if (users.length > 0) { // We'll come back to this later } else { response.actions.push({ remember: { week: questionDetails.week, }, }); response.actions.push({ say: "Thanks for taking part in the competition. Here's this week's question, plus a few more so we can get some important details from you. You must answer all questions to enter.", }); const questionText = `${questionDetails.question}\n${["A", "B", "C"] .map((letter) => `${letter}: ${questionDetails.options[letter]}`) .join("\n")}`; response.actions.push({ collect: { name: "watch_and_win", questions: [ { question: questionText, name: `week${questionDetails.week}`, validate: { allowed_values: { list: ["A", "B", "C"], }, on_failure: { messages: [ { say: `That's not one of the possible answers.\n\n${questionText}`, }, ], repeat_question: false, }, }, }, { question: "Great! What's your name?", name: "Name", validate: false, }, { question: "What is your email address? We need this to contact you if you win.", name: "Email", type: "Twilio.EMAIL", validate: { on_failure: { messages: [ { say: "Please enter a valid email address. We need this to contact you if you win.", }, ], repeat_question: false, }, }, }, { question: "Do you agree to the terms and conditions of the competition?", name: "Terms", type: "Twilio.YES_NO", validate: { allowed_values: { list: ["yes"], }, on_failure: { messages: [ { say: "You need to agree to the terms and conditions and the privacy policy to take part in the competition. Please type 'yes' to confirm that you agree.", }, ], repeat_question: false, }, }, }, ], on_complete: { redirect: { method: "POST", uri: `https://${context.DOMAIN_NAME}/answers`, }, }, }, }); } callback(null, response); }; 
Ok, what if we already have a user? We need to check whether they have already answered this week's question and if they have, tell them to come back next week. And if they haven't, we'll greet them by name, remember the ID of their user record and only ask the question. Since we have already built up a question object, we can refactor that out of the new user block and use it here too.
Here's the complete code:
const Airtable = require("airtable"); const questions = require(Runtime.getAssets()["/questions.js"].path); exports.handler = async (context, event, callback) => { const now = new Date(); const questionDetails = questions.find((question) => { return now < question.liveUntil; }); if (!questionDetails) { return callback(null, { actions: [ { say: "This competition is now over. Thank you for playing!", }, ], }); } const memory = JSON.parse(event.Memory); const user = memory.twilio.chat ? memory.twilio.chat : memory.twilio["messaging.facebook-messenger"]; const userId = user.From; const airtable = new Airtable({ apiKey: context.AIRTABLE_API_KEY }); const base = airtable.base(context.AIRTABLE_BASE_ID); const table = base.table("Users"); let users; try { users = await table .select({ maxRecords: 1, filterByFormula: `{ID} = '${userId}'`, }) .firstPage(); } catch (err) { users = []; } const response = { actions: [], }; const questionText = `${questionDetails.question}\n${["A", "B", "C"] .map((letter) => `${letter}: ${questionDetails.options[letter]}`) .join("\n")}`; const competitionQuestion = { question: questionText, name: `week${questionDetails.week}`, validate: { allowed_values: { list: ["A", "B", "C"], }, on_failure: { messages: [ { say: `That's not one of the possible answers.\n\n${questionText}`, }, ], repeat_question: false, }, }, }; if (users.length > 0) { const user = users[0]; const name = user.get("Name"); const answered = user.get(`week${questionDetails.week}`); if (answered) { response.actions.push({ say: `Hi ${name}, I see you have already entered this week's competition. Come back after the next episode for your next question.`, }); } else { response.actions.push({ remember: { userAirtableId: user.id, week: questionDetails.week, }, }); response.actions.push({ say: copy.existingPlayer(name), }); response.actions.push({ collect: { name: "watch_and_win", questions: [competitionQuestion], on_complete: { redirect: { method: "POST", uri: `https://${context.DOMAIN_NAME}/answers`, }, }, }, }); } } else { response.actions.push({ remember: { week: questionDetails.week, }, }); response.actions.push({ say: "Thanks for taking part in the competition. Here's this week's question, plus a few more so we can get some important details from you. You must answer all questions to enter.", }); response.actions.push({ collect: { name: "watch_and_win", questions: [ competitionQuestion, { question: "Great! What's your name?", name: "Name", validate: false, }, { question: "What is your email address? We need this to contact you if you win.", name: "email", type: "Twilio.EMAIL", validate: { on_failure: { messages: [ { say: "Please enter a valid email address. We need this to contact you if you win.", }, ], repeat_question: false, }, }, }, { question: "Do you agree to the terms and conditions of the competition?", name: "terms", type: "Twilio.YES_NO", validate: { allowed_values: { list: ["yes"], }, on_failure: { messages: [ { say: "You need to agree to the terms and conditions to take part in the competition. Please type 'yes' to confirm that you agree.", }, ], repeat_question: false, }, }, }, ], on_complete: { redirect: { method: "POST", uri: `https://${context.DOMAIN_NAME}/answers`, }, }, }, }); } callback(null, response); }; 
At the end of each of the Collect actions you see an on_complete block. When the user has finished answering all the questions successfully this action is evaluated. Our action will redirect the bot to a new URL, a function we are yet to write. This function will store the answers in Airtable and thank the user for entering.
Let's build that function now. Create a new file in the function directory called answers.protected.js and open it in your editor.
To start with we access our Airtable Users table, parse the Memory object and load our user identifier the same as the last function.
const Airtable = require("airtable"); exports.handler = async (context, event, callback) => { const airtable = new Airtable({ apiKey: context.AIRTABLE_API_KEY }); const base = airtable.base(context.AIRTABLE_BASE_ID); const table = base.table("Users"); const memory = JSON.parse(event.Memory); const user = memory.twilio.chat ? memory.twilio.chat : memory.twilio["messaging.facebook-messenger"]; const userId = user.From; 
This is some repetition, but there are only two functions and there's more overhead to refactoring this than just repeating ourselves twice. If we started another function which used the same functions it would become worth refactoring, but we will leave it for now.
We need to get the answers out of the memory object and into a format that we can pass to the Airtable API. The answers aren't an object of keys and strings, but keys and objects with more data about the answer. You can see some examples in the documentation here. This code formats the answers into a simple object:
const Airtable = require("airtable"); exports.handler = async (context, event, callback) => { const airtable = new Airtable({ apiKey: context.AIRTABLE_API_KEY }); const base = airtable.base(context.AIRTABLE_BASE_ID); const table = base.table("Users"); const memory = JSON.parse(event.Memory); const user = memory.twilio.chat ? memory.twilio.chat : memory.twilio["messaging.facebook-messenger"]; const userId = user.From; const answerFields = memory.twilio.collected_data.watch_and_win.answers; const userAnswers = Object.keys(answerFields).reduce((acc, field) => { acc[field] = answerFields[field].answer; return acc; }, {}); userAnswers["ID"] = userId; 
After formatting the data, all that's left to do is either update the existing user or create a new one in the Users table.
const Airtable = require("airtable"); exports.handler = async (context, event, callback) => { const airtable = new Airtable({ apiKey: context.AIRTABLE_API_KEY }); const base = airtable.base(context.AIRTABLE_BASE_ID); const table = base.table("Users"); const memory = JSON.parse(event.Memory); const user = memory.twilio.chat ? memory.twilio.chat : memory.twilio["messaging.facebook-messenger"]; const userId = user.From; const answerFields = memory.twilio.collected_data.watch_and_win.answers; const userAnswers = Object.keys(answerFields).reduce((acc, field) => { acc[field] = answerFields[field].answer; return acc; }, {}); userAnswers["ID"] = userId; const userAirtableId = memory.userAirtableId; if (userAirtableId) { await table.update(userAirtableId, userAnswers); } else { await table.create(userAnswers); } callback(null, { actions: [ { say: "Thanks for entering the competition, good luck!", }, ], }); } 
Before we test this out in the simulator, we need to make one change to the environment. Even though Twilio will be requesting the functions through the ngrok URL, the functions environment still considers its domain name, exposed as context.DOMAIN_NAME, to be localhost:3000. Add the following line to your .env file.
DOMAIN_NAME=YOUR_SUBDOMAIN.ngrok.io 
Restart the server then head back to the Autopilot simulator. Start a conversation with the bot and it should take you through all the competition questions. Try again after that and it will tell you that you have already entered. Check your Airtable base and you will find your answers.

Deploying the Functions

Head back to your code and remove DOMAIN_NAME from the .env file. On the command line, run:
twilio serverless:deploy 
Your Functions will be deployed to a new Twilio Runtime service and at the end you will be given a URL. In your Autopilot bot, update the URL for the enter_competition task to point to this new URL and build the model again.
We've written our bot, the way to handle storing and loading data from Airtable and deployed it all to the Twilio Runtime. The last thing to do is connect the bot to Facebook Messenger.
Connect the bot to Facebook Messenger
The best advice I have here is to follow the instructions in the documentation for connecting your Autopilot bot to Facebook Messenger. Once you've completed that, you will be able to talk to your bot through Messenger.
We built a competition bot
In this post we've seen how you can use Twilio Autopilot, Twilio Functions and Airtable to build a bot very similar to the bot behind the The Apprentice: ONE Championship Edition watch and win competition. The only thing left to do is decide on your winner!
All the code for this bot is available in this GitHub repo. The bot is deployed to Facebook Messenger, but you could also hook this up to an SMS number, a voice call, WhatsApp, or your own custom channel. There's more you can build with Autopilot and these channels, from the serious, like helping people get involved in local issues, to the silly, like making BuzzFeed style quiz bots.
I'm not sure this bot would win me a spot in The Apprentice final, but hopefully it's inspired you to build something. Are you building your own bots? Let me know by dropping me an email at [email protected] or shooting me a message on Twitter.