git
client installed, you can download it as follows:$ git clone https://github.com/miguelgrinberg/flask-twilio-video
master
branch in this repository already includes all the code to support the chat room feature. If you plan on coding along with this tutorial, then switch to the without-chat
branch using the following command:$ git checkout without-chat
git
client installed you can also download the complete application as a zip file. Or if you are intending to code along with the tutorial, then just the video calling portion.$ python -m venv venv $ source venv/bin/activate (venv) $ pip install -r requirements.txt
$ python -m venv venv $ venv\Scripts\activate (venv) $ pip install -r requirements.txt
pip
, the Python package installer, to install the Python packages used by this application. These packages are:TWILIO_ACCOUNT_SID="<enter your Twilio account SID here>" TWILIO_API_KEY_SID="<enter your Twilio API key here>" TWILIO_API_KEY_SECRET="<enter your Twilio API secret here>"
(venv) $ flask run * Environment: development * Debug mode: on * Running on http://127.0.0.1:5000/ (Press CTRL+C to quit) * Restarting with stat * Debugger is active! * Debugger PIN: 274-913-316
source venv/bin/activate
or venv\Scripts\activate
depending on your operating system) and then enter the following command:(venv) $ ngrok http 5000
Client
class. In the app.py file, add the twilio_client
right after the authentication credentials are loaded from the environment:from twilio.rest import Client load_dotenv() twilio_account_sid = os.environ.get('TWILIO_ACCOUNT_SID') twilio_api_key_sid = os.environ.get('TWILIO_API_KEY_SID') twilio_api_key_secret = os.environ.get('TWILIO_API_KEY_SECRET') twilio_client = Client(twilio_api_key_sid, twilio_api_key_secret, twilio_account_sid) app = Flask(__name__)
get_chatroom()
takes a chat room name and checks if a conversation resource with that name already exists. If one does not exist, then it creates it. It returns the conversation resource to the caller. To implement this function, add the following code to the app.py file.from twilio.base.exceptions import TwilioRestException # ... def get_chatroom(name): for conversation in twilio_client.conversations.conversations.list(): if conversation.friendly_name == name: return conversation # a conversation with the given name does not exist ==> create a new one return twilio_client.conversations.conversations.create( friendly_name=name)
from twilio.jwt.access_token.grants import ChatGrant from twilio.base.exceptions import TwilioRestException # ... @app.route('/login', methods=['POST']) def login(): username = request.get_json(force=True).get('username') if not username: abort(401) conversation = get_chatroom('My Room') try: conversation.participants.create(identity=username) except TwilioRestException as exc: # do not error if the user is already in the conversation if exc.status != 409: raise token = AccessToken(twilio_account_sid, twilio_api_key_sid, twilio_api_key_secret, identity=username) token.add_grant(VideoGrant(room='My Room')) token.add_grant(ChatGrant(service_sid=conversation.chat_service_sid)) return {'token': token.to_jwt().decode(), 'conversation_sid': conversation.sid}
get_chatroom()
auxiliary function defined above to get a chat room with the name “My Room”, which is also the name that we are using for the video room. In a more advanced application with support for multiple video rooms, you could also create multiple chat rooms to match.conversation.participants.create()
method is used to add the user to the chat room. For identification purposes we are using the username as the identity
value, exactly as we are doing for the video.<!doctype html> <html> <head> <link rel="stylesheet" type="text/css" href="{{ url_for('static', filename='styles.css') }}"> </head> <body> <h1>Flask & Twilio Video Conference</h1> <form> <label for="username">Name: </label> <input type="text" name="username" id="username"> <button id="join_leave">Join call</button> <button id="share_screen" disabled>Share screen</button> <button id="toggle_chat" disabled>Toggle chat</button> </form> <p id="count">Disconnected.</p> <div id="root"> <div id="container" class="container"> <div id="local" class="participant"><div></div><div class="label">Me</div></div> <!-- more participants will be added dynamically here --> </div> <div id="chat"> <div id="chat-scroll"> <div id="chat-content"> <!-- chat content will be added dynamically here --> </div> </div> <input id="chat-input" type="text"> </div> </div> <script src="https://media.twiliocdn.com/sdk/js/video/releases/2.3.0/twilio-video.min.js"></script> <script src="https://media.twiliocdn.com/sdk/js/conversations/releases/1.0.0/twilio-conversations.min.js"></script> <script src="{{ url_for('static', filename='app.js') }}"></script> </body> </html>
html, body { height: 100%; display: flex; flex-direction: column; } #root:not(.withChat) { display: block; width: 100%; height: 100%; margin-top: 20px; } #root.withChat { display: grid; grid-template-columns: 75% 25%; height: 100%; margin-top: 20px; } /* video section */ .container { width: calc(100% - 5px); height: 100%; padding-right: 5px; display: flex; flex-wrap: wrap; align-content: flex-start; } .participant { margin-bottom: 10px; margin-right: 5px; display: grid; grid-template-rows: auto 20px; } .participant div { text-align: center; } .participant div video { background-color: #eee; border: 1px solid black; } .participant div video:not(.trackZoomed) { width: 240px; height: 180px; } .participant .label { background-color: #ddd; padding: 2px; } .participantZoomed { width: 100%; height: calc(100% - 5px); grid-template-rows: auto 30px; } .participantHidden { display: none; } .trackZoomed { width: 100%; height: 100%; } .participantZoomed div video:not(.trackZoomed) { display: none; } .participantHidden div video { display: none; } .participantHidden .label { display: none; } .participantZoomed .label { margin-top: 8px; } /* chat section */ #root.withChat #chat { width: calc(100% - 10px); display: grid; grid-template-rows: auto 30px; border-left: 1px solid black; padding: 5px; } #root:not(withChat) #chat { display: none; } #chat #chat-scroll { overflow: auto; } #chat #chat-content { margin-top: 10px; margin-bottom: 10px; line-height: 0.5em; max-height: 1px; }
const root = document.getElementById('root'); const usernameInput = document.getElementById('username'); const button = document.getElementById('join_leave'); const shareScreen = document.getElementById('share_screen'); const toggleChat = document.getElementById('toggle_chat'); const container = document.getElementById('container'); const count = document.getElementById('count'); const chatScroll = document.getElementById('chat-scroll'); const chatContent = document.getElementById('chat-content'); const chatInput = document.getElementById('chat-input'); let connected = false; let room; let chat; let conv; let screenTrack;
connectChat()
function below accomplishes this by using the access token (which as you recall now contains grants for both video and chat) and the sid
identifier for the conversation to make a connection to the Conversations service. Add this function to the bottom of static/app.js:function connectChat(token, conversationSid) { return Twilio.Conversations.Client.create(token).then(_chat => { chat = _chat; return chat.getConversationBySid(conversationSid).then((_conv) => { conv = _conv; conv.on('messageAdded', (message) => { addMessageToChat(message.author, message.body); }); return conv.getMessages().then((messages) => { chatContent.innerHTML = ''; for (let i = 0; i < messages.items.length; i++) { addMessageToChat(messages.items[i].author, messages.items[i].body); } toggleChat.disabled = false; }); }); }).catch(e => { console.log(e); }); };
chat
global variable. With the client, it then retrieves the conversation object represented by the conversationSid
argument. Recall that the Flask server’s /login route is now returning this value along with the token.conv
global variable so that it can be used later when posting messages. Before returning, the function adds an event handler for the messageAdded
event, which will fire whenever a participant posts a message. It also calls the getMessages()
method to get a list of past messages for the room, which are then added to the chat panel with the addMessageToChat()
auxiliary function. We will write this addMessageToChat()
function in the next section. Note that getMessages()
returns a single page with the most recent messages.connectChat()
function can be invoked as part of the overall connection process, so it can be called from the connect()
function that runs when the user clicks the connection button. Below is the updated version of this function, with the changes highlighted:function connect(username) { let promise = new Promise((resolve, reject) => { // get a token from the back end let data; fetch('/login', { method: 'POST', body: JSON.stringify({'username': username}) }).then(res => res.json()).then(_data => { // join video call data = _data; return Twilio.Video.connect(data.token); }).then(_room => { room = _room; room.participants.forEach(participantConnected); room.on('participantConnected', participantConnected); room.on('participantDisconnected', participantDisconnected); connected = true; updateParticipantCount(); connectChat(data.token, data.conversation_sid); resolve(); }).catch(e => { console.log(e); reject(); }); }); return promise; };
connectChat()
but we do not have a handler for when this function completes. The idea is that as long as we have a video connection we are going to proceed. The connection to the chat room is going to run in the background and will enable the “Toggle Chat” button when it succeeds. If the connection were to fail, then the chat button will remain disabled, but the video call will proceed normally.addMessageToChat()
function to add it to the chat panel. Add this function to static/app.js:function addMessageToChat(user, message) { chatContent.innerHTML += `<p><b>${user}</b>: ${message}`; chatScroll.scrollTop = chatScroll.scrollHeight; }
<p>
element with the new message at the end of the chat log. The data is formatted with the username in bold, followed by the message in normal font. Since the chat log can get long, every time we append a new element we automatically scroll the container <div>
element to the bottom.withChat
CSS class. Add this function to static/app.js:function toggleChatHandler() { event.preventDefault(); if (root.classList.contains('withChat')) { root.classList.remove('withChat'); } else { root.classList.add('withChat'); chatScroll.scrollTop = chatScroll.scrollHeight; } }; toggleChat.addEventListener('click', toggleChatHandler);
Enter
key is pressed on this field, we want to capture the value of the text field and send it to the chat room. Add this function to static/app.js:function onChatInputKey(ev) { if (ev.keyCode == 13) { conv.sendMessage(chatInput.value); chatInput.value = ''; } }; chatInput.addEventListener('keyup', onChatInputKey);
keyup
event handler on the input field, and checking if it was the Enter
key that was pressed. Earlier, when the connection was established we stored the conversation object in the conv
global object. Now we can call the conv.sendMessage()
method to submit the message. The Twilio Conversations client will automatically fire a messageAdded
event in all active clients (including the one sending the message), so the message will immediately appear in the chat log for all the participants.disconnect()
function, with the changes to disconnect from Conversations highlighted:function disconnect() { room.disconnect(); if (chat) { chat.shutdown().then(() => { conv = null; chat = null; }); } while (container.lastChild.id != 'local') container.removeChild(container.lastChild); button.innerHTML = 'Join call'; if (root.classList.contains('withChat')) { root.classList.remove('withChat'); } toggleChat.disabled = true; connected = false; updateParticipantCount(); };