Thursday, 28 June, 2018 UTC
Summary
We’ve looked at how to choose cameras in JavaScript using the
mediaDevices API, particularly for the case of mobile devices with a front and back camera, and now it’s time to put that into practice in an application. In this post we will use what we learned to build a camera switcher into a video chat application using Twilio Video.Getting started
We’re going to build this off a slightly modified version of the Twilio Video quickstart application. In order to build this app you will need:
- Node.js (I’m using the current latest version; 10, but it works with older versions)
- ngrok so that you can visit your development application from your mobile device
- A Twilio account; you’ll need your account SID, available on your Twilio dashboard, and an API key and secret, which you can generate in your console
Clone the repo for this application, checking out the branch that we’ll use to build on. Then change into the directory and install the dependencies:
git clone https://github.com/philnash/mediadevices-camera-selection.git -b camera-selection-video-chat cd mediadevices-camera-selection npm install
If you built the application in the previous blog post you will need to get the
camera-selection-video-chat branch from my repo to continue.Load the application up with the credentials we gathered earlier. Copy the
.env.template file to .env and fill in the blanks with your account SID, API key and API secret. Start up the application and visit http://localhost:3000 to make sure it’s all working.npm start
Join a video room by entering the room name in the input and clicking “Join room”. Do this in another browser window and you will see yourself talking to yourself.
Now, we’re going to add a feature to allow us to change camera during the call. This is mostly useful for mobile devices which have a front and back camera, though can come in handy for any users with an external webcam too.
Showing the cameras
We’re going to use the code from the previous blog post that enumerates the video devices available and displays a drop down list that we can choose from.
First, let’s add the empty
<select> element to the page next to the Preview My Camera button. Open up video-chat/quickstart/public/index.html, find the <div> with id preview and add the <select> like this:<div id="preview"> <p class="instructions">Hello Beautiful</p> <div id="local-media"></div> <button id="button-preview">Preview My Camera</button> <select id="video-devices"></select> </div>
Now, open
quickstart/src/index.js and add the following function just before the roomJoined function:function gotDevices(mediaDevices) {
const select = document.getElementById('video-devices');
select.innerHTML = '';
select.appendChild(document.createElement('option'));
let count = 1;
mediaDevices.forEach(mediaDevice => {
if (mediaDevice.kind === 'videoinput') {
const option = document.createElement('option');
option.value = mediaDevice.deviceId;
const label = mediaDevice.label || `Camera ${count }`;
const textNode = document.createTextNode(label);
option.appendChild(textNode);
select.appendChild(option);
}
});
}
We’re going to call this function once a user joins a room. Find the
roomJoined function and add the following line to the top:function roomJoined(room) {
window.room = activeRoom = room;
navigator.mediaDevices.enumerateDevices().then(gotDevices);
// ...rest of the function
}
Restart and reload the application. Now when you join a room you will see the drop down list populated with the cameras on your device.
If you want to check this out on your mobile device with two cameras, I recommend running ngrok so you can access the code running on your laptop from your mobile device. Install ngrok, following the instructions on the site, then start it up to point to port 3000 on your localhost:
./ngrok http 3000
Enter the resulting ngrok URL into your mobile device and you will be able to interact with the site from your mobile device. (Sorry it doesn’t look great on a mobile browser!)
Switching camera
Now we have our choice of camera in the drop down box, we need to change the input camera when it is selected. First, after we call
enumerateDevices, add the code to listen to changes on the select element:function roomJoined(room) {
window.room = activeRoom = room;
navigator.mediaDevices.enumerateDevices().then(gotDevices);
const select = document.getElementById('video-devices');
select.addEventListener('change', updateVideoDevice);
Now we write the
updateVideoDevice function. Add it below the gotDevices function we already wrote. We start by getting the select element from the event and the local participant from the room variable. If the value of the select element is empty then we do nothing, otherwise we start to request access to the new video stream.function updateVideoDevice(event) {
const select = event.target;
const localParticipant = room.localParticipant;
if (select.value !== '') {
// get the video
}
}
We use the Twilio Video function
createLocalVideoTrack, passing the constraints for the video device we want. In this case we will use the exact device ID. Once we get access to the video stream we perform a number of tasks:- Unpublish the local participant’s existing video track from the room, this will trigger the
trackRemovedevent on theroomfor any other participants - Detach the existing video track from the page
- Attach the new video track to the page
- Publish the new video track to the room, triggering the
trackAddedevent on theroomfor the other participants
The code looks like this:
function updateVideoDevice(event) {
const select = event.target;
const localParticipant = room.localParticipant;
if (select.value !== '') {
Video.createLocalVideoTrack({
deviceId: { exact: select.value }
}).then(function(localVideoTrack) {
const tracks = Array.from(localParticipant.videoTracks.values());
localParticipant.unpublishTracks(tracks);
log(localParticipant.identity " removed track: " tracks[0].kind);
detachTracks(tracks);
localParticipant.publishTrack(localVideoTrack);
log(localParticipant.identity " added track: " localVideoTrack.kind);
const previewContainer = document.getElementById('local-media');
attachTracks([localVideoTrack], previewContainer);
});
}
}
We need to do one more bit of housekeeping here. Since this code only works during a call, we should stop listening to changes on the select element once the participant leaves the room.
In the room’s disconnected event, remove the event listener.
room.on('disconnected', function() {
log('Left');
if (previewTracks) {
previewTracks.forEach(function(track) {
track.stop();
});
}
detachParticipantTracks(room.localParticipant);
room.participants.forEach(detachParticipantTracks);
activeRoom = null;
document.getElementById('button-join').style.display = 'inline';
document.getElementById('button-leave').style.display = 'none';
select.removeEventListener('change', updateVideoDevice);
});
Reload the application and join a room. You will see the select element fill with your available devices and when you switch, your local preview and your remote stream will also switch.
Making video calls more useful
Giving your users the option to change camera can be useful in a number of ways; showing their surroundings or something specific from a rear camera and switching back to the front camera for face to face calls can make for a much better experience.
You can find all the code we wrote for this post in the GitHub repo here.
Is this something you can use for your video chat application? Or do you have any other ideas for it? Let me know in the comments below, on Twitter or by email at [email protected].
Switching cameras during a video chat with Twilio Video