Tuesday, 8 October, 2019 UTC
Summary
We are thrilled to announce that the Network Bandwidth Profile API is now available via Public Beta: a Programmable Video API designed to improve the quality of experience in Group Rooms. Before this release, video bandwidth was split equally between tracks, which meant that lower and higher priority tracks received the same treatment. Now with this release, developers can specify how the available network bandwidth is allocated, reallocate bandwidth to higher priority tracks, protect audio quality, and optimize battery and network resources consumption.
Why a Network Bandwidth Profile API? Programmable Video Group Rooms are based on an SFU (Selective Forwarding Unit) architecture. This means that participants publish audio and video as independent tracks to the SFU server, which in turn routes them to the rest. Hence, the number of subscribers tracks per participant grows as N-1 where N is the total number of participants.
The SFU can control the quality of the forwarded track. For this, it needs to determine what bandwidth to allocate to each track: the higher the bandwidth the higher the fidelity. However, this is a non trivial process that requires the following considerations:
- The total bandwidth must not exceed the network capacity. Otherwise, congestion will emerge.
- The bandwidth allocation must align with the end-user UI layout. For example, a track rendered using a larger UI area such as a screen share, should be assigned more bandwidth than a smaller one such as a thumbnail video.
- The bandwidth allocation must adapt to dynamically changing conditions. For example, if a participant becomes dominant speaker at a given time, her video track should also be assigned a higher relevance at that time.
- To allow developers to limit the number of displayed video tracks. For example, in a Room with 30 participants, you may want to display the 5 most relevant participants and save the bandwidth of the other 25.
- To preserve audio quality in case of severe congestion. This may require switching-off the less relevant video tracks.
- To allow developers to set the maximum bandwidth a participant may consume. This may be helpful for controlling costs and battery consumption.
A Network Bandwidth Profile is a specification defining how the available downstream bandwidth of a participant should be allocated. The Network Bandwidth Profile does not define which tracks a participant subscribes to (that’s the task of the Track Subscription API) rather, it controls the bandwidth allocation (and quality) of the subscribed tracks.
Before working with Network Bandwidth Profiles we need to understand two new concepts:
- Track Priorities: The track priority indicates the relevance of a track. Using the Track Priority API, developers can set priorities at track publishing time. Twilio Video supports three tracks priorities:
low,standard, andhigh. - Track Switch-off: Sometimes the available bandwidth is so low that it isn’t possible to relay all the tracks without generating congestion. In those situations, the Twilio SFU will switch-off the lower priority video tracks. This means that the SFU will stop forwarding traffic for these tracks, thereby reducing their bandwidth consumption to zero and allowing to preserve higher priority tracks quality.
A Network Bandwidth Profile is a JSON object containing the following properties:
mode: specifies the algorithm that must be used for the bandwidth allocation. Three values are supported:grid,collaboration, andpresentation.maxSubscriptionBitrate: specifies the maximum video bitrate the participant may consume in their network downlink.dominantSpeakerPriority: specifies the priority of the video tracks published by the dominant speaker. If a participant becomes dominant speaker, then all her tracks are upgraded to this priority.maxTracks: Specifies the maximum number of visible video tracks. Only the N most relevant tracks are kept visible. Non visible tracks are switched-off. Tracks with higher priority are considered more relevant. Within a given priority, tracks with higher speaking activity are considered more relevant.renderDimensions: specifies the desired UI display dimensions (in pixels) used to render video tracks.
To use the Network Bandwidth Profile API you need first to determine what are your higher priority video tracks. Note: audio tracks are always considered as a top priority and that cannot be changed. For example, in a collaboration application you may wish to set the screen share track a
high priority. This is done at track publication time, as the following code snippet illustrates:JavaScript SDK (Required v2.0.0-beta14+)
//You can set the track priority at publish time const localTrackPublication = await room.localParticipant.publishTrack(localTrack, { priority: 'high' //choose among 'high', 'standard' or 'low' }); If you don’t specify any priority, video tracks are published with priority
standard.Next you have to define your Network Bandwidth Profile. For this, the first step is to select the
mode. The following guidelines tend to be useful for that purpose:- If your application renders the video tracks with equal size, without having one of them more relevant than the rest, you likely need to use
grid. - If your application needs to enhance some specific tracks, but it’s important to have less relevant tracks visible, then select
collaboration. - If your application has a critical track that must be preserved by all means, then your mode is
presentation.
After that, you just need to select the rest of parameters that best fit into your use-case requirements. Check our official documentation for further guidance on how to do it.
Once you have decided your settings, you can define a Network Bandwidth Profile at connect using the following code:
JavaScript SDK (Required v2.0.0-beta14+)
//Specifying a Network Bandwidth Profile at connect time const { connect } = require('twilio-video'); const room = await connect(token, { bandwidthProfile: { video: { mode: 'collaboration', maxSubscriptionBitrate: 2400000, dominantSpeakerPriority: 'high', maxTracks: 5, renderDimensions: { high: {width: 1080, height: 720}, standard: {width: 640, height: 480}, low: {width: 320, height: 240} } } }); We want your feedback
The Network Bandwidth Profile API is essential for creating high quality Twilio Programmable Video applications. This is why we consider this API as a critical feature of our platform.
To learn more, check out the docs. Questions or feedback? Don’t hesitate to reach out to us.
We can’t wait to see what you build!