Michael Bebenita, a Research Engineer at Mozilla, recently posted a fascinating article on the development of AV1, a next-generation video codec. If you’re interested in how new media formats are created, I highly recommend reading the full article.

What caught my eye is the discussion of porting the AV1 bitstream analyzer to the Web:

The input to the analyzer is usually small (an encoded bitstream), but the output is very large. […] The ideal solution is to run the analyzer directly in the browser and thus eliminate the need to download analyzer output.

But how do you do this when the codec is written in C? One option is to manually re-implement it in JavaScript and hope you can keep up with changes in the reference implementation. The other better option is to directly re-use the C and compile it to the Web. That’s exactly what the AV1 team is doing, with the help of Emscripten.

Emscripten compiles arbitrary C/C++ into JavaScript, which makes it possible for the AV1 team to automatically compile each revision of their codec to JavaScript and post it to the Web. At which point, comparing two revisions of the codec is as easy as sharing a link.

This workflow is fast enough, but it’s not as fast as it could be. Because JavaScript does not support 64-bit integers, many of AV1’s computations have to undergo costly numeric conversions. Specifically, emulating 64-bit math is estimated to add up to a 20% performance penalty to AV1, and we’ve seen performance penalties of up to 600% in other projects that are directly attributable to this emulation.

That’s where WebAssembly comes in. WebAssembly is a new, low-level format for programs on the Web. It’s an open standard being developed by Mozilla, Microsoft, Google, and Apple, so it will eventually work in all browsers. Crucially, Bebenita explains, “WebAssembly has support for 64 bit math, and once that’s ready [the AV1 Bitstream Analyzer will] be switching over to WebAssembly.”

Fortunately, Emscripten already has experimental support for compiling to WebAssembly, so the AV1 workflow will remain the same: develop a single codebase in C and use Emscripten to compile that to the Web for testing. In this way, WebAssembly will play an integral role in the development of next generation video codecs.

More importantly, this workflow represents a fundamental shift in Web development: The wall between “native” and the Web is falling, and developers will be able to seamlessly use the same libraries in both contexts. This marks the end of tedious, manual ports of projects to JavaScript and opens the door to dramatically greater performance on the Web.