April 18, 2016 10:00 am

WebM, VP9 and Opus Support in Microsoft Edge

By

Starting with EdgeHTML 14.14291, the open-source WebM container format and the VP9 video and Opus audio codecs are supported in Microsoft Edge. These are available to websites that use Media Source Extensions (MSE) to adaptively stream video content. Windows Web Apps (built on the same Edge APIs) will also be able to use WebM streams containing VP9 and Opus. This change will be available in stable releases starting with the Windows 10 Anniversary Update.

VP9 is an efficient open source video codec developed as part of the WebM Project by Google. The VP9 and WebM project team recently joined the Alliance for Open Media effort where their work on VP9 is expected to be a significant contribution to the next generation of open-source, royalty-free video codecs.

Opus was developed by the Xiph Foundation to support both bandwidth-efficient speech/RTC applications as well as high-quality media playback – all within one codec. Opus is the primary audio codec paired with VP9 in the WebM specification.

We are adding support for these formats to simplify building interoperable websites, and to provide an alternative format for compressing 4K video content. To try them in Microsoft Edge, please install the most recent Windows 10 Insider Build.

Background

Microsoft has been providing support for Media Source Extensions (MSE) since Windows 8.1, initially through support for ISO-Base Media File Format (ISO-BMFF) content that uses fragmented MP4 containers with H.264/AVC video and AAC audio. More recently, we added support within MP4 for H.265/HEVC video and Dolby Digital multi-channel audio, formats targeted at high-performance media.

The WebM Byte Stream Format has existed for some time. It defines the specifics of using VP9 and Opus with MSE. VP9 is an open source video codec that has efficient compression well suited to HD and 4K streaming. Opus, similarly, is a very efficient open source audio codec. They are used together by some prominent websites – notably YouTube – that value them for this efficiency, especially on content with greater than 1080p resolution. Adding WebM, VP9 and Opus to Microsoft Edge means the full experience on these websites will be possible in the Edge browser.

VP9 and Opus Performance

Our implementation of VP9 will support software decoding and, when supported by the device, hardware decoding via the DXVA interface. Video decoding is computationally complex, so the best experience will be seen on computers that have VP9 hardware acceleration which is optimized for 4K decoding support, a feature which will start to become common on devices later this year. Software decoding can yield good performance, but will work best on more powerful desktop or laptop computers.

There’s a side effect of high computing load on software decode:  system power consumption is elevated. The elevation is highest on 4K content because the computing load correlates closely with video resolution. If we compare software decode to hardware accelerated decode, software decode may consume 20% more total power when viewing a 1080p video, and up to 80% more power viewing 2160p. The following charts compare hardware video decode to software video decode at different resolutions on a mid-level laptop computer:

Bar chart comparing software decode power consumption versus hardware decode power consumption. Software decode uses 80% more power at 2k resolutions, and 20% more power at 1080p resolutions.

This difference in power consumption is important because hardware acceleration is common for H.264, but not yet for VP9. If websites see VP9 is supported, they may prioritize using it and users will experience an unexplained reduction in battery life.

Bar chart showing extrapolated battery life estimates based on power consumption. When playing back 2k video, software accelerated playback could result in up to four hours less battery life.

Ideally, there would be a way for apps to confirm hardware acceleration itself is supported. Current web specifications don’t allow for this. For this reason, we’ve put VP9 behind an experimental flag in Microsoft Edge, and have provided a default setting for it that automatically enables VP9 when hardware acceleration is detected. VP9 is not supported on Windows mobile SKUs at this time.

Audio codecs, Opus included, don’t have the same power concern and are usually decoded in software. We have, however, elected to put Opus behind an experimental flag as well. Opus is supported on Windows mobile and desktop SKUs.

Managing Experimental Flags

You can manage the settings for VP9 and Opus by navigating to about:flags in the address bar.

Screen capture showing the Media Source Extensions section of the about:flags interface in Microsoft Edge.

Flags may be used to change the default settings for VP9 and Opus. These default settings are:

  • Opus: Enabled
  • VP9: Automatic

The “Automatic” setting for VP9 enables VP9 if hardware acceleration is supported on the device. We did this for the power reason previously discussed as well as the potential on less powerful computers to experience frame drops when doing software decode on higher resolution streams. The full set of VP9 flag settings and their meaning are:

  • Automatic: VP9 is enabled if hardware-accelerated VP9 decoding is supported
  • Enabled: VP9 is always enabled and available for use
  • Disabled: VP9 is always disabled

These settings allow users to have VP9 always enabled or disabled, in addition to our default setting. We encourage experimenting with these settings and look forward to your feedback.

The “Enabled” default setting for Opus means the Opus codec will be enabled and available for website use, and will show up in the MSE MediaSource.isTypeSupported capabilities query. It’s possible to disable Opus using this flag, which would make it unavailable for playback and show as unsupported via isTypeSupported().

VP8 video and Opus audio are important formats for WebRTC support, and have different considerations regarding hardware acceleration. The Edge WebRTC implementation supports VP8 and Opus regardless of the flag settings. See Roadmap update for Real Time Communications in Microsoft Edge for more information.

Integration

VP9 and Opus support can be queried by websites. With VP9 and Opus enabled, the following Media Source Extension capability checks will return “true”:

  • isTypeSupported ('video/webm; codecs="vp9"')
  • isTypeSupported ('video/webm; codecs="vp9, opus"')
  • isTypeSupported ('video/webm; codecs="opus"')
  • isTypeSupported ('audio/webm; codecs="opus"')

And these will return “false”:

  • isTypeSupported ('video/webm')
  • isTypeSupported ('audio/webm')

If VP9 is on “Automatic”, it will show up as supported on systems that have VP9 hardware acceleration, and won’t on systems that don’t.

MediaElement.canPlayType() is used to check capabilities for file-based playback. Edge will respond to any WebM canPlayType() calls with an empty string (the defined response for “not supported”) since we are only supporting WebM on MSE right now.

Microsoft has an ongoing effort to expand codec offerings in Windows. WebM, VP9, and Opus are our latest format additions. We continue to evaluate other formats and look forward to receiving feedback as we work on implementing them.

Please send us your feedback so we can further improve our media support in Microsoft Edge!

– Andy Glass, Principal Program Manager, Silicon, Graphics & Media (VP9 video)
– Neil Hinnant, Program Manager, Silicon, Graphics & Media (Opus audio)
– Jerry Smith, Senior Program Manager, Microsoft Edge (Media)

Updated April 18, 2016 9:32 pm

Join the conversation

      • Have you read the post?
        Media elements (video and audio elements) do not support VP9 and Opus at the moment. They are only supported in Media Source Extensions.

        • Of course I read the post. That’s why I’m complaining.

          I didn’t see any mention about implementing support for this without using MSE in the future.

          They only site I know that seems to use MSE is YouTube. While that is good, there is a lot of content that doesn’t use MSE, and the current implementation doesn’t support it.

        • Thanks, that worked. But I would still like to see this implemented natively in the video tag.

          This would make Edge able to play content that other browsers can play without having to write 500 lines of JavaScript code instead of one line of HTML.

          Adaptive playback is very nice, but not every web developer will bother to implement that.

  2. Thanks for the update! At Wikipedia we’re very interested in this, as due to concerns over patent licensing we do not produce media output in .mp4 or .mp3 formats.

    We currently handle media playback in Edge, IE 10/11, and Safari through a JavaScript decoder, currently decoding Ogg Theora video and Vorbis audio on the CPU while we continue to improve WebM support. This works better than one would imagine but means lower resolutions and higher CPU usage than native playback.

    I’ve done some initial testing with VP9/Opus via MSE on Edge 14, and it’s working well — eventually we’ll move to adaptive streaming with a DASH manifest and this should work well on Edge, as long as VP9 is actually enabled.

    Most importantly, we need to be able to support either VP8 or VP9 software decoding on systems without HW acceleration for them — including on mobile! — as our alternative is not accelerated H.264, but JavaScript decoding of Theora or VP8. Native software decoding will always have less overhead than anything we can rig up through JS or WASM.

    Thanks again, and let me know if there’s anything I can do to help you folks understand our use case for Wikipedia and Wikimedia Commons.

  3. Any chance WebM support will make its way to the media controls in C#/XAML UWPs? Or is my best bet to try and host a web frame within my C# app?

      • Checked on Lumia 950 running 14322 as well… for Xaml MediaElement control, in .webm file renamed to .mkv, Opus audio is working on both desktop and mobile. VP8 and VP9 video both work on desktop, but not on mobile (as noted in the post, the codec’s just not included yet on mobile). Vorbis isn’t yet supported at all.

        Note that most WebM files in the wild will probably contain VP8 video and Vorbis audio.

  4. > Ideally, there would be a way for apps to confirm hardware acceleration itself is supported. Current web specifications don’t allow for this.

    Not a big fan of non-standard extensions, but this is one of those cases where I think it makes sense … at least until a standard is pushed through. There are users that can’t pay the licensing fee and situations in which battery life isn’t an issue. You are also messing with the semantics of the API.

    • A large number of commenters to the ArsTechnica article covering this blogpost are complaining about YouTubes’s use of VP9 by default. Normally I’m a FOSS fanboy, but I really think the standard needs to be extended a bit so that browsers and service providers can intelligently pick the right codec use. This is a complex issue, but maybe in the short-term you could add a second parameter to `isTypeSupported(“codecInfo”, “auto|hw|any”)` and base the “auto” decision on whether there is GPU acceleration, mobile, plugged in, etc. That way Wikipedia and others who can’t afford to pay royalties can still use native decoding support without degrading the experience on sites that already use h.264.

  5. Support “Ogg Opus” without ogg? You kidding me? Maybe it’s first step but it is not enough.
    And what plans do you have to video/audio tags?

    Thanks.

  6. Question: Are those CODECs part of the system and by that I mean if an application uses Media Foundation that these CODECs will be supported out of the box? For example, if Groove is dependent on Media Foundation then does that mean that Groove will be able to playback Opus files? can Movies and TV playback VP9/VP8 files? Long story short, are those CODECs part of the Media Foundation framework or are they strictly bundled within Edge and can only be used by Edge?

  8. The Anniversary update was released. I have the Opus audio format enabled and VP9 always on. Even so, the feature detection on http://html5test.com/ still reports no support when the following are used.

    MediaSource.isTypeSupported(‘video/webm; codecs=”vp9″‘) at engine.js:2791
    MediaSource.isTypeSupported(‘audio/webm; codecs=”opus”‘) at engine.js:2867

    It has been some time since this blog post. What is the actual status of the functionality?

  10. doesn’t work, even after the anniversary update. love this browser, but until this is fixed it’s unusable for me.