By 2019, video traffic will be 82% of global consumer Internet traffic. In order to handle this intense load, the video codec wars between HEVC and AV1 rage on.
In particular, debates are becoming more intense on the importance of factors including open-source and royalty-free options. Open-source, royalty-free video codecs such as VP9 and AV1 are in direct competition with licensed video codecs like H.265. The competition is fierce enough that, with each new iteration of any codec, the community attempts to reduce the storage space needed for delivering the same video quality by half, at the same time increasing complexity (CPU cycles) by up to 40%.
Royalty-free and Royalty-bearing Codecs
Licensing of codecs is a hotly contested topic. This is especially true considering the codecs that are standardized as a group, where participating companies provide improvements to the codec that are endorsed by the community. There have been several organizations that have provided a framework for standardization, including ITU, ISO, and more recently the IETF (NETVC), and AOM (AV1).
Typically, depending on the standardizing organization, participating companies contribute their improvements and then create corresponding patent pools. The patent pools then license the codec. Alternatively, the organization can attempt to build a codec that actively avoids known patents or uses the Friendly and Non-discriminatory (FRAND) licensing model. Irrespective of how you build the codec (royalty-bearing or free), there are always a potential issue with an unknown patent holder. Royalty-free codecs usually use a process that attempts to minimize that possibility as much as is possible. For royalty-bearing codecs, the unknown patent holder might be a company that was not part of the consortium and independently identified a technique or algorithm that improves the compression performance of the codec. Therefore, it is still a better proposition to build on a confirmed royalty-free codec with known patent holders, as the unknown exist regardless.
In this article, we walk through the two major codec evolutions, namely, H.26x and the VPx, amongst others.
H.264 - 2003
H.264 is a widely used video codec and family of standards made up of multiple profiles. It is a block-oriented, motion-compensation-based video compression standard that is usually used for lossy compression. It supports resolutions up to 8192x4320 along with 8K UHD.
H.264 Release
H.264 was publicly released in 2003 by the ITU-T Video Coding Experts Group in conjunction with the ISO/IEC JTC1 Moving Picture Experts Group. Their goal was to create a video codec standard that provided high-quality video at a substantially lower bitrate than previous standards without increased design complexity. Additionally, they wanted to establish H.264 as a standard that could be used in many applications and scenarios across networks and systems.
H.264 Patents
H.264 is protected by a variety of patents owned by a number of different entities. The majority of these patents are handled and overseen by MPEG LA. While MPEG LA allows free access for internet video that is streamed freely to the end user, any commercial use requires payment of fees and royalties.
H.265 - 2013
H.265 is a video codec and potential successor to H.264. It is able to offer double the data compression ratio while maintaining the same level of video quality. Similar to H.264, it supports resolutions up to 8192x4320 along with 8K UHD.
H.265 Release
H.265 was publicly released in 2013 and created by the Joint Collaborative Team on Video Coding. It is considered an extension of many of the concepts in H.264.
H.265 Patents
H.265 uses technologies covered by several patents and requires a license for use. This is covered by at least 3 patent pools and multiple different terms and rates.
VP8 - 2008
VP8 is an open source and royalty-free video codec. It is a traditional block-based transform coding format similar to H.264.
VP8 Release
VP8 was publicly released in 2008 by On2 Technologies before On2 was acquired by Google in 2010. It was designed to be the successor to VP7. It was created for the internet age and specifically for mobile devices. As such, it offers high compression and low computational complexity. It is mainly used in connection with WebRTC, and is the video codec included with WebRTC.
VP8 Patents
As of May 2010, Google released a specification of the VP8 format under the Creative Commons Attribution 3.0 license. This, along with the release of libvpx under the revised BSD license, made VP8 open-source and royalty-free. The potential exists for unknown patent holders, but this potential exists regardless as stated in the introduction.
VP9 - 2009
VP9 is an open-source, royalty-free video codec. It is a traditional block-based transform coding format that enables lossless compression. VP9 offers better support for ultra-high definition (UHD), as it is specifically engineered for resolutions in excess of 1080p.
VP9 Release
VP9 was publicly released in 2009 by Google as the successor to VP8. VP9 was originally used almost exclusively on YouTube, though interest grew as the Alliance for Open Media gained popularity.
VP9 Patents
Although parts of the codec are covered by patents, anyone may use the codec freely as long as they do not engage in patent litigation. Because of its open nature, VP9 is an optional codec in the WebRTC spec. The potential exists for unknown patent holders, but this potential exists regardless as stated in the introduction.
Dirac - 2008
Dirac is an open-source, royalty-free video codec. It supports HDTV resolutions and greater, and purports to provide significant data rate and quality improvements over its competitors. Specifically, they claim it has “a two-fold reduction in bit rate over MPEG-2 for high definition video”.
Dirac Release
Dirac was publicly released in 2008 by BBC Research and Development with the goal of providing high-quality video compression for Ultra HDTV and above.
Dirac Patents
Though the BBC had previously applied for patents regarding Dirac, they let the applications lapse. No known patents are held by the BBC on Dirac, and developers have stated publicly that they aim to avoid infringing on any third party patents to empower public use. The potential exists for unknown patent holders, but this potential exists regardless as stated in the introduction.
AV1 - 2018
AV1 is a new open-source, royalty-free video codec. It is designed with a low computational footprint with the goal of bringing consistent, high-quality video. Independent member tests show that 4K UHD video is delivered at an average of 30% greater compression over competing codecs.
AV1 Release
AV1 was released on March 28, 2018 by the Alliance for Open Media. It’s main goal is to be the successor of VP9 and become the new, state-of-the-art, royalty-free video standard.
AV1 Patents
One of the main goals when developing AV1 was to ensure it was completely royalty-free. In order to achieve this, every single feature is independently double checked to ensure it does not infringe on patents of competing companies before adoption. This is a big deviation from other standards in the industry such as HEVC, where no review for intellectual property rights was included in the standardization process. The potential exists for unknown patent holders, but this potential exists regardless as stated in the introduction.
Summary
In the past, new versions of the standardized codec have come at 8-10 year intervals: H.263 (1995), H.264 (2003), H.265 (2013). In contrast, proprietary codecs have evolved faster at roughly 4 year intervals: VP7 (2005), VP8 (2008), VP9 (2012).
The Alliance for Open Media has dedicated significant time into the AV1 release in 2018, and consortium members are working towards releasing new versions of the codec at an expedited rate, targeting a new release every 2-3 years. This is in keeping with the demands of many emerging new use-cases (VR/AR), consumption of more video content (via streaming), and the desire for better quality video.
