High Efficiency Video Coding(HEVC) Algorithms and Architectures

Vivienne sze Madhukar budagavi Gary J. Sullivan Editors High Efficiency Video Coding(hevc) Algorithms and architectures 2 Springer editors Vivienne sze Madhukar budagavi Department of Electrical Engineering Texas Instruments inc and Computer Science Dallas TX USA Massachusetts Institute of Technology Cambridge MA USA Sullivan soft Corp Redmond. WA. USa ISSN1558-9412 ISBN978-3-319-06894-7 ISBN978-3-319-06895-4( e Book) DOI10.1007978-3-31906895-4 Springer Cham Heidelberg New York Dordrecht London Library of Congress Control Number: 2014930758 o Springer International Publishing Switzerland 2014 This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation broadcasting, reproduction on microfilms or in any other physical way, and transmission or information storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology now known or hereafter developed. Exempted from this legal reservation are brief excerpts in connection ith reviews or scholarly analysis or material supplied specifically for the purpose of being entered and executed on a computer system, for exclusive use by the purchaser of the work. Duplication of this publication or parts thereof is permitted only under the provisions of the Copyright Law of the Publishers location, in its current version, and permission for use must always be obtained from Springer Permissions for use may be obtained through Rightslink at the Copyright Clearance Center. Violations are liable to prosecution under the respective Copyright Lay The use of general descriptive names, registered names, trademarks, service marks, etc in this publication does not imply, even in the absence of a specific statement, that such names are exempt from the relevant protective laws and regulations and therefore free for general use While the advice and information in this book are believed to be true and accurate at the date of publication, neither the authors nor the editors nor the publisher can accept any legal responsibility for any errors or omissions that may be made. The publisher makes no warranty, express or implied, with respect to the material contained herein Printed on acid-free paper SpringerispartofSpringerScience+businessMedia(www.springer.com) Preface Advances in video compression, which have enabled us to squeeze more pixels through bandwidth-limited channels, have been critical in the rapid growth of video usage. As we continue to push for higher coding efficiency, higher resolution and more sophisticated multimedia applications, the required number of computations per pixel and the pixel processing rate will grow exponentially. The High Efficiency Video Coding(HEVC standard, which was completed in January 2013, was developed to address these challenges. In addition to delivering improved coding efficiency relative to the previous video coding standards, such as H.264/AVC, implementation-friendly features were incorporated into the HEVC standard to address the power and throughput requirements for many of todays and tomorrows video applications This book is intended for readers who are generally familiar with video coding concepts and are interested in learning about the features in HEVC (especially in comparison to H.264/MPEG-4 AVC). It is meant to serve as a companion to the formal text specification and reference software. In addition to providing a detailed explanation of the standard, this book also gives insight into the development of various tools, and the trade-offs that were considered during the design process Accordingly, many of the contributing authors are leading experts who were directly and deeply involved in the development of the standard itself. As both algorithms and architectures were considered in the development of the HEVC, this aims to provide insight from both fronts. The first nine chapters of the book focus on the algorithms for the various tools in hevc. and the techniques that were used to achieve its improved coding efficiency The last two chapters address the hevc tools from an architectural perspective and discuss the implementation considerations for building hardware to support HEVC encoding and decoding In addition to reviews from contributing authors, we would also like to thank the various external reviewers for their valuable feedback, which has helped improve the clarity and technical accuracy of the book. These reviewers include Yu-Hsin Chen Presa Chih-Chi Cheng, Keiichi Chono, Luis Fernandez, Daniel Finchelstein, Hun-Seok Kim, Hyungjoon Kim, Yasutomo Matsuba, Akira Osamoto, Rahul Rithe, Mahmut Sinangil, Hideo Tamama, Ye-Kui Wang and minhua Zhou Cambridge, MA, USA Vivienne sze Dallas. TX. USA Madhukar budagavi Redmond. WA. USa Gary Sullivan About the editors Vivienne sze is an assistant Professor at the Massachusetts Institute of Technology (MIT) in the Electrical Engineering and Computer Science Department. Her research interests include energy-aware signal processing algorithms, and low power circuit and system design for portable multimedia applications. Prior to joining Mit, she was with the r&d Center at Texas Instruments(TD), where sh represented Ti in the JCT-VC committee of ITU-T and Iso/EC standards bod during the development of HEVC(ITU-T H 265 ISO/IEC 23008-2) Within the committee, she was the primary coordinator of the core experiments on coefficient scanning and coding and Chairman of ad hoc groups on topics related to entropy coding and parallel processing. Dr. Sze received the Ph. D. degree in Electrical Engineering from MIT. She has contributed over 70 technical documents to HEVC and has published over 25 journal and conference papers. She was a recipient of the 2007 DaC/IsSCC Student Design Contest Award and a co-recipient of the 2008 A-SSCC Outstanding Design Award. In 2011, she received the Jin-Au Kong Outstanding doctoral Thesis Prize in Electrical Engineering at Mit for her thesis on"Parallel Algorithms and architectures for Low power video decoding? Madhukar Budagavi is a Senior member of the Technical Staff at Texas Instru- ments(TI) and leads Compression r&D activities in the embedded processing R&d Center in Dallas, TX, USA. His responsibilities at Ti include research and development of compression algorithms, embedded software implementation and prototyping, and video codec Soc architecture for Ti products in addition to video coding standards participation. Dr Budagavi represents Ti in ITU-T and ISo/EC international video coding standardization activity. He has been an active participant in the standardization of HEVC (iTU-T H 265 ISO/EC 23008-2) next-generation video coding standard by the jCT-vC committee of ITU-T and Iso/EC. Within the JCT-VC committee he has helped coordinate sub-group activities on spatial transforms, quantization, entropy coding, in-loop filtering, intra prediction, screen content coding and scalable HEVC (SHvC). Dr Budagavi received the Ph D degree in Electrical Engineering from Texas a&M University He has published 6 book chapters and over 35 journal and conference papers. He is a Senior Member of the ieee about the editors Gary J. Sullivan is a Video and Image Technology Architect at Microsoft Cor poration in its Corporate Standardization Group. He has been a longstanding chair- man or co-chairman of various video and image coding standardization activities in ITU-T VCEG, ISO/TEC MPEG, ISO/EC JPEG, and in their joint collabo rative teams since 1996. He is best known for leading the development of the AVC (ITU-T H.264 ISO/IEC 14496-10)and HEVC (ITU-T H 265 ISO/IEC 23008-2)standards, and the extensions of those standards for format application range enhancement, scalable video coding, and 3D/stereoscopic/multiview video coding At Microsoft, he has been the originator and lead designer of the directX Video Acceleration (DXVA) video decoding feature of the Microsoft Windows operating system. Dr Sullivan received the Ph. D. degree in Electrical Engineering from the University of California, Los Angeles. He has published approximately 10 book chapters and prefaces and 50 conference and journal papers he has received the IEeE masaru Ibuka Consumer Electronics Technical Field Award the IEEE Consumer Electronics Engineering Excellence Award, the Best Paper award of the ieee trans Csvt. the Incits Technical Excellence award the IMTC Leadership award, and the University of Louisville j. B Speed Professional Award in Engineering. The team efforts that he has led have been recognized by an ATAS Primetime Emmy Engineering Award and a pair of NaTAs Technology Engineering Emmy Awards. He is a Fellow of the IEEE and sPie Contents 1 Introduction Gary J. Sullivan 2 HEVC High-Level Syntax 13 Rickard Sjoberg and Jill Boyce 3 Block structures and parallelism Features in hevc Heiko Schwarz, Thomas Schierl, and Detlev Marpe 4 Intra-Picture Prediction in hevc 91 Jani lainema and woo-Jin han 5 Inter-Picture Prediction in hevc Benjamin Bross, Philipp Helle, Haricharan Lakshman, d Kemal ugur 6 HEVC Transform and Quantization 141 Madhukar Budagavi, Arild Fuldseth, and Gisle bjontegaard 7 In-Loop Filters in HEVC 171 Andrey Norkin, Chih-Ming Fu, Yu-Wen Huang, and shaomin lei 8 Entropy Coding in HEVC .209 Vivienne sze and detlev marpe 9 Compression Performance Analysis in HEVC∴.....275 Ali Tabatabai, Teruhiko Suzuki, Philippe hanhart Pavel Korshunov Touradj ebrahimi, Michael horowitz Faouzi Kossentini, and hassen tmar