High Efficiency Video Coding(HEVC) Algorithms and Architectures

VivienneszeMadhukarBudagavi GaryJ.Sullivan Editors HighEfficiencyVideo Coding(hevc) Algorithmsandarchitectures ② Springer editors Madhukarbudagav DepartmentofElectricalEngineering Texasinstrumentsinc andcomputerscience DallasTX.usa MassachusettsInstituteofTechnology Cambridge,MA,USA GarySullivan MicrosoftCorp Redmond.WA.usa ISSN1558-9412 ISBN978-3-31906894-7 ISBN978-3-319-06895-4(eBook) DOI10.1007/978-3-319-06895-4 SpringerChamHeidelbergNewYorkDordrechtLondon LibraryofCongressControlNumber:2014930758 oSpringerInternationalPublishingSwitzerland2014 Thisworkissubjecttocopyright.AllrightsarereservedbythePublisher,whetherthewholeorpartof thematerialisconcerned,specificallytherightsoftranslation,reprinting,reuseofillustrations,recitation broadcasting,reproductiononmicrofilmsorinanyotherphysicalway,andtransmissionorinformation storageandretrieval,electronicadaptation,computersoftware,orbysimilarordissimilarmethodology nowknownorhereafterdeveloped.Exemptedfromthislegalreservationarebriefexcerptsinconnection withreviewsorscholarlyanalysisormaterialsuppliedspecificallyforthepurposeofbeingentered andexecutedonacomputersystem,forexclusiveusebythepurchaserofthework.Duplicationof thispublicationorpartsthereofispermittedonlyundertheprovisionsoftheCopyrightLawofthe Publisher'slocation,initscurrentversion,andpermissionforusemustalwaysbeobtainedfromSpringer PermissionsforusemaybeobtainedthroughRightsLinkattheCopyrightClearanceCenter.Violations areliabletoprosecutionundertherespectiveCopyrightLaw Theuseofgeneraldescriptivenames,registerednames,trademarks,servicemarks,etcinthispublication doesnotimply,evenintheabsenceofaspecificstatement,thatsuchnamesareexemptfromtherelevant protectivelawsandregulationsandthereforefreeforgeneraluse Whiletheadviceandinformationinthisbookarebelievedtobetrueandaccurateatthedateof publication,neithertheauthorsnortheeditorsnorthepublishercanacceptanylegalresponsibilityfor anyerrorsoromissionsthatmaybemade.Thepublishermakesnowarranty,expressorimplied,with respecttothematerialcontainedherein Printedonacid-freepaper SpringerispartofSpringerScience+businessMedia(www.springer.com) Preface Advancesinvideocompression,whichhaveenabledustosqueezemorepixels throughbandwidth-limitedchannels,havebeencriticalintherapidgrowthofvideo usage.aswecontinuetopushforhighercodingefficiency,higherresolutionand moresophisticatedmultimediaapplications,therequirednumberofcomputations perpixelandthepixelprocessingratewillgrowexponentially.TheHighEfficiency VideoCoding(HEvC)standard,whichwascompletedinJanuary2013,was developedtoaddressthesechallengesInadditiontodeliveringimprovedcoding efficiencyrelativetothepreviousvideocodingstandards,suchasH.264/AVC, implementation-friendlyfeatureswereincorporatedintothehevcstandardto addressthepowerandthroughputrequirementsformanyoftodaysandtomorrows coapplications Thisbookisintendedforreaderswhoaregenerallyfamiliarwithvideocoding conceptsandareinterestedinlearningaboutthefeaturesinHEVC(especiallyin comparisontoH.264/MPEG-4AVC).Itismeanttoserveasacompaniontothe formaltextspecificationandreferencesoftware.Inadditiontoprovidingadetailed explanationofthestandard,thisbookalsogivesinsightintothedevelopmentof varioustools,andthetrade-offsthatwereconsideredduringthedesignprocess Accordingly,manyofthecontributingauthorsareleadingexpertswhoweredirectl anddeeplyinvolvedinthedevelopmentofthestandarditself. Asbothalgorithmsandarchitectureswereconsideredinthedevelopmentofthe HEVC,thisaimstoprovideinsightfrombothfrontsThefirstninechaptersofthe bookfocusonthealgorithmsforthevarioustoolsinhevc.andthetechniquesthat wereusedtoachieveitsimprovedcodingefficiencyThelasttwochaptersaddress thehevctoolsfromanarchitecturalperspectiveanddiscusstheimplementation considerationsforbuildinghardwaretosupporthEVCencodinganddecoding Inadditiontoreviewsfromcontributingauthors,wewouldalsoliketothankthe variousexternalreviewersfortheirvaluablefeedbackwhichhashelpedimprovethe clarityandtechnicalaccuracyofthebook.ThesereviewersincludeYu-HsinChen, Preface Chih-ChiCheng,KeiichiChono,LuisFernandez,DanielFinchelstein,Hun-Seok Kim,HyungjoonKim,YasutomoMatsuba,AkiraOsamoto,RahulRithe,Mahmut Sinangil,HideoTamama,Ye-KuiWangandminhuaZhou Cambridge,MA,USA Ⅴviennesze Dallas.TXUSa Madhukarbudagavi Redmond.WA.USA GaryJ.Sullivan abouttheeditors VivienneszeisanAssistantProfessorattheMassachusettsInstituteofTechnology (MIT)intheElectricalEngineeringandComputerScienceDepartment.Her researchinterestsincludeenergy-awaresignalprocessingalgorithms,andlow powercircuitandsystemdesignforportablemultimediaapplications.Priorto joiningMit,shewaswithther&dCenteratTexasInstruments(TD),whereshe representedtiintheJCT-VCcommitteeofitu-tandIso/ECstandardsbody duringthedevelopmentofHEVC(ITU-TH265ISO/IEC23008-2).Withinthe committee,shewasthePrimaryCoordinatorofthecoreexperimentsoncoefficient scanningandcodingandChairmanofadhocgroupsontopicsrelatedtoentropy codingandparallelprocessing.Dr.SzereceivedthePh.D.degreeinElectrical EngineeringfromMiT.shehascontributedover70technicaldocumentstohevc andhaspublishedover25journalandconferencepapers.Shewasarecipientof the2007DAC/ISSCCStudentDesignContestAwardandaco-recipientofthe 2008A-SSCCOutstandingDesignAward.In2011,shereceivedtheJin-AuKong OutstandingdoctoralThesisprizeinElectricalEngineeringatmitforherthesis on"ParallelAlgorithmsandArchitecturesforlowpowerVideodecoding?" MadhukarBudagaviisaSeniorMemberoftheTechnicalStaffatTexasInstru ments(TDandleadsCompressionr&dactivitiesintheEmbeddedprocessing R&dCenterinDallas,TX,USA.HisresponsibilitiesatTiincluderesearchand developmentofcompressionalgorithms,embeddedsoftwareimplementationand prototyping,andvideocodecSocarchitecturefortiproductsinadditiontovideo codingstandardsparticipation.Dr.BudagavirepresentsTiinITU-TandIso/EC internationalvideocodingstandardizationactivity.Hehasbeenanactiveparticipant inthestandardizationofHevC(ITU-TH265Iso/EC23008-2)next-generation videocodingstandardbythejCT-vccommitteeofiTU-TandIso/EC.Within thejCT-vccommitteehehashelpedcoordinatesub-groupactivitiesonspatial transforms,quantization,entropycoding,in-loopfiltering,intraprediction,screen contentcodingandscalableHEVC(SHVC).DrBudagavireceivedthePh.Ddegree inElectricalEngineeringfromTexasA&MUniversity.Hehaspublished6book chaptersandover35journalandconferencepapers.HeisaSeniorMemberof theieee VIl Abouttheeditors GaryJ.SullivanisaVideoandImageTechnologyarchitectatMicrosoftcor- porationinitsCorporateStandardizationGroupHehasbeenalongstandingchair manorco-chairmanofvariousvideoandimagecodingstandardizationactivities inITU-TVCEG,ISO/ECMPEG,ISO/ECJPEG,andintheirjointcollabo rativeteamssince1996.Heisbestknownforleadingthedevelopmentofthe AVC(ITU-TH.264ISO/EC14496-10)andHEVC(ITU-TH265ISO/EC 23008-2)standards,andtheextensionsofthosestandardsforformatapplication rangeenhancement,scalablevideocoding,and3D/stereoscopic/multiviewvideo codingAtMicrosoft,hehasbeentheoriginatorandleaddesignerofthedirectX VideoAcceleration(DXVA)videodecodingfeatureoftheMicrosoftWindows operatingsystem.DrSullivanreceivedthePh.D.degreeinElectricalEngineering fromtheUniversityofCalifornia,LosAngeles.Hehaspublishedapproximately 10bookchaptersandprefacesand50conferenceandjournalpapers.Hehas receivedtheieeemasaruibukaConsumerelectronicstechnicalfieldaward theieEeConsumerElectronicsEngineeringExcellenceAward,theBestPaper awardoftheIeeeTrans.CSvT.theINcitsTechnicalExcellenceAwardthe IMTCLeadershipAward,andtheUniversityoflouisvilleJBSpeedProfessional AwardinEngineering.Theteameffortsthathehasledhavebeenrecognizedbyan ATASPrimetimeEmmyEngineeringAwardandapairofNaTAsTechnology EngineeringEmmyAwards.HeisaFellowoftheIEEEandSPie Contents 1ntroduction GaryJ.sullivan 2HEVCHigh-LevelSyntax 13 RickardsjobergandJillboyce 3BlockStructuresandParallelismFeaturesinHEVO HeikoSchwarz,ThomasSchierl,andDetlevMarpe 4Intra-PicturePredictioninHEVC JaniLainemaandwoo-Jinhan 5Inter-PicturePredictioninHEvc 113 BenjaminBross,PhilippHelle,HaricharanLakshman, andKemalugur 6HEVCTransformandQuantization MadhukarBudagavi,ArildFuldseth,andGisleBjontegaard.............141 7In-LoopFiltersinHEVC 171 AndreyNorkin,Chih-MingFu,Yu-WenHuang, andshaominlei EntropyCodinginHEVC 209 VivienneSzeandDetlevmarpe 9CompressionPerformanceAnalysisinHEVC 275 AliTabatabai,TeruhikoSuzuki,PhilippeHanhart, PavelKorshunov,Touradjebrahimi,MichaelHorowitz Faouzikossentini.andhassentmar