PCI Express System Architecture

PCI Express体系结构.英文版本，已解密。All rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, electronic, mechanical, photocopying,recording, or otherwise, without the prior written permission of the publisher.Printed in tMind Share Learning OptionsMind shareMind shareMind shareMind shareClassroom∨ irtual classrooneLearningPCI EXPRESSSysTiMARCHITECTUHE SAIAS午NTMCO In-House TrainingVirtual In-House TrainingIntro eLearningBooksModulesPublic Training(8 Virtual Public TrainingComprehensiveeLearning Modules函eBooksClassroom TrainingVirtual Classroom TrainingeLearning Module TrainingMind share pressInvite MindShare to trainThe majority of our coursesMindShare is also an eLearningPurchase our books andyou in-house, or sign-up tolive over the web in an intercompany. Our growing list ofeBooks or publish yourattend one of our many publicactive environment with WebExinteractive eLearning modulesown content through usclasses held throughout theand a phone bridge. We deliverincludeMind Share has authoredyear and around the worldtraining cost-effectively acrossIntro to virtualizationover 25 books and the listNo more boring classes, themultiple sites and time zonesTechnologyis growing. Let us helpMind Share Experience:' isImagine being trained in yourIntro to lo Virtualizationmake your book projectsure to keep you engagedcubicle or home office andIntro to PCI Express 2.0a successful oneavoiding the hassle of travelUpdatesContact us to attend one ofPCI Express 2.0our public virtual classesUSB 2.0AMD Opteron ProcessorArchitectureVirtualization Technologyand moreEngage MindShareHave knowledge that you want to bring to life? MindShare will work with you to "Bring Your Knowledge to LifeEngage us to transform your knowledge and design courses that can be delivered in classroom or virtual classroom settings, create online eLearning modules, or publish a book that you authorWe are proud to be the preferred training provider at an extensive list of clients that includeADAPTEC·AMD·AG| LENT TECHNOLOG|ES· APPLE· BROADCOM· CADENCE·CRAY·C|SCo·DELL· FREESCALEGENERAL DYNAM|CS·HP·|BM· KODAK·Ls| LOGIC· MOTOROLA·M| CROSOFT·NASA· NATIONAL SEM| CONDUCTORNETAPP·NoK|A·NVD|A· PLX TECHNOLOGY·QLoG|C·S| EMENS·SUNM| CROSYSTEMS SYNOPSYS·T·UN|SYSMINDSHARE4285 SLASH PINE DRIVE COLORADO SPRINGS, C0 80908 USAM1.602617.112301.800.633.1440ravi@ mindshare. cowww.mindshare.comBRINGING LIFE TO KNOWLEDGEPCⅠ ExpressSustemArchitectureMIND SHARE, INCRavi budrukDon andersonTom ShanleyTechnical edit by oe winklesADDISON-WESLEY DEVELOPERS PRESSBoston· San francisco· New york· TorontoMontreal· London munich· Paris· Madrid· SydneyCape town· Tokyo· Singapore· Mexico cityMany of the designations used by manufacturers and sellers to distinguish their products are claimed as trademarks. Where those designators appear in this book, and Addison-Wesley was aware of the trademark claim the designations have been printed ininitial capital letters or all capital lettersThe authors and publishers have taken care in preparation of this book, but make noexpressed or implied warranty of any kind and assume no responsibility for errors oromissions. No liability is assumed for incidental or consequential damages in connec-tion with or arising out of the use of the information or programs contained hereinLibrary of Congress Cataloging-in-Publication DataISBN: 0-Copyright C2003 by MindShare, IncAll rights reserved. No part of this publication may be reproduced, stored in a retrievalsystem, or transmitted, in any form or by any means, electronic, mechanical, photocopy-ing, recording, or otherwise, without the prior written permission of the publisherPrinted in the United States of America. Published simultaneously in CanadaSponsoring EditorProject ManagerCover DesignSet in 10 point Palatino by MindShare, Inc123456789-MA-9998979th Printing, April, 2008Addison-Wesley books available for bulk purchases by corporations, institutions, andother organizations. For more information please contact the Corporate, Government,and Special Sales Department at(800)238-9682Find a-w Developers press on the world-Wide Web at:http://www.awl.com/devpress/Contentsabout This bookThe mind share architecture series,……,,,Cautionary note……2Intended audience∴.2Prerequisite Knowledge…...…...….….3Topics and Organization…..…Documentation Conventions····命·a·····:·····a··中·PCI ExpressHexadecimal NotationBinary notation……………………Decimal notationBits versus bytes notationBit fields…4555Active Signal States…Visit Our Web site。d。We want your feedback…Part One: The Big PictureChaptePer 1: Architectural PerspectiPveIntroduction To PCI Express....9The role of thOriginal pci solutiog10Don't throw away what is Good Keep it10Make Improvements for the Future.10Looking into the Future........11Predecessor Buses ComparedAuthors disclaimer12Bus Performances and Number of Slots Compared12PCI Express Aggregate Throughput……13Performance Per Pin Compared14I/O Bus Architecture Perspective1633 MHz PCI Bus Based system...........................................16Electrical Load Limit of a 33 MHZ PCI Bus17PCI Transaction Model -Programmed IO19PCI Transaction Model- Peer-to-PeerPCI Bus arbitration2PCI Delayed Transaction Protocol. ......23PCI Retry protocolPCI Disconnect protocol24PCI Interrupt handlin..25ContentsPCI Error Handling…26PCI Address spacc MaPCI Configuration Cycle generation79PCI Function Configuration Register Space30PCI Programming Model……………………Limitations of a 33 MHz PCi SystemLatest Generation of Intel Pci chipsets326 MHz PCI Bus Based SystemLimitations of 66 MHz pCi bus.34Limitations of PCi Architecture3466 MHz and 13 3 MHz Pci-X10 Bus based platformsPCI-X FeaturesPCI-X Requester/Completer Split Transaction Model37DDR and qdR PCi-X20 Bus Based PlatformsThe PCI Express Way………………………………….41The link -a Point-to-Point inte··41Differential Signalin41Switches Used to Interconnect Multiple devices......... 42Packet Based Protocol2Bandwidth and clocki43Address s43PCI Express transactionsPCI Express Transaction Model..43Error Handling and Robustness of Data Transfer44Quality of Service(QoS), Traffic Classes(TCs)and Virtual Channels(vCs).44Flow Control45MSI Style Interrupt handling similar to PCi-X45er manag45lot Plug Support.46PCI Compatible Software Model46Mechanical form factors…47PCl-like Peripheral Card and Connector………47Mini pcieFxpress Form Factor.47Mechanical Form Factors Pending release.47NEWCARD Form Factor47Server lo module(SIOM) Form Factor....,,…,….…47PCI EXPTopology48Enuting50PCI Express System Block Diagram……….…51Low Cost pcieChipset5High-End Server SystemPCI Express Specifications.ContentsChapter 2: Architecture OverviewIntroduction to PCI Express transactions..55PCI Express Transaction Protocol57Non- Posted read transactions58Non- Posted Read Transaction for Locked Requests……………59Non -Posted write transactions61Posted Memory Write transactions62Posted message transactionsSome examples of transactionsMemory Read Originated by CPU, Targeting an Endpoint……………Memory Read Originated by Endpoint, Targeting System MemoryIO Write Initiated by CPU, Targeting an Endpoint.67Memory Write Transaction Originated by CPU andTargeting an EndpointPCI Express Device Layers…........……….…….69Overview曹,,里里Transmit Portion of Device Layers......71Receive portion of device layers71Device Layers and their Associated Packets71Transaction Layer Packets(TLPsTLP Packet Assembly72TLP Packet Disassembly73Data link layer packets (dLLPs74DLLP Assembly垂DLLP Disassembly番番音番76Physical layer packets(PLPs)……Function of Each PCI Express Device layerDevice Core Software layerTransmit SideReceive side78Transaction Layer…Transmit sideReceiver side.…....…….......81Flow ControlQuality of Service(Qos)82Traffic ClassesTCs)and Virtual Channels(vCs)Port arbitration and vc arbitration.……Transaction Ordering…….87Power Management..87Configuration Registers………87Data Link layer.......87IXContentsTransmit Side8Receive sidoData Link layer contribution to TLPs and dlLPNon-Posted Transaction Showing acK-nak ProtocolPosted Transaction Showing ACK-NAK Protocol92Other Functions of the data Link la92Physical layerTransmit sideRe sideLink training and InitializationLink Power Management……Reset95Electrical Physical layer.96Example of a Non- Posted Memory Read Transaction……………….96Memory Read request Phase………97Completion with Data PhaseHot Plug…….101PCI Express Performance and data Transfer Efficiency............101Part Two: transaction protocolChapter 3: Address Spaces Transaction RoutingIntroduction106Receivers Check For Three Types of link Traffic107Multi-port Devices Assume the Routing Burden……Endpoints Have Limited Routing Responsibilities. ...…………107.107System routing strategy Is programmedTwo Types of Local Link traffic........……….108Ordered sets108Data Link Layer Packets(DLLPs111Transaction Layer Packet Routing Basics………,,……,………………,113TLPS Used to Access Four Address Spaces.……113Split transaction protocol Is UsedSplit transactions: Better Performance, More Overhcad,114114Write Posting Sometimes a Completion Isn't Needed ..............................115Three methods of TLP Routing..117PCI Express Routing Is Compatible with PC1…………………………117Why Were Messages Added to PCI Express Protoco.………PCI Express Adds Implicit Routing for Messages118118How Implicit Routing Helps with Messages118Header Fields Define Packet Format and Routing………∴119ContentsUSing TLP Header Information: Overview.120General………120Header Type /Format Field encodings120Applying Routing Mechanisms121Address routing……122Memory and Io Address Maps…………………….122Key TLP Header Fields in Address routing123TLPs with 3dw 32-Bit addressTLPs With 4DW, 64-Bit Address124An Endpoint checks an address- Routed tlp………125A Switch receives an address routed tlp Two checks.125125Other Notes About Switch Address-Routing127ID Routins127ID Bus number device number function number limits127Key TLP Header Fields in ID Routing1283 DW TLP, ID Routing…1284 DW TLP, ID Routing…………………129An endpoint Checks an ID-Routed TLP..130A Switch receives an id-routed tlp. two checks130Other Notes about Switch ID Routing130Implicit Routing131Only Messages May Use Implicit Routing………132Messages May Also Use Address or ID Routing132Routing Sub- Field in Header Indicates Routing Method……………132Key TLP Header Fields in Implicit Routing……132Message Type Field Summary133An endpoint Checks a TLP Routed Implicitly……………………….134A Switch Receives a TLP Routed Implicitly……134Plug-And-Play Configuration of Routing Options…135Routing Configuration Is PCI-Compatible135Two Configuration Space Header Formats:Type0,Type1……….135Routing Registers Are Located in Configuration Header135Base Address Registers(BARs): Type 0, 1 Headers136General136BAR Setup Example One: 1MB, Prefetchable memory request138BAR Setup example two: 64-Bit, 64MB Memory request...140BAR Setup Example Three: 256-Byte IO Request142Base/ Limit registers,Type1 Header Only………144144Prefetchable Memory Base/ Limit Registers………………144Non-Prefetchable Memory Base/Limit Registers146