Home Page Icon
Home Page
Table of Contents for
Who Performs Discovery?
Close
Who Performs Discovery?
by MindShare, Inc., Tom Shanley
InfiniBand Network Architecture
Copyright
PC System Architecture Series
Figures
Tables
Acknowledgments
About This Book
Who Needs This Book?
The MindShare Architecture Series
Cautionary Note
Specifications This Book Is Based On
Specification Is the Final Word
Organization of This Book
Documentation Conventions
Visit Our Web Site
We Want Your Feedback
Core Concepts
Basic Terms and Concepts
Definition of the Acronym “IBA”
Packet Field Documentation Convention
InfiniBand Advantages
Some Preliminary Terminology
Definition of a Subnet
Packet Addressing Basics
Every Packet Contains a BTH
Channel Adapters
Role of Switches and Routers
Repeater's Role
It's All About Message Passing
Intro to Attributes and Managers
Why Talk About Attributes and Managers Now?
Definition of an Attribute
Who Accesses Attributes?
MAs Handle Access Requests
MA's Response
Managers Use Special Packets Called MADs
Attribute Format and Documentation Conventions
QP: Message Transfer Mechanism
Introduction
A QP Is a Bi-Directional Message Transport Engine
Verb Layer Is an OS-Independent API
QP Context Defines QP's Operational Characteristics
Sending a Message to a Destination CA
Intro to Transport Types
Four IBA Transfer Protocol Flavors
Two Non-IBA Service Types
Intro to Send/Receive Operations
How Is a Message Transmitted or Received?
SQ Operation Types
RQ Operation Types
Division of Labor
Introduction
Brief Layer Descriptions
Layers in CAs, Switches and Routers
Simplified Example Scenarios
Physical Layer Overview
Link Layer Overview
Network Layer Overview
Transport Layer Overview
Verbs Overview
Subnet-Local Addressing
Port Numbering
LID Address Space
LID's Purpose: Packet Routing Within Subnet
Assigning Port's Base LID Address
Why Assign a LID Range to a Port?
Port's Decode of DLID Address
Port's Selection of SLID on Packet Transmit
SM Path Database
LID Rule Summary
Global Addressing
Global Routing: Source/Destination CAs in Different Subnets
IPv4 Addresses Too Limiting
IBA Global Address = IPv6 Address
IPv6 Addressing
Assignment of Port's Subnet ID and GUID(s)
Each Device Has Device-Level Identifiers
Intro to the Managers
The SM
General Services Managers
Subnet Administrator's Role
Introduction to Event Notification
Intro to Connection Establishment
Questions Addressed in This Chapter
A CA Is a Provider of Services
Locating a Specific CA
Discovering Services a CA Provides
RC/UC Connection Establishment
RD Connection Establishment
UD Connection Issues
PSN Usage
Overall Size of PSN Range
Requester QP's SQ Logic PSN Generation and Verification
Responder QP's RQ Logic Request PSN Verification
QP Creation and Operation
QP Verbs and QP State Machine
QP-Related Verbs
The QP State Machine
QP Creation
Software Control of QP State
QP Setup Is Performed in a Defined Sequence
Reset State
Initialized State
Ready to Receive State
Ready to Send State
SQ Drain (SQD) State
SQ Error State
Error State
WRs, WQEs, and CQEs
Once Posted to SQ or RQ, WR Is Called a WQE
WRs
WQE Execution and Completion Order
RDMA Read Relaxed Ordering Rules
Completion Queues (CQs) and CQEs
Asynchronous Events and Errors
Why Asynchronous?
Registering a Handler
Affiliated Asynchronous Events
Affiliated Asynchronous Errors
Unaffiliated Asynchronous Errors
Protection Mechanisms
Memory Protection
The Problems
The Solutions
Virtual-to-Physical Page Mapping Background
Memory Regions
Memory Windows
Protection Domains
Other Protection Mechanisms
The IBA Protection Mechanisms
Memory Access Protection (PD, L_Key, and R_Key)
PDs and UD Service
Partition Key (P_Key)
SM-Related Protection Mechanisms
RD Domain
Queue Key (Q_Key)
Baseboard Management Key (B_Key)
Detailed Description of the Transport Services
RC Transport Service
RC Support Requirement
RC Basic Operational Characteristics
RC Connection Establishment
Packet Opcodes
RC Message Transfer Primer
Structure of This Discussion
QP State before Any Messages Are Transferred
Standard Operation in Fast, Error-Free Environment
Traffic Reduction
Packet Delivery Delays
Packet Loss
Nak Errors
RQ Logic's Error Detecting and Handling
End-to-End Flow Control
SQ Logic Can Use MSN to Complete WQEs
Additional Reference Information
UC Transport Service
UC Support Requirement
In RC, Responses Are Expected
UC Is a Subset of RC
UC Transport Service Type's Basic Characteristics
Requester QP's SQ Logic Operation
Responder QP's RQ Logic Operation
RD Transport Service
RD Support Requirement
Introduction
Many Similarities to RC
RD Basic Operational Characteristics
The Major Differences from RC
The Scheduler
Keep RDC Operational If a QP Goes Down
Additional Reference Material
UD Transport Service
UD Support Requirement
No Responses Expected
The Only Operation Supported Is Send
Maximum Message Length Is One PMTU
Basic Operational Characteristics
Messaging with the Desired Remote Service
Address Handles
PD Check Performed before Message Is Processed
SQ Logic Operation
RQ Logic Operation
Raw Transport Service Types
Goal: Tunneling Non-IBA Packets through IBA Network
Solution: Disguise It as Special-Purpose IBA Packet
Raw QPs Are Used to Transmit/Receive Non-IBA Packets
Raw QP Support Requirement
Raw Transport Services Are Unreliable
Send and Receive Are Only Supported Operation Types
Basic Operational Description
Raw Datagrams Do Not Have an ICRC
Raw Datagrams Do Not Have a Destination QP
LRH:LNH Indicates Packet Type
Raw IPv6 Datagrams
Raw EtherType Datagrams
Additional Reference Material
Multicasting
Definition Of Multicasting
Support for Multicasting Is Optional
Only UD and Raw QPs Can Participate in Multicasting
UD Multicasting
Raw Packet Multicasting
Additional Reference Material
Automatic Path Migration
Definition of APM
CA Support Is Optional
Causes of a Path Migration
APM-related Elements
Normal Operation before APM Enabled
Enabling APM
Automatic Hardware Trigger of APM
Loading a New Path or a Tertiary Path
Additional Reference Information
Static Rate Control
How Fast Can a Port Transmit Packets?
Problem: Fast CA Port Can Cause Problems
Solution
IPD Calculation and Source
How IPD Is Provided to a QP or an EEC
Link and Physical Layer Descriptions
Detailed Description of the Link Layer
Link Layer Functional Overview
Link State Machine
Detailed Description of LRH
QoS within the Subnet: SL and VLs
Detailed Description of VL Arbitration
Link-Level Flow Control
Packet CRCs
Intro to the Packet Delimiters
Packet Receive State Machine
Data Packet Check
Link Packet (Flow Control Packet) Check
Switch Performs Packet Forwarding
Overview of Router Port's Link Layer
Detailed Physical Layer Description
Module Basics
General
Port Types
Signal Naming Conventions
Electrical Signaling and Copper Cable
Link Layer to Physical Layer Interface
Transmit Logic Functions
Receiver Logic Functions
Link Training
Physical Layer Error Handling
Repeaters
Performance Counters
The SM and SA
The SMI
Purpose of the SMI (QP0)
The SMI on a CA and Router
The SMI on Switches
Detailed Switch Handling of SMPs
Detailed CA or Router Handling of SMPs
SM Wishes to Access an Attribute in Its Local Device
SM Can Reside in a Switch
SMI Is a Privileged Resource
SMI Only Communicates with Other SMIs
Port States SMPs Can Be Sent and Received In
SMPs Never Leave the Subnet
Setting Up an HCA Port's SMI
How the SM Sends a Message and Handles a Response
SMP Source and Destination
The SMI and the Q_Key
The SMI and Partitions
Additional Reference Material
Detailed Description of MADs
Definition of a MAD
Software Times Return of MAD Response
Basic MAD Contents
SMP MADs
GMP MADs
Traps
Event Subscription and Event Forwarding
The Notice Queue
SM Methods and Attributes
SM MAD Formats
SM Methods
SM Attributes
SM Traps
SMA Notice Support
Multiple SMs
Many Issues Are Outside the Scope of the Specification
Multi-Vendor SM Failover Is Not Supported
A Subnet Can Have More Than One SM
How SM Issues Requests
Introduction to the SM States
Are You Alive?
SM Control Packets
Multiple Master SMs
SM States
Discovery
Who Performs Discovery?
How Packets Are Normally Routed
Packet Routing During Discovery
Scenario: Sweep at Startup
Accessing Device along Partially Configured Path
The GSI
The GSMs, GSAs, and GSIs
QP1 Is the GSI
Preparing an HCA GSI for Use
Required/Optional GSAs
The SA and the GSI
GMPs Use VL Buffer Determined by the GMP's SL Value
GMPs Can Transit Routers
QP1 Is a Controlled-Access QP
P_Key Insertion and Checking
Additional Reference Material
Detailed Description of SA
Purpose of the SA
SA Accessed Using GMPs
Location of the SA
Requester Access Authorization
SA Methods and Attributes
Record Identifier (RID) Definition
SubnAdmGet() Operation
SubnAdmSet() Operation
Definition of a Table
SubnAdmConfig() Operation
Database Queries Using SubnAdmGetTable()
Fetch Entire Database
Reliable Multi-Packet Transaction Protocol
Additional Reference Information
General Services
Baseboard Management
Roles of the Other Managers
The BM Reaches behind the IBA Front-End
Chassis and Module
Chassis Baseboard Management Elements
Passively Managed Chassis
Module BM Elements
Non-Module IBA Devices
BM MAD Format
BM Methods
BM Attributes
BM Sending a Command to the MME
CME Sends a Command to the BM
BM-related Traps
Performance Management
The Role of Performance Management (PM)
Required Features
Optional Features
Performance Management MAD Format
Performance Methods
Mandatory Performance Attributes
Optional PM Attributes
Communications Management
Introduction
CM MAD Format
CM Methods
CM Attributes
CM MADs
Definition of Client and Server
Definition of Active and Passive CM
Three Models Are Supported
Stale Communications Channel
Device Management
Definition of IOU and IOCs
IOU Contains a DMA
DM MAD Format
DM Methods
DM Attributes
Caveats
Glossary
Index
Search in book...
Toggle Font Controls
Playlists
Add To
Create new playlist
Name your new playlist
Playlist description (optional)
Cancel
Create playlist
Sign In
Email address
Password
Forgot Password?
Create account
Login
or
Continue with Facebook
Continue with Google
Sign Up
Full Name
Email address
Confirm Email Address
Password
Login
Create account
or
Continue with Facebook
Continue with Google
Prev
Previous Chapter
Discovery
Next
Next Chapter
How Packets Are Normally Routed
Who Performs Discovery?
The SM uses directed-route SMPs to discover the devices that populate a subnet.
Add Highlight
No Comment
..................Content has been hidden....................
You can't read the all page of ebook, please click
here
login for view all page.
Day Mode
Cloud Mode
Night Mode
Reset