Reliable Multi-Packet Transaction Protocol

Introduction

SA MAD transfers consist of multiple packets under the following circumstances:

  • A SubnAdmGetTable() operation may result in the return of a series of Subn AdmGetTableResp() response MADs.

  • A SubnAdmGetBulk() operations results in the return of the entire SA database (or, if a non-zero SA_Key issued, a subset thereof). The database is returned in a series of SubnAdmGetBulkResp() response MADs.

  • A SubnAdmConfig() operation may consists of a series of SubnAdmConfig() request MADs containing the records to be added, deleted, or edited.

In all three cases, both the SA and the requester will want to be sure that all of the MADs in the multiple-MAD sequence are received correctly. The SA MAD transfer protocol provides for verification of a reliable transfer of the entire multiple-MAD sequence. The following sections provide a detailed description of this protocol.

The Packet Fields

Table 33-10 on this page provides a description of the SA MAD fields that are related to the multi-packet transfer protocol.

Table 33-10. MAD Fields Related to a Multiple-MAD SA Transfer
FieldLengthDescription
Segment Number32 bitsIdentifies the relative position of each packet within a multi-packet request or response packet series. Segment Numbers for multi-packet requests and responses begin at segment number 1. The segment number of a single-packet request or single-packet response is set to 0.
Payload Length32 bitsValid only for:

  • First packet of a multi-packet SubnAdmConfig() request. Indicates the sum (in bytes) of the lengths of the Admin Data Areas in all request MADs which the requester is sending for the transaction.

  • First packet of a multi-packet SubnAdmGetBulk(), SubnAdmGetTable(), or SubnAdmConfig() response. Indicates the sum (in bytes) of the lengths of the Admin Data Areas in all response packets in the series.

  • Last packet of a multi-packet SubnAdmGetBulk(), SubnAdmGetTable(), or SubnAdmConfig() response. Indicates the number of valid bytes in the final packet's Admin Data Area.

In all other packets of a multi-packet request or response, the payload length field is reserved.

After the first response to a SubnAdmGetBulk() or SubnAdmGetTable() request is sent, the actual payload length may change (due to SM updates to the database during the multiple packet transfer). The payload length field in the last response packet indicates the number of valid bytes in the final packet's Admin Data Area, and the Last Packet bit (bit 2) of the Fragment Flag is set to one.
Fragment Flag8 bitsIdentifies the packet as either the first, a mid-stream, or the Last request or response packet in a multi-packet transfer. Also used to specify:

  • A Resend request.

  • An Ack packet.

  • A KeepAlive packet.

For a single-packet request or response, the Fragment Flag is set to zero. Refer to Table 33-11 on page 951.
Window16 bitsValid only in:

  • Multi-packet SubnAdmGetBulkResp() or Subn AdmGetTableResp(). Specifies the number of packets the responder may send before it pauses to receive an Ack packet. When an Ack packet is received by the responder, a number of additional packets (subsequent to the packet being Ack'd) equal to the window value may be sent.

  • Ack packet of a SubnAdmConfig() operation. Indicates the number of packets the requester may send (subsequent to the packet being Ack'd) before it pauses to receive another Ack packet. For the first burst of request packets of a Subn AdmConfig() operation, the default Window value of 64 is used by the requester. If the Window value in any Ack packet < 64, the default window value of 64 is used by the recipient.

Admin Data Area192 bytesWhen sending a packet during a multi-packet request or response, the Admin Data Areas of all packets except the last contain the full 192 bytes of data. The last packet may contain fewer bytes as indicated by its Payload Length field.

Table 33-11. Fragment Flag Description
Bit(s)NameDescription
0First PacketSet to one in the first packet of a multi-packet request or response.
1Reserved 
2Last PacketSet to one in the last packet of a multi-packet request or response.
3Resend Request

  • For a requester, this bit is set to one to request that the SA restart sending packets for a multi-packet response beginning with the Segment Number indicated in the Segment Number field.

  • For the SA, this bit is set to one to instruct the requester to start resending packets of a multi-packet request beginning with the Segment Number indicated in the Segment Number field.

  • For resend request packets, all fields in the MAD data field other than the Fragment Flag and Segment Number are ignored by the recipient.

In all other cases, bit 3 = 0.
4Reserved 
5Ack Packet

  • For a requester, this bit is set to one to Ack the receipt of all response packets up to and including the packet with the segment number indicated in the Segment Number field.

  • For the SA, this bit is set to one to Ack the receipt of all request packets up to and including the packet with the segment number indicated in the Segment Number field.

After receipt of a packet is Ack'd, the packet source no longer has to maintain a copy of the packet contents and a request to retransmit the packet is not allowed (see bit 3 in this table.) The segment number intervals at which Ack packets are sent is not specified. All fields in the MAD other than the Segment Number, Fragment Flag, and Window are ignored by the recipient.
6Reserved 
7KeepAlive PacketSet to one by the sender of a multi-packet request or response to request that the recipient re-initialize the timer for the transaction to the Segment Timeout period (see “Timeouts” on page 953). The sender may be either the requester or the SA. All fields in the MAD other than the Fragment Field are ignored by the recipient.

Timeouts

General

All GMPs are sent using the special-purpose UD QP1 (the GSI). As with any UD QP, no response is expected by the QP's SQ Logic after it transmits a packet. The QP's SQ Logic therefore has no hardware timer that is triggered on transmission of the request packet.

Rather, it is software that must start a timer when it posts a WR to the UD QP's SQ for which it expects to get a response. The amount of time that the sending software will wait for a response is based on two timeouts supplied by hardware:

  1. The SM programs each port's PortInfo.SubnetTimeout attribute element with a timeout value indicating the maximum flight time for a packet to get from any source port to any destination port in the subnet. For more information, see “Software Times Return of MAD Response” on page 781.

  2. The SA's ClassPortInfo.ResponseTimeValue attribute element indicates:

    - The maximum expected time within which the SA will transmit a response after it receives the corresponding request packet.

    - The maximum expected time within which an SA agent of a given port will Ack the receipt of the last packet allowed within a Window during the receipt of a a multi-packet request.

    - The maximum expected time between the transmission of the subsequent packets of a multi-packet SubnAdmConfig() request or between the transmission of subsequent response packets of a multi-packet SubnAdmGetTableResp(), SubnAdmGetBulkResp(), or SubnAdmConfigResp() transmission.

The above timeouts are used to calculate the Response Timeout Period and the Segment Timeout Period. The next two subsections describe these two timeouts.

Response Timeout Period
General

When a packet is sent that requires a response, the Response Timeout period is the maximum expected time within which the sender expects to receive the response. The response timeout period is used for the following responses:

- For the sender of a SubnAdmGetTable() or SubnAdmGetBulk() request, the Response Timeout period is the maximum time within which the requester expects to receive the first response from the SA.

- For the sender of a multi-packet request or multipacket response that contains more packets than allowed by the Window field, the Response Timeout period is the maximum time within which the sender expects to receive an Ack packet after it sends the last packet allowed in the window.

The value of the Response Timeout =

(2 X packet flight time) + Packet processing time =

(2 X (4.096us X 2SubnetTimeout)) + (4.096us X 2ResponseTimeValue).

Response Timeout Error

When a Response Timeout error is detected, the software that sent the packet may retry (i.e., resend) the packet a maximum of seven times. The actual number of retries is vendor-specific.

Segment Timeout Period
General

The Segment Timeout period is the maximum expected time between the receipt of subsequent packets of a multi-packet SubnAdmConfig() request packet stream, or a multi-packet SubnAdmGetTableResp() or SubnAdmGetBulkResp() response packet stream.

The recipient of the multi-packet request or response triggers the Segment Timeout period for the packet stream whenever it receives a packet of the request or response packet stream. As long as there are outstanding packets expected, the timeout is reinitialized and is restarted whenever another packet of the request or response packet stream is received (including receipt of a KeepAlive packet).

The Segment Timeout is = the packet flight time + the packet processing time:

(4.096us X 2SubnetTimeout ) + (4.096us X 2ResponseTimeValue).

Segment Error Types

A segment error is detected under two circumstances:

- The Segment Timeout expires due to the cessation of packet receipt during a multi-packet operation.

- A packet is received during a multi-packet transmission whose Segment Number is not one greater than the previous packet in the request or response packet stream.

Segment Error Recovery

The error recovery for a segment error is:

  1. Discard packets received with Segment Numbers > the segment number of the missing packet.

  2. Issue a Resend request packet with the Fragment Flag = 00010000b (see Table 33-11 on page 951). The Segment Number in the Resend request packet is set to the segment number of the packet that was expected but not received.

This causes packet transmission to restart beginning with the packet whose segment number is equal to the segment number in the Resend request packet. When the missing packet with the correct segment number is received, the recipient stops discarding packets.

If a Resend request packet is received with the segment number of a packet whose data payload is no longer available, a response MAD with its Status field indicating an Invalid Attribute is returned (refer to Table 27-2 on page 776).

The number of times that a Resend request is issued for a specific missing packet is vendor-specific, but it may not exceed seven resend requests.

Multi-Packet Response Protocol

Figure 33-2 on page 957 illustrates an example SubnAdmGetTable() or SubnAdmGetbulk() operation that results in the return of a multi-packet SubnAdmGetTableResp() series or SubnAdmGetBulkResp() series.

  1. The requester initiates the database access by sending a request MAD with:

    - The Fragment Flag = 0 (because this is a single-packet request).

    - Window field = n. The SA is to return n response packets before it pauses to await receipt of an Ack packet.

  2. The requester triggers the Response Timer and awaits the return of a response.

  3. If the SA cannot generate the first response packet within the Response Timeout, it transmits a KeepAlive packet back to the requester. Additional KeepAlive packets may be sent if additional time is required to send the first response packet.

  4. When the SA has accumulated some of or a full window's worth of records, it sends the first n response packets. Each packet is sent within the ResponseTimeValue from the time when the previous packet was sent. Except for the optional KeepAlive packet, the Segment Number of each response packet increments starting from one.

  5. When the requester receives a response packet, it initializes the Segment Timer. As long as there are outstanding responses expected, the Segment Timer is re-initialized and restarted whenever a response packet or Keep Alive packet is received. The Segment Timer is stopped when there are no outstanding responses expected.

  6. The responder continues sending packets until it has sent a full window's worth of packets. After sending the nth packet, the SA triggers the Response Timer and waits for an Ack packet.

  7. After receiving the nth packet (the last packet in the window), the requester sends an Ack packet with its Segment Number = n. This Acks receipt of the first n packets.

  8. Upon receipt of the Ack packet, the SA continues sending response packets resuming with Segment Number n+1.

  9. If the SA sends the last response packet before sending a full window's worth of packets, it sets the Fragment Flag = Last Packet. The Payload Length field in the last packet is set to the number of valid data bytes present in the Admin Data Area of the last packet.

  10. When sending the last packet, the SA triggers the Response Timer and waits for the receipt of the Ack packet from the requester.

Figure 33-2. Multi-Packet SA Response Example


Multi-Packet Request Protocol

Figure 33-3 on page 959 illustrates a multi-packet SubnAdmConfig() operation wherein multiple request packets are sent to the SA.

  1. In the figure, the requester initiates the transaction by sending the initial request with a Segment Number of 1 and the Fragment Flag = First Packet. The Payload Length = the sum (in bytes) of the lengths of the Admin Data Area fields in all packets to be sent as part of the request.

  2. The requester then sends each of the subsequent packets within a time period of ClassPortInfo.ResponseTimeValue of the previous packet. The segment number of each packet is incremented by one. The requester may send 64 packets before receiving the first acknowledgment packet from the SA.

  3. As the multi-packet request is received, the SA verifies that the Segment Number in each request packet is one higher than the Segment Number in the previous packet received. The SA also triggers the Segment Timer whenever it receives a request packet.

  4. In the example, the requester sends 64 request packets—the default Window size on a SubnAdmConfig() operation—before it pauses to await receipt of an Ack packet from the SA. When sending the 64th packet, the requester initializes the Response Timeout and waits for an Ack packet.

  5. When the SA has sufficient buffer space available to accept another window's worth of data, it sends an Ack packet with a new window value of n. This indicates to the requester that it may send another n additional packets before pausing for an another Ack packet.

  6. In the example, the requester completes sending the multi-packet request somewhere before the end of the final window. The last request packet sent has the Fragment Flag = Last packet. When sending the last packet, the requester initializes the response timer.

  7. When the SA has received the last packet, it sends a SubnAdminConfigResp() indicating that the operation is complete.

Figure 33-3. Example Multi-Packet SA Request


..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset