Dell PowerEdge 6650

Hot Plug

PCI Hot Plug Operation

The PCI Hot Plug (PHP) Controller is responsible for controlling the low-level aspects of powering up and down PCI cards as well as providing an interface to the status LEDs and buttons. The controller is implemented in a Xilinx Spartan XL FPGA and is a slave device of the Baseboard Management Controller (BMC). Refer to the ESM-3 Hardware Specification for details on the PCI Hot Plug controller.

There are three separate PCI hot-plug functions: hot replacement, hot add, and hot upgrade:

Feature

Function

Benefit

The PHP functionality of the PowerEdge 6650 incorporates LED indicators on the rear of the chassis to allow the user to see which slots are powered ON, or which need attention. These LEDs are located on the PowerEdge 6650 I/O board. Light pipes within the PCI card divider panels (dishrack) make the indicators visible from both the rear of the chassis and from above the PCI Hot Plug area when the access door is opened.

The design of the new dishrack allows either Amber or Green to be displayed, but only one color at a time. Thus a trade-off must be made when comparing to some previous systems that had two separate LEDs (one Green, one Amber).

Below is a table that shows the Hot Plug indicator patterns for the PowerEdge 6650.

Indicator

Indicator Code

The state where the connector is in the process of being powered up or powered down is generally so short that the flashing green indicator is not visible. The exception would be when the user requests a power down operation and the OS does not comply (perhaps no PHP driver).

The obvious disadvantage of one bi-color LED is that a colorblind operator cannot distinguish Amber and Green. With the PowerEdge 6650, the flashing state conveys sufficient information so that a person with that condition would still be able to ascertain what is happening.

An example of the process is:

1. Application indicates which slot needs to be hot-swapped (Green Flash - Fast)
2. Operator pushes the 'Notify' button to initiate power-down
3. If the OS doesn't have a problem, the power down state comes very quickly (Green light turns off)
4. If the OS has a problem the flash slows down (very noticeable change) and the operator must determine from the application console why the operation is not being completed.
5. After the power down the operator swaps the card and then pushes the 'Notify' button.
6. Steady Green indicates the card is powered up.
7. Flashing Amber indicates there is a power up fault that must be corrected.

• Slot Isolation Logic - Each slot provides logic to electrically isolate the PCI signals from a slot connector and to disconnect all power. The PCI Hot Plug FPGA under control of ESM Firmware controls this logic.
• Over Current Detection - Over current detection is used to determine if a PCI adapter that has been added to the system is improperly installed and shorting a power rail. This logic will automatically shut down power to a slot and indicate status on the status LEDs.
• Notify Switch - The Notify switch is located under the dishrack divider adjacent to each slot. It is actuated by pressing on the status indicator located at the end of each divider.
• Status Indicator - LEDs (Amber/Green) are located underneath the status indicator located at the end of each divider. The status indicator is a plastic light pipe that allows the status LEDs to be viewed from either the top or the rear of the system. The top of the indicator is shaped like an arrow which points to its associated slot. Pressing the indicator actuates the Notify switch.
• Dishrack Base and Dividers - The dishrack base and dividers provide an insulated barrier between each PCI card and over the I/O Planar to prevent electrical shorts during PCI Hot Plug operations. In addition the end of each divider contains the status indicator for the adjacent slot (slot to the right as seen from the rear). This status indicator doubles as the actuator for the Notify switch.
• PCI Hot Plug FPGA - The PCI Hot Plug FPGA contains the logic necessary to determine proper PCI bus speeds, isolate PCI slots, activate the status LEDs, and provide the PHP control interface to ESM3 Firmware. This FPGA can be reprogrammed using a floppy based DOS utility and provides for an embedded revision number for change control purposes.

PCI Hot Plug Operating System Support

• Support is provided by the Dell developed "Hot Plug Wizard" application.
• Hot replace of an adapter with the same adapter model is supported.
• Adapter must be teamed with another NIC and the non-active (failover) adapter may be replaced.
• Hot add of an adapter is not supported.

• Support is native to W2K using the "Unplug or Eject Hardware" utility
• Hot replace and hot add are supported
• Hot add may be restricted by available resources pre-allocated to a PCI bus

• Support is provided by the Dell developed "SBD" application.
• Hot replace of an adapter with the same adapter model is supported.
• Hot add of an adapter is not supported.

• PCI Hot Plug is not supported

PCI Hot Plug Restrictions and Rules

NetWare and NT4 only support hot replace. An adapter may be replaced with its exact equivalent which will use the same PCI resources as the original adapter.

During a "hot add" operation a PCI adapter may be added to a bus if it matches the speed and protocol of the other adapters on that bus. The PCI and PCI-X specifications provide for backwards compatibility of PCI adapters into buses. The speeds and protocols supported by PowerEdge 6600 in order of compatibility are as follows:

• PCI 33MHz
• PCI 66MHz
• PCI-X 66MHz
• PCI-X 100MHz (adapters rated at 133MHz)

If another adapter is on the bus (in the other slot) then the bus is running at the capabilities of that adapter. An adapter may not be added to a PCI bus if the adapter cannot support the bus speed and protocol of that bus. For instance, a 33MHz PCI adapter cannot be added to a bus that is already running at 66MHz PCI. A 66MHz PCI adapter may not be hot added to a bus that is already running at 66MHz PCI-X. If an attempt is made to add an adapter to a bus with higher capabilities, the status indicators will show the "Speed Fault" state.

If an adapter is hot added to a bus that is already running at a lower capability due to an existing adapter, then the added adapter will also run at that lower capability. No indication will be given for this condition.

Hot Add Rules for the PowerEdge 6650

1. Any NIC (including SSL) can be hot added to an empty PCI Bus
   1. In a CIOB 2.1 system the adapter will run at PCI-33
   2. In a CIOB 2.2 system the adapter will run at its rated capability
2. Any 2 NICs (including SSL) can be hot added to the same empty PCI Bus
   1. In a CIOB 2.1 system the adapters will run at PCI-33
   2. In a CIOB 2.2 system the adapters will run at the capability of the first one added
   3. In a CIOB 2.2 system, if the adapters are of different capabilities, the one with the lower capabilities must be added first
3. Any single-port NIC (including SSL) can be hot added to a PCI Bus that contains an existing single-port NIC of the same or lower capabilities
   1. A PCI-X single-port NIC can be added to a bus with any other single-port NIC
   2. A PCI-66 single-port NIC can be added to a bus with a PCI-66 or PCI-33 single-port NIC
   3. A PCI-33 single-port NIC can be added to a bus with a PCI-33 single-port NIC
4. NICs may not be hot added to a PCI Bus that contains any existing adapter other than a NIC.