Dolphin StorExpress
DSE4XM1
DSE4XM2
User Guide for Windows
Revision Information:
Revision 0.3
Dolphin Interconnect Solutions
Contents
Features and Performance. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
Specifications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
Half-height Bracket Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
Connecting the DXH510 to Chassis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
Powering up the DSE4XM. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Troubleshooting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
Creating a RAID Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
Common Maintenance Tasks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Uninstalling the ioDrive Windows Driver . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Upgrading the ioDrive Firmware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28
Performing the Upgrade. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
March 10, 2009
Dolphin Interconnect Solutions
3
Manual Installation Procedures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
Manual Install on Windows XP Pro or Windows Server 2003. . . . . . . . . . . . . . . . . . . . . . 36
Manual Install on Windows Vista or Windows Server 2008 . . . . . . . . . . . . . . . . . . . . . . . 38
Enabling SNMP. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
March 10, 2009
Dolphin Interconnect Solutions
4
1
Introduction
Congratulations on your purchase of a Dolphin StorExpress product. This guide pro-
vides user instructions for the following StorExpress models:
•
•
DSE4XM1
DSE4XM2
These products are part of the DSE4XM family of StorExpress products.
Dolphin’s StorExpress is a PCI Express based solid state storage system designed to
deliver superior response times and storage capacity. Providing large capacity and fast
direct storage, StorExpress connects directly to any PCI Express system. It is the ideal
solution for deploying fast solid state cache solutions to improve the performance of
database or web-based applications. StorExpress systems integrate with Dolphin’s PCI
Express cluster technology enabling businesses to easily build fast distributed storage
solutions. Capable of being connected to multiple servers, StorExpress provides a flexi-
ble and scalable storage solution which can be distributed within the enterprise to
improve access to latency sensitive data. In a multi-server configuration, StorExpress is
also capable of functioning as a clustering switch and can obviate the need for a dedi-
cated clustering fabric for inter-server communication
Offering both superior power utilization and outstanding performance, this fl ash based
solution signifi cantly lowers annual power usage while delivering outstanding perfor-
mance. StorExpress leverages the latest solid state SLC NAND technology to provide
extremely faststorage solutions that outperform traditional disk based systems.
1.1 System Requirements
The DSE4XM requires an open x4 or x8 PCI Express slot for each DXH510 Host
Adapter card.
1.2 DSE4XM Components
Your DSE4XM product includes the following items:
DSE4XM chassis
•
March 10, 2009
Dolphin Interconnect Solutions
5
•
•
•
•
DXH510 PCI Express host adapter card
DXH510 half-height bracket
2 - 1m CX4 cables
StorExpress USB Key, containing
–
–
–
DSE4XM User’s Guide for Windows
DSE4XM User’s Guide for Linux
ioManager User’s Guide
•
•
Quick Start Instructions
Configuration Sheet
1.3 Features and Performance
The DSE4XM product has the following features:
•
•
480 Gbyte or 960 Gbyte NAND Flash SLC using Fusion-IO ioDrive® technology
Performance for x4, x8 and dual x8 server connection(s), respectively:
–
–
–
800 MB/s, 1500 MB/s, and 2700 MB/s (read)
600 MB/s, 1250 MB/s, and 2500 MB/s (write)
80,000, 150,000, and 270,000 I/O operations per second (IOPS)1
•
•
< 50 microseconds read access latency2
Server connection options
–
–
–
–
–
x4 PCI Express for up to four servers
x4 or x8 PCI Express for up to two servers
x4, x8, or dual x8 PCI Express for one server
Requires PCI Express slot in server (two slots for dual x8 connection)
Additional DXH510 host adapter required for each server connection or dual
x8 connection
•
Supports external 4-channel parallel fiber-optic transceivers
Extended distance up to 100 meters
–
1. Based on 4KB packet sizes
2. Based on 4KB packet sizes
March 10, 2009
Dolphin Interconnect Solutions
6
1.4 Specifications
1.4.1 Mechanical
The following specifications are for the DSE4XM chassis:
•
•
Standard 4U rackmount chassis
Dimensions
–
–
–
Height: 6.96” (4U /176.8mm)
Depth: 18.9 (480mm)
Width: 19” (483mm)
•
Weight approximately 32lbs (14.5kg)
1.4.2 Environmental
•
•
•
Operating Temperature: 00C to 500C ambient
Storage Temperature: -200C to 700C
Relative Humidity: 15-85% non-condensing
1.4.3 Electrical
•
•
Operating Voltage:100-240 VAC at 50-60Hz
Power Supply: 300 watt dual PSU
1.4.4 Agency Approvals
•
•
•
FCC Part 15, Class A
EMC Directive CE Mark, EN55022, EN55024, 61000-3-2, 61000-3-3, EN6950-1
VCCI, Class A
1.5 Customer Support
If you continue to have issues with your system or need service, please contact Dolphin
customer support at http://www.dolphinics.com/support
March 10, 2009
Dolphin Interconnect Solutions
7
2
Hardware Installation
This chapter describes how to install your DSE4XM hardware.
Your DSE4XM includes a DXH510 PCI Express host adapter card and two 1-meter
CX4 cables. These components support a x8 connection between the DSE4XM chassis
and one server, which is the standard shipping configuration for the DSE4XM.
The DSE4XM supports additional configurations accommodating connections to up to
four servers, with a fixed amount of storage allocated to each server. Each additional
server requires a DXH510 adapter card and cable to connect to the DSE4XM chassis.
Allocation of memory to each server is specified at the time the product is ordered and
pre-configured at the factory. Dual x8 connection to a single server also requires an
additional DXH510 and factory pre-configuration. Please contact Dolphin customer
DSE4XM configuration.
2.1 DSE4XM Chassis Overview
instructions.
March 10, 2009
Dolphin Interconnect Solutions
8
Figure 2–1 DSE4XM Chassis Rear
Table 2–1 DSE4XM Chassis Rear Features
Label
Name
A
AC Power Inlet
AC Power Switch
Uplink Slot 0
B
C
D
Uplink Slot 1
E
ioDrive SSD Slot
For Manufacturing Use Only
Connector P0
F
C/D - 0
C/D - 1
Connector P1
Figure 2–2 shows the front of the DSE4XM chassis with the front cover open.
March 10, 2009
Dolphin Interconnect Solutions
9
Figure 2–2 DSE4XM Chassis Front
Caution: For protection of the StorExpress devices, we recommend that you do not attempt to
service or remove the cover of your DSE4XM chassis. However, if you do, power should
be disconnected from the chassis prior to servicing the chassis or removing the cover
for any reason. In order to disconnect power remove the power cord from its receptacle
connect switch for system power.
2.2 Installing DXH510 Card in Server
are labeled in this figure.
March 10, 2009
Dolphin Interconnect Solutions
10
Figure 2–3 DXH510 Adapter Card
Caution: Electrostatic discharge (ESD) can damage electronic components. Be sure that you are
properly grounded prior to any hardware installation procedure using an ESD
protection device such as a wrist strap.
If you are installing the DXH510 in a low-profile system and need to use the half-height
bracket, please refer to the Section 2.2.1.
1. Locate the serial number on the back of your DXH510 and record it for future refer-
ence.
2. Power off the server and disconnect the power cable.
3. Remove the server's access panel. Locate an available PCIe slot.
–
Consult your server's documentation for details on removing the panel and
identifying PCIe slots.
Note: Your DXH510 is designed for use in a x8 PCIe slot. It will work in a x16 slot, but this
does not improve performance. Your DXH510 can also work in a x2 or x4 slot but
performance may be diminished in a x2 slot, or in a x4 slot if a x8 connection is
intended.
4. Follow server manufacturer’s instructions for installing PCIe add-in cards.
5. Replace the server’s access panel
6. Plug in the server’s power cable
March 10, 2009
Dolphin Interconnect Solutions
11
You have completed the hardware installation of the DXH510 into the server. If you are
connecting multiple servers to the DSE4XM chassis, repeat this process for each server.
Each server requires a DXH510 host adapter to connect to the DSE4XM chassis. If you
are making a dual x8 connection to a single server, locate a second PCI Express slot and
install the second DXH510 adapter.
2.2.1 Half-height Bracket Installation
For installation in a low-profile server, the full-height bracket must be replaced with the
half-height bracket.
Caution: Electrostatic discharge (ESD) can damage electronic components. Be sure that you are
properly grounded prior to any hardware installation procedure using an ESD
protection device such as a wrist strap.
1. Locate the half-height bracket in the DSE4XM package
to the DXH510. Do not attempt to remove the bracket until the screws have been
fully removed.
3. Remove the bracket carefully from the device.
4. Align the connectors on the DXH510 with the openings in the half-height bracket:
Be sure the bracket tabs are on the backside of the device.
5. Adjust the bracket so that the notches on the bracket tabs fit around the guides on
the back of the card. The bracket guides are illustrated in Figure 2–4.
6. Attach the half-height bracket using a screwdriver to tighten the four screws. Do
not overtighten the screws as this can damage the device.
March 10, 2009
Dolphin Interconnect Solutions
12
Figure 2–4 DXH510 Adapter Card Rear View
2.3 Connecting the DXH510 to Chassis
A server connects to the DSE4XM chassis using the DXH510 PCI Express host adapter
card. One or two cables are used to connect the DXH510 to the DSE4XM chassis. One
cable provides a x4 (8 Gbps) connection; two cables provide a x8 (16 Gbps)
connection.
Two 1-meter cables (Model DXC1M-A) are provided as a part of the standard
DSE4XM package. Additional compatible cables are available from Dolphin for
www.dolphinics.com/support for the most up to date compatible cable offerings.
Table 2–2 Dolphin DSE4XM Compatible Cables
Part Number
DXC1M-A
Description
1 Meter copper cable
3 Meter copper cable
10 Meter fiber optic cable
50 Meter fiber optic cable
100 Meter fiber optic cable
DXC3M-A
FCDX10M-A
FCDX50M-A
FCDX100M-A
The DSE4XM chassis has two pairs of connectors (four total), which are used to cable
is located at Uplink Slot 0; the second set of connectors is located at Uplink Slot 1. The
lower connector is P0 and the upper connector is P1.
March 10, 2009
Dolphin Interconnect Solutions
13
One set of connectors can be used for a x8 connection when cabled to a single
DXH510, or for two x4 connections when each connector in the set is cabled to a
different DXH510 adapter card. Two sets of two connectors allow for connection to up
to four DXH510 adapter cards in this manner.
connectors are used for a x8 connection, or a single connector may be used for a x4
connection.
Table 2–3 shows how the DXH510 card(s) are cabled to the DSE4XM for
configurations of one to four servers.
Table 2–3 DXH510 to DSE4XM Chassis Cable Connections
One Server
Uplink Slot 0: P0 Uplink Slot 0: P0 Uplink Slot 0: P0 Uplink Slot 0: P0
Uplink Slot 0: P1‡ Uplink Slot 0: P1‡ Uplink Slot 0: P1‡
Two Servers
Three Servers
Four Servers
DXH510: 1 P0
cables to
cables to
cables to
cables to
cables to
cables to
cables to
cables to
P1
DXH510: 2 P0
P1
–
–
–
–
–
–
–
Uplink Slot 1: P0 Uplink Slot 1: P0 Uplink Slot 0: P1
Uplink Slot 1: P1‡
–
–
DXH510: 3 P0
P1
–
–
–
–
Uplink Slot 1: P1 Uplink Slot 1: P0
–
–
–
–
DXH510: 4 P0
P1
Uplink Slot 1: P1
–
‡
Only required if a x8 connection is desired. May be left unconnected if a x4 connection only is needed.
The single server dual x8 configuration is similar to the two server configuration,
however in this case both DXH510:1 and DXH510:2 adapters reside in the same server.
Attach the cable(s) to the required connectors for your configuration as specified by
March 10, 2009
Dolphin Interconnect Solutions
14
Figure 2–5 DSE4XM with One x8 Connection
Figure 2–6 DSE4XM with Four x4 Connections
Figure 2–7 DSE4XM with Two x8 Connections
March 10, 2009
Dolphin Interconnect Solutions
15
Note: Cables should be strain relieved or strapped to a cabinet/rack to ensure additional
reliability.
Note: Cables should be connected according to the configuration of your DSE4XM product.
The configuration is documented on the configuration sheet included with your
product. The standard StorExpress configuration is single server. Multiple server
configurations require additional DXH510 adapter cards and cables, which may be
2.4 Powering up the DSE4XM
1. Connect main power (100-240VAC) to the DSE4XM chassis using the provided
AC power cord or a regionally appropriate AC power cord to the DSE4XM chassis
power connector, which is located on the back of the chassis, as shown in Figure 2–
1. The AC power cord provided by Dolphin is a Northern American 3- prong plug.
A different power cord, with the appropriate power prongs, is required for other
geographies that don’t support the North American standard. The chassis accepts
any IEC 320 EN60320 C13 connector and the power supply provided with the
DXE410 accepts any voltage from 100V to 240V at a frequency of 50-60Hz.
Auxiliary power is supplied to the DSE4XM when AC power is connected.
2. An AC power switch for each power supply unit is located on the back of the chas-
placed in the on position to provide full AC power to the chassis.
This switch can be used to manually fully power up the chassis.
The DSE4XM chassis provides automatic remote power-on capabilities. Toggling
the front power button is not necessary to fully power to the chassis. As long as AC
power is switch on, the DSE4XM will fully power up when the server (or first
server) is powered on.
–
When fully powered, the DSE4XM will go through a self test. You will observe
that for 0.5 seconds the P0 and P1 LEDs on the Uplink slots will turn green.
Then for 0.5 seconds, the P0 and P1 LEDs turn yellow. If the servers connected
to the DSE4XM are powered down, or if the connectors are not cabled, then the
LEDs will turn off.
4. Power-on the server that contains the DXH510. If not already powered, after server
power is turned on, the DSE4XM will come on automatically. This is indicated by
the following:
–
–
Chassis fans will turn on
The Uplink Slot P0 and P1 LEDs will transition from yellow to off and then
green for connected ports, as described in the previous step.
5. Check the P0 and P1 LEDs for both the DXH510 and the DSE4XM Uplink slot(s)
–
–
For a x8 or a single x4 connection, the P0 LED should be illuminated
For two x4 connections, both the P0 and P1 LEDs should be illuminated
March 10, 2009
Dolphin Interconnect Solutions
16
are prompted by the operating system for a driver, click Cancel.
7. Repeat the power-on procedure for any additional servers you may have connected
to the DSE4XM.
Note: If you are running a Windows operating system, Dolphin provides drivers to support the
support
2.5 Troubleshooting
Table 2–4 Troubleshooting Tips
Symptom
Tip
No Power to board
1) Check AC power cord for power.
2) Ensure rear power switch is turned on
Both Uplink slot LEDs are off when con- 1) Check cable connections
nected to server (both server and the
DSE4XM must be powered)
LED(s) are blinking when connected to
server
Contact customer support
Contact customer support
Memory cards do not appear in device
manager or device list
Host does not boot properly
System hangs on boot
Contact customer support
Contact customer support
Red Alarm LED is illuminated
Red Alarm LED is flashing
TBD
TBD
March 10, 2009
Dolphin Interconnect Solutions
17
3
Software User Guide
3.1 Software Installation
This section describes how to install software for your DSE4XM product. The
DSE4XM storage device is the Fusion-IO ioDrive. The software will reference ioDrive.
3.1.1 New ioDrive Installation
To install the ioDrive software on a new system:
1. Make sure you have completed the DSE4XM hardware installation steps in
part of the installation.
3. Login in as Administrator or have Administrator rights.
support to your desktop or a convenient directory.
5. Run the Setup program.
6. Follow the onscreen prompts to complete the install. (If Setup requests you to ver-
ify the driver install, click Yes to continue.)
The Setup program will:
–
–
–
–
–
Create a Program Files/Fusion-io folder for the ioDrive software components.
Create shortcuts to the utilities and documentation in this folder.
Install and load the ioDrive Windows driver.
Install the ioManager administrator console.
Create an ioManager desktop icon.
It will also create shortcuts in the All Programs/Fusion-io folder for:
–
–
–
User documentation
ioManager and ioManager Remote
SNMP License Readme file
March 10, 2009
Dolphin Interconnect Solutions
18
When Setup creates the Fusion-io folder on the drive, it also creates these sub-fold-
ers:
–
–
–
–
–
–
Docs—for the ioDrive User documentation
Driver—for the latest ioDrive Windows driver
Firmware—for the latest ioDrive firmware
ioManager—for the administrator console software and manual
SNMP—for components of SNMP
Utils—for the ioDrive command line utilities
7. Proceed to the appropriate section to continue:
–
–
3.1.2 Existing ioDrive Installation
To install the latest ioDrive Windows software on an existing installation:
1. Review the Release Notes and the Errata file available for this version of the soft-
ware for additional steps that may be needed to complete the install.
part of the installation.
3. Login as Administrator (or have administrator rights).
4. Uninstall the existing Fusion-io driver using Device Manager.
support to your desktop or a convenient directory.
6. Run the Setup program.
7. Follow the onscreen prompts to complete the install. (If Setup requests you to ver-
ify the driver install, click Yes to continue.)
8. Click Restart once the Setup program finishes.
9. Once the system reboots, proceed to the appropriate section to continue:
–
–
March 10, 2009
Dolphin Interconnect Solutions
19
3.1.3 Outdated Firmware Check on Windows XP Pro or Windows 2003 Server
After installing the ioDrive software, check the Windows System Event Log for fiodrive
entries. These entries will show details of the ioDrive driver status.
To view the Windows System Event Log:
1. Choose Start > Control Panel
2. Click on Administrative Tools.
3. Click on Event Viewer.
4. Click on System in the console tree. The System Event Log appears in the dialog.
5. Click on the Source column to resort the list by source.
6. Scroll through the list to identify any fiodrive Error entries:.
7. Double-click on the error entry in the list to view its details.
Check to see if the fiodrive Error entry says “ioDrive (x) firmware is too old”. (The x is
the PCI bus number of the ioDrive you just installed.)
March 10, 2009
Dolphin Interconnect Solutions
20
If there is a fiodrive error entry warning of old firmware, follow the instructions in the
Section 3.2.4.3 later in this guide to update the firmware.
If the System Event Log shows no fiodrive error entries (as in the example above), then
your ioDrive is ready to receive a Windows file system.
3.1.4 Outdated Firmware Check on Windows Vista or Windows Server 2008
After installing the ioDrive software, check the Windows System Event Log for fiodrive
entries. These entries will show details of the ioDrive driver status.
To view the Windows System Event Log:
1. Click Start.
2. Type event viewer in the Search box.
3. Press Enter to launch the Event Viewer.
4. Click Windows Logs in the console tree.
March 10, 2009
Dolphin Interconnect Solutions
21
5. Select System in the console tree. The System Event Log appears in the dialog.
6. Click the Source column to re-sort the list by source.
7. Scroll through the list to identify any fiodrive error entries.
8. Highlight an entry in the list to view its details.
Check to see if there is a fiodrive Error entry that reads “ioDrive (x) firmware is too
old”. (The x is the number of the ioDrive you just installed.)
March 10, 2009
Dolphin Interconnect Solutions
22
If there is a fiodrive error entry warning of old firmware, as in the example above,
If the System Event Log shows no fiodrive error entries, then your ioDrive is ready to
receive a Windows file system.
If you choose to enable SNMP support, refer to Section 3.6.
3.1.5 ioDrive Naming
The ioDrive receives a name and number as part of the install process for identification.
The syntax is fctx where x is the number of the PCIe bus where you installed the
ioDrive. Use ioManager to view this bus number or you can:
1. Choose Start > Control Panel > System > Hardware > Device Manager.
2. Select Fusion-io Devices. (Select System Devices for pre-1.2.3 drivers.)
3. Click on your ioDrive in the list. The Properties dialog appears.
The Location field shows the PCIe bus number for your device (fct13 in this case):
March 10, 2009
Dolphin Interconnect Solutions
23
Note: The system manufacturer assigns bus numbers, which can range from 0 on up.
These numbers may or may not reflect the physical location of the bus. For example, the
second slot from the edge of the motherboard may be Bus 2, but it could also be Bus 16
or another arbitrary number. Checking Device Manager is one way to confirm the
specific bus number for your installation. You can also use ioManager to view this
number.)
3.1.6 Adding a File System to the ioDrive
With the ioDrive and driver installed, you can now use the Windows Disk Management
utility to make your device available to applications. Typically, Windows will detect the
new device, initialize it and display it in Disk Management. You can then add
partitions, format a volume, or create a RAID configuration on your ioDrive using the
standard Windows procedures (see the Windows Disk Management Utility documentation
for more details)
If Windows does not, you may need to initialize it manually. To initialize an ioDrive:
1. Select Start > Control Panel.
2. Click Administrative Tools.
3. Click Computer Management.
4. Click Disk Management in the Storage section of the console tree.
March 10, 2009
Dolphin Interconnect Solutions
24
5. Locate and right-click the ioDrive in the list of storage devices on the right. (If the
ioDrive does not appear in the list, choose Rescan Disks from the Action menu.
You may also need to restart your computer to display the ioDrive in the list.)
6. Click Initialize Disk.
You can now use the Disk Management Utility to add a file system to your ioDrive.
3.1.7 Creating a RAID Configuration
You can use your ioDrive as part of a RAID configuration. To do so, you must format
your ioDrive as a dynamic volume. In turn, you can then use this dynamic volume to
create multi-disk RAID configurations (spanned, striped, mirrored, or RAID 5).
Note: Windows XP Professional does not support software RAID or mirroring. It does
support spanning and striping.
For specific steps to perform a RAID configuration, see the Windows Disk Management
Utility documentation for details.
Note: If you are using RAID1/Mirroring, and one device fails, be sure to run a fio-
format on the replacement device (not the remaining good device) before rebuilding the
RAID.
3.2 ioDrive Maintenance
The ioDrive includes both software utilities for maintaining the device as well as
external LED indicators to display its status.
3.2.1 ioDrive LED Indicators
Each ioDrive slot includes three LEDs showing drive activity or error conditions. The
Figure 3–1 ioDrive Slot LEDs
March 10, 2009
Dolphin Interconnect Solutions
25
Table 3–1 ioDrive Slot LED Indicators
Amber/
Red*
OFF
Green
Yellow
Indicates
Notes
Power off
OFF
OFF
OFF
Power on
(driver not loaded)
Power on
Load driver
ON
ON
ON
OFF
OFF
(driver loaded)
Writing (rate indicates
volume of writes)
Reading (rate indicates
volume of reads)
Can appear in combination
with the Read LED
Can appear in combination
with the Write LED
Also appears during a
firmware update
ON
FLASH OFF
FLASH
ON
OFF
OFF
Location beacon
ON
ON
* Later versions of the ioDrive use an amber-colored LED instead of a red LED.
3.2.2 The ioManager Console
Your ioDrive software includes the ioManager console application. This GUI performs
the most common operations you need to do with the ioDrive. In addition, it provides a
detailed information screen on each of your installed devices.
The ioManager can perform:
•
•
•
Firmware upgrades
Low-level formatting
Attach and detach actions
The ioManager installs as part of your Windows Setup process. Details on how to use
ioManager appear in the ioManager User Guide available in the All Programs/Fusion-
io/ioManager menu.
3.2.3 ioDrive Command Line Utilities
The Windows Setup package also includes five command line utilities for managing
your ioDrive. They include:
•
•
•
•
•
fio-attach
fio-detach
fio-format
fio-status
fio-update-iodrive
Each of these utilities is described in detail in Section 3.5.
March 10, 2009
Dolphin Interconnect Solutions
26
3.2.4 Common Maintenance Tasks
The most common tasks involve drivers and firmware.
3.2.4.1 Uninstalling the ioDrive Windows Driver
To uninstall the ioDrive Windows driver:
1. Go to Start > Control Panel.
2. Click Administrative Tools.
3. Click Computer Management.
4. Click Device Manager in the console tree at the left.
5. Expand the Fusion-io Devices item. (Select System Devices for pre-1.2.3 drivers.)
6. Right-click on the desired ioDrive.
7. Click Uninstall.
Windows will uninstall the driver.
3.2.4.2 Upgrading the ioDrive Windows Driver
To upgrade the ioDrive Windows driver:
1. Refer to the Release Notes and the Errata file for this new version of the driver for
details on any additional steps to perform the upgrade.
2. Follow the steps above to uninstall the existing driver.
4. Either unzip or run the Windows package to copy the files to a convenient direc-
tory.
5. Go to Start > Control Panel.
6. Click Administrative Tools.
7. Click Computer Management.
8. Click Device Manager in the console tree at the left.
9. Expand the Fusion-io Devices item. (Select System Devices with pre-1.2.3 drivers.)
10. Right-click on the desired ioDrive.
steps to install the updated driver.
The operating system will now detect your ioDrive.
Note: Read both the release notes and the errata files that come with each new release as
well as these installation instructions to ensure no loss of data when performing upgrades.
March 10, 2009
Dolphin Interconnect Solutions
27
3.2.4.3 Upgrading the ioDrive Firmware
Caution: You should upgrade the firmware only if the System Event Log reports out-of-
date firmware or if instructed to do so by Dolphin Customer Support.
3.2.4.3.1 Viewing the Firmware Version
You can view the ioDrive’s firmware version using the Windows System Event Log.
You can then use this information to evaluate upgrading to a newer firmware release.
To view the firmware version in the System Event Log:
1. Choose Start > Administrative Tools.
2. Click Event Viewer.
3. Click System in the console tree at the left. The System Event Log appears in the
dialog.
4. Click the Source column to re-sort the list by source.
5. Scroll through the list to identify any fiodrive entries.
6. Highlight each entry for fiodrive that is of the “Information” type and view its
details. Each installed ioDrive will have an entry listing its firmware version.
Record this firmware version information for future reference.
March 10, 2009
Dolphin Interconnect Solutions
28
3.2.4.3.2 Performing the Upgrade
Warning: It is extremely important that the power not be turned off to either the server
or the DSE4XM chassis during a firmware upgrade, nor should the cable be disconnected.
Power or connectivity loss during a firmware upgrade could cause device failure. Consider
adding a UPS to the system prior to performing a firmware upgrade to prevent this from
happening.
Caution: You should back up the data on the ioDrive prior to any upgrade as a precaution.
Note: You must upgrade the ioDrive driver if you upgrade the device firmware.
Note: To correctly perform the firmware upgrade, you must power the system,
including the DSE4XM chassis, all the way down and bring it all the way back up. This
shutdown and restart is required for the upgrade to complete on the ioDrive.
Note: A firmware upgrade may in some instances require a low-level format. Refer to
the release notes for specific details.
Note: Upgrading the firmware may take several minutes. The update command
displays a progress bar to indicate the pace of the upgrade.
Typically, you use the ioManager console to perform firmware upgrades. You can also
perform the upgrade using the fio-update-iodrive command line utility.
To upgrade the firmware on the ioDrive using fio-update-iodrive:
1. Open a Command Prompt. Navigate to the Program Files\Fusion-io\utils directory.
2. Run the Detach command for your device at the Command Prompt. For example:
fio-detach /dev/fct0
detaches the ioDrive labeled “fct0”. You need to specify the name of your drive you
want to detach. (Each ioDrive is labeled /dev/fct# where # is its PCIe bus number.
Use fio-status at a Command Prompt to view this number.)
The fio-detach command will show a progress bar and percentage as it completes
the operation.
3. Type the following command at the Command Prompt:
fio-update-iodrive <path>iodrive_version.fff
where <path>is the location of the ioDrive firmware. The default path is Program
Files\Fusion-io\firmware. The version is the numerical version of the drive.
Note: All three of the ioDrive slot LED indicators will light during the update pro-
cess.
4. Completely power down the server and the DSE4XM chassis. Turn off the AC
power to the DSE4XM chassis, and turn it back on again. Restart the server, which
March 10, 2009
Dolphin Interconnect Solutions
29
will completely power up the DSE4XM chassis, to complete the firmware upgrade.
(It must be a power down and restart of both the server and the chassis to ensure a
complete reset of the+ system and the ioDrive(s). If there are multiple servers con-
nected to the DSE4XM chassis, these servers should be powered down as well, and
powered back up after the DSE4XM is powered up.)
3.2.5 Unmanaged Shutdown Issues
Unmanaged shutdowns due to power loss or other circumstances will force the ioDrive
to perform a consistency check during the reboot. This may take several minutes or
more to complete and is shown by a progress percentage during Windows startup.
You can cancel this consistency check by pressing Esc during the first 15 seconds after
the Fusion-io Consistency Check message appears at the prompt. If you choose to
cancel the check, however, the ioDrive(s) will remain unavailable to users until the
check is done. (You can perform this check later on using ioManager’s Attach
function).
Although data written to the ioDrive will not be lost due to unmanaged shutdowns,
important data structures may not have been properly committed to the drive. This
consistency check repairs these data structures.
3.2.6 Disabling Auto-Attach
The ioDrive Windows driver defaults to automatically attach (auto-attach) all installed
ioDrives to the operating system. (If the ioDrive does not attach, it will not be available
to applications or users.) You can disable auto-attach to assist in troubleshooting or
diagnostics.
To disable auto-attach:
1. Choose Start/Run.
2. Type in the command:
Regedit <path>autoattachdisable.reg<enter>
where <path>is the folder where the autoattachdisable.reg file is. The default is:
C:\Program Files\Fusion-io\Utils
March 10, 2009
Dolphin Interconnect Solutions
30
This will create a new DWORD parameter registry key called AutoAttach with a hex
value in:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\fiodrive\Parameters
Your ioDrive will now not automatically attach the next time you reboot the computer.
When you finish troubleshooting the driver issue, use ioManager to attach the
ioDrive(s) and make them available to Windows.
3.2.6.1 Enabling Auto-Attach
To re-enable auto-attach after disabling it with the autoattachenable.reg file:
1. Choose Start/Run.
2. Type in the command:
Regedit <path>autoattachenable.reg<enter>
where <path> is the folder where the autoattachenable.reg file is. The default is:
C:\Program Files\Fusion-io\Utils
This will reset the AutoAttach parameter in the Registry. The next time you reboot your
Windows system, your ioDrive will automatically attach.
March 10, 2009
Dolphin Interconnect Solutions
31
3.3 Troubleshooting Event Log Messages
The Windows System Event Log will display fiodrive messages concerning the
ioDrive—both Informational and Warnings.
Note: Each ioDrive is numbered from 0 upwards. These numbers reflect the PCIe bus
number where you installed the device. Use ioManager to view this number for your
device.
The following are the most common Event Log error messages.
Message:
Error: ioDrive(x) firmware is too old. The firmware must be updated.
Suggested Solution:
Use the firmware upgrade instructions in this guide to update the firmware.
March 10, 2009
Dolphin Interconnect Solutions
32
Message:
Error: The ioDrive(x) initialization failed with error code 0xerrorcode (where errorcode
is a number which may vary)
Suggested Solutions:
•
•
•
Reinstall the Windows driver
Remove and reseat the ioDrive device
Remove and insert the ioDrive in a different PCIe slot
If these do not correct the error, please contact Dolphin Customer Support.
Message:
Error: The ioDrive(x) was not attached. Use the fio-attach utility to rebuild the drive.
March 10, 2009
Dolphin Interconnect Solutions
33
Suggested Solution:
This error may appear if the system shut down abruptly or hung. You can use either the
fio-attach command line utility or ioManager to re-attach the ioDrive. This attach
process may take up to ten minutes as the utility rebuilds the drive.
Message:
Warning: The ioDrive(x) was not attached because auto-attach is disabled.
March 10, 2009
Dolphin Interconnect Solutions
34
Suggested Solution:
The ioDrive must attach to the Windows operating system to be available to users and
applications. (This attach normally occurs at boot time.) As part of this attach process,
the ioDrive driver checks to see if there is an AutoAttach parameter in the Windows
registry. If you create this Registry parameter to disable auto-attach, the attach
operation does not complete. The driver will then display this message.
To attach an unattached ioDrive:
1. Run ioManager.
2. Select your unattached ioDrive from the Device Tree.
3. Click the Operations tab.
4. Click Attach.
5. Confirm the Attach operation.
Your ioDrive now attaches to the Windows operating system.
March 10, 2009
Dolphin Interconnect Solutions
35
To re-enable Auto Attach at boot time, refer to the section on Enabling Auto Attach.
3.4 Manual Installation Procedures
The Windows Setup program should install your needed driver and software. However,
if the driver does not install, you can perform a manual install (or upgrade) using the
steps below for either Windows XP Pro/Server 2003 or Windows Vista/Server 2008.
3.4.1 Manual Install on Windows XP Pro or Windows Server 2003
Windows Driver Wizard will automatically detect the new ioDrive and start to locate its
driver after you restart the system. To complete the install:
1. Windows will ask you to locate the software driver.
2. If you have not done so, download and run the ioDrive Windows Setup program
3. Return to the Update Driver dialog.
4. Click Browse my computer for driver software. The Setup program will ask you
for a path to search.
March 10, 2009
Dolphin Interconnect Solutions
36
5. Click Browse next to the path field. Windows will display a file dialog.
6. Select the folder with the ioDrive driver (the default is Program Files\Fusion-
io\driver).
7. Click OK.
Windows will find the correct driver and install the device software. When the driver install
completes, Windows will display this message:
March 10, 2009
Dolphin Interconnect Solutions
37
Proceed to the appropriate Windows section in this guide to perform an outdated
firmware check.
3.4.2 Manual Install on Windows Vista or Windows Server 2008
The Windows Driver Wizard will automatically detect the new ioDrive and start to
locate its driver after you restart the system. To complete the install:
1. Windows will ask you to locate the software driver.
March 10, 2009
Dolphin Interconnect Solutions
38
2. Click Browse my computer for driver software. The Setup program will ask you
for a path to search.
March 10, 2009
Dolphin Interconnect Solutions
39
3. Click Browse next to the path field. Windows will display a file dialog.
4. Select the folder with the ioDrive driver (the default is Program Files\Fusion-
io\driver).
5. Click OK.
6. Click Next.
Windows will find the correct driver and install the device software.
When the driver install completes, Windows will display this message:
March 10, 2009
Dolphin Interconnect Solutions
40
Proceed to the appropriate section for your Windows version to perform the outdated
firmware check.
3.5 Command Line Utilities Reference
The Windows Setup package installs various utilities into the Program Files\Fusion-
io\utils folder. These provide a number of useful ways to access, test, and manipulate
the ioDrive via the command line (Command Prompt). They include:
Utility
Purpose
fio-attach
Makes an ioDrive available to Windows
Removes an ioDrive from Windows access
Used to perform a low-level format on an ioDrive
Displays information about the ioDrive
Updates the ioDrive’s firmware
fio-detach
fio-format
fio-status
fio-update-iodrive
Note: Each utility has –h (for Help) and –v (for Version) options.
March 10, 2009
Dolphin Interconnect Solutions
41
fio-attach
Description
Attaches the ioDrive and makes it available to Windows. You can then partition,
format, or set up the ioDrive as part of a RAID array using the Windows Disk
Management utility. This command displays a progress bar and percentage as it
completes the attach process.
Note: In most cases, the ioDrive driver automatically attaches the device at boot time.
You only need to run fio-attach if you ran fio-detach or if you create and set the
ioDrive’s AutoAttach parameter in the Windows Registry to 0. (See Disabling Auto-
Attach for details.)
Syntax
fio-attach <device-node> [-q, -h, -v]
where <device-node>is the name given by the ioDrive driver to your device. This
name is /dev/fctxwhere x indicates the PCIe bus number where you installed the
ioDrive. (For example, the name /dev/fct4 refers to the ioDrive installed in PCIe
Bus 4 in your Windows system. Use ioManager or fio-status to view this bus number.)
Option
-q
Quiet: Runs the Attach process without displaying the progress bar or
percentage.
fio-detach
Description
Detaches the ioDrive and removes the corresponding “fctx” ioDrive block device from
the OS. The fio-detach command will wait until the device completes all read/write
activity before executing the detach operation. The command also displays a progress
bar and percentage as it completes the detach.
Caution: Before using this utility, ensure that the device you want
to detach is not currently mounted and in use.
Syntax
fio-detach <device-node> [-i, -q, -h, -v]
March 10, 2009
Dolphin Interconnect Solutions
42
where <device-node>is the name given by the ioDrive driver to your device. This
name is /dev/fctxwhere x indicates the device number. (The number reflects the
PCIe bus for the ioDrive.) For example, the name /dev/fct4 refers to the ioDrive
installed in PCIe Bus 4 in your Windows system. (Use ioManager or fio-status to view
this number.)
Options
-i
Immediate: Causes a forced immediate detach
(does not save metadata)
-q
Quiet: Runs the Detach process without displaying the progress bar or
percentage.
fio-format
Description
Performs a low-level format of the ioDrive. (This format is distinct from a format
performed by Windows.) The fio-format utility displays a progress bar and percentage
as it completes the format.
Note: The ioDrive ships pre-formatted and does not require the use of fio-format
except to change the logical size of the device.
Caution: Use this utility with care, as it deletes all user information on the ioDrive.
Syntax
fio-format <ioDrive-device> [-q, -s <device-size>, -f, -u, -h, -v]
where <ioDrive-device>is the name given by the ioDrive driver to your device.
This name is /dev/fctxwhere x indicates the device number (this number is the
ioDrive’s PCIe bus). Use ioManager or fio-status to view this number.
March 10, 2009
Dolphin Interconnect Solutions
43
Options
-q
Quit on erase error.
-s <device-size>
Sets the storage size by a number and one of the
following units as a suffix:
•
•
•
•
% Percentage of drive for user data
t/T Number of terabytes to format
g/G Number of gigabytes to format
m/M Number of megabytes to format
For example, setting the size as either 90% or 72GB
on an 80GB ioDrive both result in the same
capacity for user data—72GB.
-f
Force size: Bypass sizing sanity checks to enable 64-
bit address space. This option also bypasses
interactive prompts in the formatting process.
-u
Quiet: Disables the display of the progress bar or
percentage.
fio-status
Description
Provides detailed information about the specified or all installed ioDrive(s). The fio-
status command requires that the Windows driver be loaded. If it finds no device, it
displays an index number instead.
Syntax
fio-status </dev/fctx> [-c, -a, -h, -v]
where /dev/fctx refers to the name of this device and the x is its PCIe bus number. If
you don’t specify a name, fio-status displays information on all installed ioDrives.
Options
March 10, 2009
Dolphin Interconnect Solutions
44
Count: Report only the number of ioDrives installed.
-c
-a
All: Print all available information for each device. The fio-status
command displays the following information:
•
•
•
•
•
•
•
•
Serial number
•
•
•
•
Hardware UID
Part number
Bus ID
Manufacturer's name
Manufacturing date
Vendor ID
Subsystem vendor and device IDs
Device ID
Firmware version at manufacturing time •
Size of the device, out of total capacity •
PCI slot number
NAND manufacturer
•
Internal temperature (avg. and max.,
since driver load) in degrees Centigrade
NAND cell type (SLC, MLC, or
unknown)
•
Ambient temperature, in degrees Centi-
grade
•
•
•
Size of the device in blocks
Size of a block in pages
•
•
•
Internal voltage: avg. and max.
Auxiliary voltage: avg. and max.
Size of a page in bytes (including ECC
overhead)
Health status: Healthy, Marginal, or
Degraded, based on the percentage of
good erase blocks on the device
•
•
Number of pads, planes and banks
Size of an ECC-protected unit (includ-
ing check-bit overhead)
•
•
•
Percentage of good data
Percentage of good blocks
Percentage of good metadata
•
FPGA ID
Sample Output
The fio-status command alone displays abbreviated information on all installed
ioDrives:
fio-status
Found 2 ioDrives in this system.
Fusion-io driver version: 1.2.4.3
fct0
Attached
ioDimm3 SN:0719 Firmware v11791
92 GBytes unformatted capacity.
fct1
Not attached
ioDimm3 SN:0299 Firmware v11791
154 GBytes unformatted capacity.
The fio-status command with the –a parameter and no device name displays all the
information available on every installed ioDrive.
fio-status -a
Found 2 ioDrives in this system.
Fusion-io driver version: 1.2.4.3
fct0
Attached
March 10, 2009
Dolphin Interconnect Solutions
45
ioDimm3 SN:0719 PN:001195011, Mfr:000 Date:20080709
Firmware v11791
92 GBytes, 8192 blocks, 256 pages, 47264 bytes/page (25 pads, 1 plane, 4
banks)
Nand: Samsung (ec) SLC
Error correction: 11 bits per 240 bytes, retire above 4 bits
FPGA ID:000 UID:0000000002cf01326845000048f00c00
PCI: 02:00.0, Slot Number:2 Vendor ID: 1aed Device ID: 1003
Ambient temperature: 40 degC
Internal temperature: avg 44.3 degC, max 50.7 degC
Media status: healthy. 99.78% blocks good. data:99.77% good, md:100%
fct12 Not attached
ioDimm3 SN:0299 PN:001160002, Mfr:AEMS Date:20080724
Firmware v11791
154 GBytes, 8192 blocks, 256 pages, 78848 bytes/page (20 pads, 1 plane, 4
banks)
Nand: Samsung (ec) SLC
Error correction: 11 bits per 240 bytes, retire above 4 bits
FPGA ID:xc5vlx110t-ff1136-1 UID:00000000012b01326854000046cd0801
PCI: 03:00.0, Slot Number:3 Vendor ID: 1aed Device ID: 1001
Ambient temperature: 34 degC
Internal temperature: avg 37.9 degC, max 42.8 degC
Media status: healthy. 99.87% blocks good. data:99.82% good, md:100%
fio-update-iodrive
Description
Updates the ioDrive’s firmware. This utility scans the PCIe bus for all attached ioDrives
and updates up to 256 total devices. Each ioDrive displays a progress bar and
percentage as the update completes. To update one or more specific devices, use the -d
option along with the PCIe bus address to identify which device to update. This utility
requires the device driver be loaded to operate. It also requires a complete power down
of the computer after using fio-update-iodrive to ensure a complete reset and
installation of the firmware.
Caution: The default action (without using the “–d” option) is to upgrade all ioDrive
devices with the updated firmware contained in the <ffffile> file. Confirm that all
devices need the upgrade prior to running the update. If in doubt, use the –p Pretend
option to view the possible results of the update.
Syntax
fio-update-iodrive <iodrive_version> [-d, -f, -l, -p, -q, -h, -v]
where <iodrive_version> is the path and firmware archive file.
March 10, 2009
Dolphin Interconnect Solutions
46
Options
-d
-f
Updates the specified device. Use /dev/fctx where x is the device’s
PCIe bus number. (Use fio-status to view this.)
Force Upgrade (Also used to perform a rollback to an earlier
firmware version.)
-l
List firmware available in the archive.
-p
Pretend: Show what updates would be done (the firmware is not
modified). This option requires the -d option.
-q
Runs the update process without displaying the progress bar or
percent completed.
Caution: Use the -f option with care, as forcing an upgrade could damage your device.
Note: All three external LED indicators will light during the update process.
3.6 Enabling SNMP
The ioDrive also supports monitoring using SNMP or Windows Management
Instrumentation (WMI). The Windows Setup package places the ioDrive SNMP
extension agent DLL and MIB into Program Files\Fusion-io\SNMP. This ioDrive agent
requires the Windows SNMP service active on the computer (see your Windows
documentation for details). Currently, WMI requires SNMP in the background to
function so you’ll need to setup SNMP for either.
To install and enable the extension agent:
1. Copy fio-snmp-win.dll into Windows\System32.
2. Open Regedit and create the following registry key:
HKEY_LOCAL_MACHINE\SOFTWARE\Fusion-io\
fio-snmp-win\CurrentVersion\Pathname = %systemroot%\system32\
fio-snmp-win.dll
3. In Regedit, locate the following registry key:
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Ser-
vices\SNMP\Parameters\ExtensionAgents
4. Modify this key to include a String Value. Number it next in the list of agents.
5. Modify this value with the following:
March 10, 2009
Dolphin Interconnect Solutions
47
SOFTWARE\Fusion-io\io-snmp-win\CurrentVersion
6. Save and exit Regedit.
7. Stop and restart the Windows SNMP service.
You can now monitor your ioDrive using SNMP. By default, the ioDrive agent uses the
cpqIODrv.mib in the Program Files\Fusion-io\SNMP folder. (Refer to the Release
Notes for further details on the MIB.)
3.6.1 Windows Management Instrumentation (WMI)
WMI is the Microsoft implementation of the Common Information Model (CIM) and
Web-Base Enterprise Management (WBEM). WMI support is an optional component.
To install it:
1. Click on Start/Control Panel.
2. Click on Add/Remove Programs (Programs and Features in Vista/Server 2008).
3. Click on the Add/Remove Windows Components button.
4. Select Management & Monitoring Tools.
5. Click on the Details button.
6. Check the box labeled WMI Windows Installer Provider.
7. Click OK.
8. Click Next.
Follow the on-screen instructions to complete the install.
3.6.1.1 Install WMITools
Next, install the WMITools package:
1. Enter “WMITools” in the Downloads search box at the top of the page at
2. Download the package to a convenient folder.
3. Click on the package to launch the setup process.
4. Follow the onscreen instructions to complete the install.
3.6.1.2 Convert the MIB File
You must now create and register the WMI .MOF version of the SNMP MIB file.
1. Download a copy of the RFC1213-MIB file from an available online resource such
2. Open a Command Prompt and navigate to this directory.
3. To create the MOF version of the MIB file, type in the command:
March 10, 2009
Dolphin Interconnect Solutions
48
C:\WINDOWS\system32\wbem\snmp\smi2smir /g RFC1213-MIB.txt
cpqHost.mib cpqIODrv.mib > cpqIODrv.mof
4. To register this MOF file with WMI, type in the command:
C:\WINDOWS\system32\wbem\mofcomp cpqIODrv.mof
Your management system is now ready for active monitoring.
March 10, 2009
Dolphin Interconnect Solutions
49
|