IBM 9077 SP Switch Router:
Get Connected to the SP Switch
Hajo Kitzhöfer, Steffen Eisenblätter, Uwe Untermarzoner
International Technical Support Organization
http://www.redbooks.ibm.com
SG24-5157-00
Download from Www.Somanuals.com. All Manuals Search And Download.
SG24-5157-00
International Technical Support Organization
IBM 9077 SP Switch Router:
Get Connected to the SP Switch
November 1998
Download from Www.Somanuals.com. All Manuals Search And Download.
Take Note!
Before using this information and the product it supports, be sure to read the general information in
First Edition (November 1998)
This edition applies to PSSP Version 2, Release 4 for use with AIX 4.3.1 and Ascend Embedded/OS
Version 1.4.6.4.
Comments may be addressed to:
IBM Corporation, International Technical Support Organization
Dept. HYJ Mail Station P099
522 South Road
Poughkeepsie, New York 12601-5400
When you send information to IBM, you grant IBM a non-exclusive right to use or distribute the
information in any way it believes appropriate without incurring any obligation to you.
© Copyright International Business Machines Corporation 1998. All rights reserved
Note to U.S Government Users – Documentation related to restricted rights – Use, duplication or disclosure is
subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corp.
Download from Www.Somanuals.com. All Manuals Search And Download.
Contents
Figures. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .ix
Tables. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xi
Preface. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .xiii
The Team That Wrote This Redbook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii
Comments Welcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xv
Part 1. Introducing and Installing the GRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
Chapter 1. Dependent Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.1 Dependent Node Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1.2 Limitations of the Dependent Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Chapter 2. Router Node . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
2.1.1 Motivation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2.1.2 Design Objectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.3 What is a Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2.1.4 Routing without the GRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
2.1.5 Routing with the GRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
2.1.6 Overview of Supported Routing Protocols. . . . . . . . . . . . . . . . . . 15
2.1.7 Media Adapters At-a-Glance. . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.8 Benefits of the GRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
2.1.9 Price Comparison. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17
2.2 GRF Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
2.2.1 IP Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19
2.2.2 Supported Routing Protocols . . . . . . . . . . . . . . . . . . . . . . . . . . . 20
2.2.3 Filtering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
2.2.4 System Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3 GRF Hardware . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.1 GRF Block Diagram . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24
2.3.2 GRF Features. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
2.3.3 IP Switch and Control Board. . . . . . . . . . . . . . . . . . . . . . . . . . . . 31
2.3.4 Memory Guidelines for the IP Switch Control Board . . . . . . . . . . 35
2.3.5 Characteristics of GRF Media Cards. . . . . . . . . . . . . . . . . . . . . . 36
2.3.6 SP Switch Router Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
2.3.7 Media Card Performance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38
2.3.8 Other Media Cards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39
2.3.9 GRF Operating Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4 PSSP Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
© Copyright IBM Corp. 1998
iii
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.1 SDR Enhancements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
2.4.2 New Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
2.4.3 Enhanced Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
2.4.4 Hardware Perspectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
2.4.5 SP Extension Node SNMP Manager. . . . . . . . . . . . . . . . . . . . . . 58
2.4.6 Dependent Node MIB . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
2.4.7 Coexistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
2.4.8 Partitioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62
2.5 Planning for the GRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
2.6 Planning for the Dependent Node. . . . . . . . . . . . . . . . . . . . . . . . . . . . 65
2.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66
Part 2. Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
Chapter 3. Installation and Configuration. . . . . . . . . . . . . . . . . . . . . . . 69
3.1 Initial Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70
3.2 Pre-Installation Assumptions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71
3.2.1 Order of Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
3.3 Installing an SP Switch Router Adapter Card . . . . . . . . . . . . . . . . . . . 75
3.3.1 Installation Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75
3.3.2 Installing the PCMCIA Spinning Disk . . . . . . . . . . . . . . . . . . . . . 76
3.4 Attaching SP Switch Router Cables . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4.1 Ethernet Cable . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79
3.4.2 SP Switch Cable. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80
3.4.3 Procedure for Connecting Cards to the SP Switch . . . . . . . . . . . 80
3.5 Configuration Required on the SP System . . . . . . . . . . . . . . . . . . . . . 81
3.5.1 Determining the Switch Connection for a Dependent Node. . . . . 82
3.5.2 Procedure to Get the Jack Number. . . . . . . . . . . . . . . . . . . . . . . 84
3.6 Multiple Frames for Multiple System Connections . . . . . . . . . . . . . . . 85
3.7 Step-by-Step Media Card Configuration . . . . . . . . . . . . . . . . . . . . . . . 86
3.7.1 Configuration Files and Their Uses. . . . . . . . . . . . . . . . . . . . . . . 86
3.8 Step 1. Check SNMP in the SP Switch Router System . . . . . . . . . . . . 88
3.9 Put SNMP Changes into Effect. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.10 Step 2. Assign IP Addresses . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89
3.10.1 Method 1: Use SP SNMP Manager - Recommended . . . . . . . . 89
3.10.2 Method 2: Edit /etc/grifconfig.conf - Optional . . . . . . . . . . . . . . 93
3.10.3 Putting grifconfig.conf Additions into Effect. . . . . . . . . . . . . . . . 95
3.11 Step 3. Change Profile Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.12 Step 4. Run dev1config . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95
3.13 Step 5. Reset SP Switch Router System to Install Files . . . . . . . . . . 96
3.13.1 Saving Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96
3.14 Verify an SP Switch Router Adapter Card on the Router . . . . . . . . . 97
3.14.1 Verify Media Card Operation Using ping. . . . . . . . . . . . . . . . . . 97
iv
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
3.14.2 Check Media Card Status Using grcard . . . . . . . . . . . . . . . . . . 98
3.14.3 Reset Media Card Using grreset. . . . . . . . . . . . . . . . . . . . . . . . 99
3.14.4 Using grstat to Display GRF Statistics . . . . . . . . . . . . . . . . . . . 99
3.15 Bringing the SP Switch Router Adapter Card Online with the SP . . 100
3.15.1 Checking Connectivity to the SP System . . . . . . . . . . . . . . . . 101
Chapter 4. Configuration of IP-Forwarding Media Cards . . . . . . . . . . 105
4.1 Ethernet 10/100Base-T Configuration. . . . . . . . . . . . . . . . . . . . . . . . 105
4.1.1 Physical and Logical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 105
4.1.2 Configuration File and Profile Overview . . . . . . . . . . . . . . . . . . 106
4.1.3 Installing Configurations or Changes . . . . . . . . . . . . . . . . . . . . 107
4.1.4 Assign IP Addresses - grifconfig.conf . . . . . . . . . . . . . . . . . . . . 107
4.1.5 Specify Ethernet Card Parameters . . . . . . . . . . . . . . . . . . . . . . 108
4.1.6 Some maint Commands for the Ethernet Media Cards . . . . . . . 109
4.2 ATM OC-3c Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110
4.2.1 Physical and Logical ATM Interfaces . . . . . . . . . . . . . . . . . . . . 110
4.2.2 Installing Configurations or Changes . . . . . . . . . . . . . . . . . . . . 113
4.2.3 Configuration Files and Profiles . . . . . . . . . . . . . . . . . . . . . . . . 113
4.2.4 Assign IP Addresses - grifconfig.conf . . . . . . . . . . . . . . . . . . . . 114
4.2.5 Specify ATM Card Parameters . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2.6 Configuring PVCs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115
4.2.7 Some maint Commands for the ATM OC-3c Media Card . . . . . 116
4.2.8 Using grrt to Display the Route Table . . . . . . . . . . . . . . . . . . . . 118
4.2.9 Using grstat to Display GRF Statistics . . . . . . . . . . . . . . . . . . . 119
4.3 ATM OC-12c Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119
4.3.1 Physical and Logical ATM Interfaces . . . . . . . . . . . . . . . . . . . . 119
4.3.2 Installing Configurations or Changes . . . . . . . . . . . . . . . . . . . . 120
4.3.3 Configuration Files and Profiles . . . . . . . . . . . . . . . . . . . . . . . . 120
4.4 FDDI Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121
4.4.1 Separate Networks versus Bridging . . . . . . . . . . . . . . . . . . . . . 126
4.4.2 Naming the FDDI Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
4.4.3 Physical Interface Numbers . . . . . . . . . . . . . . . . . . . . . . . . . . . 127
4.4.4 GRF Interface Name. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128
4.4.5 Configuration Files and Profiles . . . . . . . . . . . . . . . . . . . . . . . . 128
4.4.6 Assign IP Addresses - grifconfig.conf . . . . . . . . . . . . . . . . . . . . 129
4.4.7 Specify FDDI Card Parameters. . . . . . . . . . . . . . . . . . . . . . . . . 130
4.4.8 Installing Configurations or Changes . . . . . . . . . . . . . . . . . . . . 130
4.4.9 Some maint Commands for the FDDI Media Card . . . . . . . . . . 131
4.4.10 Using grrt to Display the Route Table . . . . . . . . . . . . . . . . . . . 132
4.4.11 Using grstat to Display GRF Statistics . . . . . . . . . . . . . . . . . . 133
4.5 HIPPI Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.1 Introduction to HIPPI . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133
4.5.2 HIPPI Configuration Options. . . . . . . . . . . . . . . . . . . . . . . . . . . 138
v
Download from Www.Somanuals.com. All Manuals Search And Download.
4.5.3 Physical and Logical Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 139
4.5.4 Configuration Files and Profiles . . . . . . . . . . . . . . . . . . . . . . . . 140
4.5.5 Installing Configurations or Changes . . . . . . . . . . . . . . . . . . . . 141
4.5.6 Some maint Commands for the HIPPI Media Card . . . . . . . . . . 141
4.6 Configuring Bridging. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
4.6.1 GRF Bridging Implementation. . . . . . . . . . . . . . . . . . . . . . . . . . 142
4.6.2 Simultaneous Routing and Bridging . . . . . . . . . . . . . . . . . . . . . 143
4.6.3 Configuration Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 143
4.6.4 Interoperability . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.6.5 Spanning Tree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.6.6 Bridge Filtering Table . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.6.7 Fragmentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144
4.6.8 Spamming . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.6.9 Bridging Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 145
4.6.10 Management Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146
4.6.11 Configuration File and Profile Overview . . . . . . . . . . . . . . . . . 148
4.6.12 Bridging ATM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 154
4.6.13 Bridging FDDI. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
4.6.14 Bridging Ethernet . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 155
Chapter 5. Single RS/6000 SP and Single SP Switch Router . . . . . . . 157
5.1 Single SP Partition and Single SP Switch Router Adapter Card . . . . 157
5.1.1 SP Switch - Ethernet Connection . . . . . . . . . . . . . . . . . . . . . . . 157
5.1.2 SP Switch - FDDI Connection. . . . . . . . . . . . . . . . . . . . . . . . . . 162
5.1.3 SP Switch - ATM Connection . . . . . . . . . . . . . . . . . . . . . . . . . . 167
5.1.4 SP Switch - FDDI Connection (Distinct FDDI Networks) . . . . . . 174
5.1.5 SP Switch - FDDI Connection in an ADSM Environment. . . . . . 185
5.2 Single SP Partition and Multiple SP Switch Router Adapter Cards . . 187
5.2.1 Configuration of a Dual SP Switch Router Connection . . . . . . . 187
5.2.2 Complex Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190
5.2.3 Recovery Procedure for an SP Switch Adapter Card Failure. . . 196
5.3 Multiple SP Partition and Multiple SP Switch Router Adapter Cards . 197
Chapter 6. Multiple RS/6000 SPs and One SP Switch Router . . . . . . 203
6.1 RS/6000 SP Switch - RS/6000 SP Switch Connection . . . . . . . . . . . 203
6.2 Sharing Network Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 207
Chapter 7. Multiple RS/6000 SPs and Multiple GRFs . . . . . . . . . . . . . 209
7.1 ATM OC-3c Backbone Connection . . . . . . . . . . . . . . . . . . . . . . . . . . 209
7.1.1 ATM OC-3c Backbone - Using One Port. . . . . . . . . . . . . . . . . . 210
7.1.2 ATM OC-3c Backbone - Using Two Ports . . . . . . . . . . . . . . . . . 215
7.2 ATM OC-12c Backbone - One Port. . . . . . . . . . . . . . . . . . . . . . . . . . 222
7.3 HIPPI Backbone Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 227
vi
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix A. Laboratory Hardware and Software Configuration . . . . 233
A.1 Node and Control Workstation Configuration . . . . . . . . . . . . . . . . . . . . . 233
A.1.1 Hard Disks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
A.1.2 Software Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239
A.1.3 Network Options and Tuning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 253
A.2 SP Switch Pool Size Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 255
A.3 7025-F50 Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256
A.4 SP IP Switch Router Configuration. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
Appendix B. GRF Configuration Files . . . . . . . . . . . . . . . . . . . . . . . . . . 261
B.1 /root/.profile. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
B.2 /etc/Release . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 263
B.3 /etc/bridged.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 264
B.4 /etc/fstab . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
B.5 /etc/grarp.conf. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 267
B.6 /etc/gratm.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 268
B.7 /etc/grclean.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274
B.8 /etc/grclean.logs.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275
B.9 /etc/grdev1.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 277
B.10 /etc/grifconfig.conf. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 282
B.11 /etc/grlamap.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284
B.12 /etc/grroute.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 285
B.13 /etc/hosts. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
B.14 /etc/inetd.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 286
B.15 /etc/motd . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
B.16 /etc/rc.local . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 287
B.17 /etc/snmpd.conf. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288
B.18 /etc/syslog.conf . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291
B.19 /etc/ttys . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292
Appendix C. Hardware and Software Information . . . . . . . . . . . . . . . . 295
C.1 The Front Panel of the SP Switch Router Adapter Card - Operational. . 295
C.2 SP Switch Router Adapter Media Card LEDs. . . . . . . . . . . . . . . . . . . . . 296
C.3 SP Switch Router Adapter Media Card - Bootup . . . . . . . . . . . . . . . . . . 297
C.4 Connectors and Receptacles for Different Media . . . . . . . . . . . . . . . . . . 298
C.5 Chip Interconnection on the TBS Board . . . . . . . . . . . . . . . . . . . . . . . . . 298
C.6 Updating Router Software . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
C.6.1 The SP Switch Router as an IBM Product . . . . . . . . . . . . . . . . . . . 299
C.6.2 Obtaining New Machine Code . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
C.6.3 Support for Code Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 300
C.6.4 Sample Steps to Upgrade the System Software . . . . . . . . . . . . . . 300
C.6.5 Sample Execution of grf_update Script . . . . . . . . . . . . . . . . . . . . . 301
vii
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix D. Special Notices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 305
Appendix E. Related Publications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
E.1 International Technical Support Organization Publications . . . . . . . . . . 309
E.2 Redbooks on CD-ROMs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
E.3 Other Publications. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 309
How to Get ITSO Redbooks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
How IBM Employees Can Get ITSO Redbooks. . . . . . . . . . . . . . . . . . . . . . . 311
How Customers Can Get ITSO Redbooks. . . . . . . . . . . . . . . . . . . . . . . . . . . 312
IBM Redbook Order Form . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 313
List of Abbreviations. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 315
Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 317
ITSO Redbook Evaluation. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 323
viii
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Figures
1. SP Switch Router. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
2. Functional Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3. Typical Router Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
4. Table-Based Routing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
5. Routing without GRF . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
6. Routing with GRF. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
7. GRF 400 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
8. Conventional Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14
9. Switched Routers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
10. Price Comparison . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18
11. GRF Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
12. GRF Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
13. Data Packet Transfer. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29
14. Routing Packet Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
15. Side View of GRF 400 Chassis with Slots Numbered . . . . . . . . . . . . . . . . 32
16. Top View of the GRF 1600 Chassis. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32
17. IP Switch Control Board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
18. System RAM . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36
19. SP Switch Router Adapter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37
20. Hardware Perspectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52
21. Action Menu. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54
22. Hardware Notebook. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55
23. System Partition Aid Perspectives. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57
24. System Partition Aid Notebook . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58
25. Coexistence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61
26. Partitioning. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63
27. The Laboratory Hardware Installation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67
28. Connecting the GRF to the SP Switch and the CWS . . . . . . . . . . . . . . . . 69
29. Connecting the GRF to the Frame. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
30. Connecting the GRF Console . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74
31. SP System Administrative Ethernet Connections . . . . . . . . . . . . . . . . . . . 80
32. Switch Port Assignments in Supported Frame Configurations . . . . . . . . . 83
33. Node Numbering for an SP System. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
34. How Frames Enable Connections to Multiple SP Switches. . . . . . . . . . . . 86
35. Components in the SP Switch Router Adapter Card’s Interface Name. . . 93
36. Components of the Ethernet Interface Name . . . . . . . . . . . . . . . . . . . . . 106
37. ATM OC-3c Physical and Logical Interfaces . . . . . . . . . . . . . . . . . . . . . . 110
38. Components in the ATM OC-3c Interface Name . . . . . . . . . . . . . . . . . . . 111
39. Components Forming a Virtual Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111
40. ATM OC-12c Physical and Logical Interfaces . . . . . . . . . . . . . . . . . . . . . 120
© Copyright IBM Corp. 1998
ix
Download from Www.Somanuals.com. All Manuals Search And Download.
41. Master/Slave Connectors for SAS Interfaces . . . . . . . . . . . . . . . . . . . . . 122
42. A/B Connectors for DAS Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
43. Allowed SAS and DAS Configurations. . . . . . . . . . . . . . . . . . . . . . . . . . . 123
44. Optical Bypass Switch Attachments . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
45. Dual Homing Configurations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126
46. Assigning Numbers to FDDI Interfaces . . . . . . . . . . . . . . . . . . . . . . . . . . 127
47. Physical Interface Numbering on the FDDI Media Card . . . . . . . . . . . . . 128
48. GRF Interface Name for FDDI Interfaces . . . . . . . . . . . . . . . . . . . . . . . . 128
49. HIPPI I-Field Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135
50. Components in the HIPPI Interface Name. . . . . . . . . . . . . . . . . . . . . . . . 139
51. Interface Name for FDDI, Ethernet and ATM OC-3c Interfaces . . . . . . . 150
52. One Card - One SP Partition Sample Configuration . . . . . . . . . . . . . . . . 157
53. SP Switch - Ethernet Connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 158
54. SP Switch - FDDI Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163
55. SP Switch - ATM Connection. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 168
56. SP Switch - FDDI Connection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 175
57. SP Switch - FDDI Connection (Bridging) . . . . . . . . . . . . . . . . . . . . . . . . . 180
58. SP Switch Router in an ADSM Environment . . . . . . . . . . . . . . . . . . . . . . 185
59. Connecting One SP Switch with Two SP Switch Router Adapter Cards. 187
60. Configuration with Dual SP Switch Router - SP Switch Connection . . . . 190
61. IP Traffic Flow When Issuing ping 192.168.13.1 on Node 6 . . . . . . . . . . 195
62. IP Traffic Flow When Issuing ping 192.168.13.1 on Node 10 . . . . . . . . . 195
63. IP Traffic Flow When Issuing ping 192.168.13.1 on Node 8 . . . . . . . . . . 196
64. Partition-to-Partition Connection with an SP Switch Router . . . . . . . . . . 198
65. Two RS/6000 SPs Connected to GRF 1600 . . . . . . . . . . . . . . . . . . . . . . 203
66. Sharing Network Resources between Two SPs . . . . . . . . . . . . . . . . . . . 207
67. Connection of Two SPs with Two SP Switch Routers . . . . . . . . . . . . . . . 209
68. SP Switch - ATM - SP Switch Connection. . . . . . . . . . . . . . . . . . . . . . . . 211
69. SP Switch - ATM Bridged - SP Switch Connection . . . . . . . . . . . . . . . . . 215
70. SP Switch - ATM OC-12c - SP Switch Connection . . . . . . . . . . . . . . . . . 223
71. SP Switch - HIPPI - SP Switch Connection . . . . . . . . . . . . . . . . . . . . . . . 228
72. Front Panel of the SP Switch Router Adapter Card with LEDs . . . . . . . . 295
73. The SP Switch Board. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 299
x
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Tables
1. Memory Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
2. DependentNode Attributes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40
3. DependentAdapter Attributes. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
4. Additional SDR Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
5. New Commands (root Executable) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43
6. New Commands (User Executable). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44
7. endefnode Command Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
8. enrmnode Command Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46
9. endefadapter Command Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
10. enadmin Command Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
11. splstnode Command Options. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49
12. splstadapter Command Options . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50
13. Enhanced Commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51
14. Configuration of SP Switch - Ethernet Connection . . . . . . . . . . . . . . . . . 159
15. Configuration of an SP Switch - FDDI Connection . . . . . . . . . . . . . . . . . 163
16. Configuration of SP Switch - ATM Connection . . . . . . . . . . . . . . . . . . . . 168
17. Configuration of SP Switch - FDDI Connection . . . . . . . . . . . . . . . . . . . . 175
18. Configuration of SP Switch - FDDI Connection (Bridging). . . . . . . . . . . . 181
19. Configuration of a Dual SP Switch Router Connection . . . . . . . . . . . . . . 187
20. Configuration of a Dual SP Switch Router - SP Switch Connection . . . . 191
21. Configuration of a Partition - Partition Connection . . . . . . . . . . . . . . . . . 199
22. Configuration of SP Switch - SP Switch Connection . . . . . . . . . . . . . . . . 204
23. Configuration of SP Switch - ATM - SP Switch . . . . . . . . . . . . . . . . . . . . 212
24. Configuration of SP Switch - ATM Bridged - SP Switch . . . . . . . . . . . . . 216
25. Configuration of SP Switch - ATM OC-12c - SP Switch . . . . . . . . . . . . . 224
26. Configuration of SP Switch - HIPPI - SP Switch . . . . . . . . . . . . . . . . . . . 228
27. Configuration of SP 21. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 234
28. Configuration of SP 2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 235
29. Hard Disk Equipment of SP 21 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 236
30. Hard Disk Equipment of SP 2 Part 1 of 2. . . . . . . . . . . . . . . . . . . . . . . . . 237
31. Hard Disk Equipment of SP 2 Part 2 of 2. . . . . . . . . . . . . . . . . . . . . . . . . 238
32. Software Levels on CWS and All Nodes Part 1 of 14 . . . . . . . . . . . . . . . 239
33. Software Levels on CWS and All Nodes Part 2 of 14 . . . . . . . . . . . . . . . 240
34. Software Levels on CWS and All Nodes Part 3 of 14 . . . . . . . . . . . . . . . 241
35. Software Levels on CWS and All Nodes Part 4 of 14 . . . . . . . . . . . . . . . 242
36. Software Levels on CWS and All Nodes Part 5 of 14 . . . . . . . . . . . . . . . 243
37. Software Levels on CWS and All Nodes Part 6 of 14 . . . . . . . . . . . . . . . 244
38. Software Levels on CWS and All Nodes Part 7 of 14 . . . . . . . . . . . . . . . 245
39. Software Levels on CWS and All Nodes Part 8 of 14 . . . . . . . . . . . . . . . 246
40. Software Levels on CWS and All Nodes Part 9 of 14 . . . . . . . . . . . . . . . 247
© Copyright IBM Corp. 1998
xi
Download from Www.Somanuals.com. All Manuals Search And Download.
41. Software Levels on CWS and All Nodes Part 10 of14. . . . . . . . . . . . . . . 248
42. Software Levels on CWS and All Nodes Part 11 of 14 . . . . . . . . . . . . . . 249
43. Software Levels on CWS and All Nodes Part 12 of 14 . . . . . . . . . . . . . . 250
44. Software Levels on CWS and All Nodes Part 13 of 14 . . . . . . . . . . . . . . 251
45. Software Levels on CWS and All Nodes Part 14 of 14 . . . . . . . . . . . . . . 252
46. Network Options of CWS and All Nodes Part 1 of 3 . . . . . . . . . . . . . . . . 253
47. Network Options of CWS and All Nodes Part 2 of 3 . . . . . . . . . . . . . . . . 254
48. Network Options of CWS and All Nodes Part 3 of 3 . . . . . . . . . . . . . . . . 255
49. Network Options of 7025-F50 Part 1 of 3 . . . . . . . . . . . . . . . . . . . . . . . . 256
50. Network Options of 7025-F50 Part 2 of 3 . . . . . . . . . . . . . . . . . . . . . . . . 257
51. Network Options of 7025-F50 Part 3 of 3 . . . . . . . . . . . . . . . . . . . . . . . . 258
52. SP Switch Router Adapter Media Card LEDs . . . . . . . . . . . . . . . . . . . . . 296
53. SP Switch Router Adapter Media Card LEDs - RX/TX . . . . . . . . . . . . . . 296
54. SP Switch Router Adapter Media Card LEDs During Bootup . . . . . . . . . 297
55. Media Card Cables and Connectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . 298
xii
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Preface
The GRF is a high-performance switched IP Router which provides
high-speed data communication links between IBM RS/6000 SP and external
networks or hosts. It acts as a special-purpose SP node that routes IP traffic
between SP nodes on the SP Switch and the outside world. Connected
directly to the SP Switch, the router offers significantly improved SP I/O
performance. When packaged with an IBM SP system, the GRF router is
referred to as the SP Switch Router.
This redbook helps you install, tailor and configure the SP Switch Router, IBM
machine type 9077. The SP Switch Router is also known as the "Gigarouter"
or High Performance Gateway Node (HPGN).
The first part of the book gives an overview of the GRF architecture and how
the router was integrated into the SP. It emphasizes the advantages of
choosing a dedicated router node in some configurations, as opposed to
using standard nodes for the routing task. This part also describes some
routing fundamentals, particularly focusing on concepts like IP- and
switch-routing.
The second part presents sample configurations that were carefully chosen to
match frequently occurring customer situations. The basic configurations
shown are building blocks for more complex networking topologies that
include the SP Switch Router and may inspire more sophisticated
configurations. All configurations described were tested and provide some
comparable performance figures.
This publication is intended to give IBM customers, system engineers, and
marketing personnel a broad understanding of this new architecture and what
it is used for.
The Team That Wrote This Redbook
This redbook was produced by a team of specialists from around the world
working at the International Technical Support Organization, Poughkeepsie
Center.
Dr Hajo Kitzhöfer is an Advisory International Technical Support
Organization (ITSO) Specialist for RS/6000 SP at the Poughkeepsie Center.
He holds a Ph.D. degree in electrical engineering from the Ruhr-University of
Bochum (RUB). Before joining ITSO, he worked as an SP Specialist at the
RS/6000 and AIX Competence Center, IBM Germany. He has worked at IBM
© Copyright IBM Corp. 1998
xiii
Download from Www.Somanuals.com. All Manuals Search And Download.
for eight years. His areas of expertise include RS/6000 SP, SMP, and
Benchmarks. He now specializes in SP System Management, SP
Performance Tuning and SP hardware.
Dr Steffen Eisenblätter is an AIX Software Specialist in the RS/6000 SP
Software Support Center, Germany. He holds a Ph.D. degree in physics from
the University of Leipzig. He joined IBM in 1997 and has focused on RS/6000
SP products and TCP/IP.
Uwe Untermarzoner is an RS/6000 SP Technical Support Specialist with
IBM Germany. He joined IBM 1989. He has ten years of experience in AIX
and five years of experience with the SP, mostly in the commercial
environment. He joined IBM at 1989. His areas of expertise include AIX,
RS/6000 SP, SMP, PSSP, Networking, Performance Tuning and Systems
Management.
Thanks to the following people for their invaluable contributions to this
project:
Ronald Linton
IBM PPS Lab Poughkeepsie
Gene Novitsky
Ascend Communications, Inc.
Frank May
IBM Worldwide RS/6000 SP Product Marketing
Wes Kinard
IBM RS/6000 Networking Technologies
Marcelo R. Barrios
International Technical Support Organization, Poughkeepsie Center
xiv
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Comments Welcome
Your comments are important to us!
We want our redbooks to be as helpful as possible. Please send us your
comments about this or other redbooks in one of the following ways:
to the fax number shown on the form.
• Use the electronic evaluation form found on the Redbooks Web sites:
For Internet usershttp://www.redbooks.ibm.com
For IBM Intranet usershttp://w3.itso.ibm.com
• Send us a note at the following address:
xv
Download from Www.Somanuals.com. All Manuals Search And Download.
xvi
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Part 1. Introducing and Installing the GRF
© Copyright IBM Corp. 1998
1
Download from Www.Somanuals.com. All Manuals Search And Download.
2
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 1. Dependent Node
This chapter provides an overview of a dependent node in RS/6000 SP. We
start by defining the dependent node and the rationale behind its design.
1.1 Dependent Node Architecture
The Dependent Node Architecture refers to a processor or node, possibly not
provided by IBM, for use with the RS/6000 SP.
Since a dependent node may not be a regular RS/6000 SP node, not all the
functions of a node can be performed on it, which is why it is called
"dependent". For example, it does not allow all the functions of the fault
service (Worm) daemon, as other RS/6000 SP nodes with access to the SP
Switch do.
The objective of this architecture is to allow the other processors or hardware
to easily work together with the RS/6000 SP, extending the scope and
capabilities of the system.
The dependent node connects to the RS/6000 SP Switch (but not to the
earlier High Performance Switch, HiPS).
The SP Switch Router Adapter is the first product to exploit the Dependent
Node Architecture.
1.2 Limitations of the Dependent Node
The following are limitations associated with use of the dependent node:
• To use the dependent node in an RS/6000 SP requires the SP Extension
Node SNMP Manager to be installed in the Control Workstation. The SP
Extension Node SNMP Manager requires UDP port 162 in the Control
Workstation. Other SNMP managers, such as Netview, also require this
port. To allow the two SNMP managers to coexist, the SP Extension Node
SNMP Manager must use an alternative UDP port.Dependent nodes are
not allowed in Node Groups.
• Only the 8-port and 16-port SP Switch are supported. The 8-port and
16-port High Performance Switch (the old SP Switch) are not supported.
• The spmoncommand on the RS/6000 SP is not enhanced to support
dependent nodes. Dependent nodes can only be viewed with the
perspectivescommand.
© Copyright IBM Corp. 1998
3
Download from Www.Somanuals.com. All Manuals Search And Download.
• The fault service daemon runs on all switch nodes in the RS/6000 SP, but
not on the dependent node. Therefore, the dependent node does not have
the full functionality of a normal RS/6000 SP Switch node.
• The dependent node requires the SP Switch’s primary node to compute its
switch routes. Therefore, the primary node must have at least PSSP 2.3
installed, otherwise the dependent node cannot work with the RS/6000
SP.
• In the RS/6000 SP, SP Switch nodes occasionally send service packets
from one node to the next to keep track of status and links. Sometimes
these packets are sent indirectly through another switch node. As the
dependent node is not a standard RS/6000 SP Switch node, it cannot be
used to forward service packets to other nodes.
4
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 2. Router Node
The first dependent node is actually a new SP Switch Router Adapter in a
router. This chapter offers more details about the implementation.
Section 2.1, “Overview” on page 5 gives you an overview of SP Switch
Router. This is probably the best to get an impression what the GRF is good
for. Also a functional- and a price-comparison between using an RS/6000 SP
node and the SP Switch Router is included.
More details about the underlaying Software and Hardware can be found in
Section 2.4, “PSSP Enhancements” on page 40 describes the enhancements
in the PSSP Software for the support of the dependent node.
Some planning considerations which should be considered can found in
2.1 Overview
The purpose of the SP Switch Router Adapter is to allow the GRF ("goes
really fast"), manufactured by Ascend, to forward SP Switch IP traffic to other
networks. The GRF was known as the High Performance Gateway Node
(HPGN) during the development of the adapter. IBM remarkets models of the
GRF that connect to the SP Switch as the SP Switch Router model 04S
(9077-04S) and model 16S (9077-16S). These models are not available
directly from Ascend.
Note: In the remainder of this book, we refer to the SP Switch Router as the
GRF.
The distinguishing feature of the GRF, when compared with other routers, is
that it has an SP Switch Router Adapter and therefore can connect directly to
© Copyright IBM Corp. 1998
5
Download from Www.Somanuals.com. All Manuals Search And Download.
IBM 9570 Disk
Array
Subsystem
SP Switch
Adapter
HIPPI
Adapter
HIPPI
Adapter
ATM
OC-12c
ATM
ATM
OC-3c
Switch
SP Switch
Adapter
SP
Switch
Processor
Nodes
8-port
E-net 10/100
SP Switch
Adapter
4-port
FDDI
SP Switch
Router
SP System
Figure 1. SP Switch Router
The RS/6000 SP software treats this adapter as an extension node. It is a
node because it takes up one port in the SP Switch and is assigned a node
number. It is described as an extension because it is not a standard RS/6000
SP node, but an adapter card that extends the scope of the RS/6000 SP.
Although the term extension node represents the node appearance of the
adapter, it does not define the connection. An extension node adapter is used
for that purpose. Each extension node has an extension node adapter to
represent its connection to the SP Switch.
2.1.1 Motivation
A thin node, which has a single microchannel, is unable to deliver more than
about 30 MB/s to or from the SP Switch. Using a wide node, this number
increases to 65 MB/s but is still unable to provide full bandwidth to even one
HIPPI interface. It is also unable to feed 4 FDDI or 4 Ethernet 100BaseTx
cards at full bandwidth.
A 135 MHz wide node’s CPU becomes saturated at about 5000
packets/second. A 10 Mb/s Ethernet uses a maximum of 1500 bytes for a
6
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
packet size. This would only enable a wide node to handle approximately 7.5
MB/s of IP traffic.
Since Ascend’s business depends on keeping pace with networking
technology, they already support the major interfaces today. The 9077 will be
able to take advantage of any new interfaces that are developed in the future
as well, with no further development time or money expended.
With some interfaces requiring up to 5 slots, even a wide node can run out of
available slots. This forces additional nodes to be added even if there are no
performance limitations in the current configuration.
Since there are no hot plug capabilities with an SP node, any failure means
downtime on all interfaces configured in that node, and at times a lengthy
maintenance procedure. Redundancy is not built into the SP node’s
architecture.
SP Node
9077
Shared
Non-blocking Crosspoint Switch
250 ns path setup
Bus
1 MCA per thin node
2 MCA per wide node
Centralized
Cache,Software Based
Cache hits <50% typical
Independent lookups per card
Hardware based, <2.5 µs
150,000 route capacity per card
Route Table
Scalability
Single port per card
Single CPU
High per card port density
Per card
Limited by shared bus
Processors
Route Tables
Lookup engines
Each card has dedicated bandwidth
5000 pps
Up to 130,000 pps
100 MB/s per card slot, full duplex
Throughput
Support
30 MB/s per thin node
65 MB/s per wide node
No support for:
HSSI
ATM OC12
Sonet
Support for:
multiple SP Switch interfaces
High-speed networks such as
HIPPI
Multiple SP Switch Adapters
Protocols
Figure 2. Functional Comparison
Router Node
7
Download from Www.Somanuals.com. All Manuals Search And Download.
2.1.2 Design Objectives
Because the dependent node is part of the RS/6000 SP, it had to be
packaged and assigned some roles consistent with other RS/6000 SP nodes.
Changes were made to the RS/6000 SP to incorporate management
requirements for the dependent node.
Ease of design and implementation were important objectives in the design.
These were accomplished by limiting the amount of switchcontrol protocol for
the dependent node.
New SDR (System Data Repository) classes were created to manage
dependent nodes. This was done to minimize the scope of the changes and
the exposure to side effects that dependent nodes may cause if they were
represented as standard nodes in the SDR.
2.1.3 What is a Router
One of the basic functions of the Internet Protocol (IP) is its ability to connect
between different networks. This is due its routing algorithm and its flexibility
to use almost any physical network below. A system that connects different
physical or logical networks and directs traffic is termed a router, although the
older term IP gateway is also used.
Again, IP routing is the passing of an IP packet from one device to another by
sending it on a physical or logical interface. routers interconnect networks so
that IP traffic can be routed between the systems in the networks, as shown
8
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Network 1
Network 3
Router
Network n
Network 2
Figure 3. Typical Router Configuration
Routers help to reduce the amount of processing required on local systems,
since they perform the computation of routes to remote systems. For
example, a system can communicate with a remote system by passing the
message (or packets) to the router. The router works out how to get to the
remote system and forwards the message appropriately.
Storing routes on the system takes up memory. But because a system does
not have to store routes to systems not in its own subnet, the route table uses
less storage space and thereby frees up memory for other work.
The use of routing reduces network traffic, because routers encourage
subnetting, which creates a smaller network of systems. By having smaller
networks, network traffic congestion is reduced and overall network
performance and traffic control are improved.
A network’s routing configuration does not always require a routing protocol.
In situations where the routing information does not change, for example,
when there is only one possible route, the system administrator usually builds
the routing table manually. Some networks have no access to any other
TCP/IP networks, and therefore do not require routing tables at all. The three
most common routing configurations are:
Router Node
9
Download from Www.Somanuals.com. All Manuals Search And Download.
• Minimal routing
A network completely isolated from all other TCP/IP networks requires
only minimal routing. A minimal routing table is usually built by ifconfig
when the network interfaces are configured. If your network does not have
direct access to other TCP/IP networks, and if you are not using
subnetting, this may be the only routing table you require.
• Static routing
A network with a limited number of gateways to other TCP/IP networks
can be configured with static routing. When a network has only one
gateway, a static route is the best choice. A static routing table is
constructed manually by the system administrator using the route
changes, so they work best where routes do not change.
Source Host
Destination Host
Application
Transport
Application
Transport
Gateway
Destination Gateway
192.168.1.0 192.168.1.5
Destination
Gateway
Destination
Gateway
192.168.1.0 192.168.12.3
192.168.1.0 192.168.1.2
192.168.12.2
192.168.12.1
192.168.12.0
default
192.168.12.3
192.168.12.1
192.168.1.5
192.168.12.0
default
default
Network Access
192.168.12.2
Network Access
192.168.12.3 192.168.1.5
Network Access
192.168.1.2
192.168.12.0
192.168.1.0
Figure 4. Table-Based Routing
• Dynamic routing
A network with more than one possible route to the same destination
should use dynamic routing. A dynamic routing table is built from the
information exchanged by the routing protocols. The protocols are
designed to distribute information that dynamically adjusts routes to reflect
changing network conditions. Routing protocols handle complex routing
10
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
situations more quickly and accurately than a system administrator can
do. Routing protocols are designed not only to switch to a backup route
when the primary route becomes inoperable; they are also designed to
decide which is the "best" route to a destination. On any network where
there are multiple paths to the same destination, a dynamic routing
protocol should be used.
2.1.4 Routing without the GRF
Before the GRF was available, there were only two ways to get IP traffic from
remote systems to reach the RS/6000 SP nodes:
1. By putting an additional IP adapter into every RS/6000 SP node.
Router
Node
Internet/Intranet
Node
ATM
Node
SP Switch
FDDI
Node
Ethernet
Figure 5. Routing without GRF
The first option was usually not chosen because it was too costly for the
following reasons:
• For systems with a large number of nodes having multiple IP adapters for
each RS/6000 SP node can be expensive.
• The number of I/O slots in the RS/6000 SP node is limited. In addition,
these slots are required to perform other tasks for the system, such as
connecting to disk or tape. Using these I/O slots to connect IP adapters
restricts the functions of the RS/6000 SP node.
Router Node 11
Download from Www.Somanuals.com. All Manuals Search And Download.
The second case has proven to be very expensive as well. The RS/6000 SP
node was not designed for routing. It is not a cost-effective way to route traffic
for the following reasons:
• It takes many CPU cycles to process routing. The CPU is not a dedicated
router and is very inefficient when used to route IP traffic (this processing
can result in usage of up to 90%).
• It takes a lot of memory to store route tables. The memory on the RS/6000
SP node is typically more expensive than router memory.
The CPU on a node can only drive the system I/O bus at less than 80
megabytes per second, which is less than what a high-end router can do.
For these reasons, the performance of routers in handling IP traffic from
remote systems to the RS/6000 SP nodes was limited.
2.1.5 Routing with the GRF
Switch Router adapter can route up to 30,000 packets per second and up to
100 MB/s into the SP Switch network in each direction simultaneously.
Internet/Intranet
Node
Node
SP Switch
ATM
Node
GRF
FDDI
Ethernet
Figure 6. Routing with GRF
12
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
interconnect its adapters. This switch is capable of 4 or 16 Gbit/s (model
dependent) and gives better performance than the MCA bus.
IP Switch Control Board
Route
4Gb/s
Crosspoint
Switch
Manager
1 Gb/s to each Media Card
Switch Engine Interface
T3-OC12
IP
Packet
Forwarding
Route
Table and
Lookup
I/O
Buffering
LAN/WAN
LAN/WAN Interfaces
Media Cards
Figure 7. GRF 400
In conventional routers, each packet is processed at each gateway (also
called hop) along a path. The processing is done at the Layer 3 level (see
Figure 8 on page 14) and requires a router’s CPU to process both the packet
and the route information. Conventional routers use shared resources, which
leads to congestion and poor scalability and performance. Software-based
route-table lookups can be very slow, if the route-table is not in cache.
Router Node
13
Download from Www.Somanuals.com. All Manuals Search And Download.
Unswitched Data
Switched Data
5
2
Router
4
Router
1
3
Router
Disadvantages:
Shared data paths
All processing done on Layer 3
Slow softwarebased processing
Router
Router
Process
Process
Layer 3
Layer 2
Packets
Packets
......
......
........
.....
.....
Figure 8. Conventional Routers
The SP Switch Router provides near wire-speed packet forwarding while
using standard routing protocols. This ensures interoperability with other
network technologies and does not require a specific network architecture,
such as ATM. It works equally well in large and small networks. At each hop
where a routing switch is used, routes are processed at Layer 3 but the
SP Router, the route processing is done through hardware, so all processing
is done at near-wire speed.
14
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Unswitched Data
Switched Data
5
2
Router
Examples:
Ascend GRF, SP Switch Router
Cisco 12000
4
Router
1
3
Router
Disadvantages:
Hardware can be hard to upgrade *
Reduced routing functions *
Switch
Switch
Advantages:
Process Route
Process Route
Layer 3
Layer 2
Behave like traditional router
Not dependent on a network architecture
Interoperability
.........
..... .....
..... .....
Figure 9. Switched Routers
Other advantages of using GRF are as follows:
• Availability of a redundant power supply
• Availability of a redundant fan
• Availability of a hot-swappable power supply
• Availability of a hot-swappable fan (model 16S only)
• Availability of hot-swappable media adapters (to connect to networks)
• Scalability of up to 4 or 16 media adapters, depending on the GRF model
Perhaps the greatest advantage of using the GRF is improved
price/performance. As previously mentioned, the GRF is a dedicated router,
and as such it is much more cost effective for routing IP traffic than using
dedicated RS/6000 SP node.
2.1.6 Overview of Supported Routing Protocols
In addition to static routes, various routing protocols are available on the
GRF, as follows:
RIPRouting Information Protocol Version 1 or 2 (RIP 1 or 2)
OSPFOpen Shortest Path First
IS-ISIntermediate System to Intermediate System (an OSI gateway protocol)
Router Node
15
Download from Www.Somanuals.com. All Manuals Search And Download.
MulticastIP Multicast and OSPF Multicast
EGPExterior Gateway Protocol
BGPBorder Gateway Protocol Version 3 or 4 (BGP 3 or 4)
More details about the various protocols are in Section 2.2.2, “Supported
2.1.7 Media Adapters At-a-Glance
Available IP forwarding media cards are:
• 1-port 100 Mbyte/s Switch Adapter
• 8-port 10/100 Mbit/s Ethernet cards
• 2-port 155 Mbit/s OC-3 ATM (Asynchronous Transfer Mode UNI 3.0/3.1)
• 1-port 622 Mbit/s OC-12 ATM
• 4-port 100 Mbit/s FDDI (Fiber Distributed Data Interface)
• 1-port 800 Mbit/s HIPPI (High Performance Parallel Interface)
• 2-port 52 Mbit/s or 45 Mbit/s DS-3 HSSI (High Speed Serial Interface)
• 1-port 155 Mbit/s IP/SONET OC-3c
More details are in Section 2.3.8, “Other Media Cards” on page 39.
2.1.8 Benefits of the GRF
The crosspoint switch is a nonblocking crossbar. This architecture is faster
than an RS/6000 SP node, in which media adapters communicate through a
shared microchannel bus.
To take advantage of the fast I/O provided by the crosspoint switch, fast route
table access time is required. The GRF can store up to 150,000 routes in
memory on each media card, while an RS/6000 SP node can store only
hundreds. It is said that you need about 50,000 routes for the whole Internet.
This means that the GRF is able to retrieve a route faster than an RS/6000
SP node.
The GRF is able to route up to 2.8 million packets per second for the 4-slot
model and 10 million packets per second for the 16-slot model.
All the media adapters on the GRF are hot-pluggable. This differs from using
an RS/6000 SP node as your router. Should any network adapter on the
RS/6000 SP node fail, the node has to be brought down to replace the faulty
16
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
adapter. As a result, other network adapters are brought down as well.
Bringing down the router will impact all the networks in the location.
Each RS/6000 SP is allowed to connect to multiple SP Switch Router
Adapters, and it does not matter if these adapters are on different GRFs.
Connecting multiple SP Switch Router adapters to either different partitions in
an RS/6000 SP or to different RS/6000 SPs allows them to communicate with
each other and with the other GRF media adapters via the SP Switch.
Attention
The SP Switch Router model 04S can support four media cards such as
FDDI or ATM. The SP Switch Router model 16S can support 16. All card
slots could be occupied by SP Switch Router adapters; this means a
maximum of 4 SP Switch Router adapters for model 04S and a maximum
of 16 SP Switch Router adapters for model 16S.
Note: The number of packets that the GRF can route per second depends on
the following:
• The type of media adapter
• The size of the packet
2.1.9 Price Comparison
Figure 10 on page 18 shows a price comparison between an RS/6000 SP
node solution and a GRF based solution for three sample configurations.
Router Node
17
Download from Www.Somanuals.com. All Manuals Search And Download.
9077-04S
with one SP Switch Router Adapter
HIPPI Adapter
$
53,000
13,500
66,500
$
53,000
19,000
48,000
3,200
1,950
595
10,000
17,500
135 MHz Wide Node
64 MB memory
4.5 GB Disk
u
72,000
$
53,000
20,000
48,000
3,200
1,950
595
10,000
15,980
Ethernet
SP Switch Adapter
73,000
HIPPI Adapter
48,000
3,200
1,950
595
10,000
5,390
SP Switch Adapter
81,245
4 FDDI 1-port SAS Adapters *
2 ATM 155 Adapters *
75,730
69,135
Figure 10. Price Comparison
These price comparisons are based on US prices as of March 1998. In other
countries these prices may be different. The basic message of these charts is
that the solutions based on the GRF could be quite competitive and will quite
often be cheaper than the conventional configurations.
Let us look, for example for a solution connecting an RS/6000 SP via HIPPI to
a mainframe system. The first chart shows that a GRF solution is cheaper
than adding a dedicated node for the HIPPI connection to your system, apart
from the fact that the GRF solution is the better choice from a performance
point of view.
It is nearly the same if you need a connection to an FDDI network. One GRF
FDDI media card offers four independent singlering connections. An offer
based on an dedicated SP node is more expensive than a GRF solution.
Our example on the third chart focuses on an ATM connection. In this case,
the RS/6000 SP node based solution is the more reasonable solution, if you
consider only the price. But the difference is not that much; the growth path
with the GRF-based offer will be better than the node solution.
2.2 GRF Software
The software functionality of the GRF is distributed between the Router
Manager on the IP Switch Control Board (see also Section 2.3.2, “GRF
Features” on page 26) and the individual media cards. While the Route
18
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Manager updates the system routing tables and performs other administrative
functions, the intelligent processors on each media card perform all routing
functions. This design supports efficient distributed processing of router
operations.
2.2.1 IP Protocol
The GRF supports IP datagram routing between major types of standard
media. The implementation conforms with IP Version 4 and routing
specifications described in Internet RFCs.
Each media card has a complete set of route and Address Resolution
Protocol (ARP) information contained in the program memory space of the
card’s on-board processor. IP packets are buffered in large transmit and
receive buffers from which they are transmitted across the central switch
fabric to the destination media. Any difference in MTU size (large MTU to
smaller MTU) is handled by packet fragmentation as specified in the IP
standard. Logic on the destination media is responsible for any
media-specific processing of the packet, such as producing 53-byte cells for
ATM.
Data Forwarding
Individual media cards maintain their own route tables, perform lookups, and
autonomously handle the passing of datagrams to other media cards for
export, without intervention of the Router Manager. Layer-3 decisions are
local to each card.
Route Table Implementation
Critical to providing sustained performance in a highly dynamic environment
are the cacheless route table and route lookup implementation. Each card
carries a complete copy of the route table and can support up to 150,000
entries.
Keeping pace with significant advances in routing, the GRF also supports
variable-length subnet masking and route aggregation.
Router Node
19
Download from Www.Somanuals.com. All Manuals Search And Download.
Subnet Masking/Supernetting
Variable length subnet masking is a classless addressing scheme for
interdomain IP packet routing. It is a way to more efficiently manage the
current 32-bit IP addressing method. Subnet masks let sites configure the
size of their subnets based on the site needs, not on the arbitrary Class A, B,
and C structure originally used in the Internet addressing. Class-based
addressing restricts the boundary to the 8-bit boundaries and is implicitly
derived from the first eight bits of the network address. The new addressing
method allows the network portion of an IP address to be separated from the
host portion of the address at any point within the 32-bit address space. This
expanded boundary is called the "netmask" and is explicitly provided to the
router along with the network address information. Class-based addressing
restricts the boundary to the 8-bit boundaries and is implicitly derived from
the first eight bits of the network address.
Subnet masking offers a number of benefits by extending the current address
space. By eliminating implicit netmask assignments, addresses can now be
assigned from any unused portion of the entire 32-bit address range rather
than from within a specific subset of the space previously called a class.
Since it hides multiple subnets under a single network address, this method is
called supernetting.
Classless addressing allows the network administrator to further apportion an
assigned address block into smaller network (or host) segments based on
powers of two (2, 4, 8, 16 networks, for example). Knowledge of the
apportioned segments need not be communicated to exterior peers. They
need only a single pointer to the entire address block. Not only does subnet
masking better utilize address space, but implemented properly it results in
significantly smaller routing tables.
2.2.2 Supported Routing Protocols
In the days of a single Internet, core groups of independent networks were
called autonomous systems. We will use the term autonomous systems (AS)
in the following description of protocols. The routing protocols supported on
the GRF can be divided into two classes: Interior routing protocols or interior
gateway protocols (IGPs) and Exterior routing protocols (EGPs).
• Interior routing protocols
Interior routing protocols are used to exchange routing information
between routers within a single autonomous system. They are also used
by routers that run exterior protocols to collect network reachability
20
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
information for the autonomous system. Here is the list of interior
protocols supported by the GRF:
• RIP
The Routing Information Protocol (RIP), as delivered with most UNIX
systems, is run by the routing deamon routed. During the startup of
routed a request for routing updates is issued. After that, the daemon
listens for responses to the request. Systems that are configured to
supply RIP information hear this request and respond with update
packets based on the information in the system’s routing table. The
update packets contain the destination addresses from the routing
table and the routing metrics associated with each destination. Update
packets are send on request and periodically to keep routing
information accurate.
• OSPF
Open Shortest Path First (OSPF) is defined by RFC 2178 (Request for
Comments). It is a link-state protocol and very different from RIP. A
router running RIP shares information about the entire network with its
neighbors. A router running OSPF shares information about its
neighbors with the entire network. The "entire network" means, at
most, a single autonomous system. OSPF further defines a hierarchy
of routing areas within an autonomous system:
• Areas
These are sets of networks within a single autonomous system that
have been grouped together. The topology of an area is hidden from
the rest of the autonomous system, and each area has a separate
topology database. Routing within the autonomous system takes
place on two levels, depending on whether the source and
destination of a packet reside in the same area (intra-area routing)
or different areas (inter-area routing).
Intra-area routing is determined only by an area’s own topology.
That is, the packet is routed solely on information obtained within
the area.
Inter-area routing is always done via a backbone.
The dividing of an autonomous system into areas enables a
significant reduction in the volume of routing traffic required to
manage the routing database for a large autonomous system.
• Backbone
The backbone consists of those networks not contained in any area,
their attached routers, and routers that belong to multiple areas.
Router Node
21
Download from Www.Somanuals.com. All Manuals Search And Download.
Every area must connect to the backbone, because the backbone is
responsible for the distribution of routing information between
areas. The backbone itself has all the properties of an area. Its
topology is separate from that of other areas.
• Subarea
A subarea has only one area border router, which means that there
is only one route out of the area. In this case, the area border router
does not need to advertise external routes to the other routers
within the subarea. It can simply advertise itself as the default route.
• The sequence of operations performed by OSPF routers is as follows:
1. Discovering OSPF neighbors
2. Electing the Designated Router
3. Forming adjacencies
4. Synchronizing databases
5. Calculating the routing table
6. Advertising Link States
Routers go through these steps when they first come up, and repeat
them in response to the events that occur in the network. Each router
must perform each of these steps for each network it is connected to,
except for the calculation of the routing table. Each router generates
and maintains a single routing table for all networks.
• Multicast
• IP Multicast
The GRF supports IP multicast routing per RFC 1112 and some
components of RFCs 1301 and 1469. The implementation includes
the IP multicast kernel modifications, dynamic route support and
mrouted (multicast route daemon).
Data that arrives at a GRF interface is duplicated and forwarded to
multiple destination interfaces. The multicast packet’s destination
address is a Class D address. A lookup of the multicast route table
returns a list of Virtual Interfaces (VIFs) to which the packet is sent.
• OSPF Multicast
The GRF uses the multicast capability of OSPF Version 2, as
described in RFC 1583 and RIP Version 2, for communications
between routers.
22
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Host extensions for IP multicasting as described in RFC 1112 are
also provided. The Router Manager acts as a host and uses the
Internet Group Management Protocol (IGMP), version 2, to add and
delete its membership in multicast groups. Accordingly, the Route
Manager "joins" the appropriate routing protocol host groups for
OSPF and RIP.
• IS-IS
Intermediate System to Intermediate System (an OSI gateway
protocol) is a protocol similar to OSPF: it also uses a Link State,
Shortest Path First algorithm. However, IS-IS is an OSI protocol used
for routing Connectionless Network Protocol (CLNP) packets within a
routing domain. CLNP is the OSI protocol most comparable to IP.
• Exterior Routing Protocols
Exterior Routing Protocols are used to exchange routing information
between routers in different autonomous systems. Here is the list of
exterior routing protocols supported by the GRF:
• EGP
The Exterior Gateway Protocol (EGP) is the protocol used for
exchange of routing information between exterior gateways (not
belonging to the same autonomous system).
EGP gateways may only forward reachability information for networks
within their autonomous system. This routing information must be
collected by the EGP gateway, usually via an GP).
• BGP
Border Gateway Protocol (BGP) is the leading exterior routing protocol
of the Internet and is replacing EGP as the exterior protocol of choice.
It is based on the OSI InterDomain Routing Protocol (IDRP). BGP
supports policy-based routing, which uses nontechnical reasons (for
example, organizational, political, or security considerations) to make
routing decisions. Thus, BGP enhances an autonomous system’s
ability to choose between routes and to implement routing policies
without relying on a central routing authority.
2.2.3 Filtering
IP filtering supports specific permit or deny decisions for each instance of a
filter (per logical interface). The criteria within each filter may include any
combination of the following:
• Protocol (ICMP, TCP, UDP)
Router Node
23
Download from Www.Somanuals.com. All Manuals Search And Download.
• Source address
• Destination address
• Protocol port number (single number, or range, or ranges) for TCP and
UDP
• Established TCP connections
2.2.4 System Management
The GRF currently supports the Simple Network Management Protocol
(SNMP) Version 1, which provides a mechanism for remote query or setting
operational parameters for the device. Media cards collect network statistics,
which can be reported to network management packages via the Router
Manager on the IP Switch Control Board. In addition to collection statistics
and other management information, the SNMP agent is also capable of
issuing traps. For more information, see Section 2.4.5, “SP Extension Node
2.3 GRF Hardware
As already mentioned, the unique GRF switching architecture is specially
designed for high-performance packet forwarding. The following sections give
you more details about the various hardware components.
2.3.1 GRF Block Diagram
Figure 11 on page 25 shows the two models: the 4-slot and the 16-slot model.
24
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
19"
Cooling Fan Drawer
21"
19"
IP Switch Control Board
IP Forwarding Media Card
IP Forwarding Media Card
IP Forwarding Media Card
IP Forwarding Media Card
Power
Supply
Power
Supply
Power
Supply
Power
Supply
5.25"
GRF 400
GRF 1600
Figure 11. GRF Models
The SP Switch Router model 9077 04S (GRF 400) can accommodate up to
four media adapters.
The SP Switch Router model 9077 16S (GRF 1600) can accommodate up to
16 media adapters.
Each adapter enables the GRF to connect to one or more networks.
Each of the models has an additional slot for the IP Switch Control Board,
which is used to control the router.
GRF 400
PartDescription
Cooling FansThese are located on the right side of the chassis and cannot
be accessed without bringing down the GRF. The
fans are redundant, allowing service to be
deferred until it is convenient to bring down the
GRF.
Media CardsThere are four media card slots on this chassis. They are
slotted horizontally and are located at the bottom
of the chassis.
IP Switch Control BoardThis board is located at the top of the four media
slots and is also slotted horizontally.
Router Node
25
Download from Www.Somanuals.com. All Manuals Search And Download.
Power SupplyThe left side of the chassis is reserved for the two power
supplies that are required for redundancy.
The failed power supply can be hot-swapped out of the GRF chassis.
GRF 1600
Part Description
Cooling FansThese are located at the top of the chassis, and can be
accessed separately from the other parts of the
GRF. The fan tray contains redundant fans and is
hot-swappable.
Media CardsThere are 16 media card slots on this chassis. They are slotted
vertically. Eight of the cards are on the left side of
the chassis, and eight are on the right side.
IP Switch and
Control BoardThese boards are located in the middle of the 16 media slots
and are also slotted vertically.
Power SupplyThe base of the chassis is reserved for the two power supplies
that are required for redundancy. The failed power
supply can be hot-swapped out of the GRF
chassis.
2.3.2 GRF Features
GRF has the following features:
• Redundant power supply
Should any power supply fail, a message is sent to the control board. The
power supply will automatically reduce its output voltage if the
temperature exceeds 90 °C (194 °F). If the voltage falls below 180V, the
GRF automatically shuts down.
• Hot-swappable power supply
The faulty power supply can be replaced while the GRF is in operation.
• Redundant fan
For the GRF 1600 model, if one fan breaks down, a message is sent to the
control board.
For both models, when the temperature reaches 53°C (128 °F), an audible
alarm sounds continuously, and a message is sent to the console and
logged into the message log.
26
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
If the temperature exceeds 57.5 °C (137 °F), the GRF does an automatic
system shutdown.
• Hot-swappable fan
For the GRF 16S model, the cooling fan can be replaced while the GRF is
in operation.
• Hot-swappable adapters
There are two types of adapters on the GRF: the media adapters and the
IP Switch Control Board.
The media adapters are independent of each other and can be replaced or
removed without affecting any other adapter or the operation of the GRF.
However, the IP Switch Control Board is critical to the GRF. Should this
board be unavailable, the router fails.
• Crosspoint switch
The crosspoint switch is a 16x16 (16Gb per second) or 4x4 (4Gb per
second) crossbar switch for the GRF 16S and GRF 04S, respectively, see
Figure 12 on page 27. It is the I/O path used when the media adapters
need to communicate with each other.
Serial Interface
16MB Rx / 16MB Tx Buffers
Speed decoupling
WAN delays
QBRT Route Table Lookup
Times range from 1-2.5 µs
On-Board Processor
IP Header
QBRT
CPU
Media Interface
RT
Route decisions
4 Gb/s Switch
CPU
............
..........
Combus
80 Mb/s bus
Out-of-Band
User Interface
Dynamic Routing Engine (Route Manager)
Configuration/Control
Figure 12. GRF Architecture
Router Node
27
Download from Www.Somanuals.com. All Manuals Search And Download.
1. Normally, all media cards have a 4 MB send buffer and a 4 MB receive
buffer, except the SP Switch Adapter card, which has a 16 MB buffer
size for each buffer. See also Section 2.3.5, “Characteristics of GRF
2. Quick Branch Routing Technology (QBRT) is a hardware-assisted
route table lookup. Route lookup times range from 1 - 2.5 µs with up to
150,000 next-hop routes in the table. Not all media cards use QBRT.
Cards that do not use QBRT use a microcode lookup.
The benefit of this architecture is that the entire route table can be
stored locally on the media card and searched quickly. In the traditional
cached route table method, a small number of routes can be stored and
searched locally. However, when a large number of routes is desired,
or the kind of traffic one would see on the Internet backbone arises,
caching is inadequate. Inevitably, cache misses occur, and route table
lookups are performed at a limited, central, shared resource.
Performance is enhanced even further with parallel processing of table
lookups occurring on each media card, which is another technique that
helps assure linear scalability. The router manager on the controller
board, which also contains the switch fabric, maintains the master
route table and distributes updates simultaneously to all installed
media cards, even as the cards continue their forwarding functions.
3. On-Board Processor
4. Router management takes place on the IP Switch Control Board (see
based on a 166 Mhz Pentium processor. It is responsible for system
monitoring, configuration management and the user interface.
5. The GRF communications bus (Combus) is an "out-of-band" data path
for configuration, control and monitoring of media cards. The Combus
connects the IP Switch Control Board to the media cards independent
of the switch connection to each card. It is not used for routed data
between the cards. Route update packets received on any media card
are also sent across the Combus to the Route Manager and, therefore,
do not have to compete with normal IP traffic. The Combus is a serial
bus with a transferrate of 80 Mb/s and is FIFO-buffered. The Arbitration
Logic is on the IP Switch Control Board.
2.3.2.1 Data Packet Processing
With the knowledge about the local routing functions of the media cards, we
from one media card to another.
28
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
External
Interface
External
Interface
DMA
DMA
CPU
CPU
QBRT
QBRT
DMA
DMA
Switch
Control Board
Internal
Interface
Internal
Interface
Dynamic
Route
Manager
Figure 13. Data Packet Transfer
The routing can be divided into the following steps:
1. A data packet is received by the media card.
2. The packet is transferred to the receive buffer by the DMA engine.
3. The CPU examines the header and gives the destination address to the
route lookup hardware.
4. The QBRT finds the next hop in the route table.
5. A special header is added to the packet by the CPU. This header contains
information for the downstream card to process the packet, as well as the
next hop found by the QBRT.
6. The packet is then transferred to the internal serial interface by the DMA
engine.
7. A connection to the downstream card is set up through the switch.
8. The serial stream is converted back to parallel format by the downstream
card.
9. The packet is transferred to the transmit buffer by the DMA engine.
Router Node
29
Download from Www.Somanuals.com. All Manuals Search And Download.
10.The header is examined by the CPU, which uses the information to build a
new header that will deliver the data across the media interface.
11.The DMA engine transfers the packet to the media interface.
12.The packet is transferred across the media.
2.3.2.2 Routing Packet Processing
The processing of packets with routing information is a little bit different from
the data packet processing procedure as you can see in Figure 14.
External
Interface
External
Interface
DMA
DMA
CPU
CPU
QBRT
QBRT
DMA
DMA
Switch
Control Board
Internal
Interface
Internal
Interface
Dynamic
Route
Manager
Figure 14. Routing Packet Processing
These are the steps for processing routing packets:
1. A routing packet is received by the media card.
2. The packet is transferred to the receive buffer by the DMA engine.
3. The CPU examines the header and gives the destination address to the
route lookup hardware.
4. The QBRT finds the next hop in the route table.
5. A special header is added to the packet by the CPU. This header contains
information for the downstream card. The result of the hardware lookup
determines whether the packet should be forwarded to the Router
Manager.
30
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
6. The packet is then transferred to the Combus interface by the DMA
engine.
7. The packet is sent to the IP Switch Control Board’s Router Manager
across the Combus.
8. The Route Manager receives the packet and passes it to the dynamic
routing software.
9. The packet is processed and global routing information is determined.
10.Route updates are broadcast across the Combus to all media cards
simultaneously.
11.Each card receives the update packet and makes changes to its route
tables.
12.The packet is transferred across the media.
To ensure that dynamic routing packets are not dropped during times of
heavy congestion, precedence features are used. Routing packets are given
a high-priority tag and a user-configurable threshold for Tx buffers is
maintained for high-priority traffic.
2.3.3 IP Switch and Control Board
The control board, also known as the IP Switch Control Board, is accessed
through Telnet or a locally attached VT100 terminal. The IP Switch Control
Board is supplied with the GRF and is necessary for its operation. The VT100
terminal is not supplied with the GRF. It is only needed for the installation of
the GRF.
Using terminal emulation software instead of looking for a real VT 100
terminal may be an alternative. You can use your Control Workstation or one
of your SP nodes. Install the ATE package (advanced terminal emulation) on
your RS/6000 and establish a serial connection between the system and the
router.
After installation, all future access to the GRF can be through Telnet to the IP
Switch Control Board’s administrative Ethernet.
The IP Switch Control Board is identified as slot 66 in both GRF models. A
sideview of the GRF 400 slot numbering scheme is shown in Figure 15 on
Router Node
31
Download from Www.Somanuals.com. All Manuals Search And Download.
Backplane
IP Switch Control Board
66
3
Slot numbers in decimal
for media cards
2
1
0
Figure 15. Side View of GRF 400 Chassis with Slots Numbered
The GRF 1600 has 16 media slots. The control board is located in slot 66, as
shown in Figure 16.
Backplane
Media cards
Slot numbers
0 1 2 3 4 5 6 7 66
8 9 10 11 12 13 14 15
Switch board
Control board
Figure 16. Top View of the GRF 1600 Chassis
The CPU in the IP Switch Control Board is a 166MHz Pentium processor and
runs a variant of BSD UNIX as its operating system. For this reason, the GRF
administrator is assumed to be proficient in UNIX.
The IP Switch Control Board is used to install, boot, and configure the router
and its media adapters.
It is also used for the logging of messages, the dumping of memory and
status, and to perform diagnostic checking of both the GRF and the media
adapters.
32
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.3.3.1 Route Manager
As already mentioned, the router management takes place on the IP Switch
Control Board.
Specific functions of the Route Manager are:
• It processes all dynamic routing packets.
• It synchronizes the route tables on the media cards.
• It controls the media cards: issues interrupts and resets to individual
media cards and downloads executable programs and connection
information.
2.3.3.2 IP Switch Control Board Components
Let us examine the IP Switch Control Board in more detail.
Figure 17 shows all the components of the IP Switch Control Board:
Flash
Memory
(85 MB)
32MB
32MB
32MB
32MB
32MB
32MB
32MB
32MB
System Bus
CPU
Pentium
166MHz
PCMCIA PCMCIA
Card Card
Admin Ethernet
(de0)
Figure 17. IP Switch Control Board
Router Node
33
Download from Www.Somanuals.com. All Manuals Search And Download.
ItemDescription
MemoryThe IP Switch Control Board comes standard with 128 MB of
memory (the four shaded blocks of 32 MB of
memory in the upper left corner).
The memory can be upgraded to 256 MB, in increments of 64 MB (the four
white blocks of memory).
The system uses the first 32 MB of memory for file system storage. The
top half is used for applications such as the SNMP
agent, the gated daemon, and for the operating
system.
Flash memoryThis memory (the 85 MB ATA flash memory on the system) is
used to store the operating system information
and the configuration information for the GRF.
System busThis bus is used by the IP Switch Control Board components to
communicate with each other.
Pentium processorThis 166 MHz processor drives the IP Switch Control
Board and the GRF. As previously mentioned, this
processor runs a variant of BSD UNIX, and so it is
useful for the GRF administrator to have UNIX
management skills.
Administrative EthernetThis Ethernet is known to the GRF as de0. This port
supports the 10BaseT or the 100BaseT Ethernets
and switches between them automatically,
depending on the type of network used.
To use 10Base2 or 10Base5, the user must add a transceiver (supplied by
the user).
PCMCIA cardsThe two white blocks at the bottom right corner of the figure
are PCMCIA slots.
There are two types of PCMCIA cards:
• Slot A on the 9077 is configured with a 520 MB disk drive. This disk is
used to hold the log of the GRF. It may also be
used to backup the configuration of the GRF.
Making a backup is strongly recommended.
• The PCMCIA modem card, also available as an optional device, allows
the user to dial into the GRF through a modem
to administer it remotely.
Note: For the initial setup, the console must be available locally, not
through the modem.
34
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Additionally, the RS232 port (which is not shown in the figure) allows you to
connect the VT100 console by using an RS232 null modem cable. The
console and cable must be supplied by the user.
2.3.4 Memory Guidelines for the IP Switch Control Board
As already mentioned, the GRF base system comes with 128 MB of memory.
In all GRF memory configurations, 32 MB are used for the file system and the
remainder is used for system operations. For example, in the base system,
there are 128 MB of total memory, 32 MB of available memory and 78 MB of
configuration information). Up to six additional 32 MB DRAM SIMMS may be
added to support larger dynamic routing tables and larger numbers of peers
(for a total of 256 MB, 204 MB usable).
The following table provides guidelines for memory configuration. All media
cards can hold up to 150 KB route entries. The control board, depending on
memory configuration, can hold 35,805 to 521,730 route prefixes. Select the
amount of memory according to your routing environment. Additional memory
may be required for higher average numbers of routes per BGP peer.
Table 1. Memory Configuration
Customer
Profile
Total
Mem.
Avail.
Usable No. of
Route
No. of
Route
No. of
Peer
Entries
on
Prefixes
in
Sess.
Media
Cards
Dynamic
Database
Static Routing
Small POP
64 MB
32 MB
14 MB
78 MB
150 KB
150 KB
35805
0
3
7
128 MB 96 MB
199485
362165
Medium POP/
ISP Backbone
192 MB 160 MB 142 MB 150 KB
Large POP/
256 MB 224 MB 204 MB 150 KB
521730
10
Exchange Point/
Route Reflection
Server
Figure 18 gives an overview of the memory layout and the possible memory
extensions.
Router Node
35
Download from Www.Somanuals.com. All Manuals Search And Download.
64MB RAM
64MB RAM
64MB RAM
64MB RAM
- System software
- Config files
32MB
(fixed size)
- Gated binary
- Route table
20MB
- Kernel runs
- Gated runs
8-12MB
(fixed size)
= expandable area of RAM
Figure 18. System RAM
2.3.5 Characteristics of GRF Media Cards
All GRF media cards (media adapters) are self-contained and independent of
other media adapters.
Each media card has an onboard processor that is responsible for IP
forwarding on the media adapter.
Each media card has two independent memory buffers, a 4 MB send buffer
and a 4 MB receive buffer. These buffers are necessary to balance the speed
differences between the media adapters, because they have different transfer
rates.
Each onboard processor has local memory that can contain a local route
table with up to 150,000 entries, to be used for routing on the media adapter.
Because these route entries are in local memory, access to them is very fast.
When the media adapter is started up, it gets its initial route entries from the
IP Switch Control Board.
2.3.6 SP Switch Router Adapter
Switch Router Adapter in detail. This adapter allows the GRF to connect
directly into the SP Switch.
36
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Crosspoint
Switch
Media Board
Receive
TBIC
16MB
Buffer(1)
FIFO
(1)
3
2
7
Receive
Proc & C
6
5
Send
Proc & C
1
0
Serial
Daughter
Card
FIFO
(2)
Send
TBIC
16MB
Buffer(2)
4
SP Switch
Figure 19. SP Switch Router Adapter
The SP Switch Router Adapter is made up of two parts: the media board and
a serial daughter card.
The serial daughter card is an interface for the media board into the
crosspoint switch. This switch is the medium by which the GRF (media)
adapters talk to each other.
The purpose of the media board is to route IP packets to their intended
destination through the GRF. The SP Switch Router adapter described here is
used for routing IP packets to and from the SP Switch to other systems
connected directly or indirectly to the GRF. A brief description of the
components on the media board follows.
Receive TBICThis component receives data segments from the SP Switch
and notifies the Receive Controller and Processor that
there is data to be transferred to the buffer.
Router Node
37
Download from Www.Somanuals.com. All Manuals Search And Download.
Receive Controller
and ProcessorThis component recognizes the SP Switch segments and
assembles them into IP packets in the 16 MB buffer.
Up to 256 IP datagrams can be handled
simultaneously. When a complete IP packet has been
received, the Receive Controller sends the packet to
the FIFO (1) queue for transfer to the serial daughter
card.
Buffer (1)This component is segmented into 256 64 KB IP packet buffers. It
is used to reassemble IP packets before sending them
to the FIFO queue, as switch data segments may
arrive out of order and interleaved with segments
belonging to different IP packets.
FIFO (1)This component is used to transfer complete IP packets to the serial
daughter card and even the flow of data between the
SP and GRF crosspoint switch.
FIFO (2)This component receives IP packets from the serial daughter card
and transfers them to Buffer (2).
Buffer (2)This buffer is used to temporarily store the IP packet while its IP
address is examined and a proper SP Switch route is
set up to transfer the packet through the SP Switch.
Send Controller
and ProcessorThis component is notified when an IP packet is received in
the FIFO (2) queue and sets up a DMA transfer to
send the packet to Buffer (2). The Send Processor
looks up the IP address in the packet header and
determines the SP Switch route for the packet, before
notifying the Send Controller to send the packet to the
Send TBIC from Buffer (2).
Send TBICThis component receives data from Buffer (2) and sends it in SP
Switch data segments to the SP Switch.
2.3.7 Media Card Performance
The SP Switch Router adapter has the following performance characteristics:
• It is able to transfer up to 100 MB per second. The limiting factor is the
crosspoint switch connection bandwidth.
38
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
• It is able to transfer up to 30,000 packets per second. At 20,000 packets
per second, each packet needs to be at 5 KB in order to achieve the 100
MB per second transfer rate mentioned.
• As previously mentioned, each adapter stores its own route tables in
memory. Therefore, route table lookup is very fast, that is, less than 2.5
µs.
• Finally, each media adapter has a 1 Gbit per second dedicated link into
the crosspoint switch. That is why the 4-port and 16-port models have an
aggregate bandwidth of 4 Gbit and 16 Gbit per second, respectively, for
the crosspoint switch.
2.3.8 Other Media Cards
The following are other media cards and adapters currently supported on the
GRF:
EthernetThe 10/100 Mb Ethernet media adapter consists of eight
10/100BaseT Ethernet ports. All ports support only shielded
twisted pair (STP) and unshielded twisted pair (UTP-5) copper
cables. Other types of cables require the user to supply the
appropriate transceivers.
ATM OC-3cThe ATM OC-3c media adapter allows the user to connect up to
two connections into the ATM network at 155 Mb/s.
ATM OC-12cThe ATM OC-12c media adapter allows the GRF to connect to a
single ATM network at speeds of up to 622 Mb per second.
FDDIThe FDDI media card provides four ports in the card. These ports allow
the media card to be connected into the Fiber Distributed Data
Interchange (FDDI). The four ports can be configured such
that they support the following:
• Two dual-ring FDDI networks
• One dual-ring and two single-ring FDDI networks
• Four single-ring FDDI networks
HIPPIThe HIPPI media adapter is a single-port card that allows the GRF to
connect to a High Performance Parallel Interface (HIPPI)
network at speeds of up to 800 or 1600 Mb/s. After deducting
the overhead, this medium can support connections of up to
100 MB/s.
HSSIThe High Speed Serial Interface (HSSI) is a dual-ported media adapter
that can connect to two serial networks simultaneously. Each
port is capable of up to 45 Mb per second.
Router Node
39
Download from Www.Somanuals.com. All Manuals Search And Download.
IP/SONETThe IP/SONET OC-3c is a single-ported card that allows the user
to connect to a digital network using a transmission format
known as Synchronous Optical Network protocol (SONET).
This standard is increasingly popular in the
telecommunications industry.
2.3.9 GRF Operating Environment
As previously mentioned, the operating temperature should not exceed 53 °C
(128 °F). Even though there is a buffer between the operating temperature
and the warning temperature, it is best to keep the temperature within the
operating level in order to minimize the possibility of damage to GRF
components.
2.4 PSSP Enhancements
This section discusses the enhancements made to PSSP to accommodate
the Dependent Node Architecture.
2.4.1 SDR Enhancements
Data Repository (SDR) needed to be extended to support the dependent
node architecture. Two classes have been added to the SDR.
• DependentNode
• DependentAdapter
2.4.1.1 DependentNode Attributes
Table 2. DependentNode Attributes
User Defined
System Defined
switch_node_number
switch_number
node_number
extension_node_identifier
reliable_hostname
switch_chip
management_agent_hostname
snmp_community_name
switch_chip_port
switch_partition_number
40
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
The attributes of the DependentNode class are described in detail as follows:
AttributeDescription
node_numberThis user-supplied node number represents the node position
of an unused SP Switch port used for the SP
Switch router adapter.
extension_node_identifierThis is a 2-digit slot number that the SP Switch
router adapter occupies on the GRF. Its
range is from 00 to 15.
reliable_hostnameThe hostname of the administrative Ethernet, de0, is the
GRF’s hostname. Use the long version of
the hostname when DNS is used.
management_agent_hostnameThis attribute is the hostname of the SNMP
agent for the GRF. For the GRF dependent
node, this is the same as the
reliable_hostname.
snmp_community_nameThis field contains the SNMP community name that
the SP Extension Node SNMP Manager and
the GRF’s SNMP Agent will send in the
corresponding field of the SNMP messages.
This value must match the value specified in
the /etc/snmpd.conf file. If left blank, a
default name found in the SP Switch Router
Adapter documentation is used.
The following attributes are derived by the RS/6000 SP system when the
SDR_config routine of endefnodeis invoked.
AttributeDescription
switch_node_numberThe switch port that the dependent node is attached
to.
switch_numberThe switch board that the dependent node is attached to.
switch_chipThe switch chip that the dependent node is attached to.
switch_chip_portThe switch chip port that the dependent node is attached
to.
switch_partition_numberThe partition number to which the dependent node
belongs.
Router Node
41
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.1.2 DependentAdapter Attributes
Table 3. DependentAdapter Attributes
User Defined
node_number
netaddr
System Defined
-
-
-
netmask
The attributes of the DependentAdapter class are described in detail as
follows:
AttributeDescription
node_numberThis user-supplied node number represents the node position
of an unused SP Switch port to be used by the SP Switch
router adapter.
netaddrThis is the IP address of the SP Switch Router adapter.
netmaskThis is the netmask of the SP Switch Router adapter.
2.4.1.3 Additional Attributes
and the Switch_partition classes:
Table 4. Additional SDR Attributes
Syspar_map class
Switch_partition class
...
...
node_type
switch_max_ltu
switch_link_delay
Details of these attributes follow:
42
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
AttributeDescription
node_typeThis attribute is set to dependent for GRF and to standard for all
other RS/6000 SP nodes.
switch_max_ltuThis specifies the maximum packet length of data on the SP
Switch; the default is 1024. Do not change this value for
any reason.
switch_link_delaySpecifies the delay for a message to be sent between the
two furthest points on the switch; the default is 31. Do
not change this value for any reason.
2.4.2 New Commands
To support the dependent node architecture, seven new commands were
added. These commands can be divided into two groups. The first group
must only be executed with root permission on the Control Workstation. The
second group can be executed by any user on any standard RS/6000 SP
node.
Table 5. New Commands (root Executable)
Command
endefnode
enrmnode
Description
Define or change an dependent node
Remove a dependent node
endefadapter
enrmadapter
enadmin
Define or change an dependent node
Remove an dependent node adapter
Reconfigure or reset the dependent node
The first four commands all have the same characteristics, which are as
follows:
• The are part of the ssp.basic fileset.
• They must only be executed on the Control Workstation.
• They can only be executed by the root user.
• They only affect the current active partition.
• They only affect the SDR, unless the -r option is specified (this option is
not applicable to enrmadapter).
• They return a code of 0 if successful, 1 if failed.
Router Node
43
Download from Www.Somanuals.com. All Manuals Search And Download.
The enadmincommand is used to change the administrative state of a
dependent node in the GRF; it has the following characteristics:
• It is part of the ssp.spmgr fileset.
• It must only be executed on the Control Workstation.
• It can only be executed by the root user.
• The -roption from endefnodeand endefadapter triggers enadmin -a
reconfigure, while the -roption from enrmnode triggers enadmin -a reset.
• The return code is 0 if successful, 1 if failed.
any user on any standard node):
Table 6. New Commands (User Executable)
Command
Description
splstnodes
splstadapters
List SP nodes
List SP adapter
These commands have the following characteristics:
• They are part of the ssp.basic fileset.
• They can be executed on any standard RS/6000 SP node.
• They can be executed by any user.
• They only affect the current active partition unless the -G option is used.
The following sections describe the commands in more detail.
2.4.2.1 The endefnode Command
The endefnodecommand can be executed using smit. The fast path for smit is
enter_extnode. This command is used to add or change an extension node in
44
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 7. endefnode Command Options
Flags
SMIT Option
Description
-a
Administrative hostname This is the hostname of the GRF, and the IP name
of the GRF’s administrative Ethernet, de0. Use
long names if DNS is used in the network.
-c
SNMP community name
This field contains the SNMP community name
that the SP extension node SNMP Manager and
the GRF’s SNMP agent send in the
corresponding field of the SNMP messages. This
value must match the value specified in the
/etc/snmpd.conf file on the GRF. If left blank, a
default name found in the SP Switch Router
Adapter documentation is used.
-i
Extension node
This field contains the two-digit slot
number of the SP Switch Router Adapter
on the GRF. The value for this field is from
00-15 and is shown on the slots of the
GRF.
-s
-r
SNMP Agent hostname
This field refers to the hostname of the
processor running the SNMP Agent for the
GRF. In the current version of the GRF, this
value is equivalent to that of the
Administrative Hostname.
Reconfigure the
extension node
This field specifies whether the enadmin
command is to be activated after the
endefnodecommand completes. It is placed
here so that the user does not have to
explicitly issue the enadmincommand. If the
specification is yes, the -r option is part of
the command. If the specification is no, the
-roption is not part of the command.
Node number
This is the node number the extension
node logically occupies in the RS/6000 SP.
This command adds attribute information for the extension node. The
endefadapter command adds IP information, such as the IP address and
netmask for the extension node. Together, these two commands define the
extension node.
Router Node
45
Download from Www.Somanuals.com. All Manuals Search And Download.
Attention
Note that this command only affects the SDR, unless the -r option is used.
The -r option should be issued only if endefadapterhas been executed for
the extension node.
When the GRF is properly configured and powered on, with the SP Switch
Router Adapter inside, it periodically polls the Control Workstation for
configuration data. The -r option or enadmincommand is not required to
activate the polling here.
2.4.2.2 The enrmnode Command
The enrmnodecommand is used to remove an extension node from the SDR
DependentNode class and can also be executed using smit. The fast path for
Table 8. enrmnode Command Options
Flags
SMIT Option
Description
-r
Specifies whether the enadmincommand is to be
activated after the enrmnodecommand
Reset the
extension node
completes. With this option the user does not
have to explicitly issue the enadmincommand. If
the specification is yes, the -roption is part of the
command. If the specification is no, the -roption
is not part of the command.
Node number
This is the node number the extension node
logically occupies in the RS/6000 SP.
Attention
Note that this command only affects the SDR, unless the -roption is used.
This command should be issued with a -rflag, because the enadmin
command is not available for the extension node after enrmnodeis executed,
since the extension node has been removed from the SDR.
46
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.2.3 The endefadapter Command
The endefadaptercommand is used to add or change the extension node
adapter IP information in the SDR DependentAdapter object, and can be
executed using smit. The fast path for smitis enter_extadapter. The command
Table 9. endefadapter Command Options
Flags
SMIT Option
Description
-a
Network
address
Specifies the IP address of the extension node.
-m
-r
Network
netmask
Specifies the netmask for the extension node.
Specifies if the enadmincommand is to be
activated after the endefadaptercommand
completes. With this option, the user does not
have to explicitly issue the enadmincommand. If
the specification is yes, the -roption is part of the
command. If the specification is no, the -roption
is not part of the command.
Reconfigure
the extension
node
Node number
This is the node number the extension node
logically occupies in the RS/6000 SP.
Attention
Note that this command only affects the SDR, unless the -roption is
issued. The -r option should be issued only if the endefnodehas been
executed for the extension node.
When the GRF is properly configured and powered on, with the SP Switch
Router Adapter inside, it periodically polls the Control Workstation for
configuration data. The -roption or enadmincommand is not required to
activate the polling here.
2.4.2.4 The enrmadapter Command
The enrmadaptercommand is used to remove the SDR DependentAdapter
object, and can also be executed using smit. The fast path for smit is
delete_extadapter.
Router Node
47
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.2.5 The enadmin Command
The enadmincommand is used to change the status of the SP Switch router
adapter in the GRF and can also be executed using smit. The fast path for
Table 10. enadmin Command Options
Flags
SMIT Option
Description
-a
Either resetor reconfigure. A resetis sent to the
extension node SNMP Agent to change the
target node to a down state (not active on the SP
Switch). A reconfigureis sent to the extension
node SNMP Agent to trigger reconfiguration of
the target node, which causes the SNMP Agent
to request new configuration parameters from
the SP extension node SNMP Manager, and to
reconfigure the target node when the new
parameters are received.
Actions to be
performed on
the extension
node
Node number
This is the node number the extension node
logically occupies in the RS/6000 SP.
48
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.2.6 The splstnodes Command
The splstnodescommand is used to list the node attributes of all nodes in the
SDR, and can also be executed using smit. The fast path for smit is
Table 11. splstnode Command Options
Flags
Description
-h
Outputs usage information.
-G
Ignores partition boundaries for that output.
Inhibits header record in the output.
Uses the <delimiter> between its attributes in the output.
-x
-d <delimiter>
-p <string>
Uses the <string> value in the output in place of an
attribute that has no value.
-s <attr>
Sorts the output using the <attr> value. In SMIT, this field
is known as Sort Attribute.
-t <node_type>
Uses standardto list RS/6000 SP nodes, or dependent. If
none is specified, it displays standard and dependent. In
SMIT, this field is known as Node Type.
-N <node_grp>
<attr==value>
Restricts the query to the nodes belonging to the node
group specified in <node_grp>. If the <node_grp>
specified is a system node group, the -G flag is implied.
This operand is used to filter the output, such that only
nodes with attributes that are equivalent to the value
specified are displayed. In SMIT, this field is known as
Query Attribute.
<attr>
This is a list containing attributes that are displayed by the
command. If none is specified, it defaults to node number.
This list of attributes can be found in the DependentNode
class. In SMIT, this field is known as Attribute.
Router Node
49
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.2.7 The splstadapters Command
The splstadaperscommand is used to list the adapter attributes of all nodes in
the SDR, and can also be executed using smit. The fast path for smit is
Table 12. splstadapter Command Options
Flags
Description
-h
Outputs usage information.
-G
Ignores partition boundaries for its output.
Inhibits header record in the output.
Uses the <delimiter> between its attributes in the output.
-x
-d <delimiter>
-p <string>
Uses the <string> value in the output in place of an
attribute that has no value.
-t <node_type>
<attr==value>
Uses standardto list RS/6000 SP nodes, or dependent. If
none is specified, it displays standard and dependent. In
SMIT, this field is known as Node Type.
This operand is used to filter the output, such that only
nodes with attributes that are equivalent to the value
specified are displayed. In SMIT, this field is known as
Query Attribute.
<attr>
This is a list containing attributes that are displayed by the
command. If none is specified, it defaults to node number.
This list of attributes can be found in the Adapter and
DependentAdapter class. In SMIT, this field is known as
Output Attribute.
50
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.3 Enhanced Commands
introduction of the dependent node:
Table 13. Enhanced Commands
Command Comment
Eprimary
Estart
The dependent node cannot be the Primary node.
The dependent Node depends on the Primary node to calculate the
routes.
Efence
Enhanced for dependent node support.
Enhanced for dependent node support.
Eunfence
Here is a more detailed description about the modifications:
•Eprimary
This command has been modified so that dependent nodes will not be
able to act as a Primary or Primary Backup node for the SP Switch in the
partition. The dependent node does not run the RS/6000 SP Switch codes
like standard RS/6000 SP nodes and therefore does not have the ability to
act as the Primary or Primary Backup node.
•Estart
This command functions as it does with normal nodes. It was enhanced to
support the depend node in the RS/6000 SP.
•Efence
This command functions as it does with normal nodes in the RS/6000 SP.
In addition, the dependent node can be fenced from the SP Switch with
autojoin like any other standard RS/6000 SP node.
•Eunfence
This command functions as it does with normal nodes in the RS/6000 SP.
In addition, the dependent node can rejoin the SP Switch network with this
command, if that node was previously removed from the switch network
due to failures or Efence.
Router Node
51
Download from Www.Somanuals.com. All Manuals Search And Download.
2.4.4 Hardware Perspectives
In Perspectives IP Node is used as a convenient and short descriptive term
easily displayed in the GUI. It conveys the role and functions of the
dependent node. Currently, this is the only dependent node.
introduction of the IP Node. The changes are restricted to the Hardware and
System Partition Aid Perspectives.
1
2
3
4
Figure 20. Hardware Perspectives
This figure shows the Hardware Perspectives, which can be started using the
command perspectivesand selecting the Hardware icon. Alternatively, it can
be started directly via the command sphardware.
The Hardware Perspective consists of the following four parts:
1. Menu bar
2. Toolbar
52
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
3. Nodes pane (Frame or Icon View)
4. Information area
The most obvious change is the addition of the IP Node icon as seen in the
Nodes pane. (The figure above shows the Frame View.) The default label for
this icon is IP Node <node number>.
The IP Node icon is also located on the side of the frame, where a standard
node with that node number would be. In this figure the IP Nodes are 7, 14
and 15.
When switch_responds is monitored, it shows the IP Node in two states:
• Green when working with the SP Switch.
• Marked with a red cross when fenced or not operating due to hardware or
configuration problems.
In the figure, IP Node 7 and 15 are working, while IP Node 14 is down.
2.4.4.1 Action Menu
pane, and Actions->Nodes is selected in the menu bar (1). We see that only
the following five actions are available:
Router Node
53
Download from Www.Somanuals.com. All Manuals Search And Download.
1
2
3
4
Figure 21. Action Menu
• View
This will bring up the IP Node’s hardware notebook, shown in the next
figure.
• Fence/Unfence...
This will bring up another window to allow us to either fence or unfence an
IP Node. If we are fencing the IP Node, we can use the option of autojoin.
• Create Node Group...
This will bring up another window to allow us to add the RS/6000 SP
nodes to a Node Group. This action does not affect the IP Node, even
though it is selectable.
• 3 Digit Display
54
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
This will bring up a window to show the three-digit display of all RS/6000
SP standard nodes in the current partition. This action does not apply to
the IP Node, even though it is selectable.
• Open Administrative Session...
This action will open a window that is a Telnet session to the GRF, using
the reliable_hostname attribute specified in the DependentNode class.
In addition, the Nodes pane in this figure shows the Icon View. In this view,
the IP Node icons are always located after all the standard RS/6000 SP node
icons. The results of monitoring the IP Nodes and the icon labels are the
same as those of Frame View, mentioned in the previous figure.
2.4.4.2 Hardware Notebook
Figure 22 shows the IP Node hardware notebook. This notebook can be
triggered by selecting the Notebook icon on the Hardware Perspective
toolbar (2), or selecting Action->Nodes->View in the menu bar (1).
Figure 22. Hardware Notebook
The notebook has three tabs: Configuration, All Dynamic Resource Variables,
and Monitored Conditions. This figure shows the Configuration tab.
Router Node
55
Download from Www.Somanuals.com. All Manuals Search And Download.
These are the attributes listed in the Configuration tab:
• Node number
• Hostname
• Management agent hostname
• SNMP community name
• System partition
• Extension node identifier
• Dependent node IP address
• Dependent node netmask
• Switch port number
• Switch number
• Switch chip
• Switch chip port
• Switch partition number
• Switch responds
The All Dynamic Resource Variables tab only shows the state of the Switch
Responds, and the Monitored Conditions tab only shows the value of the
Switch Responds if it is being monitored.
2.4.4.3 System Partition Aid Perspectives
two panes, the Nodes pane and the System partitions pane. The Nodes pane
(3) in this figure shows the Icon view. Notice that the IP Nodes are displayed
after all the standard RS/6000 SP nodes.
56
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
1
2
3
4
Figure 23. System Partition Aid Perspectives
The IP Nodes can only be assigned to a partition here. This is done either by
using the Assign icon in the toolbar (2), or by selecting
Action->Nodes->Assign Nodes to System Partition on the menu bar (1).
Except for the System Partition Notebook, discussed in the next figure, all
other actions, though selectable, do not apply to the IP Node.
2.4.4.4 System Partition Aid Notebook
Figure 24 on page 58 shows the IP Node System Partition Aid Notebook. This
notebook can be triggered by selecting the Notebook icon on the Hardware
Perspective toolbar (2), or selecting Action->Nodes->View on the menu bar
(1).
The notebook only has the Node Information tab shown in this figure.
Router Node
57
Download from Www.Somanuals.com. All Manuals Search And Download.
Figure 24. System Partition Aid Notebook
These attributes are listed in the Node Information tab:
• Node number
• Switch port number
• Assigned to system partition
2.4.5 SP Extension Node SNMP Manager
The SP Extension Node SNMP manager is contained in the ssp.spmgr file set
of PSSP. This file set must be installed on the Control Workstation in order for
the GRF to function as an extension node.
The SP Extension Node SNMP manager is an SNMP manager administered
by the System Resource Controller. The purpose of the SNMP manager is to
communicate with the SNMP agent on the GRF. The SNMP manager and the
agent adhere to Version 1 of the SNMP protocol. The SNMP manager sends
configuration data for an extension node to the SNMP agent on the GRF. The
SNMP agent applies the configuration data to the SP Switch Router Adapter
represented by the extension node. The SNMP agent also sends
asynchronous notifications in the form of SNMP traps to the SNMP Manager
when the extension node changes state. The following commands are
available to control the SP Extension Node SNMP Manager:
•startsrc
58
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
•stopsrc
•lssrc
•traceson
•tracesoff
2.4.6 Dependent Node MIB
IBM has defined a dependent node SNMP Management Information Base
(MIB) called ibmSPDepNode. This MIB contains definitions of objects
representing configuration attributes of each dependent node and its state.
The GRF Agent maintains the state and configuration data for each
dependent node using the MIB as a conceptual database.
The MIB defines a single table of up to 16 entries representing the adapter
slots in the GRF. When a slot is populated by an SP Switch Router Adapter,
the entry in the table, accessed using the extension node identifier, contains
the configuration attribute and state values for the adapter in the slot. Also
included in the MIB are the definitions of trap messages sent by the GRF
agent to the SP Extension Node SNMP Manager. A copy of the MIB is
contained in the file /usr/lpp/ssp/config/spmgrd/ibmSPDepNode.my on the
Control Workstation.
Other SNMP managers in the network can query this MIB table to validate the
configuration and status of the dependent node and GRF. However, only an
SNMP manager using the correct SNMP community name can change the
values in the MIB table.
Following is a listing of MIB entries:
EntryDefinition
ibmSPDepNodeObject identifier for the dependent node in the MIB
database.
ibmSPDepNodeTableTable of entries for dependent nodes.
ibmSPDepNodeEntryA list of objects comprising a row and a clause in
ibmSPDepNodeTable. The clause indicates
which object is used as an index into the table to
obtain a table entry.
ibmSPDepNodeNameThe extension_node_identifier attribute in the
DependentNode class.
ibmSPDepNodeNumberThe node_number attribute in the DependentNode
class.
Router Node
59
Download from Www.Somanuals.com. All Manuals Search And Download.
ibmSPDepSwTokenA combination of switch_number, switch_chip and
switch_chip_port attributes from the
DependentNode class.
ibmSPDepSwArpThe arp_enabled attribute in the Switch_partition class.
ibmSPDepSwNodeNumberThe switch_node_number attribute in the
DependentNode class.
ibmSPDepIPaddrThe netaddr attribute in the DependentAdapter class.
ibmSPDepNetMaskThe netmask attribute in the DependentAdapter class.
ibmSPDepIPMaxLinkPktThe switch_max_ltu attribute in the Switch_partition
class.
ibmSPDepIPHostOffsetThis attribute stores the difference between the host
portion of a node’s IP address and its
corresponding switch node number. When ARP
is disabled on the SP Switch network, this offset
is subtracted from the host portion of the IP
address to calculate the switch node number.
ibmSPDepConfigStateThe six config states of the dependent node are:
notConfigured, firmwareLoadFailed,
driverLoadFailed, diagnosticFailed,
microcodeLoadFailed, and fullyConfigured, for
use in configuring the adapter.
ibmSPDepSysNameThe syspar_name attribute in the Syspar class.
ibmSPDepNodeStateThe value of nodeUp or nodeDown, to show the status
of the dependent node.
ibmSPDepSwChipLinkThe switch_chip_port attribute in the DependentNode
class.
ibmSPDepNodeDelayThe switch_link_delay attribute in the Switch_partition
class.
ibmSPDepAdminStateThe value of up, down, or reconfigure, indicating the
desired state of the dependent node. If the
dependent node is not in its desired state, the
SNMP agent on the GRF will trigger the
appropriate action to change its state.
2.4.7 Coexistence
Figure 25 shows a single-frame RS/6000 SP in a single partition with a
connection to the GRF. Nodes 1 and 2 are installed with PSSP 2.4. The other
nodes are installed with any other version of PSSP that can coexist with
60
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
PSSP 2.4 to represent coexistence. Also, note that Node 16 is empty,
because the SP Switch port for this node is used by the SP Switch router
adapter in the GRF.
PSSP 2.4 or PSSP2.3 and IX70649 on
CWS
Primary switch node
Backup switch node
PTFs for all other nodes
PSSP
*.*
PSSP
*.*
PSSP 12 PSSP
*.*
PSSP
*.*
15
13
11
16
14
*.*
PSSP
for PSSP2.1 IX71246
9
10
PSSP
*.*
PSSP
*.*
PSSP
*.*
PSSP
*.*
PSSP
2.4
*.*
for PSSP2.2 IX71245
ssp.spmgr file set installed on CWS
Must be an SPS or SPS-8 switch
7
5
8
PSSP
*.*
6 PSSP
*.*
4 PSSP
*.*
2 PSSP
2.4
3
1
IP Switch
Control
Board
SP Switch
SP Switch Router Cable
Crosspoint
Switch
Switch
4-port FDDI
Frame
Ethernet
Cable
CWS
GRF 400
RS232
Cable
PSSP 2.4
Figure 25. Coexistence
The dependent node is only supported in PSSP 2.3 and higher PSSP
versions. To use it with nodes with PSSP versions less than 2.3 requires the
use of coexistence. The following conditions are required for the dependent
node to communicate with nodes with a lower version than 2.3 using
coexistence:
• The Control Workstation must be at PSSP 2.3 or higher to manage
dependent nodes.
• The Primary node of the SP Switch must be at PSSP 2.3 or higher, as the
Primary node needs to perform some tasks for the dependent node and
these functions are only available in PSSP 2.3 and higher PSSP versions.
• The Primary Backup node of the SP Switch should be PSSP 2.3 or higher
so that if the Primary node fails, the dependent node can continue to
function in the RS/6000 SP when the Backup node takes over.
Router Node
61
Download from Www.Somanuals.com. All Manuals Search And Download.
• All RS/6000 SP nodes with a version less than PSSP 2.3 in the partition
need to maintain the right level of fixes (PTFs) in order for coexistence
with PSSP 2.4 to take place.
• The ssp.spmgr file set must be installed on the Control Workstation.
• Because the SP Switch router adapter will only work with the 8-port or
16-port SP Switch, make sure that the switch used in the RS/6000 SP is
not a High Performance Switch (HiPS).
• There must be at least one free SP Switch port to install the SP Switch
router adapter.
Important
When the Primary switch node fails, the Primary Backup Switch node take
over as the new Primary switch node. The new Primary Backup switch
node, selected from the current partition, can be a node with a PSSP level
below 2.3, even though another node with a PSSP level of 2.3 or higher
may exist in that partition. The only way to ensure that the new Backup
switch node is at PSSP 2.3 or higher is to manually check the RS/6000 SP
system. If the PSSP Version of the Primary Backup switch node is below
Version 2.3 you have to chose a node with PSSP 2.3 or higher as the
Primary Backup switch node.
If a node running a version of PSSP earlier then 2.3 is selected as the new
primary node, the SP Switch Router Adapter will be fenced from the switch.
2.4.8 Partitioning
Figure 26 on page 63 shows a single-frame RS/6000 SP broken into two
partitions, Partition A and Partition B. Each partition has seven standard
RS/6000 SP nodes and one dependent node. Only seven nodes are allowed
in each partition, as a single-frame RS/6000 SP has only 16 SP Switch ports,
and two of them are used for the SP Switch router adapter, one for each
partition.
62
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Cross-partition communication
through the SP Switch
IP Switch
Control
Board
SP Switch
SP Switch
4-port FDDI
Crosspoint
Switch
Partition B
Partition A
GRF 1600/400
Partition A
Partition B
SDR
Switch
Frame
Ethernet
Cable
CWS
RS232
Cable
PSSP 2.4
Figure 26. Partitioning
Normally, RS/6000 SP nodes in different partitions cannot communicate with
each other through the SP Switch. The GRF plays a unique role here by
allowing RS/6000 SP nodes to communicate across partitions, when each
partition contains at least one SP Switch router adapter, and these adapters
are interconnected by TCP/IP.
The requirements for partitioning are the same as those for coexistence, with
the addition of having at least one free SP Switch port per partition, to
connect to the SP Switch router adapter.
2.5 Planning for the GRF
Before acquiring any model of the SP Switch Router, ensure that there are
SP Switch ports available in the designated partition, and that the switch used
in the RS/6000 SP is the 8-port or 16-port SP Switch.
Router Node
63
Download from Www.Somanuals.com. All Manuals Search And Download.
Next, ensure that the following parameters are defined:
ParametersDescriptions
GRF IP addressThe IP address for the GRF administrative Ethernet.
GRF netmaskThe Netmask for the GRF administrative Ethernet.
GRF Default routeThe default route of the GRF.
SNMP community nameThis attribute describes the SNMP community
name that the SP Extension Node SNMP Manager
and the GRF’s SNMP Agent will send in the
corresponding field of the SNMP messages. This
value must match the value specified for the same
attribute of the corresponding dependent node
definition on the SP system. If left blank, a default
name found in the SP Switch Router Adapter
documentation is used.
CWS IP addressThe Control Workstation’s IP address. When a GRF
contains multiple SP Switch router adapters that
are managed by different SNMP managers on
different RS/6000 SP CWS, each of the Control
Workstation IP addresses should be defined along
with a different community name for each Control
Workstation.
DNSThe DNS server and domain name, if used.
SP Extension Node SNMP
Manager port #The SNMP port number used by the SP Extension Node
SNMP Manager to communicate with the SNMP
agent on the GRF.
This port number is 162 when the SP Extension
Node SNMP Manager is the only SNMP manager
on the Control Workstation. Otherwise, another
port number not used in the /etc/services of the
Control Workstation is chosen.
64
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.6 Planning for the Dependent Node
Next, for each dependent node on the RS/6000 SP, define the following:
ParametersDescriptions
Node #A user-supplied dependent node number representing the node
position of an unused SP Switch port to be used
by the SP Switch Router Adapter.
Slot #The slot number on which the SP Switch Router Adapter is located in
the GRF.
GRF hostnameThe hostname for the GRF administrative Ethernet. A long
hostname is recommended if the domain name
service (DNS) is used in the network. This
represents both the Administrative and SNMP
agent hostname of the dependent node.
SNMP community nameThis attribute describes the SNMP community
name that the SP Extension Node SNMP Manager
and the GRF’s SNMP Agent will send in the
corresponding field of the SNMP messages. This
value must match the value specified in the
/etc/snmpd.conf file on the GRF. If left blank, a
default name found in the SP Switch Router
Adapter documentation is used.
SP Extension Node SNMP
Manager port #The SNMP port number used by the SP Extension Node
SNMP Manager to communicate with the SNMP
agent on the GRF.
This port number is 162 when the SP Extension
Node SNMP Manager is the only SNMP manager
on the Control Workstation. Otherwise, another
port number not used in the /etc/services of the
Control Workstation is chosen.
Then, for the dependent node adapter, define these parameters:
ParameterDescriptions
IP addressThe IP address of this adapter.
NetmaskThe netmask of this adapter. Use the same format as that for
standard RS/6000 SP nodes.
Router Node
65
Download from Www.Somanuals.com. All Manuals Search And Download.
2.7 Conclusion
The SP Switch Router 9077-04S has an aggregate bandwidth of 800 MB/s.
An SP wide node by contrast is capable of no more than about 65 MB/s of
sustained throughput. A wide node’s CPU hits a wall at about 5000
packets/second, whereas the 9077 is capable of an aggregate of 2.8 million
packets/second. All this is achieved in part because of the non-blocking
crosspoint switch with four 100 MB/s, full duplex connection points. This
enables multiple paths to operate at full speed simultaneously.
Unlike the SP nodes, the SP Switch Router is designed with high availability
in mind. It provides balanced, fully redundant power supplies that can be hot
swapped in case of failure. It provides the ability for redundant paths to an SP
Switch to be configured on a single 9077; with dynamic routing protocols, a
second 9077 can be used to provide a backup path in case of system failure
of the primary router. In either case, each media card is hot swappable and
autoconfigured after the initial install has been completed.
With its high port count on interfaces such as FDDI and Ethernet and its
highly scalable performance, the SP Switch Router provides a very cost
effective solution. With each media card you get a nearly linear scaling of
performance with very little cost increase. An SP node by comparison runs
out of CPU cycles and/or slots very quickly requiring the purchase of another
entire node.
Since the 9077 (or rather the Ascend GRF) was originally designed for ISP’s,
it has a full set of protocols, including dynamic routing protocols such as
OSPF, BGP4 and RIPv2. It also has the memory required to hold up to
150,000 routes and the speed to access a table of this size without
performance degradation. Support for media types not supported by the SP
nodes also enables the SP to now be connected into networks that will be
important for its future. These include support for HSSI and Sonet, which are
important for the SP’s ever-growing role as a Web server or online
transaction manager.
66
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Part 2. Scenarios
This part presents some sample configurations of an RS/6000 SP system
with an SP Switch Router. It is beyond the scope of this book to represent all
possible applications of an SP Switch Router. Nevertheless, the basic
configurations shown are building blocks for more complex networking
topologies that include the SP Switch Router and may inspire more complex
configurations.
All following sample scenarios were carefully chosen to match frequently
occurring customer situations. They can be easily configured and modified to
apply to customers’ needs. All configurations described were tested in our
laboratory with the available hardware and software. We used two RS/6000
SPs, each consisting of a control workstation (CWS) and one frame with an
installed with AIX 4.3.1 and PSSP 2.4. Additionally, a GRF1600 and a
GRF400 running Ascend Embedded/OS V1.4.6.4. were used. Only static
routing was applied in our tests. For using and configuring gated on an SP
Switch Router refer to GRF Reference Guide 1.4, GA22-7367. For detailed
configuration information, refer to Appendix A, “Laboratory Hardware and
Software Configuration” on page 233, and the scenario sections.
SP 21
SP 2
GRF 400
GRF 1600
CWS 2
CWS 21
Figure 27. The Laboratory Hardware Installation
But first let us get physical and see how the SP Switch Router media cards
are configured.
© Copyright IBM Corp. 1998
67
Download from Www.Somanuals.com. All Manuals Search And Download.
68
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 3. Installation and Configuration
The SP Switch Router functions as an IP router to provide high-speed data
communication links between SP processor nodes and external networks or
hosts. The SP Switch Router Adapter media card connects to the SP Switch
SP Control Workstation
Administrative network =
Ethernet hub or bridge
Control
board
SP Switch Router
Primary node
for SP Switch
Processor
node
Processor
node
Switch
SP Switch
Router Adapter
media card
SP Switch
to/from other networks and hosts
Figure 28. Connecting the GRF to the SP Switch and the CWS
The SP Switch Router Adapter card also transmits data to/from other types of
media cards across the SP Switch Router’s internal switch core. These media
cards include HIPPI, HSSI, FDDI, ATM OC-3c, ATM OC-12c, 100Base-T
(Fast Ethernet), and other SP Switch Router Adapter cards. The SP system
manages the SP Switch Router Adapter card as a dependent node, under the
control of the SP SNMP Manager running on the SP Control Workstation and
the Primary node of the SP Switch. To learn more about dependent nodes,
up, the SP Switch Router can be configured and managed remotely, either via
a site’s administrative network, or using Telnet from the CWS.
Information about procedures performed from the SP CWS are found in the
"Managing Extension Nodes" chapter in RS/6000 SP: Administration Guide
Version 2 Release 4, GC 23-3897.
© Copyright IBM Corp. 1998
69
Download from Www.Somanuals.com. All Manuals Search And Download.
The intent of this chapter is to provide, or refer you to, the necessary
information to enable you to attach an SP Switch Router to an IBM SP
system. Coverage is provided as follows:
• Information to configure the SP Switch Router Adapter card as required
for SP Switch Router functionality is complete in this chapter.
• Information to physically connect the two independent systems across
cables is complete in this chapter.
• Information to start up, configure, and begin operations on the SP Switch
Router is contained in GRF 400/1600 Getting Started 1.4, GA22-7368.
• Information to configure the SP Switch Router Adapter card as required
for SP system functionality is only partially described in this chapter.
Detailed information is contained in the "Managing Extension Nodes"
chapter in RS/6000 SP: Administration Guide Version 2 Release 4, GC23-
3897.
3.1 Initial Configuration
When a new, unconfigured GRF is powered on an initial configuration script
runs automatically. You will be asked a series of questions about the
configuration, as shown in the following screen.
Welcome to Ascend Embedded/OS system configuration..
Host name for this machine ?
Do you wish to configure the maintenance Ethernet interface ?
Which interface type ? (TP BNC AUI)
IP address of this machine ?
Netmask for this network ?
IP address of router (‘none’ for no default route)?
Do you wish to go through the questions again ?
Which region is this machine located in ?
Which time zone ?
Next you are prompted to change the local password for root.Changing the
root password is the end of the configuration script. If you need to change
these parameters later, run the config_netstat script again. More details about
the initial configuration are in GRF Configuration Guide 1.4, GA22-7366.
If you log onto the GRF, a super>prompt appears. This indicates that you are
in the command-line interface (CLI). This CLI is different compared to other
Installation and Configuration
70
Download from Www.Somanuals.com. All Manuals Search And Download.
UNIX systems. On most of the UNIX systems you are working on the shell
layer after you logged onto the system.
Many system management and configuration commands are now available.
Enter a question mark (?) to retrieve a list of CLI commands. To edit
configuration files, you must be in the UNIX shell. The shcommand opens the
UNIX shell you use to modify configurations. The following screen shows how
to do this.
August 22 14:18:31 grf16 kernel: ge027: GRF Ethernet, GRIT address 0:2:7
super>
super>ls
super>no profile was specified
super>sh
Copyright 1992, 1993, 1994, 1995, 1996 Berkeley Software Design, Inc.
Copyright (c) 1980, 1983, 1986, 1988, 1990, 1991, 1993, 1994
The Regents of the University of California. All rights resevered.
Ascend Embedded/OS GR TA1.4.6 Kernel #1 (nit): Fri Jan 30 13:08:03 CST 1998
Ascend Embedded/OS 1.4.6
Copyright 1992,1993,1994,1995,1996,1997,1998 Ascend Communications, Inc.
IMPORTANT: By use this software you become subject to the terms and
conditions of the license agreement on file /etc/license and any other
License agreements previously provided to you bt Ascend Communications.
#ls
.chsrc .klogin .login .profile
#
We found it more convenient to work directly at the shell layer. Therefore, we
modified the .profile and commented the appropriate part that starts the CLI.
For more details, see Appendix B.1, “/root/.profile” on page 261.
3.2 Pre-Installation Assumptions
We assume the following:
• The RS/6000 SP Switch Router is powered on and has a VT-100 or
administrative Ethernet network connected to its control board.
connections and proper setup.
• The SP Switch Router’s basic system parameters, such as the IP address
and host name, were configured during the first power-on configuration
script. You use the terminal or network to log in to the SP Switch Router
Installation and Configuration
71
Download from Www.Somanuals.com. All Manuals Search And Download.
system and enter these basic configuration parameters. Procedures for
starting and setting up the SP Switch Router are found in GRF 400/1600
Getting Started V.14, GA22-7368.
Ignore the prompts for network logging, since we will configure logging to
a PCMCIA device; just press Enter when asked to enter the remote
logging host name or its IP address.
• Remote Telnet access is working. To enable it, you have to edit the /etc/
ttys file on the SP Switch Router and modify the appropriate entries, as
follows:
#name getty
ttyp0 none
ttyp1 none
ttyp2 none
ttyp3 none
type
status
comments
network secure
network secure
network
network
Adding secureto a ttypX stanza opens it for Telnet, so in this example, only
two Telnet sessions at a time were allowed.
Use the following command to connect to the SP Switch Router:
xterm -sb -geometry 80x65 -e telnet <hostname of SP Switch Router>
This gives you a large window with a scroll bar at the left, so that you may
retrieve information easier. See Appendix B.1, “/root/.profile” on page 261
for a modified .profile for the root user.
Hint: If you just use -e telnetand give the hostname at the telnet>
prompt, the window will survive reboots of the GRF.
Installation and Configuration
72
Download from Www.Somanuals.com. All Manuals Search And Download.
Standard Switch Cable of 10m
Other Switch Cables
5 m
(f/c 9305)
10 m (f/c 9310)
15 m (f/c 9315)
20 m (f/c 9320)
CWS
Ethernet
Cable
RS232
Cable
PSSP 2.4
IP Switch
Control
Board
10BaseT
Crosspoint
Switch
SP Switch
SP Switch Cable
Grounding Cable
Figure 29. Connecting the GRF to the Frame
• You are ready to configure media cards. Procedures to configure media
cards are in this redbook; complete information is in the GRF
Configuration Guide 1.4, GA22-7367.
Installation and Configuration
73
Download from Www.Somanuals.com. All Manuals Search And Download.
Admin Ethernet
(de0)
IP Switch
cws
Control
Board
Terminal Settings
Crosspoint
Switch
SP Switch
9600 baud
RS232
(Null
Modem
Cable)
GRF 400
No parity
Eight data bits
One stop bit
VT100
terminal
GRF Console (optional)
Figure 30. Connecting the GRF Console
• The IBM SP system is up and operating.
• The SP system administrator has given you one of these pieces of
information:
• The node number assigned to each SP Switch Router Adapter card to
be attached to an SP Switch port
• The port location on each SP Switch reserved for specific SP Switch
Router Adapter cards
3.2.1 Order of Information
Here is the sequence of steps you have to complete:
1. An installation overview of tasks involving the SP Switch Router, the SP
Switch Router Adapter card, and the SP system
2. The configuration procedure for the PCMCIA 520 MB disk, which also
initiates system logging
3. A description of which cables to attach between the SP Switch Router and
the SP control workstation, and between the SP Switch Router Adapter
card and the SP Switch
Installation and Configuration
74
Download from Www.Somanuals.com. All Manuals Search And Download.
4. Methods to determine node number and SP Switch port for an SP Switch
Router Adapter card
5. A step-by-step configuration of an SP Switch Router Adapter card
6. A list of ways to verify that the SP Switch Router Adapter card is correctly
installed in the SP Switch Router
7. A description of what needs to occur to bring the card online to the SP
system
3.3 Installing an SP Switch Router Adapter Card
This section contains the procedure for physical installation and minimal
configuration of the SP Switch Router Adapter card for use as an SP
dependent node. This includes cabling the GRF to the SP CWS and the
appropriate SP Switch port.
Note: There needs to be a network path between Ethernet twisted-pair
interface on the SP Switch Router control board and the SP control
workstation. This is most easily done through an Ethernet hub (or bridge to
the 10BaseT SP LAN). However, it can also be done through a connection to
a network external to the SP.
3.3.1 Installation Overview
IBM support personnel who install the SP Switch Router (9077) perform the
physical installation and minimal configuration described below with help from
the customer’s system administrator. The system administrator must
complete the following basic configuration steps:
1. Locate all the components of the SP Switch Router chip group.
2. Perform the complete physical installation of the SP Switch Router unit as
described in the "Power On and Initial Configuration" chapter of GRF 400/
1600 Getting Started 1.4, GA22-7368. Make sure that when the "First-time
power on configuration script" runs at system boot, the required
configuration information is provided by or entered by the customer. This
information includes the SP Switch Router unit IP address and hostname.
As stated before, ignore script references to network or syslog logging.
3. Perform the procedure to configure the PCMCIA disk. The procedure is
Installation and Configuration
75
Download from Www.Somanuals.com. All Manuals Search And Download.
4. Route the Ethernet twisted-pair cable between the SP Switch Router unit
and the Ethernet hub, then connect the cable to the SP Switch Router
control board and the Ethernet hub.
5. Verify that the SP CWS has a connection to this same Ethernet hub. If the
SP CWS Ethernet adapter is configured by the system administrator, then
a pingtest from the SP CWS to the configured SP Switch Router Ethernet
address should be done to test Ethernet connectivity.
Physical installation and minimal configuration are complete at this point.
connecting the SP Switch Router Adapter card cables to the SP Switch ports
specified for this configuration.
3.3.2 Installing the PCMCIA Spinning Disk
Your system is shipped with a PCMCIA disk that is required to collect the
system log files. This disk can hold up to 520 MB of data regarding your
model of the SP Switch Router.
You can install the disk any time after the SP Switch Router is powered on
and running. Logging is not enabled until you install the disk and complete
this configuration procedure. Logged messages might be very helpful while
you are configuring media cards. The configuration is done only once to set
up local logs and dumps, and is not affected by software updates or system
reboots. System logs include: gritd.packets, grinchd.log, gr.console,
gr.conferrs, gr.boot and mib2d.log.
The procedure formats and initializes an external flash (/dev/wdXa), where
the X is normally a 1, denoting the number of the device. You get the actual
number from the mountfcommand. We have /dev/wd3a in our example and
this value will be used for the rest of this chapter. The procedure then mounts
the flash temporarily on /mnt and creates subdirectories, symbolic links and a
permanent site file for storing the symbolic links.
Proceed as follows:
1. Insert the PCMCIA disk into slot A on the SP Switch Router control board
(the width of the disk requires it to be installed in slot A).
2. Log in as root to the SP Switch Router, start the UNIX shell, and execute
the following commands from the shell:
Installation and Configuration
76
Download from Www.Somanuals.com. All Manuals Search And Download.
prompt> sh
#
# cd /
# iflash -A
May 29 15:54:18 grf16 kernel: wd2: no disk label
# mountf -A -w -m /mnt
Device /dev/wd3a mounted on /mnt
# mkdir /mnt/crash
# mkdir /mnt/portcards
# cd /var
# mv crash crash.orig
# mv portcards portcards.orig
# ln -s /var/log/portcards /var/portcards
# ln -s /var/log/crash /var/crash
# grsite --perm portcards crash
Device /dev/wd0a mounted on /flash.
Device /dev/wd0a unmounted.
#
# cd /var/log
# pax -rw -pe -v . /mnt
/mnt/.
/mnt/./cron
/mnt/./daemon.log
/mnt/./lastlog
/mnt/./maillog
/mnt/./messages
/mnt/./secure
/mnt/./wtmp
/mnt/./grclean.log
/mnt/./mibmgrd.log
/mnt/./cli.log
# umountf -A
Device /dev/wd3a unmounted.
#
3. Edit the file /etc/rc.boot and see if the line mount /dev/wd3a /var/log is
present; if not, add this line at the end of the file.
4. Edit the file /etc/fstab and add this line as shown in the following excerpt:
/dev/wd3a /var/log ufs rw 0 2 # PCMCIA slot A
## Each line is of the form:
## device mount_point type flags dump fsck_pass
/dev/rd0a/ ufs rw 0 0
/dev/wd3a /var/log ufs rw 0 2 # PCMCIA slot A
Installation and Configuration
77
Download from Www.Somanuals.com. All Manuals Search And Download.
5. Edit the file /etc/syslog.conf to specify the location where the logs will be
kept. Uncomment the local log configuration lines in the “Log messages to
Disk” section by removing #disk# from each line, and specify /var/log as
the directory for each log. The entries should now look like the following:
*.err;*.notice;kern.debug;lpr,auth.info;mail.crit /var/log/messages
cron.info /var/log/cron
local0.info /var/log/gritd.packets
local1.info /var/log/gr.console
local2.* /var/log/gr.boot
local3.* /var/log/grinchd.log
local4.* /var/log/gr.conferrs
local5.* /var/log/mib2d.log
6. Check that /etc/grclean.conf and /etc/grclean.logs.conf have entries
pointing to files in /var/log.
The /etc/grclean.conf file entries should look like the following:
###################################################################
# port card dump files.
###################################################################
hold=4
size=1
remove=y
local=y
logfile=/var/portcards/grdump.*
###################################################################
# cleanup our own log file, if necessary.
###################################################################
DEFAULTS
hold=2
local=y
size=10000
logfile=/var/log/grclean.log
The /etc/grclean.logs.conf file entries should look like the following:
Installation and Configuration
78
Download from Www.Somanuals.com. All Manuals Search And Download.
***********************************************************************
* Log files that used to be archived by the /etc/{daily|weekly|monthly}
* scripts.
***********************************************************************
size=150000
logfile=/var/log/gr.console
size=11000
logfile=/var/log/gr.boot
7. Save all changes and reboot:
# grwrite -v
# reboot
8. After the SP Switch Router is up and running again, use csconfig -ato
verify that the PCMCIA interface is available and the PCMCIA disk are up.
For a quick test, run grconslog -vf. If the setup is correct, grconslog will
not complain about a missing /var/log/gr.console file, but instead will show
all entries and stay up running, giving updates of new entries to the file
onto the screen. To stop grconslog, use Ctrl+C.
3.4 Attaching SP Switch Router Cables
Three types of cables must be attached:
• The administrative Ethernet LAN cable
• The SP Switch Router Adapter card to SP Switch cable(s)
• The ground strap to the SP frame
3.4.1 Ethernet Cable
Route the Ethernet twisted-pair cable between the SP Switch Router unit and
the Ethernet hub, then connect the cable to the SP Switch Router control
be accomplished.
Installation and Configuration
79
Download from Www.Somanuals.com. All Manuals Search And Download.
SP Control Workstation
Hub
SP Switch Router
Administrative Ethernet network
Control board
Figure 31. SP System Administrative Ethernet Connections
3.4.2 SP Switch Cable
The SP Switch Router Adapter card provides one full-duplex attachment and
requires a specific cable with 50-pin connector ends, obtainable from IBM.
The cable has a unique signal wiring map, and is not replaceable by a 50-pin
HSSI cable, for example. SP Switch Router Adapter card cables are available
in 10- and 20-meter lengths (32 or 65 feet). Excess cable lengths should be
bound in a figure-eight pattern. Do not wind excess cable into circular coils.
Each connector end has 50 fragile pins. Pins can become bent when making
the connection to the media card if alignment is wrong. If an SP Switch
Router Adapter card link does not work after cabling, check both ends of the
cable for bent pins. When not connected, keep the plastic caps on the ends.
3.4.3 Procedure for Connecting Cards to the SP Switch
This procedure connects the SP Switch Router Adapter card(s) to the SP
Switch. Before the SP Switch Router unit can begin full operation, all other
router media cards must be configured with appropriate customer
configuration information.
Make sure you have labeled the SP Switch cable to show which media card
and SP Switch port it will be connected to. Keep in mind that for any work
done on the SP Switch you should have shut down and powered off the SP
System and also turned off the central power supply switch at the left front
edge of the SP.
Execute the following steps to make the connections:
1. If there are any terminators on the media card or the switch assembly
where you need to attach the switch cable, remove them now.
Installation and Configuration
80
Download from Www.Somanuals.com. All Manuals Search And Download.
2. Using appropriate frame entry and exit holes for cable management, route
the SP Switch cable between the SP Switch Router unit and the SP
Switch.
3. Connect the SP Switch cable to both the media card and the correct SP
Switch port, as follows:
• Connection to media card
The EMI shielding fitted inside the connector end can make insertion
difficult, so be sure to insert the connector end as perpendicular as
possible. (Pins can be damaged when the connector is inserted at too
much of an angle.) Seat the connector firmly so the spring clips engage.
• Connection to SP Switch port
The cable ends should click onto the connectors. Determining the correct
Switch port is described in Section 3.5.1, “Determining the Switch
4. Make sure both ends of the cable are firmly seated by pulling on them
lightly.
At this point, the SP Switch Router Adapter card configuration information
must be entered on the SP CWS to enable the PSSP code and SP Switch to
recognize the adapter. These tasks are discussed in Section 3.5,
3.5 Configuration Required on the SP System
This section describes the SP Switch Router-related configuration
information that should be defined by the SP administrator and then entered
from the SP CWS before the SP Switch Router Adapter card is configured.
The SP Switch Router-related configuration information includes the
following:
• The SP Switch Router Ethernet IP address
• The SP Switch Router Ethernet hostname (that is, the SP Switch Router’s
administrative Ethernet hostname)
• Unique node numbers for SP Switch Router Adapter cards
The SP Switch Router Adapter card configuration information enables the
PSSP code and the SP Switch to recognize and communicate with this card.
Installation and Configuration
81
Download from Www.Somanuals.com. All Manuals Search And Download.
3.5.1 Determining the Switch Connection for a Dependent Node
The SP Switch Router Adapter connection replaces an SP node connection
to the SP Switch. Each SP Switch Router Adapter media card is referred to
as a dependent node, and is assigned a node number that corresponds to its
specific connection on the SP Switch.
The node number is determined by the SP system administrator based on an
understanding of how node numbers are assigned in the SP System and the
rules for choosing a valid, unused SP Switch port.
are assigned and how to look for the best suited port number for the actual
scenario.
Note: Look out for the change in numbering regarding Frame n+2 and
Frame n+3!
of whether the frame is fully populated.
If proper planning has been done to assign the node number, the system
administrator knows which SP frame, Switch board, and node slot
corresponds to a dependent node. Given this information, you can determine
which jack on the Switch board should be used by consulting the Switch
Cable Charts for the SP Switch in RS/6000 SP: Maintenance Information,
Volume 1, Installation and Customer Engineer Operations, GC23-3903.
Do not attempt to connect an SP Switch Router Adapter to the SP Switch until
proper planning has been done to assign the node number.
Once the node number is assigned, the SP system administrator can define
the corresponding dependent node using SMIT as described in the
"Managing Extension Nodes" section of RS/6000 SP: Administration Guide
Version 2 Release 4, GC23-3897.
After defining new dependent nodes on the SP, the administrator should use
the Eannotatorcommand to annotate the SP Switch topology file. With the file
annotated, even if the administrator is not sure of the frame, switch board, or
node slot for the dependent node, you can determine the corresponding
switch connection with the procedure described in Section 3.5.2, “Procedure
Installation and Configuration
82
Download from Www.Somanuals.com. All Manuals Search And Download.
15
13
11
14
12
10
8
9
7
5
3
6
4
2
0
1
Switch
Frame n
14
12
10
15
13
11
8
6
4
2
9
7
5
3
0
1
No Switch
Switch
Frame n+1
Frame n
14
12
10
8
6
4
2
13
9
14
10
6
5
1
2
0
Switch
No Switch
Frame n+1
No Switch
Frame n+2
Frame n
12
8
13
9
15
11
7
14
10
6
4
5
0
1
3
2
Switch
No Switch
No Switch
Frame n+2
No Switch
Frame n+3
Frame n
Frame n+1
Figure 32. Switch Port Assignments in Supported Frame Configurations
Installation and Configuration
83
Download from Www.Somanuals.com. All Manuals Search And Download.
31
29
27
25
23
21
19
17
47
45
43
41
39
37
35
33
15 16
61
57
53
13
11
9
14
12
10
8
7
5
3
1
6
4
2
49
Switch 2
Switch 1
Switch 3
Frame 1
Frame 2
Frame 3
Frame 4
Figure 33. Node Numbering for an SP System
3.5.2 Procedure to Get the Jack Number
Following are the steps required to get the jack number:
1. From the SP control workstation, determine the dependent node’s number
by entering SDRGetObjects DependentNode node_number. This produces
output that looks like the following:
# SDRGetObjects DependentNode node_number
node_number
4
#
2. From the SP CWS, determine the switch_node_number by entering
SDRGetObjects DependentNode node_number==n switch_node_number,where n
is the node_number of the dependent node you just got from the previous
command. This gives the following output:
# SDRGetObjects DependentNode node_number==4 switch_node_number
switch_node_number
3
#
3. From the SP CWS, determine the host name of the SP Switch primary
node by entering splstdata -s | grep -p primary. The output looks like
this:
Installation and Configuration
84
Download from Www.Somanuals.com. All Manuals Search And Download.
# splstdata -s | grep -p primary
switch_part
number
topology
filename
primary
name
arp switch_node
enabled nos._used
-------------------------------------------------------------------------
1 expected.top.an sp21n07 yes no
#
In this case, the primary node host name is sp21n01.
4. Log into the primary node by entering telnet node_hostname,where
node_hostname is the host name of the primary node.
5. From the primary node, enter pg /var/adm/SPlogs/css/out.top.
6. In the out.top file, look for the line(s) containing Dependent Node. In this
line you will find the term tb3, which is immediately followed by the value
for the switch_node_number. For switch_node_number 3, the following
line identifies the SP Switch jack E01-S17-BH-J25 (frame 1, Switch,
bulkhead, jack J25):
s 16 1 tb3 3 0 E01-S17-BH-J25 to E01-N4 # Dependent Node
If you need help interpreting this identifying string, see PSSP Maintenance
Information, Volume 2, Maintenance Analysis Procedures and Parts
Catalog for an explanation of the naming standard for RS/6000 SP
components.
3.6 Multiple Frames for Multiple System Connections
SP Switch Router Adapter cards in an SP Switch Router can connect to
different switch boards in the same SP system. A configuration problem could
arise in which the SP Switch Router Adapter cards are assigned the same
node number if each card plugs into the same port position on each switch
board.
The use of frames removes the configuration problem. The following example
demonstrates the organization of three SP frames (1, 2, and 3) with switch
boards in each.
Figure 34 on page 86 shows how the frame numbering differentiates each SP
Switch Router connection:
• The SP Switch Router card connected to port J31 of SP Switch A1 is node
number 9.
Installation and Configuration
85
Download from Www.Somanuals.com. All Manuals Search And Download.
• The SP Switch Router card connected to port J31 of SP Switch A2 is node
number 25.
• The SP Switch Router card connected to port J31 of SP Switch A3 is node
number 41.
• The SP Switch Router card connected to port J15 of SP Switch A1 is node
number 16.
SP system A
Frame 1
Frame 2
Frame 3
J7
J23
J15
J23
J15
J23
J15
J7
J7
SP Switch A1
SP Switch A2
SP Switch A3
J31
J31
J31
gt010
Node 16
gt000
Node 9
gt020
Node 25
gt030
Node 41
SP Switch Router
Figure 34. How Frames Enable Connections to Multiple SP Switches
3.7 Step-by-Step Media Card Configuration
This section provides a configuration overview and the steps required to
configure an SP Switch Router Adapter media card.
3.7.1 Configuration Files and Their Uses
These are the configuration files found in /etc on the SP Switch Router,
discussed in this chapter:
• grifconfig.conf - identifies each logical interface on a media card
• snmpd.conf - enables SNMP capabilities
• grdev1.conf - configures SP Switch Router Adapter cards
Refer to GRF Reference Guide 1.4, GA22-7367 for templates of all
configuration files.
Installation and Configuration
86
Download from Www.Somanuals.com. All Manuals Search And Download.
3.7.1.1 Overview of the Steps to Configure a Media Card
A detailed discussion of these steps follows this overview.
1. Edit the SNMP configuration file and start the SNMP daemon on the SP
Switch Router.
2. Assign an IP address and other parameters to the SP Switch Router
Adapter interface.
There are two ways to configure these parameters:
• We recommend using the procedures documented in the "Managing
Extension Nodes" chapter of the RS/6000 SP: Administration Guide
Version 2 Release 4, GC23-3897.
• As an alternative, you can log in to the SP Switch Router and use a
UNIX editor to edit the /etc/grifconfig.conf file. These assignments are
entered in the SP Switch Router’s /etc/grifconfig.conf file:
• Interface name (gt0y0)
• IP address
• Netmask
• Broadcast address
• Argument (MTU size)
3. Change the default boot diagnostics and dump settings in profiles
(optional).
To change the defaults for one card, change the settings in the appropriate
Card profile; to change the defaults for all installed SP Switch Router
Adapter cards, change the settings in the Dump and Load profiles.
4. Run the dev1configcommand while logged into the SP Switch Router.
View the /etc/grdev1.conf file to verify configuration data.
For the SP Switch Router Adapter card to operate, the SP Switch Router
requires a specific configuration file, /etc/grdev1.conf. The dev1config
command creates this skeleton file using configuration information passed
to the router in either of two ways:
• We recommend using the procedures documented in the “Managing
Extension Nodes” chapter of the RS/6000 SP: Administration Guide
Version 2 Release 4, GC23-3897 to install the parameters. (This is the
same configuration information as described in the recommended
method of step 2.) These parameters will only be available to
dev1configafter the SP Switch Router Adapter card is activated on the
SP Switch.
Installation and Configuration
87
Download from Www.Somanuals.com. All Manuals Search And Download.
• As an alternative, you can log on to the SP Switch Router and use a
UNIX editor to enter the parameters in the /etc/grdev1.conf file.
5. Reboot the SP Switch Router unit so that the altered configuration files are
installed and used. Remember that you must use grwrite -vto preserve
the modifications of files in /etc, before a reboot.
Details about each step are provided in the next sections.
3.8 Step 1. Check SNMP in the SP Switch Router System
Check the /etc/snmpd.conf file to see if a management station and community
are defined, and if traps are enabled. Network monitoring devices
(management stations) can request or access the SP Switch Router’s SNMP
information.
Follow the procedure described in Chapter 2, "How to configure SNMP" of the
GRF Configuration Guide 1.4, GA22-7366. Note that you must not remove
the ALLOW and COMMUNITY public statements that are already in /etc/
snmpd.conf.
Here are excerpts from our /etc/snmpd.conf file appropriate for the SP
Switch Router connected to an SP system.
This configuration assumes that one SP CWS manages the SP Switch Router
Adapter card. In the example, the IP address of the CWS is 192.168.4.137.
You will see it defined in the example as MANAGER.
# Default Agent Configuration File
ALLOW SUBAGENT 1.3.6.1.4.1.1080.1.1.1
WITH OTHER PASSWORD
USE 15 SECOND TIMEOUT
COMMUNITY public
ALLOW GET, TRAP OPERATIONS
USE NO ENCRYPTION
MANAGER 192.168.4.137
SEND ALL TRAPS
TO PORT 162
WITH COMMUNITY spenmgmt
COMMUNITY spenmgmt
ALLOW ALL OPERATIONS
USE NO ENCRYPTION
Installation and Configuration
88
Download from Www.Somanuals.com. All Manuals Search And Download.
3.9 Put SNMP Changes into Effect
To have changes to /etc/snmpd.conf take effect, kill snmpd. It will be
automatically restarted.
Log in as root, find the snmpd PID (process ID), and then kill the SNMP
daemon, as follows:
# ps -ax | grep snmpd
326 00- S
# kill 326
0:00.17 snmpd /etc/snmpd.conf /var/run/snmpd.NOV
# Jun 13 16:13:18 grf16 mib2d[397]: mib2d: terminated by master agent
Jun 13 16:13:18 grf16 root: grstart: snmpd exited status 143; restarting.
Jun 13 16:13:18 grf16 root: grstart: mib2d exited status 0; restarting.
#
3.10 Step 2. Assign IP Addresses
Assign an IP address and other parameters to the SP Switch Router Adapter
interface. As already mentioned, there are two ways to accomplish this task
(one recommended, one optional).
3.10.1 Method 1: Use SP SNMP Manager - Recommended
This is the method recommended for configuring the SP Switch Router
Adapter card. From a system point of view, it is appropriate to treat the SP
Switch Router Adapter card as an extension node in the SP system. All
configuration parameters should be entered using the SMIT panels.
Remember that if you enter configuration information into SP Switch Router
configuration files, you will also need to access the SMIT panels and reenter
information those panels require.
Refer specifically to the "Managing Extension Nodes" chapter in the RS/6000
SP: Administration Guide Version 2 Release 4, GC23-3897 for information
about setting up SNMP to monitor the SP Switch Router system and
configure the SP Switch Router Adapter media card.
Following are the SMIT commands along with the data entered for the GRF
1600 in our lab environment. Refer to Appendix A, “Laboratory Hardware and
Software Configuration” on page 233 for detailed information about our setup.
• Command: smitty enter_extnode
Installation and Configuration
89
Download from Www.Somanuals.com. All Manuals Search And Download.
Enter Extension Node Information
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
[grf16.msc.itso.ibm.com]
[spenmgmt]
Administrative Hostname
SNMP Community Name
Extension Node Identifier
SNMP Agent Hostname
[03]
[grf16.msc.itso.ibm.com]
Reconfigure the extension node?
* Node Number
no
[4]
+
#
Note: The Extension Node Identifier (the number of the slot in the GRF,
the SP Switch Router Adapter card is seated) must be given as a two-digit
number, so slots 0-9 must be entered as 00-09!
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
Before command completion, additional instructions may appear below.
The endefnode command has completed successfully.
• Command: smitty enter_extadapter
Enter Extension Node Adapter Information
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Network Address
[192.168.14.4]
Network Netmask
Reconfigure the extension node?
* Node Number
[255.255.255.0]
no
[4]
+
#
Installation and Configuration
90
Download from Www.Somanuals.com. All Manuals Search And Download.
COMMAND STATUS
stdout: yes stderr: no
Command: OK
Before command completion, additional instructions may appear below.
The endefadapter command has completed successfully.
• Command: smitty annotator
Topology File Annotator
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
Input Topology File Name
Output Topology File Name
Save Output File to SDR
[/etc/SP/expected.top.1nsb.0isb.0]
[/etc/SP/expected.top.annotated]
[yes]
+
/etc/SP/expected.top.annotated now has a new line:
s 16 1 tb3 3 0
E01-S17-BH-J25 to E01-N4 # Dependent Node
Installation and Configuration
91
Download from Www.Somanuals.com. All Manuals Search And Download.
• Command: smitty manage_extnode
Extension Node Management
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
reconfigure
[4]
Action to be performed on the extension node
* Node Number
+
#
COMMAND STATUS
Command: OK
stdout: yes
stderr: no
Before command completion, additional instructions may appear below.
enadmin command: sending a reconfigure request to the agent managing
extension node 03
enadmin command: : response received from agent indicates requested action
accepted for processing - command complete
Use SDRGetObjects switch_respondsto see the actual status of the SP
Switch:
root@sp21en0:/ SDRGetObjects switch_responds
node_number switch_responds autojoin
isolated
adapter_config_status
1
5
6
7
8
1
1
1
1
1
1
1
1
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0 css_ready
0 css_ready
0 css_ready
0 css_ready
0 css_ready
0 css_ready
0 css_ready
0 css_ready
0 css_ready
0 css_ready
1 not_configured
9
10
11
13
15
4
Before the extended node can be successfully Eunfenced, more work has
to be done.
Installation and Configuration
92
Download from Www.Somanuals.com. All Manuals Search And Download.
3.10.2 Method 2: Edit /etc/grifconfig.conf - Optional
Edit the /etc/grifconfig.conf file to assign an IP address to each logical SP
Switch Router interface. You can also provide other information about the
logical IP network to which that interface is physically attached.
Each logical interface is identified in /etc/grifconfig.conf by these properties:
• Its interface name (an SP Switch Router convention, defined below)
• Its Internet address
• Its netmask
• Its broadcast/destination address
• An argument field
The format for an entry in the /etc/grifconfig.conf file is:
# name address
gt030 192.168.14.4
netmask
255.255.255.0 -
broad_dest
arguments
mtu 65520
Remember that if you enter configuration information into SP Switch Router
configuration files, you also need to access the SMIT panels and reenter
information those panels require.
Interface name
The SP Switch Router interface name has five components that describe
an individual interface in terms of its physical slot location in the chassis,
and its specific and virtual locations on a media card.
In the SP Switch Router, the SP Switch Router Adapter card interface
names look like this: gt000, gt030, gt0a0, gt0f0 (only the slot number,
represented as a hexadecimal digit, changes).
Figure 35 shows the definitions of the components that comprise the SP
Switch Router Adapter card interface names:
g t 0 y 0
1st:
always "g" for GRF
2nd:
3rd:
4th:
5th:
media type, t (SP Adapter), f (FDDI), h (HIPPI), e (100Base-T), etc.
chassis number, always "0" (zero)
slot number in hex
logical interface number in hex, always "0" (zero)
Figure 35. Components in the SP Switch Router Adapter Card’s Interface Name
Installation and Configuration
93
Download from Www.Somanuals.com. All Manuals Search And Download.
Internet address
The Internet address is the 32-bit IP address for the specified logical
interface. The address is in standard dotted-decimal (octet) notation:
xxx.xxx.xxx.xxx.
Netmask
Netmask is the 32-bit address for the logical IP network on the physical
network to which the specific SP Switch Router or media card physical
interface is attached. The netmask is entered in standard dotted-decimal
(octet) notation. If no destination/broadcast address is supplied, a
netmask is required.
Broadcast or destination address
The broadcast or destination address is the 32-bit address for this
network. Enter the broadcast or destination address in standard dotted-
decimal (octet) notation. When a broadcast IP address is assigned to a
logical interface, the netmask value is ignored. A dash (-) can be entered
in the netmask column, or it can be left blank.
The connection of the SP Switch Router Adapter card to the SP system is
point-to-multipoint.
When you assign an IP address to a logical interface on a point-to-point
media such as HIPPI or ATM, the destination address is entered in the
broadcast address field.
Note that any entry in the broadcast address field for HIPPI or ATM makes
it a point-to-point connection to that address. If you remove the broadcast
address, you create a nonbroadcast, multiaccess (NBMA) interface.
Argument field
This field is optional for SP Switch Router Adapter cards. The argument
field specifies a Maximum Transmission Unit (MTU) value different from
the coded default value of 65520. When the command grifconfigruns, it
passes the arguments to the ifconfigcommand that it spawns.
Default MTU values
Default MTUs for SP Switch Router media are:
• SP Switch Router Adapter: 65520 bytes
• HIPPI: 65280 bytes
• FDDI: 4352 bytes
• ATM OC-3c: 9180 bytes
• ATM OC-12c: 9180 bytes
• 10/100Base-T: 1500 bytes
Installation and Configuration
94
Download from Www.Somanuals.com. All Manuals Search And Download.
Default MTUs for framing protocols are:
• Frame Relay: 4352 bytes
• HDLC: 4352 bytes
• Point-to-Point Protocol: 1496 bytes
MTU discovery facility
MTU sizes are generally selected at the host end of the route. This is
accomplished by turning on the host’s MTU discovery facility and allowing
the host to send packets. The MTU discovery facility operates by default
on the SP Switch Router. AIX4.3.1 provides MTU discovery; however, you
have to enable it with the /etc/nocommand.
In effect, the discovery facility tells the router not to fragment, but instead
to advise the host when the packet size is larger than the given path can
handle. This allows the host to discover the largest packet that the most
restrictive of the media components within the same path can handle.
Once "discovered", the host then sends only packets in sizes matching the
reported maximum, and so packets are not fragmented.
3.10.3 Putting grifconfig.conf Additions into Effect
Additions made to /etc/grifconfig.conf after first-time installation take effect
only after the file is reloaded and the media card reset.
Use the grresetcommand to reset each configured SP Switch Router
Adapter card by specifying the slot number where each card is installed, as
follows:
grreset <card_slot_number>
3.11 Step 3. Change Profile Settings
As mentioned, this step is optional and explained in detail in GRF 400/1600
Configuration Guide 1.4, GA22-7366. The SP Switch Router will work happily
with the default settings shipped with the system and there really is no need
to change anything at this time.
3.12 Step 4. Run dev1config
If you followed the advice in Section 3.10.1, “Method 1: Use SP SNMP
should be all that needs to be done.
Installation and Configuration
95
Download from Www.Somanuals.com. All Manuals Search And Download.
Nevertheless, check the file /etc/grdev1.conf. It must contain an entry for the
slot in which the media card is installed. As we have our card in slot 3, the
entry looks as follows:
# CARD 3 Interface 0
#
2.21.4.1.1.1
2.21.4.1.1.2
2.21.4.1.1.3
2.21.4.1.1.4
2.21.4.1.1.5
2.21.4.1.1.6
2.21.4.1.1.7
2.21.4.1.1.8
2.21.4.1.1.9
2.21.4.1.1.10
2.21.4.1.1.11
2.21.4.1.1.12
2.21.4.1.1.13
2.21.4.1.1.14
2.21.4.1.1.15
#
"03"
4
# Node Name
# Node Number
"00:00:00:01:00:00:00:06:00:01" # Switch Token
2
3
x192.168.14.4
x255.255.255.0
1024
1
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
# Node State
# Switch Chip Link
# Node Delay
1
sp21en0
2
1
31
1
# Admin Status
These entries show up after the SP Switch Router Adapter card gets
configured on the SP Switch.
3.13 Step 5. Reset SP Switch Router System to Install Files
To install the system configuration files, first save the files and then reboot the
SP Switch Router.
Save the files after you complete changing the system parameters, and again
after you configure the media cards and any network services, such as
filtering or dynamic routing, using grwrite -v. To check if any files need to be
written to permanent storage, use grwrite -vn.
Note: grwriteonly saves files in the /etc directory. If you changed files in
different directories, use grsiteand grsite --perm,respectively.
Use reboot -i to reset the system.
3.13.1 Saving Configuration Files
Use the grwrite -v command to save the configuration files in the /etc
directory from RAM to a flash device. This preserves the configuration files
over a reboot.
Installation and Configuration
96
Download from Www.Somanuals.com. All Manuals Search And Download.
To save an alternate configuration on the internal flash based upon the
currently running configuration on the internal flash device, use
grsnapshot -si -di=revision,version.
For more information about these commands, see GRF Reference Guide 1.4,
GA22-7367.
3.14 Verify an SP Switch Router Adapter Card on the Router
This section describes tools available with the SP Switch Router system
software to check out newly installed media cards. These tools are to be used
on the SP Switch Router:
• The pingcommand tests whether an SP Switch Router Adapter media
card can process and return a message.
• The grcard[port_number]command tells you the operating state of an
installed SP Switch Router Adapter card.
• The grresetcommand allows you to reset the card.
Note: Output from logs and other system reporting functions refer to the SP
Switch Router Adapter card as DEV1, DEV_V1 or dev1.
3.14.1 Verify Media Card Operation Using ping
Check SP Switch Router Adapter media card viability using the ping
command. This UNIX command is modified to support SP Switch Router
board components. This use of pingonly tests internal communication
between the SP Switch Router control board and the specified media card. It
does not test message routing between media cards or communication
between media cards and external devices.
Note: The pingcommand does not disturb normal SP Switch Router
operations.
The ping -P grid <slot_number>command sends a message to a specified SP
Switch Router Adapter card asking the card to respond back with another
message.
Follow these steps:
1. Log in as root to the SP Switch Router.
Installation and Configuration
97
Download from Www.Somanuals.com. All Manuals Search And Download.
2. Enter a pingcommand. Specify the appropriate media card by its chassis
slot number; for example, to act on the SP Switch Router Adapter media
card in slot 3, enter ping -c4 -P grid 3.
This is what you see when the media card responds:
# ping -c4 -P grid 3
GRID ECHO 3 (0:0x3:0): 64 data bytes, 3 packets
68 bytes from 0:0x3:0: time=0.619 ms
68 bytes from 0:0x3:0: time=0.498 ms
68 bytes from 0:0x3:0: time=0.640 ms
68 bytes from 0:0x3:0: time=0.640 ms
--- 3 GRID ECHO statistics ---
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0.498/0.580/0.640 ms
#
To act on the GRF control board, enter ping -P grid 66.
Refer to GRF Reference Guide 1.4, GA22-7367 for a description of the
pingcommand.
3.14.2 Check Media Card Status Using grcard
The grcardcommand returns information about the status of all installed
media cards. Output from logs and other system reporting functions refers to
the SP Switch Router Adapter card as DEV1, DEV1_V1 or dev1.
Here is a sample of the slot, media, and state information returned from the
grcardcommand:
# grcard -v
Slot
HW type
-------
FDDI_V2
ATM_OC3_V2
ETHER_V1
DEV1_V1
State
-----
running
running
running
running
----
0
1
2
3
#
The SP Switch Router Adapter card resides in slot 3 and its state is reported
as running.
Refer to the command descriptions in the GRF Reference Guide 1.4, GA22-
7367 for a description of grcardand the meaning of the different media states.
Installation and Configuration
98
Download from Www.Somanuals.com. All Manuals Search And Download.
3.14.3 Reset Media Card Using grreset
Use the grresetcommand to reset a media card from the UNIX prompt:
1. Log in as root on the SP Switch Router.
2. Enter the grresetcommand.
Specify the appropriate media card by its chassis slot number. To reset all the
media cards, enter grreset all; to reset the media cards in slot 0, enter
grreset 0; to reset the card in slot 4 and dump its memory, use grreset -D 4;
to reset the card in slot 4 and return debug information, enter grreset -d 4.
Note: The grreset command can be used on a media card without disturbing
normal SP Switch Router function.
You should, however, fence the SP Switch Router Adapter card before you
issue a grresetcommand against it. This prevents some unexpected
behavior of the SP primary node. Use the following command on the CWS:
Efence -autojoin node_number.
With the autojoin flag set, the SP Switch Router Adapter card is supposed to
integrate automatically into the SP Switch network after a grresetcommand.
Without the autojoin flag, Eunfenceor Estartneed to be issued.
Refer to the command section in the GRF Configuration Guide 1.4, GA22-
7366 for a description of grreset. Refer toRS/6000 SP: Command Reference
Version 2 Release 4, GC23-3900, for a description of Efence, Eunfenceand
Estart.
3.14.4 Using grstat to Display GRF Statistics
You might want to periodically watch what is going on on the SP Switch
Router Adapter card. Use the command grstat -w70 all gt0?0to obtain
information about packets sent, received, forwarded, dropped or fragmented.
Also, this command will give you an overview of the last errors that caused
packets to be dropped or resent.
Installation and Configuration
99
Download from Www.Somanuals.com. All Manuals Search And Download.
Below is an actual example:
# grstat -w70 all gt030
gt030
ipstat
count description
11095886 total packets received
51 packets dropped
3678330 packets forwarded normally
4 packets forwarded locally to card
1214 packets handled by the card
ipdrop
last
last
count
3
source addr
192.168.14.6 192.168.13.15 TTL expired
dest addr reason
48 192.168.14.15 192.168.13.15 needed to frag packet but DF set
icmperr
last
count type
last
code
last
val
last
source addr
last
dest addr error
icmpin
last
code
last
source addr
192.168.14.1
last
dst addr
10.10.1.13
count type
4 ECHO
icmpout
last
code
last
source addr
192.168.14.1
last
dst addr
10.10.1.13
count type
4 ECHOREPLY
48 UNREACH
3 TIMXCEED
NEEDFRAG
INTRANS
192.168.14.4 192.168.14.15
192.168.14.4 192.168.14.6
#
3.15 Bringing the SP Switch Router Adapter Card Online with the SP
After the SP Switch Router Adapter media card completes initialization, its
state machine enters the Configured state. The media card sends an up-trap
request to mib2d. mib2d sends the SP Switch manager a pair of
switchNodeUp and switchConfigState (ConfigState=FullyConfigured) trap
messages.
The SP system administrator now decides which action is required to bring
the SP Switch Router Adapter’s interface online.
If the SP Switch Router Adapter was previously fenced from the Switch
network with the -autojoinoption, the SP SNMP manager will automatically
unfence the adapter. Otherwise, the SP system administrator must perform
Installation and Configuration
100
Download from Www.Somanuals.com. All Manuals Search And Download.
one of the following actions to bring the SP Switch Router Adapter card
online:
• A switch initialization
• An unfencing sequence
• Another Switch management sequence
The appropriate action depends on what state the SP system is in with
respect to the dependent node. For example, if no Estartcommand has been
issued to reinitialize the SP Switch since the dependent node (the SP Switch
Router Adapter) was installed, then an Estartcommand is needed. If the
dependent node was fenced from the SP Switch without the -autojoinoption,
then an Eunfencecommand should suffice.
Many different states are possible. Consult RS/6000 SP: Installation and
Migration Guide Version 2 Release 4, GC23-3898 and RS/6000 SP:
Administration Guide Version 2 Release 4, GC23-3897 for descriptions of the
administrative actions needed to bring extension nodes online (dependent
nodes are specific types of extension nodes). See RS/6000 SP: Diagnosis
and Messages Guide Version 2 Release 4, GC23-3899 for information on
diagnosing extension node configuration problems.
The SP Switch Router Adapter media card remains in
ConfigState=FullyConfigured,until it is brought online via a switch
initialization or unfencing sequence.
Should the SwitchNodeUp-trap message not reach the SP SNMP Manager,
use the grcardcommand to check the card’s readiness and state. The grcard
command must return runningfor the SP Switch Adapter card before the card
can be brought online.
3.15.1 Checking Connectivity to the SP System
The procedure in this section is useful when a problem is suspected with the
SP Switch Router Adapter media card, its connection to the SP Switch, or its
connection to the SP Switch Router hardware. This section is intended for
hardware service personnel, although parts may be applicable to customer
problem determination.
Before beginning this procedure, it may be helpful to verify the configuration
of the media adapter. If you are unable to find a configuration problem or are
unable to correct the configuration due to potential hardware problems, this
procedure should be used to check the connection to the SP Switch.
Installation and Configuration
101
Download from Www.Somanuals.com. All Manuals Search And Download.
Each SP Switch Router Adapter media card is considered a dependent node
for the SP System. Each dependent node has a node_number and other
configuration and status information that is unique to that dependent node.
3.15.1.1 Procedure
The following steps might give you guidance in solving some of the most
common connectivity problems:
1. Check the SP Switch cable for obvious problems such as a loose or
disconnected connector, missing shielding or bent pins.
2. Check the 10Base-T twisted-pair connection between the SP Switch
Router control board and the SP CWS. This connection is normally routed
through an Ethernet hub.
3. If there is no terminal directly attached to the SP Switch Router, check the
SP Switch Router host name from the SP CWS. From the CWS, enter
SDRGetObjects DependentNode node_number reliable_hostname. This will
return the node numbers and the corresponding host names for the SP
Switch Router systems.
4. Test Ethernet connectivity by performing a pingtest from the SP CWS to
the SP Switch Router administrative Ethernet address.
5. Check the status of the SP Switch Router Adapter LEDs.
Use the tables in the "SP Switch Router Adapter LEDs" section in
Appendix C to determine the state of the card.
Generally, RX ST0/ST1/ERR and TX ST0/ST1/ERR indicate a problem.
The problem might be due to connection, configuration, hardware, or
software.
To further test the SP Switch Router Adapter card hardware, you can reset
or reseat the card, and then use the tables under "LED activity during
boot" in Appendix C to interpret the results.
6. From the CWS, use an Eunfenceand/or Estartcommand to bring the
dependent node back into the configuration.
From the CWS, issue the command SDRGetObjects switch_respondsand
check for correct values. If switch_responds is 1 or shows up green in
Perspectives, then the dependent node is active again.
7. You may need to log in to the SP Switch Router to perform additional
analysis before determining whether any hardware needs replacement.
8. If problems remain, you will have to contact the next level of Customer
Support for further direction. They may log into the SP Switch Router to
perform additional analysis. If you were directed here by the RS/6000 SP
Installation and Configuration
102
Download from Www.Somanuals.com. All Manuals Search And Download.
Maintenance Information Manual Dependent Node MAP, return to that
procedure.
For more information about configuration as related to the SP, see RS/6000
SP: Administration Guide Version 2 Release 4, GC23-3897 and RS/6000 SP:
Command and Technical Reference Version 2 Release 4, GC23-3900.
For additional information on troubleshooting your configuration, see RS/
6000 SP: Diagnosis and Messages Guide Version 2 Release 4, GC23-3899.
Installation and Configuration
103
Download from Www.Somanuals.com. All Manuals Search And Download.
Installation and Configuration
104
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 4. Configuration of IP-Forwarding Media Cards
This chapter covers the installation and configuration of selected IP
Forwarding media cards in an SP Switch Router. For detailed information
refer to GRF Configuration Guide 1.4, GA22-7366.
4.1 Ethernet 10/100Base-T Configuration
This section provides the information needed to configure the Ethernet
10/100Base-T media card. It comes in two flavors, one with four ports and the
other with eight ports available. Configuration is the same for each type of
card. Each physical interface on them is capable of connecting to either 10 or
100 Mbits/s (autosensing and autonegotiation) and may operate half duplex
(HDX) or full duplex (FDX). So this is a point to take care of, as the ports of
the connecting devices must meet FDX or HDX setting, if they do not, a high
rate of collisions will be reported.
As Ethernet seems to be best known all over the world and configuring this
type of card is really straightforward, this will was the easiest way to get
interfaces other than switch connected to the GRF.
4.1.1 Physical and Logical Interfaces
Physical Interfaces
The dual-speed Ethernet media card provides either four or eight physical
interfaces. An interface can run in either full duplex or half duplex mode.
Additionally, an interface can operate at 10 or 100 Mbits/s, as needed. This
enables the GRF Fast Ethernet media card to interoperate with 10Base-T and
100Base-T devices. These capabilities can be configured to perform in a
specific mode and at a specific transfer rate, or to autosense the mode and
rate capacity of the connected host or network.
Logical Interfaces
A logical interface is configured by its entry in the grifconfig.conf file, where it
is assigned an IP address and netmask. A logical interface is uniquely
identified by its Ethernet interface name.
Interface Name
The generic form of an Ethernet interface name is ge0xy. See Figure 36 on
page 106 for the naming conventions on the Ethernet card.
© Copyright IBM Corp. 1998
105
Download from Www.Somanuals.com. All Manuals Search And Download.
g e 0 x y
1st:
2nd:
3rd:
4th:
5th:
always "g" for GRF
media type, e (Ethernet)
chassis number, always "0" (zero)
slot number in hex
logical interface number in hex
Figure 36. Components of the Ethernet Interface Name
The interface name is used in the /etc/grifconfig.conf file to specify an IP
interface. Interface 7 on the Ethernet card in slot 3, for example, would be
added to this file as:
#name address
ge037 xxx.xxx.xxx.xxx 255.255.255.0 -
netmask
broad_dest
arguments
mtu 1500
Note: Interface names are case sensitive. Always use lower case letters
when defining interface names.
4.1.2 Configuration File and Profile Overview
These are the steps to configure Ethernet interfaces:
1. Identify each logical interface
Edit /etc/grifconfig.conf to identify each logical interface by assigning:
• An IP address
• The GRF interface name
• A netmask, as required
• A destination or broadcast address, as required
• A Maximum Transmission Unit (MTU), if needed
2. Specify Ethernet card parameters in the card profile
These are the configurable items in a card profile for an Ethernet media
card (all but the first optional):
• Configure interface mode: autonegotiate, 10 or 100 Base-T, full or half
duplex
• Specify verbose option for messages from the Ethernet card
• Specify ICMP throttling settings
• Specify selective packet discard percentage
106
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
• Change run-time binaries
• Change dump variables
3. Load profile
Global executable binaries are set in the Load profile in the hw-table field.
These only change when you want to execute new run-time code in every
Ethernet card. If you want to change the run-time code in one Ethernet
card (per interface), make the change in the Card profile, in the load field.
4. Dump profile
Global dump settings are in the Dump profile. These settings are usually
changed only for debug purposes. The keep-count field specifies how
many dumps are compressed and stored at one time for each media card.
The file system accommodates the default setting of zero (0), which
actually stores two dumps per day (the current dump and the first dump of
the day). Use caution if you change the recommended default.
If you want to change dump settings for one Ethernet card (per interface),
make the change in the Card profile, in the dump field.
For a detailed discussion of these steps, refer to GRF Configuration Guide
1.4, GA22-7366.
4.1.3 Installing Configurations or Changes
In the command line interface (CLI), which you can recognize by its super>
prompt, use setand writecommands to install configuration parameters.
To save the files in the /etc configuration directory, use grwrite -v.
Additionally, when you enter configuration information or make changes, you
must also reset the media card for the changes to take place.
Enter grreset <slot_number>to make this happen.
Hint: We have found situations where grresetwas not able to reconfigure
cards, ports or the running kernel. So if you are in doubt, it is always safe to
reboot -ithe GRF.
4.1.4 Assign IP Addresses - grifconfig.conf
Configure each logical interface in the grifconfig.conf file by assigning it an IP
address, a GRF interface name, and, if required, a netmask and destination
or broadcast address. Here is an entry from our /etc/grifconfig.conf file:
Configuration of IP-Forwarding Media Cards
107
Download from Www.Somanuals.com. All Manuals Search And Download.
#name address
netmask
broad_dest
arguments
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
ge070 10.20.30.1
#ge071 10.20.30.1
#ge072 10.20.30.1
#ge073 10.20.30.1
#ge074 10.20.30.1
#ge075 10.20.30.1
#ge076 10.20.30.1
#ge077 10.20.30.1
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
4.1.5 Specify Ethernet Card Parameters
As already mentioned, modifying the following profiles is optional.
• Card Profile - Card Parameters
• Load Profile - Run-time Code
• Dump Profile - Dump Defaults
Only for the configuration of the interface mode these profiles need to be
modified. You would at least like to know whether FDX and HDX settings are
suited to your environment and change them, if needed.
We advise that you set the ports explicitly to the values you expect them to
have according to your network layout. We prefer to have control over the
network, instead of it controlling us (which might be wishful thinking).
Set the Negotiation or Transfer Rate
Set the negotiation or transfer rate in the ports/ether field. By default, the
setting for each interface is autonegotiate:
super> read card 7
CARD/7 read
super>list ports 1
port_num = 1
cisco-hdlc = { off on 10 3 }
fddi = { single off }
sonet = { "" "" 1 sonet internal-oscillator 0 207 }
hssi = { 0 16-bit }
ether = { autonegotiate }
hippi = {1 32 no-mode 999999 4 incremental 5 300 10 10 03:00:0f:c0 disab+
super> list ether
if-config = autonegotiate
At this level, use the setcommand to look at the interface options:
super> set if-config ?
108
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
if-config:
Ethernet interface configuration.
Enumerated field, values:
autonegotiate: autonegotiate
10-half: 10 BaseT Half Duplex
10-full: 10 BaseT Full Duplex
100-half: 100 BaseT Half Duplex
100-full: 100 BaseT Full Duplex
super> set if-config = 100-full
super> write
CARD2/ written
super> quit
#
The configuration of the Ethernet card is now completed. Issue grwrite -v
and grreset <slot_number>. Communication between the GRF’s Ethernet card
and attached devices should work now. To monitor the card, use the maint
commands we found to be useful in the following section.
4.1.6 Some maint Commands for the Ethernet Media Cards
The maintcommands operate on the SP Switch control board and require the
card’s slot number, which you may find with the grcardcommand.
Prepare to use maintwith the following steps:
• First, switch to maint’s GR 66> prompt with the grrmbcommand.
• At the GR66>prompt, change to the prompt for the specific card you are
interested in. For a card in slot 7, this would be the command port 7.
• As a result, the following message along with the changed prompt is
returned:
Current port card is 7
GR 7>
• At this prompt maintcommands can be issued.
• To leave the GR 7>prompt, enter quit.
The following maintcommands were most useful for us:
maint 1-to view the list of receive side (RX) maintcommands.
maint 101-to view the list of transmit side (TX) maintcommands.
maint 3-to display configuration and status of all ports.
maint 4-to display media statistics for both the input side and the output side
for one or all eight interfaces. If you do not specify one interface,
Configuration of IP-Forwarding Media Cards
109
Download from Www.Somanuals.com. All Manuals Search And Download.
you see the output for all eight. The input port side is reported on
first.
maint 5-to return GRF switch statistics.
maint 7-to clear the current collected statistics.
maint 8-to display the ARP table for one interface or, if no interface is
specified, for all interfaces.
4.2 ATM OC-3c Configuration
This section provides information needed to configure the ATM OC-3c media
card. The GRF can be configured in point-to-point or point-to-multipoint ATM
topologies with either switches or hosts. The OC-3c media card provides two
independent physical ATM interfaces, each of which supports 110 logical
interfaces and operates at 155Mbit/s full duplex.
4.2.1 Physical and Logical ATM Interfaces
Figure 37 shows the organization of physical and logical ATM interfaces on
the ATM OC-3c media card.
Logical Interfaces
0-7f (range)
80-ff (range)
220/card
VPI
VCI
Total # of active VCs
Physical Interface 0
(top)
0
1-15
0-32767
0-511
512
0
1-15
Physical Interface 1
(bottom)
0-32767
0-511
512
1024/card
Figure 37. ATM OC-3c Physical and Logical Interfaces
Physical Interfaces
The ATM OC-3c media card supports two physical interfaces, each of which
supports the assignment of 110 logical interfaces out of a range of 128.
Logical Interfaces
Logical interfaces provide a simple way of mapping many IP addresses onto
a single ATM port. A logical interface serves as the connection between ATM
and IP. Each logical interface is assigned a unique IP address in
grifconfig.conf. All interface names are case sensitive. Always use lower case
letters when defining interface names.
110
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
g a 0 x yz
1st:
always "g" for GRF
2nd:
media type, a (ATM)
3rd:
chassis number, always "0" (zero)
slot number in hex
4th:
5th-6th:
logical interface number in hex
Figure 38. Components in the ATM OC-3c Interface Name
Virtual Circuits
A virtual circuit (VC) exists between two ATM devices. It is the point-to-point
connection between them and is of no significance to other ATM devices.
Each VC is identified by a pair of numbers, representing a virtual path
identifier (VPI) and a virtual circuit identifier (VCI). A slash (/) is used to
separate the two numbers, for example, 0/2645. The VPI/VCI must be unique
on a link. Because it is acceptable to use the same VPI/VCI on different links,
a GRF can have the same VPI/VCI active on each physical interface.
The ATM OC-3c media card supports up to 1024 active VCs as defined in the
ATM Forum UNI3.0 specification. VCs can be divided between the two
physical interfaces in any manner required by the site, with 512 VCs active at
any one time on each interface. Each VC has an associated IP address.
VPI/VCIs are assigned to logical interfaces in /etc/gratm.conf, and provide the
bridge between ATM and IP.
Virtual Paths
A virtual path (VP) connects two end stations, which may be separated by
one or more network devices such as a router or switch. A path consists of
Virtual Path
Virtual
Circuit
Virtual
Circuit
Figure 39. Components Forming a Virtual Path
Configuration of IP-Forwarding Media Cards
111
Download from Www.Somanuals.com. All Manuals Search And Download.
VPIs 0 through 15 are available for configuration use. VPIs are assigned in
the /etc/gratm.conf file with regard to their VCI.
VCIs
VCIs name VCs. VCIs are also assigned in the /etc/gratm.conf file. On VPI 0,
VCI 0 through VCI 32767 can be used; on VPIs 1-15, VCI 0 through VCI 511
can be used.
Note: Virtual circuits 0-31 on each VPI are reserved for signaling.
Permanent Virtual Circuits
Permanent virtual circuits (PVCs) are created statically. PVCs are configured
in /etc/gratm.conf. The GRF supports Inverse ATM ARP (InATMARP) to
determine the IP address of the other end of the VPI/VCI. If the other device
does not support InATMARP, an ARP entry for the IP and VPI/VCI of the other
device must be made in /etc/grarp.conf.
Note: VPI/VCI pairs must be unique per physical port, and you cannot have
two or more circuits with the same VPI/VCI in the same logical interface.
Switched Virtual Circuits
Switched virtual circuits (SVCs) are created or destroyed dynamically using
standard signaling protocols. These protocols allow ATM devices to create or
destroy connections in response to traffic demands. The VPI/VCI for a given
SVC is determined at the time of the connection setup, and thus requires no
manual configuration in /etc/gratm.conf. However, it is necessary to specify
which of the UNI signaling standards (UNI3.0 or UNI3.1) you wish to use, and
to assign an ARP server. Both sides of the ATM link must use the same
version of the signaling protocol. If you do not wish to use SVCs, set the
signaling for that interface to NONE.
UNI signaling uses ATM Format NSAP addresses, not IP addresses, and
requires the use of an ARP server. The ARP server maps an IP address to an
NSAP address so that ATM signaling can create or use the appropriate SVCs
for traffic destined for the given IP address. The ARP server’s NSAP address
must be configured in the Service section of /etc/gratm.conf.
PVCs and SVCs can be used simultaneously on the same physical interface
(port). PVCs and SVCs can also coexist on the same logical interface.
Verifying an ATM Configuration
maintcommands enable you to verify an ATM configuration. They are
Media Card” on page 116; some examples are provided there.
112
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.2.2 Installing Configurations or Changes
In the command line interface (CLI), use setand writecommands to install
configuration parameters.
To save the /etc configuration directory, use grwrite -v.
Additionally, when you enter configuration information or make changes, you
must also reset the media card with the command grreset <slot_number>for
the change to take place.
4.2.3 Configuration Files and Profiles
Following are the steps to configure an ATM card. They are listed here
complete, although you can bring up an ATM connection with a subset. For
detailed information, see GRF Configuration Guide 1.4, GA22-7366.
Proceed as follows:
1. Identify each logical interface.
Edit grifconfig.conf, to identify each logical interface, by assigning:
• An IP address
• The GRF interface name
• A netmask, as required
• A destination or broadcast address, as required
• An MTU, if needed
for the details.
2. Specify ATM card parameters in the Card profile (all optional):
• Specify ICMP throttling settings
• Specify a selective packet discard percentage in the spd-tx-thresh field
• Change run-time binaries
• Change dump variables
3. Configure PVCs and SVCs in /etc/gratm.conf.
Edit the file /etc/gratm.conf and add entries for the following keywords for
a minimum configuration:
• Traffic_Shape
• Interface
• PVC
The following screen shot shows an excerpt from our scenario:
Configuration of IP-Forwarding Media Cards 113
Download from Www.Somanuals.com. All Manuals Search And Download.
Traffic_Shape name=high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
Traffic_Shape name=low_speed_high_quality \
peak=15500 qos=high
Interface ga010 traffic_shape=high_speed_high_quality
Interface ga0180 traffic_shape=low_speed_high_quality
PVC ga010 0/132 proto=ip traffic_shape=high_speed_high_quality
PVC ga0180 0/134 proto=ip traffic_shape=low_speed_high_quality
4. Load profile (optional).
Global executable binaries are set in the Load profile in the hw-table field.
These only change when you want to execute new run-time code in every
ATM card.
If you want to change the run-time code in one ATM card (per physical
interface), make the change in the Card profile, in the load field.
5. Dump profile (optional).
Global dump settings are in the Dump profile. These settings are usually
changed only for debug purposes. The keep-count field specifies how
many dumps are compressed and stored at one time for each media card.
The default setting is zero (0), which actually stores two dumps per day
(the current dump and the first dump of the day). Use caution if you
change the recommended default.
If you want to change dump settings for one ATM card (per interface),
make the change in the Card profile, in the dump field.
4.2.4 Assign IP Addresses - grifconfig.conf
Edit the grifconfig.conf file to assign an IP address to each logical ATM
interface. You can also provide other information about the logical IP network
to which that interface is physically attached, or specify a different MTU in the
arguments field, for example.
When you configure a logical interface on a point-to-point media such as
ATM, enter the destination IP address in the broad_dest address field. An
entry in the broad_dest address field for an ATM interface creates a
point-to-point connection to that address. If you do not specify a broadcast
address, you create a non-broadcast, multiaccess (NBMA) interface. The
optional arguments field is currently used to specify MTU values on a logical
114
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
interface basis. This field is also used to specify ISO when an ISO address is
being added to an interface’s IP address. Specify the MTU value as mtu xxxx.
Leave the arguments field blank if you are not using it.
The following excerpt from our /etc/grifconf.conf file shows the format of an
entry:
# name address
ga010 10.1.1.1
ga0180 10.1.2.1
netmask
255.255.255.0 -
255.255.255.0 10.1.2.2
broad_dest
arguments
mtu 9180
As you can see, we have defined the first logical interface to be broadcast
and the second interface to be point-to-point.
4.2.5 Specify ATM Card Parameters
Changing the following profile is not mandatory. You can safely use the
defaults. Refer to GRF Configuration Guide 1.4, GA22-7366 if you intend to
change settings.
4.2.6 Configuring PVCs
To configure a logical interface that supports PVCs, make entries in these
configuration files:
1. /etc/grifconfig.conf
Assign an IP address to the logical interface.
2. /etc/grarp.conf
Supply IP-to-physical address mapping information for the ARP service.
Put an entry into /etc/grarp.conf only if the remote destination does not
support InATMARP, which the GRF does.
3. /etc/gratm.conf
• Traffic shaping section
Set the traffic shaping name and quality of service parameters. The
name may be any meaningful string you like.
• Signaling section
Set protocol=NONEon the signaling entry for the appropriate card and
connector combination.
Configuration of IP-Forwarding Media Cards 115
Download from Www.Somanuals.com. All Manuals Search And Download.
• Interface section
Define the traffic shaping profile for the logical interface to which the
media card’s PVCs are assigned.
• PVC section
Specify characteristics for each PVC, including:
• Assigned logical interface name
• VPI/VCI
• Protocol supported
• Whether AAL is used
• Assigned traffic shaping profile (only if it would be different from the
profile given previously to the logical interface in the Interface
section)
Templates of these configuration files are in GRF Reference Guide 1.4,
GA22-7367; traffic shaping is discussed in detail in GRF Configuration Guide
1.4, GA22-7366.
Hint: Use gratm -n ga0<slot_the_ATM_card_is_in>to check for any errors in
/etc/gratm.conf.
The actual data for configurations we tested will be presented in Section
OC-3c Backbone Connection” on page 209, respectively.
4.2.7 Some maint Commands for the ATM OC-3c Media Card
The maintcommands display a range of information about the media card.
The ATM OC-3c card has individual processors for the transmit and receive
sides, and two sets of maintcommands. One set covers the receive (RX) side
and includes some commands applicable to the card overall. The second set
covers the transmit (TX) side. Transmit side counterparts of receive side
commands use the same number but are 100-based. For example, the
receive side maint 8is transmit side maint 108.
To use maint, follow these steps:
• First, switch to the maintprompt with the grrmbcommand. A new prompt,
GR 66>, will appear.
• Then change the prompt port to the ATM media card you are working with.
For example, if you are working with a card in slot 3, enter GR 66> port 3.
116
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
The following message is returned along with the changed prompt:
Current port card is 3
GR 3>.
• To leave any maintprompt and return to the shell, enter quit.
Following are just a few maintcommands we have found useful; of course,
your experiences may vary.
To see the list of maintcommands for the receive side, enter maint 1; to see
the list of maintcommands for the transmit side, enter maint 101.
The maint 3command gives you useful options for looking at a variety of
active interfaces. To look at media information per port, use maint 4and the
port number, 0or 1. Use maint 4 0 to see the information for the card’s port
0.
To display switch statistics, use maint 5, which will give you information about
the number of packets to and from the switch (in the GRF).
You can return information about VPI/VCIs on a per port, per side basis with
maint 13 0or maint 13 1,respectively. For information about traffic statistics
use maint 14 0or maint 14 1, respectively.
Configuration of IP-Forwarding Media Cards 117
Download from Www.Somanuals.com. All Manuals Search And Download.
4.2.8 Using grrt to Display the Route Table
Use the grrt -S -p<slot>command to display the current contents of the ATM
OC-3c card’s route table. The following is an actual screen shot:
# grrt -S -p1
default
0.0.0.0
10.1.1.0
10.1.1.1
10.1.1.255
10.1.2.1
10.10.1.0
10.10.1.13
10.10.1.255
127.0.0.0
127.0.0.1
192.168.4.0
192.168.13.0
192.168.14.0
192.168.14.4
1
192.168.4.137
0.0.0.0
inx 0
inx 0
ga010
ga010
ga010
ga0180
bg0
RMS
DROP
FWD
LOCAL
BCAST
LOCAL
BFWD
BLOCAL
BBCAST
FWD
RMS
RMS
FWD
FWD
LOCAL
BCAST
MCAST
MCAST
MCAST
BCAST
255.255.255.255 8
255.255.255.0
16 0.0.0.0
255.255.255.255 15 0.0.0.0
255.255.255.255 14 0.0.0.0
255.255.255.255 19 0.0.0.0
255.255.255.0
7
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
127.0.0.1
255.255.255.255 6
255.255.255.255 5
bg0
bg0
255.0.0.0
4
inx 0
inx 0
inx 0
ga010
gt030
gt030
gt030
inx 0
inx 0
inx 0
inx 0
255.255.255.255 3
255.255.255.0
255.255.255.0
255.255.255.0
10 0.0.0.0
17 10.1.1.2
13 0.0.0.0
255.255.255.255 12 0.0.0.0
192.168.14.255 255.255.255.255 11 0.0.0.0
224.0.0.0
224.0.0.0
224.0.0.0
240.0.0.0
255.0.0.0
2
2
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
255.255.255.255 2
255.255.255.255 255.255.255.255 9
#
118
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.2.9 Using grstat to Display GRF Statistics
Use the grstat -w70 all <interface>command to display the current statistics
of the ATM OC-3c card’s IP stack. The following is an actual screen shot:
# grstat -w70 all ga010
ga010
ipstat
count description
1955290 total packets received
1955269 packets forwarded normally
21 packets handled by the card
ipdrop
last
last
count
icmperr
source addr
dest addr reason
last
count type
icmpin
last
code
last
val
last
source addr
last
dest addr error
last
code
last
source addr
last
dst addr
count type
icmpout
last
code
last
source addr
last
dst addr
count type
#
4.3 ATM OC-12c Configuration
This section provides information needed to configure the ATM OC-12c
media card. The GRF can be configured in point-to-point or
point-to-multipoint ATM topologies with either switches or hosts.
The ATM OC-12c card is very similar to the OC-3c card introduced in Section
4.2, “ATM OC-3c Configuration” on page 110, so only the differences between
the two cards will be handled here.
The OC-12c media card provides one physical ATM interface, which supports
220 logical interfaces and operates at 622Mbit/s full duplex.
4.3.1 Physical and Logical ATM Interfaces
Figure 40 on page 120 shows the organization of physical and logical ATM
interfaces on the ATM OC-12c media card.
Configuration of IP-Forwarding Media Cards 119
Download from Www.Somanuals.com. All Manuals Search And Download.
Logical Interfaces
0-ff (range)
VPI
VCI
Total # of active VCs
1408
Physical Interface 0
(center)
0
1-3
0-1024
0-127
Figure 40. ATM OC-12c Physical and Logical Interfaces
Physical Interfaces
The ATM OC-12c media card supports one physical interface. It supports the
assignment of 220 logical interfaces out of a range of 256.
Virtual Circuits
The ATM OC-12c media card supports up to 1408 active VCs.
Virtual Paths
VPIs 0 through 3 are available for configuration use.
VCIs
On VPI 0, VCI 0 through VCI 1023 can be used; on VPIs 1-3, VCI 0 through
VCI 127 can be used.
Note: Virtual circuits 0-31 on each VPI are reserved for signaling.
4.3.2 Installing Configurations or Changes
In the command line interface (CLI), use setand writecommands to install
configuration parameters.
To save the /etc configuration directory, use grwrite -v.
Additionally, when you enter configuration information or make changes, you
must also reset the media card with the command grreset <slot_number>for
the change to take place.
4.3.3 Configuration Files and Profiles
The following steps to configure an ATM OC-12c card are the minimum
subset required to successfully bring up a connection. See Section 4.2.3,
GA22-7366.
Proceed as follows:
120
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
1. Identify each logical interface.
Edit /etc/grifconfig.conf, to identify each logical interface by assigning:
• An IP address
• The GRF interface name
• A netmask, as required
• A destination or broadcast address, as required
• An MTU, if needed
2. Configure PVCs and SVCs in /etc/gratm.conf.
Edit the file /etc/gratm.conf and add entries for the following keywords for
a minimum configuration:
• Traffic_Shape
• Interface
• PVC
The following screen shot shows an excerpt from our scenario:
Traffic_Shape name=bigg_speed_high_quality \
peak=622000 sustain=622000 burst=2048 qos=high
Interface ga020 traffic_shape=bigg_speed_high_quality
PVC ga020 0/132 proto=ip traffic_shape=bigg_speed_high_quality
Note: the ATM OC-12c card supports up to 622 Mbit/s, as opposed to the
155 Mbit/s of the ATM OC-3c card. So you have to define new traffic
shapes and assign them to the interface and the PVC.
Hint: Use gratm -n ga0<slot_the_ATM_card_is_in>to check for any errors in
/etc/gratm.conf.
The actual data for the configuration we tested will be presented in Section
4.4 FDDI Configuration
This section provides information needed to configure the FDDI media card.
The card has four physical interfaces that can be connected to either
switches, hubs or hosts, and may be set up as four single attached stations
(SAS), as two SASs and one dual attached station (DAS), or as two DASs. It
operates at 100Mbit/s.
Configuration of IP-Forwarding Media Cards
121
Download from Www.Somanuals.com. All Manuals Search And Download.
It might be useful to give a short overview of possible FDDI connection
options, namely SAS, DAS, optical bypass and dual homing, and show a
picture explaining these scenarios.
Single Attach (SAS)
Single attach FDDI interfaces can be either master (M) ports or slave (S)
ports. They require a cable with a corresponding master or slave connector.
Single attach cables have an M connector on one end and an S connector on
the other. With no key installed, both M and S connectors fit the FDDI
interface. So if you do not use the colored keys that come with every FDDI
cable and mark the connectors and receptacles as to their use correctly, you
may well be unable to get a working physical connection, because you have
connected the wrong fiber ports together.
To key a connector, fit the red type A, the blue type B or the green type M key
inserts accordingly. An unkeyed connector is of type S.
A single attach FDDI interface on the GRF is a master port when it directly
connected to the master port of an FDDI concentrator which, in turn, connects
to the slave ports of SAS workstations.
S
S
M
A0
B0
M
S
M
M
M
S
S
S
M
M
M
S
S
S
S
FDDI
Concentrator
FDDI
A1
B1
M
M
Figure 41. Master/Slave Connectors for SAS Interfaces
Dual Attach (DAS)
Dual attach interfaces connect to form two unbroken counter-rotating rings.
Each interface, or station, has both an A and a B port.
Dual attach cables have an A connector on one end and a B connector on the
downstream neighbor; the B port connects a station to its upstream neighbor.
To create a logical ring, A must connect to B and B must connect to A.
Otherwise, the network does not operate as a logical ring, but segments into
unconnected subrings.
122
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Station
1
Station
2
A
B
A0
B0
A
B
B
A
B
A
B
A
B
A
Logical ring
FDDI
Station
N...
A1
B1
Station
3
...
A
B
A
B
A
B
A
B
Figure 42. A/B Connectors for DAS Interfaces
Configuring SAS versus DAS
Only the top or bottom pair of FDDI interfaces can be set to dual attach.
Interfaces 1 and 2, for example, must not be paired. It is recommended to set
unused FDDI interfaces to single in the Card profiles (which is the default
anyway).
All possible FDDI configurations are shown in Figure 43.
4 single attach:
2 dual attach:
FDDI
1 dual, 2 single attach:
2 single, 1 dual attach:
0
0
1
2
3
0
1
0
1
1
2
3
FDDI
FDDI
FDDI
2
2
3
3
Figure 43. Allowed SAS and DAS Configurations
In the Card profile, specify SAS or DAS as singleor dualin the ports/fddi
field. This example shows how to set interfaces 0 and 1 as DAS and do a
writeat the end to save the changes. The following shows the path to change
the setting of a port from SAS to DAS.
You have to exitfrom the #prompt to the super>prompt of the GRF and read
card <slot_number>. We assume the FDDI card seated in port 0, therefore the
command is read card 0.
The system responds with:
CARD/0 read
super>
Now provide the lower number of the two ports you want to assign DAS to,
with list ports 0,and then issue list fddito get the following from the
system:
Configuration of IP-Forwarding Media Cards
123
Download from Www.Somanuals.com. All Manuals Search And Download.
super> list ports 0
port_num = 0
cisco-hdlc = { off on 10 3 }
fddi = { single off }
sonet = { "" "" 1 sonet internal-oscillator 0 207 }
hssi = { 0 16-bit }
ether = { autonegotiate }
hippi = { 1 32 no-mode 999999 4 incremental 5 300 10 10 03:00:0f:c0 disabled
super>
super> list fddi
single-dual = single
optical-bypass = off
super>
As you can see, the single-dual field is preset to single and the optical-bypass
is preset to off. The following sets the FDDI interface 0 to DAS and saves the
new settings:
super> set single-dual = dual
super> cd ..
port_num = 0
cisco-hdlc = { off on 10 3 }
fddi = { dual off }
sonet = { "" "" 1 sonet internal-oscillator 0 207 }
hssi = { 0 16-bit }
ether = { autonegotiate }
hippi = { 1 32 no-mode 999999 4 incremental 5 300 10 10 03:00:0f:c0 disabled
super> list fddi
single-dual = dual
optical-bypass = off
super>
super> write
CARD/0 written
super>
You have to go through the same procedure again and replace references to
port 0 with port 1.
Use shto return to the operating system of the GRF and grresetthe card, or
use exitto log off from the router.
Once you get more familiar with the GRF, you may prefer the following quick
way:
super> read card 0
CARD/0 read
super> set port 0 fddi single-dual = dual
super> set port 1 fddi single-dual = dual
124
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
super> write
CARD/0 written
super>
Optical bypass
Optical bypass capability has to be provided externally. The FDDI face plate
has a six-pin DIN connector to directly attach a single bypass switch.
Y-cable adapter. The Y-cable is required to reconcile control pin assignments
between the GRF and the external switch module. Through the Y-cable, an
optical bypass switch module attaches to a pair of media interface connectors
on the FDDI card.
A bypass switch allows the GRF to remove itself from the dual ring during a
failure or maintenance without causing the ring to "wrap" at upstream and
downstream neighbors. Should a GRF failure occur, the bypass switch
connects upstream and downstream neighbors on both the primary and
secondary rings, and allows the GRF node to remove itself from the ring
gracefully, while still retaining ring continuity.
A node failure without a bypass switch causes the dual ring to "wrap". A
wrapped ring absorbs the secondary ring into the primary ring and no longer
has a backup ring.
Ring 1
FDDI
Switch Module
media card
Ring 1
FDDI
Switch Module 1
media card
Ring 2
Switch Module 2
Figure 44. Optical Bypass Switch Attachments
Dual homing
Dual homing provides redundant connectivity between an FDDI media card
and a single ring.
Configuration of IP-Forwarding Media Cards
125
Download from Www.Somanuals.com. All Manuals Search And Download.
Configure the FDDI media card for dual attach, but use two single attach
can be on either one or two FDDI concentrators on the ring.
A0
B0
A1
B1
M
M
Concentrator
Ring
FDDI
A0
B0
A1
B1
M
M
Concentrator 2
Concentrator 1
Ring
FDDI
Figure 45. Dual Homing Configurations
4.4.1 Separate Networks versus Bridging
With all that said, it should be very clear now that every port needs to be
attached to a distinct network, either physical or logical (divided by subnet
masks). This gives you distinct paths to every network and requires setting up
routes.
If you are running a flat FDDI backbone and have the need to connect all four
ports into it, you must configure the card’s interfaces to use transparent
bridging, thus "bundling up" the bandwidth of the four interfaces.
The actual steps to implement bridging will be covered later; refer to Section
4.4.2 Naming the FDDI Interfaces
Each interface may be named or numbered in four different ways:
• By its physical location on the FDDI card
• By a site-specified SAS/DAS setting name in the Card profile, singleor
dual
126
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
• By a logical interface number assigned after the SAS/DAS settings are
numbered (used in the /etc/grifconfig.conf file)
• By a unique IP address assigned to each logical interface
Figure 46 shows files where various numbers are used to configure the
interfaces on an FDDI media card.
Card
face plate
numbers
SAS/DAS
options in
Card profile
Logical interface
numbers in
grifconfig.conf
IP addresses
used in
grifconfig.conf
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
A0
B0
A1
B1
0
1
2
3
4 single attach
2 dual attach
FDDI
FDDI
FDDI
FDDI
A0
B0
A1
B1
0
1
2
3
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
A0
B0
A1
B1
0
1
2
3
xxx.xxx.xxx.xxx
1 dual
2 single attach
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
xxx.xxx.xxx.xxx
A0
B0
A1
B1
0
1
2
3
2 single
1 dual attach
xxx.xxx.xxx.xxx
Figure 46. Assigning Numbers to FDDI Interfaces
4.4.3 Physical Interface Numbers
The physical interface number identifies the specific FDDI fiber optic
attachment component according to its location on the media card; it may be
from 0–3.
Starting at the top of the media card, each physical interface is numbered
consecutively, beginning with 0, as shown in Figure 47 on page 128.
Configuration of IP-Forwarding Media Cards
127
Download from Www.Somanuals.com. All Manuals Search And Download.
Port
SNMP
0
1
2
3
1
2
3
4
A0
A1
B0
B1
FDDI
media
card
Figure 47. Physical Interface Numbering on the FDDI Media Card
The diagram shows physical interface numbering to be 0-based (0–3),
whereas SNMP numbering is 1-based (1–4).
4.4.4 GRF Interface Name
The GRF interface name has five components that describe an individual
FDDI interface in terms of its place on the media card and in the GRF router.
interface names are case sensitive and that you must use all lowercase.
g f 0 x y
1st:
always "g" for GRF
2nd:
3rd:
4th:
5th:
media type, f (FDDI)
chassis number, always "0" (zero)
slot number in hex
logical interface number in hex
Figure 48. GRF Interface Name for FDDI Interfaces
The interface name and IP address are specified in the /etc/grifconfig.conf
file.
4.4.5 Configuration Files and Profiles
Following are the steps to configure FDDI cards. They are listed here
complete, although you can bring up an FDDI connection with a subset. For
detailed information, see GRF Configuration Guide 1.4, GA22-7366.
Proceed as follows:
1. Identify each logical interface.
128
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Edit /etc/grifconfig.conf to identify each logical interface by assigning:
• An IP address
• The GRF interface name
• A netmask, as required
• A destination or broadcast address, as required
• An MTU, if needed
2. Specify FDDI card parameters in the Card profile. All but the first two are
optional and default to the most common settings, so normally you should
be just fine omitting this step.
• Specify SAS and DAS settings as singleor dual, with single being the
default.
• Manually enable optical bypass onor offwith off being the default.
• Specify ICMP throttling settings.
• Change run-time binaries.
• Change dump variables.
3. Load profile (optional).
Global executable binaries are set at the Load profile in the hw-table field.
These only change when you want to execute new run-time code in every
FDDI card.
If you want to change the run-time code in one FDDI card (per physical
interface), make the change in the Card profile.
4. Dump profile (optional).
Global dump settings are in the Dump profile. These settings are usually
changed only for debug purposes. The keep-count field specifies how
many dumps are compressed and stored at one time for each media card.
The default setting is zero (0), which actually stores two dumps per day
(the current dump and the first dump of the day). Use caution if you
change the recommended default.
If you want to change dump settings for one FDDI card (per physical
interface), make the change in the Card profile, in the dump/config field.
4.4.6 Assign IP Addresses - grifconfig.conf
Edit the /etc/grifconfig.conf file to assign an IP address to each logical FDDI
interface. You can also provide other information about the logical IP network
to which that interface is physically attached, or specify a different MTU in the
arguments field, for example.
Configuration of IP-Forwarding Media Cards
129
Download from Www.Somanuals.com. All Manuals Search And Download.
The arguments field is also used to specify ISO when an ISO address is
being added to an interface’s IP address. Specify the MTU value as mtu xxxx.
Leave the arguments field blank if you are not using it.
The following excerpt from our /etc/grifconf.conf file shows the format of an
entry:
#name address
gf000 10.2.1.15
gf001 10.3.1.16
gf002 10.4.1.17
netmask
broad_dest
arguments
mtu 4352
255.255.255.0 -
255.255.255.0
255.255.255.0
As you can see, we used three interfaces and left out the specification for the
MTU size on the last two, as 4352 is the default for FDDI.
4.4.7 Specify FDDI Card Parameters
The following steps are, as mentioned, optional, and in an SAS environment
without optical bypass you can safely use the defaults.
If there is a need to set ports from SAS to DAS, refer to GRF Configuration
DAS” on page 123 for quick help.
4.4.8 Installing Configurations or Changes
In the command line interface (CLI), which is the working environment on the
GRF with the super>prompt, use the setand writecommands to install
configuration parameters onto the media card.
If you apply changes to files in the /etc directory, do not forget to issue
grwrite -v to have these changes written to flash, so that they are still in
effect after a rebootof the GRF.
Additionally, when you enter configuration information or make changes, you
must also reset the media card for the changes to take place. Enter
grreset <slot_number> to do so.
Hint: We found that some changes did not go into effect until the GRF was
rebooted, although the documentation indicated otherwise. So do not hesitate
to rebootif things seem not to work as they should.
130
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
This completes the procedure to configure FDDI cards, and as with the ATM
card, we would like to introduce some of the maintcommands we found to be
useful.
4.4.9 Some maint Commands for the FDDI Media Card
The maintcommands display a range of information about the FDDI media
card. The FDDI card has an individual processor for the transmit and receive
side, and two sets of maintcommands. One set, the maint 1commands,
covers the receive (RX) side, while the second set, the maint 70commands,
covers the transmit (TX) side.
To use maint, follow these steps:
• First, switch to the maintprompt with the grrmbcommand. A new prompt,
GR 66>, will appear.
• Then change the prompt port to the FDDI media card you are working
with. For example, if you are working with a card in slot 0, enter port 0 at
the GR> 66 prompt.
The following message is returned along with the changed prompt:
Current port card is 0
GR 0>
• To leave any maintprompt and return to the shell, enter quit.
Following are just a few maintcommands we have found useful; of course,
your needs may vary.
To obtain a list of FDDI CPU0 maintcommands, enter maint 1. Enter the
maint 70command to switch to the set of CPU1 commands. The maint 3
command returns configuration information for each interface; to list statistics
per FDDI interface, use maint 4; to list switch interface statistics, enter
maint 5.
Clear all statistics with the maint 7 command. Display the current ARP table’s
content with maint 70 8. Reset individual FDDI interfaces using maint 12and
the physical interface number (0–3); maint 12 0will reset port 0.
Configuration of IP-Forwarding Media Cards
131
Download from Www.Somanuals.com. All Manuals Search And Download.
4.4.10 Using grrt to Display the Route Table
Use the grrt -S -p<slot>command to display the current contents of the
FDDI card’s route table, as follows:
# grrt -S -p0
default
0.0.0.0
10.1.1.0
10.1.1.1
1
192.168.4.137
0.0.0.0
inx 0
inx 0
ga010
ga010
ga010
ga0180
bg0
RMS
DROP
FWD
LOCAL
BCAST
LOCAL
BFWD
BLOCAL
BBCAST
FWD
RMS
RMS
FWD
FWD
LOCAL
BCAST
MCAST
MCAST
MCAST
BCAST
255.255.255.255 8
255.255.255.0
16 0.0.0.0
255.255.255.255 15 0.0.0.0
255.255.255.255 14 0.0.0.0
255.255.255.255 19 0.0.0.0
10.1.1.255
10.1.2.1
10.10.1.0
10.10.1.13
10.10.1.255
127.0.0.0
127.0.0.1
192.168.4.0
192.168.13.0
192.168.14.0
192.168.14.4
255.255.255.0
7
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
127.0.0.1
255.255.255.255 6
255.255.255.255 5
bg0
bg0
255.0.0.0
4
inx 0
inx 0
inx 0
ga010
gt030
gt030
gt030
inx 0
inx 0
inx 0
inx 0
255.255.255.255 3
255.255.255.0
255.255.255.0
255.255.255.0
10 0.0.0.0
17 10.1.1.2
13 0.0.0.0
255.255.255.255 12 0.0.0.0
192.168.14.255 255.255.255.255 11 0.0.0.0
224.0.0.0
224.0.0.0
224.0.0.0
240.0.0.0
255.0.0.0
2
2
0.0.0.0
0.0.0.0
0.0.0.0
0.0.0.0
255.255.255.255 2
255.255.255.255 255.255.255.255 9
132
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.4.11 Using grstat to Display GRF Statistics
Use the grstat -w70 all <interface>command to display the current statistics
of the FDDI card’s IP stack. The following is an actual screen shot:
# grstat all gf001
gf001
ipstat
count description
2958246 total packets received
3 packets dropped
2958226 packets forwarded normally
17 packets handled by the card
ipdrop
last
source addr
10.10.1.10
last
count
3
dest addr reason
192.168.14.1 TTL expired
icmperr
last
count type
icmpin
last
code
last
val
last
source addr
last
dest addr error
last
last
last
count type
icmpout
code
source addr
dst addr
last
code
INTRANS
last
source addr
10.10.1.13
last
dst addr
10.10.1.10
count type
3 TIMXCEED
#
4.5 HIPPI Configuration
This section provides the steps to configure the High Performance Parallel
Interface (HIPPI) media card of a GRF.
4.5.1 Introduction to HIPPI
HIPPI poses interesting configuration problems because of the number of
ways HIPPI connections can be established. Several addressing schemes
can be used, depending upon how a site needs to organize and connect
equipment to support a range of user needs.
Not only are there several addressing schemes, but a HIPPI media card can
be configured to process all of them. The HIPPI media card is capable of
handling both HIPPI-SC switching protocol and IP packet routing, and, based
on information field (I-field) indicators, can dynamically alternate between
Configuration of IP-Forwarding Media Cards
133
Download from Www.Somanuals.com. All Manuals Search And Download.
these modes. Hosts must pass on the appropriate information for the GRF
media cards and other HIPPI devices to operate in the desired way.
HIPPI offers many configuration options. The ANSI HIPPI standards and
RFCs describe implementation details, that support source routing, logical
addressing, IP routing, and raw (switch) mode operations.
4.5.1.1 Connection Processing
The GRF processes connections; it does not process data. It accepts data
and establishes a connection point to which it can transfer data.
The HIPPI media card establishes connections and transfers packets. A
HIPPI media card processes one HIPPI connection at a time. It does not
begin another process until the first connection completes.
There are two types of connections:
• IP connections
• Raw connections
In internetworking, the main difference between the two connections is that
data from a HIPPI host can be transferred to any other IP-capable media via
IP routing. Raw mode is HIPPI to HIPPI, and is only used to transfer data
from one HIPPI device to another HIPPI device. The HIPPI I-field tells the
media card which type of connection is being requested.
So the toughest step in the configuration of a HIPPI media card is to compose
the correct HIPPI I-field.
4.5.1.2 Starting a HIPPI Connection
The HIPPI standard requires that a HIPPI connection be established between
a HIPPI source and a HIPPI destination before any data is transmitted. Every
connection "request" signal sent to a HIPPI media card is accompanied by a
HIPPI I-field.
The sequence that establishes a HIPPI connection is as follows:
1. The HIPPI source activates the REQUEST line to a destination while at
the same time placing a 32-bit word, (the I-field), on the data lines.
2. The HIPPI destination sees the REQUEST signal, reads the I-field, and
accepts the connection by activating its CONNECT line back to the
source.
Data coming from an external HIPPI I/O channel may be formatted into
standard IP packets. Embedded in the front of each IP packet is an IP
134
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
header. The media card reads the header only if told to do so by information
in the HIPPI I-field. If the I-field tells the card to read the IP header, then an IP
connection is established.
4.5.1.3 How the I-field is Used
The I-field tells the GRF how to process the connection, and where to send
control information is in the leftmost 8 bits, addressing takes up the other 24
bits.
Addressing
12 11
Control
Logical Address Host B
Logical Address Host A
L
31
L
VU W D PS
C
24 23
0
= Locally administered bit (L=0)
VU = Vendor unique bits (not used)
W
D
= Double wide bit (not used)
= Direction bit
PS = Path selection bits
= Camp-on bit
C
Figure 49. HIPPI I-Field Components
4.5.1.4 Camp-on Bit
The camp-on (C) bit is set to 1 or 0, which translates to on or off. The HIPPI
source host uses camp-on to tell a HIPPI device (switch or router) to wait until
a busy destination becomes available and to keep trying to make the
connection.
4.5.1.5 Path Selection Bits
The path selection (PS) bits have four settings directing the HIPPI media card
how to read the 24-bit destination address.
00 Source Routing
When the path selection is set to 00, that is source routing, the HIPPI source
has selected the exact route to the destination, because the HIPPI host
knows the specific path through some number of devices (switches or
routers) to the endpoint host. In fact, the rightmost bits of the I-field (bits
0–23) contain the physical output slots for each switch or router in the path. In
source routing, a return path is automatically "built" by the network device at
each point of data transfer.
Configuration of IP-Forwarding Media Cards
135
Download from Www.Somanuals.com. All Manuals Search And Download.
01 Logical Address
When PS is set to 01, that is, logical address, the host does not know or want
to specify the actual physical route to the target endpoint. The host supplies a
logical address for the endpoint host. In this case, all switches and the GRF
must be programmed to route the connection.
The structure of the I-field is different when logical addressing is used. The
24-bit destination addresses are divided into two 12-bit fields. The rightmost
12 bits of the I-field contain the logical address of the target endpoint, the
leftmost 12 bits contain the logical address of the source host.
Each switch or router has to look up the destination host’s logical address in
its own tables and decide which of its output slots it will transfer the data to. In
the table, there may be several output slots that could be used in the route.
PS set to 01 specifies that the first entry in the list of possible output ports
must be used.
IP connection: An I-field containing a special logical address and a PS=01 or
PS=11 setting is used to establish an IP connection with a GRF HIPPI card.
In the I-field, the PS bits are set to 01 or 11, and bits 0–11 contain a
designated destination logical address (0xfc0) that is mapped to slot 64.
After the IP connection is established, the data packets arriving at the GRF
are routed to the appropriate output slot using the default or a site-specified
IP destination logical address. Therefore, data is transferred using a table
based on IP addresses rather than HIPPI addresses.
IP routing is discussed later in this chapter. Example 2 describes how to
configure logical addresses.
10 Unused
This PS setting is not currently defined for use by the HIPPI-SC standard.
11 Logical Address
The PS=11 setting is the same as the 01 setting, except that the switch or
GRF can choose any output slot from a list of valid slots for this logical
address. (Remember, with PS=01, the first port in the list is used.)
Again, the host does not know the route and instead supplies a logical
address for the endpoint host. All switches and the GRF must be
programmed to route the connection.
136
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.5.1.6 Direction Bit
HIPPI hosts set the direction bit (D). This bit determines how a switch or
router reads the 24-bits of destination address information. Figure 49 on page
135 and the previous descriptions of source routing and logical addressing
have the destination address information organized as if the host has set the
destination bit to 0.
If the destination bit is set to 1, the information in the 24-bit destination
addressing is read starting from the left, and the logical addresses of host A
and B change places. The destination bit makes it easy for source and
destination hosts to reply and reverse transfer data to one another, or it can
serve as a means to trace where a connection originates.
4.5.1.7 L, VU, and W Bits
The L and VU control bits are not used by the GRF, and the HIPPI media
cards do not support double-wide HIPPI connections.
It will reject the connection if the W bit is set on.
4.5.1.8 IP Routing
In an IP connection, data coming from a HIPPI I/O channel is formatted into
standard IP packets. Embedded in the front of each IP datagram is the IP
header. The media card reads the header only if the information in the HIPPI
I-field indicates an IP connection.
The header contains the Internet address of the host sending the datagram
and the Internet address of the target IP media host for which the datagram is
intended. This target host can be attached to any media that supports IP, or
be reached via that attached media. Because the GRF is a router, it creates
and updates an IP routing table that describes paths to destination
addresses. This is the basis of IP routing.
Each GRF media card holds a copy of this IP routing table. When processing
an IP connection, a HIPPI media card “opens” the datagram’s header, reads
the address of the target host, and decides to which GRF media card the IP
datagram is transferred.
4.5.1.9 IP Routing and the I-field
A HIPPI host’s I-field table can be used to direct the GRF HIPPI media card to
do IP routing.
In the I-field, the PS bit needs to be set to 01 or 11 and a designated
destination logical address (0xfc0) must be placed in bits 0–11. The mapping
Configuration of IP-Forwarding Media Cards
137
Download from Www.Somanuals.com. All Manuals Search And Download.
of this address to slot 64 in the file /etc/grlamap.conf indicates the IP
connection to the receiving media card and causes it to read the IP header.
The logical address 0xfc0 is preset as a default in the logical address table.
This logical address maps to the nonexistent slot 64. HIPPI cards are
programmed to accept a connection and extract the destination IP address
from the first datagram’s header when they look up an address that points to
slot 64.
A site can set any logical address to designate an IP connection if it maps this
address to destination slot 64 in the /etc/grlamap.conf file. As noted, the
address 0xfc0 is preset in this file.
4.5.1.10 Using the IP Address
The receiving HIPPI card looks up the destination IP address in the routing
table, finds the corresponding GRF output media card, and forwards the
datagram across the crosspoint switch to it.
The output media card just forwards the data unless it is a HIPPI card
connected to a HIPPI switch. In this case, the output HIPPI card needs an
I-field to forward when it requests a HIPPI connection to the switch. You need
to supply the I-field by editing the /etc/grarp.conf file.
4.5.1.11 HIPPI in a Bridging Environment
HIPPI does not bridge! On the GRF, you can route IP to a bridge group from a
HIPPI routing domain, but there is no encapsulated bridging across a HIPPI
connection.
4.5.1.12 MTU
The HIPPI MTU is 65280 bytes. A smaller MTU can be specified in the
/etc/grifconfig.conf file in the arguments field.
4.5.1.13 ARP
HIPPI ARP tables for remote devices connected to GRF HIPPI interfaces are
manually configured. Entries in the /etc/grarp.conf file map an IP address to a
32bit HIPPI I-field.
4.5.2 HIPPI Configuration Options
GRF Configuration Guide 1.4, GA22-7366 gives examples how to set up
various configurations by programming a HIPPI media card and, when
necessary, a HIPPI host I-field. Most of them deal with HIPPI to HIPPI
configurations. HIPPI IPI-3 and the IBM HIPPI connection option, H0 HIPPI,
are covered, too.
138
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Section 7.3, “HIPPI Backbone Connection” on page 227, describes the steps
to configure the GRF’s HIPPI media card to do IP forwarding, so you have to
refer to the Ascend documentation if you need to set up a different
configuration.
4.5.3 Physical and Logical Interfaces
The HIPPI media card provides a single duplex attachment and operates at a
speed of 100 MB/s. It requires a pair of 100-pin copper cables to connect to
another HIPPI device.
Physical Interfaces
The upper HIPPI interface (RCV or DESTINATION interface) receives data
from a host. The lower interface (XMT or SOURCE interface) transmits data
to a host.
Logical Interfaces
A logical interface is configured by its entry in the /etc/grifconfig.conf file,
where it is assigned an IP address and netmask. A logical interface is
uniquely identified by its HIPPI interface name.
Interface Name
naming conventions on the HIPPI card.
g h 0 x 0
1st:
2nd:
3rd:
4th:
5th:
always "g" for GRF
media type, h (HIPPI)
chassis number, always "0" (zero)
slot number in hex
logical interface number in hex, always "0" (zero)
Figure 50. Components in the HIPPI Interface Name
The interface name is used in the /etc/grifconfig.conf file to specify an IP
interface. The following is an entry from our actual configuration:
#name address
gh000 10.50.1.2
netmask
255.255.255.0 -
broad_dest
arguments
mtu 65280
Note: Interface names are case sensitive. Always use lower case letters
when defining interface names.
Configuration of IP-Forwarding Media Cards
139
Download from Www.Somanuals.com. All Manuals Search And Download.
4.5.4 Configuration Files and Profiles
The following are the steps to configure HIPPI cards. For detailed
information, see GRF Configuration Guide 1.4, GA22-7366.
1. Identify each logical interface.
Edit the /etc/grifconfig.conf to identify each logical interface by assigning:
• An IP address
• The GRF interface name
• A netmask, as required
• A destination or broadcast address, as required
• An MTU, if needed
2. Check the I-field shift in the System profile.
There is one configurable item in the System profile for HIPPI. By default,
the I-field shift is set to 5 bits.
3. Specify HIPPI card parameters in the Card profile (all optional):
• Specify ICMP throttling settings.
• Change run-time binaries.
• Change dump variables.
4. Load profile (optional).
Global executable binaries are set in the load profile in the hw-table field.
These only change when you want to execute new run-time code in every
HIPPI card.
If you want to change the run-time code in one HIPPI card (per physical
interface), make the change in the Card profile, in the load field.
5. Dump profile (optional).
Global dump settings are in the Dump profile. These settings are usually
changed only for debug purposes.
The keep-count field specifies, how many dumps are compressed and
stored at one time for each media card. The default setting is zero (0),
which actually stores two dumps per day (the current dump and the first
dump of the day). Use caution if you change the recommended default.
If you want to change dump settings for one HIPPI card (per physical
interface), make the change in the Card profile, in the dump/config field.
140
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.5.5 Installing Configurations or Changes
In the command line interface (CLI), which is the working environment on the
GRF with the super>prompt, use the setand writecommands to install
configuration parameters onto the media card.
If you apply changes to files in the /etc directory, do not forget to issue
grwrite -v to have these changes written to flash, so that they are still in
effect after a rebootof the GRF.
Additionally, when you enter configuration information or make changes, you
must also reset the media card for the change to take place. Enter
grreset <slot_number> to do so.
Hint: We found that some changes did not go into effect until the GRF was
rebooted, although the documentation indicated otherwise. So do not hesitate
to rebootif things seem not to work as they should.
This completes the procedure to configure HIPPI cards, and to close this
chapter, we would like to introduce some of the maint commands we found to
be useful.
4.5.6 Some maint Commands for the HIPPI Media Card
The maint commands operate on the SP Switch control board and require the
card’s slot number, which the grcardcommand provides.
Prepare to use maintwith the following steps:
• First, switch to maint’s GR 66>prompt with the command grrmb.
• At the GR66>prompt you have to change to the prompt for the specific card.
For a card in slot 0, this would be the command port 0.
• A new prompt, along with the following message, will appear:
Current port card is 0
GR 0>
• Finally, at this prompt maintcommands can be typed in.
• To leave the GR 0>prompt, enter quit.
The following maint commands were most useful for us:
To obtain a list of maint commands, use maint 1. To see IP statistics, use
maint 133. Observe IP routing statistics with maint 130 13and control switch
error counts with maint 141. Peek at the ARP table entries with maint 156 and
get the IEEE address with maint 128.
Configuration of IP-Forwarding Media Cards
141
Download from Www.Somanuals.com. All Manuals Search And Download.
4.6 Configuring Bridging
This Chapter describes the GRF bridging implementation and provides
configuration information.
4.6.1 GRF Bridging Implementation
The GRF implements IEEE 802.1d transparent bridging on GRF Ethernet and
FDDI interfaces, and on ATM OC-3c interfaces using RFC-1483
encapsulated bridging over PVCs.
Transparent bridging provides a mechanism for interconnecting stations
attached to physically separate Local Area Networks (LANs) as if they are
attached to a single LAN. This interconnection happens at the 802 MAC layer
and is transparent to protocols operating above this boundary in the Logical
Link Control (LLC) or Network layers. Participating stations are unable to
identify that peers are on anything other than the directly attached physical
media.
The GRF implementation consists of the transparent bridging function
described in 802.1d, and does not include any capability for Source Route or
Source Route Transparent (SRT) bridge operation.
Summary of bridging features:
• Bridging on FDDI, Ethernet, and ATM OC-3c per the 802.1d standard
• Participation in 802.1d spanning tree protocol
• Layer-2 transparent bridging of MAC frames through the GRF from one
interface to another
• Conversion of frames between Ethernet and FDDI formats as necessary
• Fragmentation of IPv4 frames if necessary
• Simultaneous bridging and routing over the same interface (a GRF
interface participating in a bridge group can still route normally)
• Routing IP to or from a bridge group from any GRF media
• RFC-1483 encapsulated bridging over ATM OC-3c PVCs with either
VC-based multiplexing or LLC encapsulation
• Up to 16 bridge groups per GRF
• Up to 32 GRF interfaces per bridge group
142
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.6.2 Simultaneous Routing and Bridging
Ascend’s transparent bridging does not preclude the use of IP packet routing
on the same physical interface.
Bridging as well as IP version 4 (IPv4) routing can both be enabled on the
same physical interface. In this circumstance, the GRF exchanges traffic
between bridging domains and routing domains that exist on the same
physical media.
A GRF interface may simultaneously bridge layer-2 frames and route layer-3
packets. That is, it can forward frames destined to a system attached to
another LAN at the MAC layer, but still receive IP packets destined for a
remote system attached to a non-broadcast GRF interface and route those
packets at the IP layer.
This unique capability eliminates the need for separate pieces of routing
equipment to transport packets between domains.
To perform the simultaneous functions, the GRF bridging interface examines
the destination MAC address of each arriving frame. If the address is other
than a GRF MAC address for any interface participating in the assigned
bridge group, the packet is submitted to the bridging engine for forwarding.
When the MAC address is a GRF MAC address, the packet is forwarded to
the GRF protocol forwarding engine for routing at the protocol layer.
4.6.3 Configuration Options
The GRF supports the configuration items specified in IEEE 802.1d. A GRF
functioning as a bridge will interoperate with other bridges, including
equipment of vendors in conformance with the IEEE 802.1d standard, to
allow forwarding of frames across multiple LAN hops.
Additionally, the GRF supports 16 active IEEE 802.1 bridge groups, and will
separate traffic between groups. For example, on a GRF with six attached
FDDI rings, rings A, B, and C could form one bridge group, rings D and E
could form a second bridge group, and ring F could stand alone, using only IP
routing for its packets.
A GRF functioning as a bridge will also interoperate with other bridges to
forward frames from one bridge to the other over ATM. This will allow two
independent bridged LANs at remote locations to function as one logical
network transparently connected by ATM. This encapsulated bridging follows
the Internet standard specification in RFC-1483.
Configuration of IP-Forwarding Media Cards
143
Download from Www.Somanuals.com. All Manuals Search And Download.
4.6.4 Interoperability
The following table gives an overview of the GRFs interoperability features.
FDDIFrame forwarding is compatible with any station sending and receiving
FDDI LLC frames.
EthernetFrame forwarding is compatible with any station using either DIX
Ethernet or IEEE 802.3 frames.
ATM OC-3cFrame forwarding is compatible with any remote bridge using
RFC-1483 bridging encapsulation.
Spanning treeGRF transparent bridging will interoperate with any other
bridge (including other GRFs) compliant with the IEEE
802.1d spanning tree protocols.
4.6.5 Spanning Tree
The GRF implementation supports the full Spanning Tree Algorithm specified
in the IEEE 802.1d standard.
Using the spanning tree, network topologies can contain cycles that can be
used as redundant or backup links. The spanning tree controls the bridge’s
flow of traffic over all potential links to prevent packet storms (bridges
repeating a packet or packets to each other, without end).
4.6.6 Bridge Filtering Table
Media card bridge interfaces forward new MAC source addresses to the
operating system for insertion in the global bridge filtering table that is
maintained on the control board. Each bridging media card type (FDDI,
Ethernet, and ATM OC-3c) also has a copy of this table.
Bridge interfaces also "age" entries according to a site-specified timeout
value. When no activity is associated with a MAC address for the specified
timeout interval, the interface sends the operating software a delete request
and the address is removed first from the global bridge filtering table and
then, via update packets, from the media cards’ tables.
The timeout is specified in seconds in the /etc/bridged.conf file.
4.6.7 Fragmentation
IPv4 frames are fragmented as necessary, as when bridging a FDDI frame of
more than 1500 bytes to an Ethernet interface.
144
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
A frame may be too large for the maximum transmission unit (MTU) of the
sending GRF interface. One example is when forwarding a 4500-byte frame
from FDDI to an Ethernet interface with an MTU of 1500 bytes. The GRF
bridge will attempt to break such a frame into fragments that will fit the
sending interface. This is possible if the frame contains an IP datagram; then
the GRF may use the fragmentation rules of IP to split the frame. Otherwise,
the GRF must drop the frame.
4.6.8 Spamming
Spamming is when a bridging interface forwards a frame to all active
interfaces in the bridge group. On the GRF, spamming is done when a
broadcast address is received or when a frame arrives whose destination
address is not in the bridge filtering table.
4.6.9 Bridging Components
The following subsections discuss the various bridging components.
4.6.9.1 Bridging Daemon – Bridged
The bridging daemon, bridged, is used to configure and manipulate bridging
interfaces on the GRF. It operates the spanning tree algorithm specified in
IEEE 802.1d and ensures interoperability with other 802.1d bridges.
Bridged reads the /etc/bridged.conf configuration file to build an initial
bridging topology.
Bridged is started by the system script /etc/grstart. This script monitors the
bridged daemon and restarts it if it stops.
4.6.9.2 Configuration File – bridged.conf
The bridging configuration file is /etc/bridged.conf. A utility, bredit, is used to
access the file and create bridge groups and bridging settings.
Parameters in bridged.conf can be set to do the following:
• Name bridge groups
• Assign logical interfaces to a group
• Assign priority, starting state, root path cost, and forwarding addresses to
individual logical interfaces
• Assign hello time and forwarding delay values, priority, maximum age, and
discard addresses to individual groups
Configuration of IP-Forwarding Media Cards
145
Download from Www.Somanuals.com. All Manuals Search And Download.
4.6.9.3 Editing Utility – Bredit
The breditutility is used to access and edit the /etc/bridged.conf
configuration file.
At this point, bredit runs a script in which you are asked if you want to make
the changes permanent. The script also gives you the option of signaling
bridged to reread the updated file immediately. When this option is taken,
bridged restarts as if it were stopped and restarted for the first time. If you
change the file in vi but do not choose either of the script options, bredit
tells you that your changes were not committed.
If bridged is not running when bredit is used, the user is given the option of
saving changes to the configuration in /etc/bridged.conf so that the next time
bridged is started, the new changes take effect.
See an actual screen shot in Section 7.1.2, “ATM OC-3c Backbone - Using
4.6.10 Management Tools
A set of tools are provided to manage bridging, primarily through bridged.
Brief descriptions are provided here; more details are given in the GRF
Configuration Guide 1.4, GA22-7366.
These tools include the brstat command which displays relevant bridged
status and bridging information and the brinfocommand which displays
relevant kernel-based bridging information.
4.6.10.1 brstat
The brstatcommand provides a snapshot of state information directly from
bridged.
Output will look similar to the following:
Bridged Information:
Debug Level: 7, Trace Mask: 0xffffffff, Spanning Tree: Enabled
Log File: "/var/tmp/bridged.trace", Config File: "/etc/bridged.conf"
bridged started at: Wed Jun 17 14:13:16 1998
Bridge Group bg0
Spanning Tree:
Enabled
Designated Root: [me]
Bridge ID:
32768 00:c0:80:89:2d:f2
Root Port: None
Topology Change Detected: No
Root
Max Age: 20, Hello Time: 2, Forward Delay: 15
Bridge Max Age: 20, Hello Time: 2, Forward Delay: 15, Hold Time: 1
Path Desig Desig
Desig
146
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Interface Port ID Con State
Cost Cost Bridge
Port
--------- ------- --- ---------- ----- ----- ----------------------- -------
gf000
gf001
gf002
gf003
128 1
128 2
128 3
128 4
Yes Forwarding 10
Yes Forwarding 10
Yes Forwarding 10
Yes Forwarding 10
0
0
0
0
[me]
[me]
[me]
[me]
128 1
128 2
128 3
128 4
Bridge Group bg1
Spanning Tree:
Enabled
Designated Root: 32768 00:c0:80:84:8c:eb
Bridge ID: 32768 00:c0:80:96:38:68
Root Port: ga010, Root Path Cost: 10
Topology Change Detected: No
Root
Max Age: 20, Hello Time: 2, Forward Delay: 15
Bridge Max Age: 20, Hello Time: 2, Forward Delay: 15, Hold Time: 1
Path Desig Desig
Cost Cost Bridge
Desig
Port
Interface Port ID Con State
--------- ------- --- ---------- ----- ----- ----------------------- -------
*ga010
ga0180
128 1
128 2
Yes Forwarding 10
Yes Blocking 10
0
0
32768 00:c0:80:84:8c:eb 128 1
32768 00:c0:80:84:8c:eb 128 2
4.6.10.2 brinfo
The brinfocommand is used to retrieve bridging interface information for
administrative debugging and other situations where a simple checking of
bridge port information is needed.
The brinfocommand prints the list of bridge groups and bridge ports
(underlying interfaces) that are members of the specified groups. If no groups
are specified, all groups are reported on by default. Remember that brinfo
gets its information directly from the BSD kernel, whereas brstatgets its
information from bridged.
See the brinfooutput for the same configuration as used for the brstatoutput
before:
Bridged Daemon: Running
Bridge group name: bg0
Flags:(0x3043) up broadcast running
Ports: 4
Port gf000: State (0xf) Forwarding
Flags: 0xb043 up broadcast running link0 link1 multicast
Bridging media: fddi bpdu
MAC address: 0:c0:80:89:2d:f2
Port gf001: State (0xf) Forwarding
Flags: 0xb043 up broadcast running link0 link1 multicast
Bridging media: fddi bpdu
MAC address: 0:c0:80:89:2d:f3
Port gf002: State (0xf) Forwarding
Flags: 0xb043 up broadcast running link0 link1 multicast
Bridging media: fddi bpdu
MAC address: 0:c0:80:89:2d:f4
Port gf003: State (0xf) Forwarding
Configuration of IP-Forwarding Media Cards
147
Download from Www.Somanuals.com. All Manuals Search And Download.
Flags: 0xb043 up broadcast running link0 link1 multicast
Bridging media: fddi bpdu
MAC address: 0:c0:80:89:2d:f5
Bridge group name: bg1
Flags:(0x43) up broadcast running
Ports: 2
Port ga010: State (0x1) Running
Flags: 0xa043 up broadcast running link1 multicast
Bridging media: ethernet fddi bpdu
Max MTU: 4352
MAC address: 0:c0:80:f8:43:0
Port ga0180: State (0xf) Blocking
Flags: 0xa043 up broadcast running link1 multicast
Bridging media: ethernet fddi bpdu
Max MTU: 4352
MAC address: 0:c0:80:f8:44:80
4.6.11 Configuration File and Profile Overview
When a new GRF system is installed or a site upgrades to a bridging software
release, the /etc/bridged.conf file does not exist. The bridging daemon,
bridged, will not start without this file. The grstartprogram periodically
checks to see if the /etc/bridged.conf file exists; when it finds the file, grstart
then starts bridged. The following are the steps to configure bridging. For
more information, refer to GRF Configuration Guide 1.4, GA22-7366.
1. Create /etc/bridged.conf.
A template file for /etc/bridged.conf is provided in
/etc/bridged.conf.template. Copy the template file into /etc/bridged.conf.
2. Create bridge groups in /etc/bridged.conf
Run breditto create and name the bridge groups, and assign bridging
parameters to each.
3. Assign an IP address to each bridge group.
Edit /etc/grifconfig.conf to identify each bridge group by assigning:
• An IP address
• The GRF interface name
• A netmask, required
• A destination or broadcast address, as required
• An MTU value, if needed
4. Create ATM OC-3c PVCs for encapsulated bridges.
148
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
If you are going to configure an encapsulated bridge on an ATM circuit,
edit the /etc/gratm.conf file to create a PVC on the ATM OC-3c logical
interface.
5. Specify ARP service in the /etc/grarp.conf file, if needed.
4.6.11.1 Starting Bridged
The grstartprogram regularly checks for the presence of /etc/bridged.conf
and starts the bridged daemon when the file is found.
In the UNIX shell, copy the template for the bridging configuration file to
/etc/bridged.conf:
# cp /etc/bridged.conf.template /etc/bridged.conf
You will see this message:
# Jun 14 23:56:53 grf16 root: grstart: starting bridged.
Now you can use the breditcommand to open bridged.conf and edit it:
# bredit
#
# NetStar $Id
#
# Configuration file for Bridge Daemon (bridged).
#
# Note: bridged will not start if it finds an error while
# trying to parse this file. Use the "-d" option on the
# command line with bridged to find proximity of the offending
# line.
#
If you have not already copied /etc/bridged.conf, or if grstartcannot find the
file, you see this message:
Could not find default config file : /etc/bridged.conf
This seems to be the first time bridged is being configured.
Do you want to use the template configuration file ? [y/n] [n]
If you respond “yes” (y) to the breditquery, the file is opened for you in the
UNIX vi editor. Comments in the file describe how to configure a group and its
members, and define the bridging options and any defaults.
Exit the file with the vi command, :q. You do not need to write the file, bredit
will do that.
Configuration of IP-Forwarding Media Cards
149
Download from Www.Somanuals.com. All Manuals Search And Download.
4.6.11.2 Creating Bridge Groups in bridged.conf
The only required parameter is the list of FDDI, ATM or Ethernet interfaces
you are assigning to the group.
The format of the group list is:
bridge_group bgA {
port interface_name;
};
where interface_name is in the standard GRF interface name format gx0yz
that uniquely describes a logical FDDI, Ethernet, and ATM OC-3c interface
A simple bridge group entry is:
bridge_group bg0 {
port ge003;
port gf010;
};
g x 0 y 0
1st:
2nd:
3rd:
4th:
5th:
always "g" for GRF
media type, a (ATM), f (FDDI) or e (Ethernet)
chassis number, always "0" (zero)
slot number in hex
logical interface number in hex, always "0" (zero)
Figure 51. Interface Name for FDDI, Ethernet and ATM OC-3c Interfaces
4.6.11.3 Assign IP Addresses to Bridge Groups
Assign an IP address to each bridge group in the /etc/grifconfig.conf file:
#name address
#
netmask
broad_dest argument
bg0
bg2
192.168.1.1
192.168.2.1
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
gf022 222.222.80.2
ga030 222.222.02.4
Note: A netmask entry is required for each bridge group.
4.6.11.4 Create an ATM PVC for an Encapsulated Bridge
Bridging over ATM can be configured in two ways:
• LLC Encapsulation (RFC 1483, section 4)
150
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
• VC-Based Multiplexing (RFC 1483, section 5)
When LLC Encapsulation is used, a single PVC is configured to carry all
bridged traffic. The same PVC can also carry nonbridged traffic such as
routed IP datagrams.
When VC-Based Multiplexing is used, multiple PVCs are defined for the
logical interface. Each PVC carries a specific type of traffic. For example, one
PVC carries Ethernet while another carries FDDI.
Configuration in gratm.conf
The next three steps describe ATM bridging configuration requirements and
options. Examples of configured PVCs follow.
1. In the Traffic Shaping section of the /etc/gratm.conf file, set traffic shaping
name and quality of service (qos) parameters. Choose a name of your
choice for each type of service that will be assigned:
# Traffic shaping parameters
# Lines beginning with the keyword "Traffic_Shape" define
# traffic shapes which may be used to configure the performance
# characteristics of ATM Virtual Circuits.
#
Traffic_Shape name=myown_high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
2. To configure a logical interface for bridging, you create an Interface entry
in the Interface section of the /etc/gratm.conf file. This entry must include
the intended bridging method. Specify it with the bridge_method=
keyword.
Here is a sample Interface entry:
Interface ga030 traffic_shape=myown_high_speed_high_quality \
bridge_method=vc_multiplexed
There are two types of bridging methods to specify:
• VC-Based Multiplexing, bridge_method=vc_multiplexed
The configuration must include one or more PVCs for this interface
specified in the PVC section and defined with proto=vcmux_bridge.
• LLC Encapsulation, bridge_method=llc_encapsulated
The configuration must include one PVC for this interface specified in
the PVC section and defined with proto=llc,bridging. Media and
transmission restrictions can also be specified with this keyword.
3. One or more PVCs must be defined in the PVC section for each logical
interface specified for bridging in the Interface section.
Configuration of IP-Forwarding Media Cards
151
Download from Www.Somanuals.com. All Manuals Search And Download.
A bridging PVC is assigned a protocol value. This value must be
consistent with the bridging method defined for the logical interface.
Bridging PVCs are assigned either of the following protocol values:
1. proto=llc,bridging,xxxx
This type of PVC is used for logical interfaces defined with
bridge_method=llc_encapsulated, as well as logical interfaces not used
for bridging. This PVC uses LLC encapsulation for each Protocol Data
Unit (PDU).
xxxx represents a second protocol qualifier required for the proto=
parameter.
This PVC entry enables bridging on an LLC PVC:
PVC ga030 0/32 proto=vcmux_bridge,bpdu \
traffic_shape=high_speed_high_quality
2. proto=vcmux_bridge,yyyy
This type of PVC is used only for logical interfaces defined with
bridge_method=vc_multiplexed. The PVC carries bridged traffic of a
single type.
yyyy represents a second protocol qualifier required for the proto=
parameter. This qualifier defines the type of bridged traffic the PVC can
carry. Traffic types include:
• proto=vcmux_bridge,ether_fcs
Each PDU is an Ethernet frame, including a Frame Check
Sequence (fcs).
• proto=vcmux_bridge,ether_nofcs
Each PDU is an Ethernet frame, without fcs.
• proto=vcmux_bridge,fddi_fcs
Each PDU is an FDDI frame, including fcs.
• proto=vcmux_bridge,fddi_nofcs
Each PDU is an FDDI frame, without fcs.
• proto=vcmux_bridge,bpdu
Each PDU is an 802.1d Bridging Protocol Data Unit (BPDU).
152
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Following are some PVC configuration examples:
LLC Encapsulated:
# Traffic shape
Traffic_Shape name=high_speed_high_quality peak=155000
sustain=155000 \
burst=2048 qos=high
# Logical interface
Interface ga030 traffic_shape=high_speed_high_quality \
bridge_method=llc_multiplexed
# PVC
PVC ga030 0/32 proto=llc,bridging
Note: The single PVC defined here can carry any kind of bridged frame,
as well as routed IP traffic.
LLC Encapsulated, restricted to Ethernet:
# Traffic shape
Traffic_Shape name=high_speed_high_quality peak=155000 \
sustain=155000 burst=2048 qos=high
# Logical interface
Interface ga030 traffic_shape=high_speed_high_quality \
bridge_method=llc_multiplexed,broute_to_ether
# PVC
PVC ga030 0/32 proto=llc,bridging
Note: Any IP routed traffic transmitted on the PVC will be encapsulated as
an Ethernet frame.
VC-Based Multiplexing options:
# Traffic shape
Traffic_Shape name=high_speed_high_quality peak=155000 \
sustain=155000 burst=2048 qos=high
# Logical interface
Interface ga030 traffic_shape=high_speed_high_quality \
bridge_type=vc_multiplexed
# PVCs for bridging
Configuration of IP-Forwarding Media Cards
153
Download from Www.Somanuals.com. All Manuals Search And Download.
PVC ga030 0/32 proto=vcmux_bridge,ether
PVC ga030 0/33 proto=vcmux_bridge,ether_fcs
PVC ga030 0/34 proto=vcmux_bridge,bpdu
# PVC for IP (RFC 1577)
PVC ga030 0/35 proto=llc
Note: IP routed traffic is transmitted on its own PVC. If the separate IP
PVC were not defined, then routed IP datagrams would be encapsulated
as Ethernet frames.
4.6.11.5 Configuration for ARP Service - grarp.conf
If needed, supply IP-to-physical address mapping information for ARP
service.
Put an entry into /etc/grarp.conf only if the remote destination does not
support InATMARP, which the GRF does.
A sample entry would be:
#[ifname]
ga020
host
hwaddr
[temp] [pub] [trail]
172.0.130.111 0/32
4.6.11.6 Installing Configuration Changes
When you enter configuration information or make changes, you must do a
grwritecommand to save the /etc directory to permanent storage. In the CLI,
or from the UNIX shell, enter grwrite -v.
You must also reset the media card for the changes to take place. Type
grreset <slot_number>.
Note: We found that grresetwould not be enough to get things up correctly,
especially if the interfaces were already in use before.
We recommend using grwrite -v,followed by reboot -i.
Do not start hunting for bugs before you have tried these two commands.
4.6.12 Bridging ATM
The information in the previous chapter was used in implementing bridging
with two ATM ports between two GRFs; see Section 7.1.2, “ATM OC-3c
154
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
4.6.13 Bridging FDDI
Transparent bridging is especially useful for day-to-day customer
environments, where several FDDI backbones meet in the computing center
and must be connected to an SP. Up to now, the SP was connected to the
FDDI switch or router with one or two FDDI adapters, thus causing a
bottleneck.
Now, with the GRF connecting directly to the SP Switch, the FDDI backbones
can deliver their data at maximum speed. See Section 5.1.4.2, “SP Switch -
FDDI Connection with Bridging” on page 179 for the setup.
4.6.14 Bridging Ethernet
What was discussed for FDDI applies to 100 Mbit/s Ethernet backbones,
also.
We provide no scenario or setup for this case.
Configuration of IP-Forwarding Media Cards
155
Download from Www.Somanuals.com. All Manuals Search And Download.
156
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 5. Single RS/6000 SP and Single SP Switch Router
This section provides several sample configurations that are possible with a
single RS/6000 SP and a single SP Switch Router. Sample configurations
range from using the SP Switch Router as a conventional high performance
router up to the connection of two SP partitions, allowing high-speed Switch
communication between the partitions.
5.1 Single SP Partition and Single SP Switch Router Adapter Card
In this configuration, a single SP Switch Router Adapter card is connected to
a single SP partition and works as a conventional high performance router.
customer environments were tested:
1. SP Switch - Ethernet Connection
2. SP Switch - FDDI Connection
3. SP Switch - ATM Connection
4. SP Switch - FDDI Connection (distinct backbones)
5. SP Switch - FDDI Connection (ADSM environment)
Partition
Net
192.168.13.0
Workstation
SP
Switch
Router
ATM
IP 192.168.13.4
SP processor node
SP Switch Router
Adapter card
SP Switch
FDDI
SP processor node
Mask 255.255.255.0
SP node
Mask 255.255.255.0
Figure 52. One Card - One SP Partition Sample Configuration
5.1.1 SP Switch - Ethernet Connection
This scenario might be appropriate in customers’ environments where 100
Mbit Ethernet is the choice for the backbone network. Up to eight computers,
Ethernet switches or Ethernet hubs can be connected directly to the SP
Switch.
© Copyright IBM Corp. 1998
157
Download from Www.Somanuals.com. All Manuals Search And Download.
Configuration assumptions:
• The SP Switch Router Ethernet media card has been installed according
works properly.
• The SP Switch Router Adapter card has been installed according to
works properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the Switch node numbers. Refer to
PSSP Planning, Volume 2, Control Workstation and Software Environment
for details.
Configuration:
A RS/6000 model F50 with a PCI Ethernet adapter card is connected to port 1
of an Ethernet media card in the GRF 1600. The GRF 1600 SP Switch Router
Adapter card is attached to the SP Switch of SP21, as shown in Figure 53
GRF 1600
SP node
1
SP Switch Router
F50
Adapter card1
SP node
SP node
Figure 53. SP Switch - Ethernet Connection
158
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 14. Configuration of SP Switch - Ethernet Connection
Adapter
IP Address
Ethernet interface in F50 (en0 on ent0) 10.20.30.50
SP Switch Router Ethernet media card 10.20.30.1
(port 1)
SP Switch Router Adapter card 1
SP processor nodes in SP21
192.168.14.4
192.168.14.1 - 192.168.14.15
To successfully run this configuration, no routes need to be set on the SP
Switch Router. The F50 and the processor nodes in SP21 require attention,
though.
You should be able to pingthe en0 interface on the F50:
(0)f50:/ 14$ ping 10.20.30.50
PING 10.20.30.50: (10.20.30.50): 56 data bytes
64 bytes from 10.20.30.50: icmp_seq=0 ttl=255 time=13 ms
64 bytes from 10.20.30.50: icmp_seq=1 ttl=255 time=0 ms
64 bytes from 10.20.30.50: icmp_seq=3 ttl=255 time=0 ms
^C
----10.20.30.50 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/3/13 ms
(0)f50:/ 15$
And you should also be able to ping the ge071 port of the GRF:
(0)f50:/ 15$ ping 10.20.30.1
PING 10.20.30.3: (10.20.30.1): 56 data bytes
64 bytes from 10.20.30.1: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 10.20.30.1: icmp_seq=1 ttl=255 time=0 ms
64 bytes from 10.20.30.1: icmp_seq=2 ttl=255 time=0 ms
^C
----10.20.30.1 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
(0)f50:/ 16$
To add the needed routing information, follow these steps:
1. On the F50, add the following route to the nodes in SP21:
route add -net 192.168.14 -netmask 255.255.255.0 \
Single RS/6000 SP and Single SP Switch Router
159
Download from Www.Somanuals.com. All Manuals Search And Download.
-mtu 1500 10.20.30.1
2. Check for correct routing entry:
(0)f50:/ 26$ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
9.12.1.30
9.12.1.50
10.1.1.3
10.20.30.50
127.0.0.1
10.20.30.1
UG
U
U
U
U
4 288844 tr0
- -
- -
- -
- -
- -
9.12.1/24
10.1.1/24
10.20.30/24
127/8
24
0
1
41728 tr0
0 at1
0 en0
5
753 lo0
192.168.14/24
UG
0 2889408 en0 1500 -
Route Tree for Protocol Family 24 (Internet v6):
::1
::1
UH
0
0 lo0 16896 -
(0)f50:/ 27$
3. On the nodes in SP21 that are supposed to communicate with the F50,
add the following route:
route add -net 10.20.30.50 -netmask 255.255.255.0 \
-mtu 1500 192.168.14.4
4. Check for correct routing entry:
root@sp21n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.20.30/24
127/8
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.137
192.168.14.4
127.0.0.1
192.168.4.15
192.168.14.4
192.168.14.15
UG
UG
U
U
UG
U
0 176719 en0
0 2639453 css0 1500 -
- -
8
529 lo0
- -
- -
11 393829 en0
0 6323257 css0 9180 -
2 183033 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1
::1
UH
0
0 lo0 16896 -
root@sp21n01:/
The -mtu parameter is optional but should be set to ensure optimal packet
size on this route.
5. Issue some pingcommands to check the connection:
On the F50, pingthe SP Switch interface of a chosen node in SP21:
160
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
(0)f50:/ 29$ ping 192.168.14.15
PING 192.168.14.15: (192.168.14.15): 56 data bytes
64 bytes from 192.168.14.15: icmp_seq=0 ttl=254 time=0 ms
64 bytes from 192.168.14.15: icmp_seq=1 ttl=254 time=0 ms
64 bytes from 192.168.14.15: icmp_seq=2 ttl=254 time=0 ms
^C
----192.168.14.15 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
(0)f50:/ 30$
On the chosen nodes in SP21, pingthe ATM interface of the F50:
root@sp21n01:/ ping f50
PING f50: (10.20.30.50): 56 data bytes
64 bytes from 10.20.30.50: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 10.20.30.50: icmp_seq=1 ttl=254 time=0 ms
64 bytes from 10.20.30.50: icmp_seq=2 ttl=254 time=0 ms
^C
----f50 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/1 ms
root@sp21n01:/
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe GRF Ethernet media card or the GRF SP
Switch media card to find out which part is failing:
ping 192.168.14.4 (on chosen SP21 processor nodes)
ping 10.20.30.1(on F50)
If any errors occur, check cabling, the configuration of SP Switch Router
media cards (See Section 3.7, “Step-by-Step Media Card Configuration”
page 105) and network adapters in the F50 and the SP nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved in this
scenario, we used ftp to conduct several file transfers of a 300 MB file from
the F50 to the chosen nodes in SP21. We sent this file to /dev/null on the
SP21 nodes to eliminate any hard disk influence on the receiver side:
Single RS/6000 SP and Single SP Switch Router
161
Download from Www.Somanuals.com. All Manuals Search And Download.
0)f50:/itso/space 32$ ftp 192.168.14.1
Connected to 192.168.14.1.
220 sp21n01 FTP server (Version 4.1 Tue Mar 17 14:00:13 CST 1998) ready.
Name (192.168.14.1:root): root
331 Password required for root.
Password:
230 User root logged in.
ftp> bin
200 Type set to I.
ftp> put bos.obj.ssp.itso /dev/null
200 PORT command successful.
150 Opening data connection for /dev/null.
226 Transfer complete.
299878400 bytes sent in 31.95 seconds (9166 Kbytes/s)
local: bos.obj.ssp.itso remote: /dev/null
ftp> quit
221 Goodbye.
(0)f50:/itso/space 33$
As you see in the screen shot, the F50’s 9.2 MB/s comes very close to the
theoretical maximum throughput of about 10-11 MB/s. So we decided to put
some stress on the network, as follows:
On two SP21 nodes, we started ftp put commands to the F50, while the F50
was busy, putting data to other SP21 nodes, and observed an aggregate
throughput over the Ethernet adapter of up to 15 MB/s, with more than 35%
collision on the Ethernet and the F50 23% busy.
5.1.2 SP Switch - FDDI Connection
This is quite an easy scenario. It is used to connect workstations equipped
with an FDDI interface card, FDDI hubs, and similar FDDI network devices to
the SP Switch.
Configuration assumptions:
4.4, “FDDI Configuration” on page 121, and works properly.
properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP addresses (strongly recommended!).
162
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the Switch node numbers. Refer to
PSSP Planning, Volume 2, Control Workstation and Software Environment
for details.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
To establish this scenario, the FDDI interface of node 10 in SP21 was
connected to the SP Switch Router FDDI media card, port A0. The SP Switch
Router Adapter card is attached to the SP Switch of SP2, as shown in Figure
GRF 400
SP node
A0
SP Switch Router
Adapter card 1
SP node
SP node
node10
FDDI
SP2
Figure 54. SP Switch - FDDI Connection
Table 15. Configuration of an SP Switch - FDDI Connection
Adapter
IP Address
FDDI interface in node 10
10.2.1.1
SP Switch Router FDDI media card 10.2.1.2
(port A0)
SP Switch Router Adapter card 1
SP processor nodes in SP2
192.168.13.4
192.168.13.1 - 192.168.13.15 (See
To successfully run this configuration, no routes need to be set on the SP
Switch Router. Node 10 and the processor nodes in SP2 require attention,
Single RS/6000 SP and Single SP Switch Router
163
Download from Www.Somanuals.com. All Manuals Search And Download.
though. To add the needed routing information and check for proper work,
follow these steps:
1. On node 10 in SP21, add the following route to the switch network of SP2:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 4352 10.2.1.2
2. Check for correct routing entry:
root@sp21n10:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.2.1/24
127/8
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.137
10.2.1.1
127.0.0.1
192.168.4.10
10.2.1.2
192.168.14.10
UG
U
U
U
UG
U
0
1
8
482 en0
22 fi0
894 lo0
- -
- -
- -
- -
8 603782 en0
0 4985719 fi0 4352 -
1 576716 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
3. On the nodes in SP2 that are supposed to communicate with node 10 in
SP21, add the following route:
route add -net 10.2.1 -netmask 255.255.255.0 -mtu 4352 192.168.13.4
The -mtu parameter is optional, but should be set to ensure optimal packet
size on this route.
4. Check for correct routing entry:
root@sp2n09:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
192.168.3.37
192.168.13.4
10.10.1.9
UG
UG
U
1
0
0
8
204 en0
0 css0 4352 -
5 fi0
767 lo0
- -
10.2.1/24
10.10.1/24
127/8
- -
- -
- -
- -
127.0.0.1
U
192.168.3/24
192.168.13/24
192.168.3.9
192.168.13.9
U
U
10 948987 en0
2 560984 css0
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
5. On GRF 400 check /etc/grifconfig.conf for the following entry:
gf000 10.2.1.2 255.255.255.0 - mtu 4352
164
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
6. On the CWS of SP2 check if the SP Switch Router Adapter card is
configured. See if the SP Switch Router Adapter card shows up green in
perspectives or enter SDRGetObjects switch_responds. Use Eunfence if
needed.
7. Issue some pingcommands to check the connection:
On the chosen SP2 nodes, pingthe FDDI interface of node 10 in SP21, for
example:
root@sp2n09:/ ping 10.2.1.1
PING 10.2.1.1: (10.2.1.1): 56 data bytes
64 bytes from 10.2.1.1: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 10.2.1.1: icmp_seq=1 ttl=255 time=0 ms
^C
----10.2.1.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
On node 10 in SP21, pingthe SP Switch interfaces of the chosen nodes in
SP2, for example:
root@sp21n10:/ ping 192.168.13.9
PING 192.168.13.9: (192.168.13.9): 56 data bytes
64 bytes from 192.168.13.9: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 192.168.13.9: icmp_seq=1 ttl=255 time=0 ms
^C
----192.168.13.9 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router FDDI media card or the
SP Switch Router Adapter card to ascertain the failing part:
ping 192.168.13.4 (on chosen processor nodes in SP2)
ping 10.2.1.2 (on node 10 in SP21)
If any errors occur, check cabling, the configuration of the SP Switch
121) and also the network adapters in the nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved in
such a scenario, we performed the following tests:
1. We obtained of tsock program (a derivative of the public domain ttcp
program developed by T.C. Slattery, USNA, improved by Mike Muuss,
Single RS/6000 SP and Single SP Switch Router
165
Download from Www.Somanuals.com. All Manuals Search And Download.
BRL, and ported to AIX by Prof. Peter Haas, University of Stuttgart). tsock
is a program that uses the UltraNet socket emulation library to perform
numerous network exercises, among them to measure the pure network
performance by eliminating any hard disk or processor load influence on
the data transfer rate by sending data packets directly from memory to
memory. We selected to send bursts of packets ranging from 1 byte to 50
kilobytes, as this might be a more realistic approach than just shifting
large files around. In this scenario an average data transfer rate of about
12 MB/s was achieved which corresponds to the theoretical maximum
data transfer rate of an FDDI connection (12.5 MB/s=100 Mb/s).
2. We used ftp to conduct several file transfers of a 300 MB file from different
nodes in SP2. We sent this file to /dev/null on node 10 in SP21, to
eliminate any hard disk influence on the receiver side:
root@sp2n09:/ ftp 10.2.1.1
Connected to 10.2.1.1.
220 sp21n10 FTP server (Version 4.1 Tue Mar 17 14:00:13 CST 1998) ready.
Name (10.2.1.1:root):
331 Password required for root.
Password:
230 User root logged in.
ftp> put bos.obj.ssp.itso /dev/null
200 PORT command successful.
150 Opening data connection for /dev/null.
226 Transfer complete.
299878400 bytes sent in 65.95 seconds (4441 Kbytes/s)
local: bos.obj.ssp.itso remote: /dev/null
ftp> quit
221 Goodbye.
root@sp2n09:/space
As can be seen in the screen shot, the slow internal SCSI disks of the
nodes in SP2 would not allow the transfer rate to exceed 4.5 MB/s. So we
decided to start four ftp programs on four different nodes in SP2. We
expected these four file transfers to sum up to an aggregate throughput of
about 18 MB/s over the net. This would be beyond the throughput of an
FDDI adapter, so we expected to reach the limits of the FDDI adapter,
either in the GRF or on node 10 in SP21.
With this scenario, we measured an average transfer rate of about 10-11
MB/s (observed with the freeware tool monitor from Jussi Maki). The
limiting factor now was the CPU of node 10 in SP21, which was not able to
handle more data simultaneously (100% busy, as seen with monitor).
166
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
5.1.3 SP Switch - ATM Connection
This scenario might be used quite often to attach a single computer with an
ATM interface to the SP Switch of an SP system. This could, for example, be
an RS/6000 model S70 acting as an ADSM server or as a database server in
an SAP or BAAN environment. It could, as well, be a connection to another
already existing ATM Switch.
Configuration assumptions:
• An SP Switch Router ATM media card has been installed according to
properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP Addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the Switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
In this scenario, we chose an RS/6000 model F50 with a PCI ATM adapter
card to be connected to an ATM port on an ATM OC-3c media card in the
GRF 1600. The ATM interface of the F50 is connected to the GRF 1600 SP
Switch Router ATM OC-3c media card’s port 80. The GRF 1600 SP Switch
Router Adapter card is attached to the SP Switch of SP21, as shown in
interfaces is 255.255.255.0.
Single RS/6000 SP and Single SP Switch Router
167
Download from Www.Somanuals.com. All Manuals Search And Download.
GRF 1600
SP node
SP node
SP node
80
SP Switch Router
Adapter card 1
F 50
ATM
SP21
Figure 55. SP Switch - ATM Connection
Table 16. Configuration of SP Switch - ATM Connection
Adapter
IP Address
ATM interface in F50 (at0 on atm0)
10.1.2.3
SP Switch Router ATM media card 10.1.2.1
(port 80)
SP Switch Router Adapter card 1
SP processor nodes in SP21
192.168.14.4
192.168.14.1 - 192.168.14.15 (See
To successfully run this configuration, no routes need to be set on the SP
Switch Router. The F50 and the processor nodes in SP21 require attention,
though. To configure the ATM adapter in the F50, follow these steps:
1. On the F50, check that all the required software for ATM support is
installed. There is no need for ATM LAN Emulation software support,
though. If the ATM adapter is in the "available" state (check with
lsdev -C -c adapter), you are probably fine.
2. To avoid any pitfalls, set the signaling protocol for the ATM adapter to
UNI3.0, as this is the default for the GRF and was probably not changed in
the file /etc/gratm.conf on the GRF. You can check and look for a line
where Signalling card= is not commented with a #.
It escapes our understanding why we had to bother with this, anyhow, as
we could not set up and use SVC between the F50 and the GRF, because
168
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
of the lack of an ARP server; signaling protocols should only matter with
SVCs and not with PVCs.
3. On the F50, use smitty chg_atm, select the ATM device and change the
field SVC UNI Version from auto_detectto uni3.0:
Change / Show Characteristics of an ATM Adapter
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
ATM Adapter
atm0
Description
Status
Location
IBM PCI 155 Mbps ATM A>
Available
30-68
no
[0x0]
[100]
[32]
[1024]
uni3.0
[0x30]
[0]
Enable ALTERNATE ATM MAC address
ALTERNATE ATM MAC address
Software Transmit Queue size
Minimum Guaranteed VCs Supported
Maximum Number of VCs Needed
SVC UNI Version
Minimum 4K-byte pre-mapped receive buffers
Sonet or SDH interface
Provide SONET Clock
+
+#
+#
+#
+
+#
+#
+
[0]
4. Use smitty mkinetatto get to the Add an "ATM Network Interface" menu
and fill in the blanks, as follows:
Add an ATM Network Interface
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
* INTERNET ADDRESS (dotted decimal)
Network MASK (hexadecimal or dotted decimal)
Network Interface
Connection Type
ATM Server Address
Alternate Device
Idle Timer
Best Effort Bit Rate (UBR) in Kbits/sec
* ACTIVATE the Interface after Creating it?
[10.1.2.3]
[255.255.255.0]
[at0]
pvc
[]
[]
[]
[155000]
yes
+
+
You may leave the Network Interface field blank, as at0is the default
interface on the device atm0, but you must change the Connection type
from svc_sto pvc. As stated, we had no ARP server available, which is a
prerequisite for any SVC type connection. Consider a PVC to be a
permanent point-to-point connection; we will have to give the identification
of the partner in the next SMIT screen.
Single RS/6000 SP and Single SP Switch Router
169
Download from Www.Somanuals.com. All Manuals Search And Download.
5. Have smittycarry out its work, exit smitty and then run smitty mkatmpvc:
Add a PVC for IP over an ATM Network
Type or select values in entry fields.
Press Enter AFTER making all desired changes.
[Entry Fields]
PVC Description (Optional)
* VPI:VCI
Network Interface
Destination IP Address
Automatically Discover Destination IP Address
LLC Encapsulation
[F50toGRF]
[0:134]
[at0]
[]
yes
yes
+
+
In the optional PVC Descriptionfield, do not use blanks or underscores. In
the VPI:VCIfield, use the numbers you get from the PVC statement in the
/etc/gratm.conf file on the GRF 1600. For your convenience, the relevant
line is shown here again:
PVC ga0180 0/134 proto=ip traffic_shape=high_speed_high_quality
Note: Be very careful about the different naming conventions on the GRF
and on AIX! Whereas the numbers are separated by a slash (/) in
/etc/gratm.conf, you must use a colon (:) to separate them in SMIT!
Because we give the VPI:VCInumbers of the GRF ATM port and leave the
field Automatically Discover Destination IP Addresson its yesdefault,
there is no need to fill in the Destination IP Address.
After smittyis done, you should be able to pingthe at0 interface on the
F50:
(0)f50:/ 14$ ping 10.1.2.3
PING 10.1.2.3: (10.1.2.3): 56 data bytes
64 bytes from 10.1.2.3: icmp_seq=0 ttl=255 time=13 ms
64 bytes from 10.1.2.3: icmp_seq=1 ttl=255 time=0 ms
64 bytes from 10.1.2.3: icmp_seq=3 ttl=255 time=0 ms
^C
----10.1.2.3 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/3/13 ms
(0)f50:/ 15$
You should also be able to ping the ga0180 port of the GRF:
170
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
(0)f50:/ 15$ ping 10.1.2.1
PING 10.1.2.1: (10.1.2.1): 56 data bytes
64 bytes from 10.1.2.1: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 10.1.2.1: icmp_seq=1 ttl=255 time=0 ms
64 bytes from 10.1.2.1: icmp_seq=2 ttl=255 time=0 ms
^C
----10.1.2.1 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
(0)f50:/ 16$
To add the needed routing information, follow these steps:
1. On the F50, add the following route to the nodes in SP21:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 9180 10.1.2.1
2. Check for correct routing entry:
(0)f50:/ 26$ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
9.12.1.30
9.12.1.50
10.1.2.3
127.0.0.1
10.1.2.1
UG
U
U
U
UG
3
41
2
3043 tr0
34248 tr0
41 at0
- -
- -
- -
- -
9.12.1/24
10.1.2/24
127/8
5
315 lo0
192.168.14/24
0 4558293 at0 9180 -
Route Tree for Protocol Family 24 (Internet v6):
::1
::1
UH
0
0 lo0 16896 -
(0)f50:/ 27$
3. On the nodes in SP21 that are supposed to communicate with the F50,
add the following route:
route add -net 10.1.2 -netmask 255.255.255.0 -mtu 9180 192.168.14.4
4. Check for correct routing entry:
Single RS/6000 SP and Single SP Switch Router
171
Download from Www.Somanuals.com. All Manuals Search And Download.
root@sp21n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.1.2/24
127/8
192.168.4.137
192.168.14.4
127.0.0.1
UG
UG
U
1
7410 en0
- -
0 3449576 css0 9180 -
8
757 lo0
- -
- -
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.1
192.168.14.4
192.168.14.1
U
UG
U
13 1132000 en0
1 13501009 css0 9180 -
8 618178 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1
::1
UH
0
0 lo0 16896 -
root@sp21n01:/
The -mtu parameter is optional but should be set to ensure optimal packet
size on this route.
5. Issue some pingcommands to check the connection:
On the F50, pingthe SP Switch interface of a chosen node in SP21:
(0)f50:/ 29$ ping 192.168.14.1
PING 192.168.14.1: (192.168.14.1): 56 data bytes
64 bytes from 192.168.14.1: icmp_seq=0 ttl=254 time=0 ms
64 bytes from 192.168.14.1: icmp_seq=1 ttl=254 time=0 ms
64 bytes from 192.168.14.1: icmp_seq=2 ttl=254 time=0 ms
^C
----192.168.14.1 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
(0)f50:/ 30$
On the chosen nodes in SP21, pingthe ATM interface of the F50:
root@sp21n01:/ ping f50
PING f50: (10.1.2.3): 56 data bytes
64 bytes from 10.1.2.3: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 10.1.2.3: icmp_seq=1 ttl=254 time=0 ms
64 bytes from 10.1.2.3: icmp_seq=2 ttl=254 time=0 ms
^C
----f50 PING Statistics----
3 packets transmitted, 3 packets received, 0% packet loss
round-trip min/avg/max = 0/0/1 ms
root@sp21n01:/
172
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router ATM media card or the SP
Switch Router Adapter card to which part is failing:
ping 192.168.14.4 (on chosen SP21 processor nodes)
ping 10.1.2.1 (on F50)
If any errors occur, check cabling, the configuration of SP Switch Router
network adapters in the F50 and the SP nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved in this
scenario, the following tests were performed:
1. Again we used tsock to get the performance figures for a mix of packets
ranging from 1 byte to 50 kilobytes. This time an average data transfer
rate of about 13.5 MB/s was achieved, which corresponds well enough to
the theoretical maximum data transfer rate of an ATM connection (17.5
MB/s=155 Mb/s).
2. We used ftp to conduct several file transfers of a 300 MB file from the F50
to the chosen nodes in SP21. We sent this file to /dev/null on the SP21
nodes to eliminate any hard disk influence on the receiver side:
(0)f50:/itso/space 32$ ftp 192.168.14.1
Connected to 192.168.14.1.
220 sp21n01 FTP server (Version 4.1 Tue Mar 17 14:00:13 CST 1998) ready.
Name (192.168.14.1:root): root
331 Password required for root.
Password:
230 User root logged in.
ftp> bin
200 Type set to I.
ftp> put bos.obj.ssp.itso /dev/null
200 PORT command successful.
150 Opening data connection for /dev/null.
226 Transfer complete.
299878400 bytes sent in 31.05 seconds (9431 Kbytes/s)
local: bos.obj.ssp.itso remote: /dev/null
ftp> quit
221 Goodbye.
(0)f50:/itso/space 33$
As you see in the screen shot, although the F50 has very fast disks
attached, they still limit the file transfer rate to 9.4 MB/s. So we decided to
start two ftp programs on the F50 in parallel. This way we could increase
the throughput of the ATM adapter to about 16.5 MB/s, with the F50 being
about 50% busy (all data again observed with the freeware tool monitor).
Single RS/6000 SP and Single SP Switch Router
173
Download from Www.Somanuals.com. All Manuals Search And Download.
An ATM adapter establishes a duplex connection to its partner, so the 16.5
MB/s write throughput should be accompanied by another 16.5 MB/s read
throughput. To prove this, we started two ftp put commands from the same
SP21 node. At the same time we started two ftp put commands an the F50
and observed an aggregate throughput over the ATM adapter of up to 28
MB/s, with the CPU of the SP node nearly 80% busy and the F50 100%
busy.
5.1.4 SP Switch - FDDI Connection (Distinct FDDI Networks)
5.1.4.1 SP Switch - FDDI Connection without Bridging
This scenario is rather similar to the one described in Section 5.1.2, “SP
Switch - FDDI Connection” on page 162 and is quite often met in customer
environments. It might be used to connect four different FDDI backbones to a
SP Switch, for instance in an ADSM environment.
Configuration assumptions:
• An SP Switch Router FDDI media card has been installed according to
Section 4.4, “FDDI Configuration” on page 121 and works properly.
properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP Addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the Switch node numbers. Refer to
PSSP Planning, Volume 2, Control Workstation and Software Environment
for details.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
To establish this scenario, the FDDI interfaces of node 9, node 10, node 11
and node 12 in SP2 were connected to the SP Switch Router FDDI media
card’s port B1, port A1, port B0 and port A0, respectively. (This "weird"
connection had to be chosen because of limited cable length in our lab
environment . Any other cabling might be used.). The SP Switch Router
Adapter card is attached to the SP Switch of SP21, as shown in Figure 56 on
174
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
255.255.255.0.
node 9
GRF 1600
SP node
A0
node 10
A1 FDDI
SP Switch Router
Adapter card 1
SP node
B0
B1
node 11
node 12
SP node
SP21
Figure 56. SP Switch - FDDI Connection
Table 17. Configuration of SP Switch - FDDI Connection
Adapter
IP Address
10.5.1.9
FDDI interface in node 9
FDDI interface in node 10
FDDI interface in node 11
FDDI interface in node 12
10.3.1.10
10.4.1.11
10.2.1.12
SP Switch Router FDDI media card 10.2.1.15
(port A0)
SP Switch Router FDDI media card 10.3.1.16
(port A1)
SP Switch Router FDDI media card 10.4.1.17
(port B0)
SP Switch Router FDDI media card 10.5.1.18
(port B1)
SP Switch Router Adapter card 1
SP processor nodes in SP21
192.168.14.4
192.168.14.1 - 192.168.14.15 (See
Single RS/6000 SP and Single SP Switch Router
175
Download from Www.Somanuals.com. All Manuals Search And Download.
To successfully run this configuration, all four ports of the SP Switch Router
FDDI media card have to be assigned IP addresses in different subnets.
Otherwise, only port A0 will be activated.
Note: Every port of the SP Switch Router FDDI media card has to be
assigned to a different subnet when bridging is not configured (see Section
In this sample configuration no routes need to be set on the SP Switch
Router. Node 9-12 on SP2 and the processor nodes in SP21 require
attention, though. To add the needed routing information, follow these steps:
1. On all four nodes in SP2 with an FDDI interface, add the route to the SP
Switch network of SP21:
• On node 9 in SP2, add this route:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 4352 10.5.1.18
• On node 10 in SP2, add this route:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 4352 10.3.1.16
• On node 11in SP2, add this route:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 4352 10.4.1.17
• On node 12 in SP2, add this route:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 4352 10.2.1.15
2. Check for correct routing entries on all four nodes, for example:
root@sp2n09:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.5.1/24
127/8
192.168.3/24
192.168.13/24
192.168.14/24
192.168.3.37
10.5.1.9
127.0.0.1
192.168.3.9
192.168.13.9
10.5.1.18
UG
U
U
U
U
0
1
8
9
390 en0
2 fi0
906 lo0
85376 en0
- -
- -
- -
- -
- -
2 731853 css0
0 10460318 fi0 4352 -
UG
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
176
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
3. On the nodes in SP21 that are supposed to communicate with the different
FDDI backbones, add the necessary routes:
route add -net 10.2.1 -netmask 255.255.255.0 -mtu 4352 192.168.14.4
route add -net 10.3.1 -netmask 255.255.255.0 -mtu 4352 192.168.14.4
route add -net 10.4.1 -netmask 255.255.255.0 -mtu 4352 192.168.14.4
route add -net 10.5.1 -netmask 255.255.255.0 -mtu 4352 192.168.14.4
The -mtu parameter is optional but should be set to ensure optimal packet
size on this route.
4. Check for correct routing entries, for example:
root@sp21n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
192.168.4.137
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
127.0.0.1
UG
UG
UG
UG
UG
U
1
76666 en0
- -
10.2.1/24
10.3.1/24
10.4.1/24
10.5.1/24
127/8
0 7407273 css0 4352 -
0 1413386 css0 4352 -
0 1083727 css0 4352 -
0 5046891 css0 4352 -
8
399 lo0
- -
- -
- -
192.168.4/24
192.168.14/24
192.168.4.1
192.168.14.1
U
U
7 534581 en0
5 110190 css0
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
5. On GRF 1600, check /etc/grifconfig.conf for the following entries:
gf000 10.2.1.15
gf001 10.3.1.16
gf002 10.4.1.17
gf003 10.5.1.18
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
mtu 4352
mtu 4352
mtu 4352
mtu 4352
6. On GRF 1600, check whether all four port interfaces are created
successfully. Use netstat -inand look for lines starting with gf000, gf001,
gf002 or gf003. They must not have an asterisk beside the interface name.
If only gf000 appears, examine if all IP addresses assigned to the four
FDDI ports really from different subnets and change IP addresses or
network masks, if necessary.
Single RS/6000 SP and Single SP Switch Router
177
Download from Www.Somanuals.com. All Manuals Search And Download.
/root netstat -in
Name
de0
de0
Mtu Network
1500 <link1>
1500 <Bridge>
Address
00:c0:80:96:38:68
00:c0:80:96:38:68
Ipkts Ierrs
Opkts Oerrs Coll
10290
10290
10290
0
0
0
74
74
0
0
0
0
42228
42228
42228
41439
41439
8387
8387
8387
20
0 182
0 182
0 182
de0
1500 192.168.4 192.168.4.4
rmb0
rmb0
lo0
616 <link2>
616 <GRIT>
1536 <link3>
1536 127
00:00:00:00:00:00 170556
0:0x40:0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
170556
8387
8387
8387
19
lo0
127.0.0.1
0:0x48:0
00:c0:80:89:2d:f2
lo0
1536 <GRIT>
gf000 4352 <link10>
gf000 4352 10.2.1/24 10.2.1.15
gf001 4352 <link20>
gf001 4352 10.3.1/24 10.3.1.16
gf002 4352 <link11>
gf002 4352 10.4.1/24 10.4.1.17
gf003 4352 <link12>
gf003 4352 10.5.1/24 10.5.1.18
19
0
20
12
12
13
00:c0:80:89:2d:f3
12
12
0
0
00:c0:80:89:2d:f4
13
13
0
0
13
19
19
00:c0:80:89:2d:f5
13
13
0
0
7. On the CWS of SP21, check if the SP Switch Router Adapter card is
configured. To perform this check, look if the SP Switch Router Adapter
card shows up green in perspectives or enter SDRGetObjects
switch_responds. Use Eunfence if needed.
8. Issue some pingcommands to check the connection:
On the chosen SP21 nodes, pingall four FDDI interfaces of the nodes in
SP2, for example:
root@sp21n01:/ ping 10.2.1.12
PING 10.2.1.12: (10.2.1.12): 56 data bytes
64 bytes from 10.2.1.12: icmp_seq=0 ttl=254 time=10 ms
64 bytes from 10.2.1.12: icmp_seq=1 ttl=254 time=1 ms
^C
----10.2.1.12 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/5/10 ms
On node 9-12 in SP2, pingthe SP Switch interfaces of the chosen nodes
in SP21, for example:
root@sp2n12:/ ping 192.168.14.1
PING 192.168.14.1: (192.168.14.1): 56 data bytes
64 bytes from 192.168.14.1: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 192.168.14.1: icmp_seq=1 ttl=254 time=1 ms
^C
----192.168.14.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms
178
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
If these pingcommands fail, check routing settings and IP address
assignment again. If everything is as it should be, try to pingthe GRF
FDDI media card ports or the GRF SP Switch media card to find the failing
part:
ping 10.2.1.15 (on node 12in SP2)
ping 10.3.1.16 (on node 10 in SP2)
ping 10.4.1.17 (on node 11 in SP2)
ping 10.5.1.18 (on node 9 in SP2)
ping 192.168.14.4 (on nodes in SP21)
If any errors occur, check cabling, the configuration of the SP Switch
121) and also the network adapters in the SP nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved in
such a scenario, the following test was performed:
We used ftp to transfer several 300 MB large files from different nodes in
SP21 to the four FDDI equipped nodes in SP2 and vice versa. We sent these
files to /dev/null to eliminate any hard disk influence on the receiver side.
The slow internal SCSI disks in two of our four nodes in SP2 would not allow
the transfer rate to exceed 4.5 MB/s. Both remaining nodes in SP2 contain
faster SSA hard disks that allowed a transfer rate of 7.5 MB/s. Nevertheless,
this would not be enough to exceed the FDDI bandwidth. So we decided to
start several ftp programs on nodes in SP2 and SP21 to sum up the transfer
rates.
With this scenario, we measured a cumulative transfer rate of up to 44 MB/s
(observed with the freeware tool monitor) that is close to the maximum
theoretical transfer rate of 4x12.5 MB/s=50 MB/s. Every node FDDI interface
yielded an overall transfer rate of about 11 MB/s (sending and receiving). The
limiting factor once again was the CPU on each of the four nodes that was not
able to handle more data simultaneously (100% busy, as seen with monitor).
5.1.4.2 SP Switch - FDDI Connection with Bridging
This scenario corresponds closely to the one described in Section 5.1.4.1,
used to transparently connect four physically separated FDDI backbones to
one large LAN that is connected to the SP Switch.
Single RS/6000 SP and Single SP Switch Router
179
Download from Www.Somanuals.com. All Manuals Search And Download.
Configuration assumptions:
• An SP Switch Router FDDI media card has been installed according to
Section 4.4, “FDDI Configuration” on page 121 and works properly.
• The SP Switch Router Adapter card has been installed according to
works properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP Addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the Switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
To build this scenario, the FDDI interfaces of node 9, node 10, node 11 and
node 12 in SP2 is connected to the SP Switch Router FDDI media card’s port
B1, port A1, port B0 and port A0, respectively. The SP Switch Router Adapter
18 on page 181. The netmask for all interfaces is 255.255.255.0.
node 9
GRF 1600
SP node
A0
node 10
A1
SP Switch Router
FDDI
SP node
B0
B1
Adapter card 1
node 11
node 12
SP node
SP21
bridge group
Figure 57. SP Switch - FDDI Connection (Bridging)
180
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 18. Configuration of SP Switch - FDDI Connection (Bridging)
Adapter
IP Address
10.10.1.9
FDDI interface in node 9
FDDI interface in node 10
FDDI interface in node 11
FDDI interface in node 12
Bridge Group bg0
10.10.1.10
10.10.1.11
10.10.1.12
10.10.1.13
192.168.14.4
SP Switch Router Adapter card 1
SP processor nodes in SP21
192.168.14.1 - 192.168.14.15 (See
To successfully run this configuration, make sure that all four different FDDI
backbones are logically located in one subnet. Otherwise, bridging does not
work and routing has to be used. Nevertheless, the GRF simultaneously
supports routing and bridging which means that an interface included in a
bridge group is able to handle bridge layer-2 frames and route layer-3
packets simultaneously. For details, refer to Section 4.6, “Configuring
Note: All FDDI backbones have to be logically located in a single IP subnet
for proper bridging. The GRF supports simultaneous routing and bridging
over one interface.
In this sample configuration, no routes need to be set on the SP Switch
Router. Node 9-12 on SP2 and the processor nodes in SP21 require
attention, though. Additionally, the bridge group has to be configured. To
perform this configuration, follow these steps:
1. Configure the bridge group on the GRF 1600.
• Define the bridge group:
Open the file /etc/bridged.conf with bredit or vi. If /etc/bridged.conf
does not exist, a new file can be created or the
/etc/bridged.conf.template can be renamed to /etc/bridged.conf. Enter
the necessary bridge group information:
Single RS/6000 SP and Single SP Switch Router
181
Download from Www.Somanuals.com. All Manuals Search And Download.
bridge_group bg0 {
port gf000 gf001 gf002 gf003;
};
This is the simplest bridge definition that worked in our scenario. Many
additional parameters can be set. See Section 4.6.9.3, “Editing Utility –
Bredit” on page 146 for further details.
• Assign an IP address:
Open /etc/grifconfig.conf. Comment out all former FDDI interface
definitions by inserting a #at the beginning of the respective lines.
Define the new bridge group and assign an IP address that is in the
same subnet as all four FDDI backbones:
bg0
10.10.1.13
255.255.255.0 -
mtu 4352
Save /etc/grifconfig.conf. Issue grwrite -vto make the changes
permanent. Reboot the GRF.
Note: According to GRF Configuration Guide 1.4, GA22-7366, just grconfig
or grreset <slot_number> should completely disable the now commented
FDDI interfaces and enable the bridge setting. In the current Ascend
Embedded/OS Version 1.4.6.4, this does not work (maybe in later releases).
The new bridge settings are enabled but the commented interface definitions
are not removed from the kernel. In this case, you will receive error messages
because the bridge daemon cannot add the interfaces to the bridge group as
long as IP addresses are assigned to them. Therefore, reboot -iis needed.
2. Check for correct bridge interface creation on GRF 1600:
grf16:/root netstat -in
Name
rmb0
rmb0
lo0
lo0
lo0
bg0
bg0
bg0
Mtu Network
616 <link2>
616 <GRIT>
1536 <link3>
1536 127
1536 <GRIT>
4352 <link37>
4352 <Bridge>
Address
00:00:00:00:00:00
0:0x40:0
Ipkts Ierrs
Opkts Oerrs Coll
9643
1
1
0
0
0
0
0
0
0
0
0
0
0
0
0
0
9487
9487
8271
8271
8271
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
0
9643
8271
127.0.0.1
0:0x48:0
8271
8271
0
00:c0:80:89:2d:f2
0
0
0
0
0
0
4352 10.10.1/24 10.10.1.13
gf000 4352 <link10>
gf000 4352 <Bridge>
gf001 4352 <link20>
gf001 4352 <Bridge>
gf002 4352 <link11>
gf002 4352 <Bridge>
gf003 4352 <link12>
gf003 4352 <Bridge>
00:c0:80:89:2d:f2
00:c0:80:89:2d:f2
00:c0:80:89:2d:f3
00:c0:80:89:2d:f3
00:c0:80:89:2d:f4
00:c0:80:89:2d:f4
00:c0:80:89:2d:f5
00:c0:80:89:2d:f5
210
210
209
209
209
209
209
209
0
0
0
0
0
0
182
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
3. Add the route to the Switch network of SP21 on all four nodes of SP2 with
an FDDI interface.
On node 9-12 in SP2, add the following route:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 4352 10.10.1.13
4. Check for correct routing entries on all four nodes, for example:
root@sp2n09:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.10.1/24
127/8
192.168.3/24
192.168.13/24
192.168.14/24
192.168.3.37
10.10.1.9
127.0.0.1
192.168.3.9
192.168.13.9
10.10.1.13
UG
U
U
U
U
1
1
8
212 en0
189 fi0
444 lo0
- -
- -
- -
- -
- -
8 232781 en0
1 169010 css0
0
UG
0 fi0 4352 -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
5. On the nodes in SP21 that are supposed to communicate with the four
different FDDI backbones (that are logically only one LAN), add the
necessary route:
route add -net 10.10.1 -netmask 255.255.255.0 -mtu 4352 192.168.14.4
The -mtu parameter is optional but should be set to ensure optimal packet
size on this route.
6. Check for correct routing entries, for example:
root@sp21n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
192.168.4.137
192.168.14.4
192.168.14.4
192.168.14.4
127.0.0.1
UG
UG
UG
UG
U
0
0
0
0
8
515 en0
65 css0
0 css0 4352 -
37 css0
446 lo0
- -
- -
10.1.1/24
10.10.1/24
10.50.1/24
127/8
- -
- -
- -
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.1
192.168.14.4
192.168.14.1
U
UG
U
9 229977 en0
1 512857 css0 65280 -
5 176327 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
Single RS/6000 SP and Single SP Switch Router
183
Download from Www.Somanuals.com. All Manuals Search And Download.
7. On the CWS of SP21, check if the SP Switch Router Adapter card is
configured. To perform this check, look if the SP Switch Router Adapter
card shows up green in perspectives or enter SDRGetObjects
switch_responds. Use Eunfence if needed.
8. Issue some pingcommands to check the connection:
On the chosen SP21 nodes, pingall four FDDI interfaces of nodes in SP2,
for example:
root@sp21n01:/ ping 10.10.1.9
PING 10.10.1.9: (10.10.1.9): 56 data bytes
64 bytes from 10.10.1.9: icmp_seq=0 ttl=254 time=2 ms
64 bytes from 10.10.1.9: icmp_seq=1 ttl=254 time=1 ms
^C
----10.10.1.9 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/2 ms
On node 9-12 in SP2, pingthe SP Switch interfaces of the chosen nodes
in SP21, for example:
root@sp2n12:/ ping 192.168.14.1
PING 192.168.14.1: (192.168.14.1): 56 data bytes
64 bytes from 192.168.14.1: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 192.168.14.1: icmp_seq=1 ttl=254 time=1 ms
^C
----192.168.14.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms
If these pingcommands fail, check routing settings, bridging settings and
IP address assignment again. If everything is as it should be, try to ping
the bridge group and the SP Switch Router media card to find the failing
part:
ping 10.10.1.13 (on nodes 9-12 in SP2)
ping 192.168.14.4 (on nodes in SP21)
If any errors occur, check cabling, the configuration of SP the Switch
121) and also the network adapters in the nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved in
such a bridging scenario, we again used ftp to transfer several 300 MB files
from different nodes in SP21 to the four FDDI-equipped nodes in SP2 and
184
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
vice versa. We sent these files to /dev/null to eliminate any hard disk
influence on the receiver side.
The hardware requisites for this test are the same as described in Section
slow internal SCSI disks in two of our four nodes in SP2 would not allow the
transfer rate to exceed 4.5 MB/s. Both remaining nodes in SP2 contain faster
SSA hard disks that allow a transfer rate of 7.5 MB/s. Nevertheless, the
overall achievable data transfer rate will not exceed the bandwidth of the
FDDI connection. So we decided to start several ftp programs on nodes in
SP2 and SP21 to sum up the transfer rates.
With this scenario, we again measured a cumulative transfer rate of up to 44
MB/s (observed with the freeware tool monitor) that is close to the maximum
theoretical transfer rate of 4x12.5 MB/s=50 MB/s. Every node’s FDDI
interface contributed an overall transfer rate of about 11 MB/s (sending and
receiving). The limiting factor once again was the CPU on the four nodes in
SP2 that was not able to handle more data simultaneously (100% busy, as
seen with monitor). We could not find any significant influence of bridging on
the measured data transfer rate.
5.1.5 SP Switch - FDDI Connection in an ADSM Environment
To get a view into a "real world" scenario and to check corresponding
performance data, we established a simple ADSM environment. Four nodes
in SP2 with FDDI interfaces stand for four FDDI backbones in a possible
customer environment. These backbones send ADSM data via SP Switch
installed.
node 9
GRF 1600
A0
node 10
A1
SP Switch Router
Adapter card 1
B0
node 1
FDDI
B1
node 11
ADSM server
node 12
SP21
SP2
bridge group
Figure 58. SP Switch Router in an ADSM Environment
Single RS/6000 SP and Single SP Switch Router
185
Download from Www.Somanuals.com. All Manuals Search And Download.
Configuration assumptions:
• The SP Switch Router FDDI media card has been installed according to
Section 4.4, “FDDI Configuration” on page 121, and works properly.
• The SP Switch Router Adapter card has been installed according to
works properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP Addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
• The ADSM server is installed on node1 in SP21; ADSM clients are
installed on Nodes 9-12 in SP2.
Configuration:
The configuration of this scenario is exactly as described in Section 5.1.4.2,
additional step has to be carried out, as follows:
1. Open /usr/lpp/adsm/bin/dsm.sys.
2. Change the parameter TCPServeraddress to the IP address of the Switch
interface of node 1:
TCPServeraddress 192.168.14.1
Performance:
To get a rough overview of the data transfer rates that can be achieved during
an ADSM backup session in our configuration, we started an incremental
backup of a filesystem that contained a 300 MB file on nodes 9-12 in SP2.
We measured a transfer rate of up to 7 MB/s one ADSM client to the ADSM
server. Starting another backup session on the other three nodes did not
influence the transfer rate of the other nodes. Every node was able to transfer
up to 7 MB/s independent of other nodes backup activity, summing up to 28
MB/s on the server side. This does not approach the maximum bandwidth of
the SP Switch Router. Limiting factor in this scenario is the ability of the
ADSM server to handle such data transfer rates (processor performance) and
to write all data fast enough to storage media.
186
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
5.2 Single SP Partition and Multiple SP Switch Router Adapter Cards
It is frequently necessary to maintain the data transfer even when an SP
Switch Router Adapter fails. This scenario describes how to setup a dual, not
truly redundant, connection between SP Switch Router and SP Switch (see
Figure 59) and how to recover from an adapter or cable failure.
5.2.1 Configuration of a Dual SP Switch Router Connection
Configuration assumptions:
• ARP should be enabled on the SP Switch network to provide the greatest
flexibility in assigning IP addresses (strongly recommended!).
SP Switch Router
FDDI
SP processor node
SP processor node
Adapter card 1
GRF 1600
SP Switch
SP Switch Router
Adapter card 2
SP processor node
ATM
SP21
Figure 59. Connecting One SP Switch with Two SP Switch Router Adapter Cards
Table 19. Configuration of a Dual SP Switch Router Connection
Adapter
IP Address
Netmask
SP Switch Router Adapter card 1 192.168.14.4
SP Switch Router Adapter card 2 192.168.14.129
255.255.255.128
255.255.255.128
255.255.255.0
SP processor nodes in SP21
192.168.14.1 -
192.168.14.15
Configuration:
To establish this scenario, both SP Switch Router Adapter cards have to be
configured correctly. We installed them according to Section 3.7,
“Step-by-Step Media Card Configuration” on page 86, keeping in mind these
additional points:
Single RS/6000 SP and Single SP Switch Router
187
Download from Www.Somanuals.com. All Manuals Search And Download.
1. Each SP Switch Router Adapter card interface has to be in a different
subnet.
2. Netmasks have to be used to create different subnets on the router side.
3. Logical subnetting is only required on the router. The SP switch sees a
single network.
4. Each SP Switch Router Adapter card must have a unique IP address. An
alias IP address cannot be used on two active cards on the same router
system.
Note: Be careful that the subnet mask does not, in effect, create a single
subnet. Assigning subnet masks of 255.255.255.0 and 255.255.255.128 to
the SP Switch Router Adapter cards on the router side would set both SP
Switch Router Adapter cards in the same subnet. This configuration does not
work.
To check if both SP Switch Router Adapter cards work properly, do the
following:
1. See if the SP Switch Router Adapter cards show up green in perspectives
or use SDRGetObjects switch_responds. Use Eunfence if needed.
2. Issue a pingto both SP Switch Router Adapter cards from any node of
SP21:
root@sp21n06:/ ping 192.168.14.4
PING 192.168.14.4: (192.168.14.4): 56 data bytes
64 bytes from 192.168.14.4: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 192.168.14.4: icmp_seq=1 ttl=255 time=0 ms
^C
----192.168.14.4 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
root@sp21n06:/ ping 192.168.14.129
PING 192.168.14.129: (192.168.14.129): 56 data bytes
64 bytes from 192.168.14.4: icmp_seq=0 ttl=255 time=0 ms
64 bytes from 192.168.14.4: icmp_seq=1 ttl=255 time=0 ms
^C
----192.168.14.129 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
If any errors occur, check cabling, the configuration of the SP Switch Router
Adapter cards (especially subnet mask setting), and also the network
adapters in the nodes.
188
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
If the SP Switch Router Adapter cards work, the can both be used to route IP
traffic from the SP Switch. Add the following routes to the SP nodes of SP21:
route add -net xxx.xxx.xxx.xxx -netmask yyy.yyy.yyy.yyy -mtu \
zzz 192.168.14.4
to route outgoing traffic (coming from the SP node) via SP Switch Router
Adapter card 1 and:
route add -net xxx.xxx.xxx.xxx -netmask yyy.yyy.yyy.yyy -mtu \
zzz 192.168.14.129
to route outgoing traffic (coming from the SP node) via SP Switch Router
Adapter card 2. xxx.xxx.xxx.xxxdescribes the destination network,
yyy.yyy.yyy.yyythe corresponding netmask and zzza proper MTU size for the
chosen route.
Note: With Ascend Embedded/OS 1.4.6.4 (and lower versions) this
configuration has an unpleasant side effect. When both SP Switch Router
Adapter cards are unfenced, the maintenance Ethernet and the RS232
connection are down. No login to the router is possible and existing login
sessions do not accept any command. The router itself works fine but no
changes in router configuration can be made. A workaround is to fence one
SP Switch Router Adapter card for maintenance work. This problem should
be solved in Ascend Embedded/OS 1.4.8.
Nevertheless, the possibility to choose which SP Switch Router Adapter card
should route outgoing traffic allows a certain degree ofload leveling. For best
performance, you should divide your expected traffic in to two equal parts and
set the corresponding routes.
So far, so good. But what happens when one SP Switch Router Adapter card
fails and why is it possible at all, to ping192.168.14.129, netmask
255.255.255.128, from node 6 in SP21 with IP address 192.168.14.1,
netmask 255.255.255.0? Node 6 in SP21 is able to reach the IP address
192.168.14.129 because this address is in the same subnet as its own IP
address. But the SP Switch Router Adapter card cannot reach node 6 in SP
21, because 192.168.14.1 is not in its subnet. So there is no way back and
pingshould not work. But it works fine.
The next sections throw light on both questions.
Single RS/6000 SP and Single SP Switch Router
189
Download from Www.Somanuals.com. All Manuals Search And Download.
5.2.2 Complex Configuration
In this section, we describe the setup of a complete scenario with a dual SP
Switch Router connection, and explain why node 8 in SP21 in this setup is
on page 191), although the way back apparently does not exist.
SP Switch Router
Adapter card 1 - gt030
Node 6
HIPPI
HIPPI
SP Switch Router
Adapter card 3
Node1
SP2
Node 8
GRF 400
GRF 1600
SP Switch Router
Adapter card 2 - gt050
Node 10
SP21
Figure 60. Configuration with Dual SP Switch Router - SP Switch Connection
Configuration assumptions:
both work properly.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP Addresses (strongly recommended!).
“HIPPI Backbone Connection” on page 227). We do not care about HIPPI
connections in this chapter.
• All routes on the SP Switch Routers are assumed to be set correctly.
190
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 20. Configuration of a Dual SP Switch Router - SP Switch Connection
Adapter
IP Address
Netmask
SP Switch Router 192.168.14.4
Adapter card 1
255.255.255.128
SP Switch Router 192.168.14.129
Adapter card 2
255.255.255.128
255.255.255.0
SP Switch Router 192.168.13.4
Adapter card 3
Node 1 in SP2
Node 6 in SP21
Node 8 in SP21
Node 10 in SP21
192.168.13.1
192.168.14.6
192.168.14.8
192.168.14.130
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
Configuration:
For this scenario we chose the following route settings on our three test
nodes in SP21 and on node 1 in SP2:
1. On node 1 in SP2, add the following route to the Switch network of SP21:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 65280 \
192.168.13.4
The MTU size of 65280 is the optimum size for the HIPPI connection.
2. Check for correct routing entry:
root@sp2n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
127/8
192.168.3/24
192.168.4/24
192.168.13/24
192.168.14/24
192.168.3.37
127.0.0.1
192.168.3.1
192.168.13.4
192.168.13.1
192.168.13.4
UG
U
U
UG
U
UG
1
8
3
2
2
0
1103 en0
408 lo0
19063 en0
452 css0
- -
- -
- -
- -
- -
10135 css0
0 css0 65280-
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
Single RS/6000 SP and Single SP Switch Router
191
Download from Www.Somanuals.com. All Manuals Search And Download.
3. On node 6 in SP21, add the following route to the Switch network of SP2:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 65280 \
192.168.14.4
The -mtu parameter is again optional but should be set to ensure optimal
packet size on this route.
4. Check for correct routing entry:
root@sp21n06:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
127/8
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.137
127.0.0.1
192.168.4.6
192.168.14.4
192.168.14.6
UG
U
U
UG
U
0
8
14 en0
402 lo0
- -
- -
- -
6 100681 en0
0
2
0 css0 65280 -
9973 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
5. On node 8 in SP21, add the following route to the switch network of SP2:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 65280 \
192.168.14.129
6. Check for correct routing entry:
root@sp21n08:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
127/8
192.168.4.137
127.0.0.1
UG
U
0
8
16 en0
420 lo0
- -
- -
- -
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.8
192.168.14.129
192.168.14.8
U
UG
U
6 101781 en0
0
3
0 css0 65280 -
10714 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
7. On node 10 in SP21, add the following route to the switch network of SP2:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 65280 \
192.168.14.129
192
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
8. Check for correct routing entry:
root@sp21n10:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.2.1/24
127/8
192.168.4/24
192.168.13/24
192.168.14/24
192.168.4.137
10.2.1.1
127.0.0.1
192.168.4.10
192.168.14.129
192.168.14.10
UG
U
U
U
UG
U
0
0
8
19 en0
8 fi0
414 lo0
- -
- -
- -
- -
6 101692 en0
0
3
0 css0 65280 -
10936 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
9. On GRF 1600, check /etc/grifconfig.conf for the following entries (or
similar):
gt030 192.168.14.4
gt050 192.168.14.129 255.255.255.128 -
255.255.255.128 -
mtu 65520
mtu 65520
10.On the CWS of SP21 check if SP Switch Router Adapter cards are
configured. See if both SP Switch Router Adapter cards show up green
in perspectives or enter SDRGetObjects switch_responds. Use Eunfence if
needed.
11.Issue some pingcommands to check the connection:
On node 8, node 6 and node 10 of SP21, pingthe switch interface of
node 1 in SP2, for example:
root@sp21n06:/ ping 192.168.13.1
PING 192.168.13.1: (192.168.13.1): 56 data bytes
64 bytes from 192.168.13.1: icmp_seq=0 ttl=253 time=0 ms
64 bytes from 192.168.13.1: icmp_seq=1 ttl=253 time=0 ms
^C
----192.168.13.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
Single RS/6000 SP and Single SP Switch Router
193
Download from Www.Somanuals.com. All Manuals Search And Download.
On node 1 in SP2, pingthe SP Switch interfaces of the chosen nodes in
SP21, for example:
root@sp2n01:/ ping 192.168.14.6
PING 192.168.14.6: (192.168.14.6): 56 data bytes
64 bytes from 192.168.14.6: icmp_seq=0 ttl=253 time=1 ms
64 bytes from 192.168.14.6: icmp_seq=1 ttl=253 time=1 ms
^C
----192.168.14.6 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router Adapter cards and check
the HIPPI connection to find the failing part.
If any further errors occur, check cabling, the configuration of SP Switch
Router media cards (See Section 3.7, “Step-by-Step Media Card
Switch Router Connection” on page 187) and also the network adapters in
the nodes.
5.2.2.1 How the Traffic Flows
Why is it possible to ping 192.168.14.129, netmask 255.255.255.128 from
node 6 in SP21 with IP address 192.168.14.6, netmask 255.255.255.0?
This question arose in Section 5.2.1, “Configuration of a Dual SP Switch
Router Connection” on page 187. We treat this problem in a more general
happens when our three chosen nodes in SP21 pingnode 1 in SP2? The next
three figures take a close look at the IP traffic flow. All three figures contain a
192.168.13.1 on node 6 in SP21. All packets are first forwarded to the SP
Switch Adapter card with IP address 192.168.14.4, according to the routing
settings. From this SP Switch Adapter card all packets are forwarded via
HIPPI connection to the only SP Switch Adapter card in GRF 400, which
forwards the traffic to node 1 in SP2. The way back follows the same route.
The SP Switch Adapter card with IP address 192.168.14.4 has no problem
delivering IP packets to node 6 in the same subnet.
194
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
192.168.14.4
255.255.255.128
GRF 400
HIPPI
GRF 1600
HIPPI
SP Switch
Adapter
SP Switch
Adapter
node 1
node 6
SP Switch
Adapter
192.168.13.1
255.255.255.0
192.168.14.6
255.255.255.0
192.168.14.129
255.255.255.128
Figure 61. IP Traffic Flow When Issuing ping 192.168.13.1 on Node 6
192.168.13.1 on node 10 in SP21. All packets are first forwarded to the SP
Switch Adapter card with IP address 192.168.14.129 corresponding to the
routing settings. From this SP Switch Adapter card all packets are forwarded
via HIPPI connection to the only SP Switch Adapter card in GRF 400, which
forwards the traffic to node 1 in SP2. The way back follows the same route.
The SP Switch Adapter card with IP address 192.168.14.129 has no problem
delivering IP packets to node 10 that is in the same subnet.
192.168.14.4
255.255.255.128
GRF 400
HIPPI
GRF 1600
HIPPI
192.168.13.1
255.255.255.0
192.168.14.130
255.255.255.0
SP Switch
Adapter
SP Switch
Adapter
node 10
node 1
SP Switch
Adapter
192.168.14.129
255.255.255.128
Figure 62. IP Traffic Flow When Issuing ping 192.168.13.1 on Node 10
Single RS/6000 SP and Single SP Switch Router
195
Download from Www.Somanuals.com. All Manuals Search And Download.
in SP21. All packets are first forwarded to the SP Switch Adapter card with IP
address 192.168.14.129 corresponding to the routing settings. From this SP
Switch Adapter card all packets are forwarded via HIPPI connection to the
only SP Switch Adapter card in GRF 400, which forwards the traffic to node 1
in SP2. The way back follows first the same route. The internal logic of the
GRF 1600 determines that the SP Switch Adapter card with IP address
192.168.14.129 is not able to deliver received packets back to node 8
because their IP address is in another subnet. So GRF 1600 uses the SP
Switch Adapter card with IP address 192.168.14.4 to forward the IP traffic to
node 8, because this is the only way to IP address 192.168.14.8.
192.168.14.4
255.255.255.128
GRF 400
HIPPI
GRF 1600
HIPPI
SP Switch
Adapter
192.168.13.1
255.255.255.0
192.168.14.8
255.255.255.0
SP Switch
Adapter
node 8
node 1
SP Switch
Adapter
192.168.14.129
255.255.255.128
Figure 63. IP Traffic Flow When Issuing ping 192.168.13.1 on Node 8
5.2.3 Recovery Procedure for an SP Switch Adapter Card Failure
What happens, when one of the SP Switch Adapter cards fails? A detailed
1. When the SP Switch Adapter card with IP address 192.168.14.4 fails,
neither node 6 nor node 8 can communicate with node 1 in SP2. Node 10
is not interrupted.
2. When the SP Switch Adapter card with IP address 192.168.14.129 fails,
neither node 10 nor node 8 can communicate with node 1 in SP2. Node 6
is not interrupted.
Note: Node 8 is always affected, independent of the failing SP Switch
Adapter card. Try to avoid routing settings that need both SP Switch Adapter
cards for proper functioning.
196
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
To recover from a failing SP Switch Adapter card, you have to define an alias
IP address for the surviving card. Follow these steps when gt030fails:
1. Login as root to the SP Switch Router.
2. Remove the interface of gt030from active status:
ifconfig gt030 delete
3. Assign the alias IP address to gt050:
ifconfig gt050 alias 192.168.14.4 netmask 255.255.255.128
Follow these steps, when gt050fails:
1. Login as root to SP Switch Router.
2. Remove the interface of gt050from active status:
ifconfig gt050 delete
3. Assign the alias IP address to gt030:
ifconfig gt030 alias 192.168.14.129 netmask 255.255.255.128
After setting the alias address, verify with the grarpcommand that the arp
table shows the correct IP addresses and corresponding physical SP Switch
addresses, for example:
grf16:/root grarp 192.168.14.129
???
???
(0): 192.168.14.129 at 0:0:0:0:0:f
(0): 192.168.14.4 at 0:0:0:0:0:f
Both IP addresses have to show the same physical SP Switch address.
5.3 Multiple SP Partition and Multiple SP Switch Router Adapter Cards
A partitioned RS/6000 SP has some advantages: It is possible to separate
production and development systems that is, test new software in one
partition without crashing the entire system when one partition crashes. Last
but not least, several partitions can run several software versions that are
incompatible with each other in one partition.
But one large disadvantage remains: There is no high-speed connection
between several partitions. Using an IBM 9077 SP Switch Router with two SP
Switch Router Adapter cards lets you overcome this problem. You can
partition your SP and set up a high-speed connection between several
partitions. This scenario describes how to establish such a
199).
Single RS/6000 SP and Single SP Switch Router
197
Download from Www.Somanuals.com. All Manuals Search And Download.
Configuration assumptions:
• The RS/6000 SP is partitioned in to two or more partitions (two partitions
for this scenario).
Note: Be careful when choosing the partition layout. In every partition, a
free Switch chip port to connect the SP Switch Router Adapter card is
necessary.
• Every partition establishes a single subnet.
Note: Be careful when subnetting your SP partitions. Although not
necessary when you simply partition your SP, it is required that every
partition in effect form in a single subnet to establish a connection via an
SP Switch Router. All other subnetting does not work.
• ARP should be enabled on the SP Switch network to provide the greatest
flexibility in assigning IP addresses (strongly recommended!).
SP Switch
SP processor node
SP Switch Router
Adapter card 1
SP processor node
SP processor node
partition 1
partition 1
partition 2
GRF 400
Node 11
Node 12
SP Switch Router
Adapter card 2
partition 2
Node 15
SP2
Figure 64. Partition-to-Partition Connection with an SP Switch Router
198
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 21. Configuration of a Partition - Partition Connection
Adapter
IP Address
Netmask
SP Switch Router Adapter card 1
SP Switch Router Adapter card 2
Node 11 in SP2
192.168.13.4
192.168.13.129
192.168.13.130
192.168.13.131
192.168.13.132
255.255.255.128
255.255.255.128
255.255.255.128
255.255.255.128
255.255.255.128
255.255.255.128
Node 12 in SP2
Node 15 in SP2
All other processor nodes in SP2
192.168.13.1 -
192.168.13.15
Configuration:
For this scenario, the following routes have to be set:
1. On node 11, node 12 and node 15 in SP2, add the following route to the
switch network of partition 1 of SP2:
route add -net 192.168.13.0 -netmask 255.255.255.128 -mtu 65520 \
192.168.13.129
The MTU size of 65520 is the optimum size for the SP Switch network.
2. Check for correct routing entries, for example:
root@sp2n11:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.10.1/24
127/8
192.168.3/24
192.168.13/25
192.168.13.128/25 192.168.13.130
192.168.3.37
10.10.1.11
127.0.0.1
192.168.3.11
192.168.13.129
UG
U
U
U
UG
U
1
0
8
8
0
1188 en0
16 fi0
377 lo0
83717 en0
12107 css0 65520 -
49 css0 - -
- -
- -
- -
- -
1
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
3. On all nodes in partition 1 in SP2, add the following route to the switch
network of partition 2 in SP2:
route add -net 192.168.13.128 -netmask 255.255.255.128 -mtu 65520 \
192.168.13.4
Single RS/6000 SP and Single SP Switch Router
199
Download from Www.Somanuals.com. All Manuals Search And Download.
The -mtu parameter is again optional but should be set to ensure optimal
packet size on this route.
4. Check for correct routing entry:
root@sp2n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
127/8
192.168.3.37
127.0.0.1
UG
U
0
8
8
1
30 en0
397 lo0
86147 en0
4 css0
- -
- -
- -
- -
192.168.3/24
192.168.13/25
192.168.3.1
192.168.13.1
U
U
192.168.13.128/25 192.168.13.4
UG
0
3082 css0 65520 -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
5. On GRF 400, check /etc/grifconfig.conf for the following entry (or one
similar):
gt020 192.168.13.129 255.255.255.128
gt030 192.168.13.4
255.255.255.128
6. On the CWS of SP2, check if SP Switch Router Adapter cards are
configured. See if both SP Switch Router Adapter cards show up green in
perspectives or enter SDRGetObjects switch_responds. Use Eunfence if
needed.
7. Issue some pingcommands to check the connection:
On node 11, node 12 and node 15 of SP2, pingthe SP Switch interface of
any node in partition 1 in SP2, for example:
root@sp2n11:/ ping 192.168.13.1
PING 192.168.13.1: (192.168.13.1): 56 data bytes
64 bytes from 192.168.13.1: icmp_seq=0 ttl=254 time=0 ms
64 bytes from 192.168.13.1: icmp_seq=1 ttl=254 time=0 ms
^C
----192.168.13.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 0/0/0 ms
On any node in partition 1 in SP2, pingthe SP Switch interfaces of any
node in partition 2 in SP2, for example:
200
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
root@sp2n01:/ ping 192.168.13.132
PING 192.168.13.132: (192.168.13.132): 56 data bytes
64 bytes from 192.168.13.132: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 192.168.13.132: icmp_seq=1 ttl=254 time=1 ms
^C
----192.168.13.132 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms
If these pingcommands fail, check routing settings again. If everything is
as it should be try to pingthe SP Switch Router Adapter cards to find the
failing part:
ping 192.168.13.4 (on nodes in partition 1)
ping 192.168.13.129 (on nodes in partition 2)
If any further errors occur, check cabling, the configuration of SP Switch
Router media cards (See Section 3.7, “Step-by-Step Media Card
Configuration” on page 86) and also the network adapters in the nodes.
Performance:
We did no performance measurement for this scenario. This setup is rather
similar to the one described in Section 6.1, “RS/6000 SP Switch - RS/6000
SP Switch Connection” on page 203. There is no good reason why these
scenarios should achieve a different data transfer rate. So we expect a data
transfer rate of about 80 MB/s between the partitions, as before.
Single RS/6000 SP and Single SP Switch Router
201
Download from Www.Somanuals.com. All Manuals Search And Download.
202
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 6. Multiple RS/6000 SPs and One SP Switch Router
In this configuration, two RS/6000 SP systems are connected to a single SP
Switch Router. This enables both SPs to communicate deploying the SP
Switch data transfer rate and/or to share other network resources. See Figure
65 for an overview.
partition
SP processor node
SP Switch
SP Switch Router
Adapter card 1
SP processor node
SP 2
GRF 1600
partition
SP Switch Router
Adapter card 2
SP processor node
SP Switch
SP processor node
SP 21
Figure 65. Two RS/6000 SPs Connected to GRF 1600
6.1 RS/6000 SP Switch - RS/6000 SP Switch Connection
RS/6000 SPs to exploit the SP Switch data transfer rate. Because of the
switch cable length limit of 20m, this scenario is only applicable when both
SPs have a maximum distance of 40m. In all other cases you will need two
routers and a corresponding high speed connection (refer to Chapter 7,
Configuration assumptions:
• Both SP Switch Router Adapter cards have been installed according to
work properly.
Note: When configuring the second SP Switch Router Adapter card, ensure
that both Control Workstations are defined as SNMP managers in
© Copyright IBM Corp. 1998
203
Download from Www.Somanuals.com. All Manuals Search And Download.
/etc/snmpd.conf on the GRF 1600. To check, open this file and look for a
stanza similar to the following:
MANAGER
192.168.4.137
SEND ALL TRAPS
TO PORT 162
WITH COMMUNITY spenmgmt
This stanza is needed twice, one for each CWS. Otherwise, the SP Switch
Router Adapter card will hang in state loading. Additionally, when you assign
an IP address to the second SP Switch Router Adapter card using SMIT,
ensure that this IP address can be resolved by name service, be it DNS or
/etc/hosts based. Otherwise the card cannot be unfenced.
• The SP Switch Router Adapter card and SP processor node switch
adapters are in the same IP subnet on both SPs.
• ARP should be enabled on the SP Switch networks to provide the most
flexibility in assigning IP addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
To establish this scenario, on the GRF 1600, an SP Switch Router Adapter
card (slot 4) is attached to the SP Switch of SP2, a second card (slot 3) to the
netmask for all interfaces is 255.255.255.0.
Table 22. Configuration of SP Switch - SP Switch Connection
Adapter
IP Address
SP Switch Router Adapter card 1
SP Switch Router Adapter card 2
SP processor nodes in SP21
SP processor nodes in SP2
192.168.13.16
192.168.14.4
192.168.14.1 - 192.168.14.15
192.168.13.1 - 192.168.13.15
To successfully run this configuration, no routes need to be set on the SP
Switch Router. Just set the corresponding routes on all nodes of SP2 and
SP21:
1. On nodes in SP21, add the following route to the switch network of SP2:
204
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 65280 \
192.168.14.4
2. Check for correct routing entry:
root@sp21n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
192.168.4.137
192.168.14.4
192.168.14.4
192.168.14.4
127.0.0.1
192.168.4.1
192.168.14.4
192.168.14.1
UG
UG
UG
UG
U
U
UG
U
0
0
0
0
8
515 en0
65 css0
2 css0 4352 -
37 css0
518 lo0
- -
- -
10.1.1/24
10.10.1/24
10.50.1/24
127/8
192.168.4/24
192.168.13/24
192.168.14/24
- -
- -
- -
10 345052 en0
0 1360145 css0 65280 -
6 283472 css0
- -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
3. On nodes in SP2, add the following route to the switch network of SP21:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 65280 \
192.168.13.16
The -mtu parameter is optional but should be set to ensure optimal packet
size on this route.
4. Check for correct routing entry:
root@sp2n01:/ netstat -rn
Routing tables
Destination
Gateway
Flags Refs
Use If PMTU Exp Groups
Route Tree for Protocol Family 2 (Internet):
default
10.1.1/24
127/8
192.168.3/24
192.168.13/24
192.168.14/24
192.168.3.37
192.168.13.4
127.0.0.1
192.168.3.1
192.168.13.1
192.168.13.16
UG
UG
U
U
U
1
0
8
398 en0
94 css0
541 lo0
- -
- -
- -
- -
- -
11 1065877 en0
76677 css0
2
UG
0 431146 css0 65520 -
Route Tree for Protocol Family 24 (Internet v6):
::1 ::1 UH
0
0 lo0 16896 -
5. On GRF 1600, check /etc/grifconfig.conf for the following entry:
gt030 192.168.14.4
gt040 192.168.13.16 255.255.255.0 -
255.255.255.0 -
mtu 65520
mtu 65520
6. On the CWS of SP2 and on the CWS of SP21, check if SP Switch Router
Adapter cards are configured. Check if the SP Switch Router Adapter
Multiple RS/6000 SPs and One SP Switch Router 205
Download from Www.Somanuals.com. All Manuals Search And Download.
cards shows up green in perspectives or enter SDRGetObjects
switch_responds. Use Eunfence if needed.
7. Issue some pingcommands to check the connection:
On the SP2 nodes, pingthe SP Switch interface of any node in SP21:
root@sp2n01:/ ping 192.168.14.1
PING 192.168.14.1: (192.168.14.1): 56 data bytes
64 bytes from 192.168.14.1: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 192.168.14.1: icmp_seq=1 ttl=254 time=1 ms
^C
----192.168.14.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms
On the SP21 nodes, pingthe SP Switch interface of any node in SP2:
root@sp21n01:/ ping 192.168.13.1
PING 192.168.13.1: (192.168.13.1): 56 data bytes
64 bytes from 192.168.13.1: icmp_seq=0 ttl=254 time=1 ms
64 bytes from 192.168.13.1: icmp_seq=1 ttl=254 time=1 ms
^C
----192.168.13.1 PING Statistics----
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max = 1/1/1 ms
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingboth SP Switch Adapter cards to find the failing
part:
ping 192.168.13.16 (on SP2 nodes)
ping 192.168.14.4 (on SP21 nodes)
If any errors occur, check cabling, the configuration of the SP Switch
Router cards (see Section 3.7, “Step-by-Step Media Card Configuration”
on page 86) and also the switch adapters in the nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved
between the SPs, we again used ftp to transfer several 300MB files from
different nodes in SP2 to several nodes in SP21. We sent these files to
/dev/null to eliminate any hard disk influence on the receiver side.
The requisites for this test are the same as described in Section 5.1.4.1, “SP
Switch - FDDI Connection without Bridging” on page 174. We started ftp
transfers on all available nodes of SP2 and on some nodes of SP21.
206
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
With this scenario and without further tuning (refer to Appendix A, “Laboratory
parameters), we measured a stable cumulative transfer rate of up to 83 MB/s
(observed with the freeware tool monitor). This was the maximum transfer
rate achievable in our environment. We are confident that further tuning and
faster nodes can achieve higher transfer rates up to the maximum of GRFs
crosspoint switch which is 100 MB/s.
6.2 Sharing Network Resources
This scenario is a combination of the scenario described in Section 6.1,
or more of the scenarios described in Section 5.1.1, “SP Switch - Ethernet
page 174. In an extended network, resources are always rare. This scenario
might help to share these resources between several SPs and at the same
partition
SP processor node
SP Switch
Ethernet
SP Switch Router
Adapter card 1
SP processor node
FDDI
SP 2
GRF 1600
partition
ATM
SP Switch Router
Adapter card 2
SP processor node
HIPPI
SP Switch
SP processor node
SP 21
Figure 66. Sharing Network Resources between Two SPs
SP Switch - RS/6000 SP Switch Connection” on page 203 first. If all tests are
successful, set up the connection from each SP and its nodes to the required
network resources. Follow the steps given in Section 5.1, “Single SP Partition
Multiple RS/6000 SPs and One SP Switch Router 207
Download from Www.Somanuals.com. All Manuals Search And Download.
interference between data transfers from each SP to different SP Switch
Router Adapter media cards except the one caused by limited bandwidth of
the different network types (FDDI, ATM).
208
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Chapter 7. Multiple RS/6000 SPs and Multiple GRFs
In this section, sample configurations with two SP systems connected with
two SP Switch Routers are presented. The routers, in turn, are connected by
any kind of high speed network supported by the GRF. Preferable selections
are dedicated high performance networks such as FDDI, ATM or HIPPI.
Figure 67 on page 209 will help to understand the setup.
This scenario might be suitable to connect two SP systems that are installed
in different locations - such as different buildings - and cannot be connected
directly with one SP Switch Router, as in Chapter 6, “Multiple RS/6000 SPs
SP
partitition 1
net 192.168.13.0
Switch
Router 1
IP 192.168.13.4
SP processor node
SP Switch Router
Adapter card
SP Switch 1
mask 255.255.255.0
SP processor node
mask 255.255.255.0
net 192.168.14.0
partition 2
SP
Switch
Router 2
IP 192.168.14.4
SP processor node
SP Switch Router
Adapter card
SP Switch 2
SP processor node
mask 255.255.255.0
mask 255.255.255.0
Figure 67. Connection of Two SPs with Two SP Switch Routers
7.1 ATM OC-3c Backbone Connection
ATM media card has two ports on it. So why would one use only one of them
to connect two GRFs?
Remember, the GRF is a router and as such expects every port on any media
card to be in a different network (logical or physical), otherwise there would
be no need to route. This means that without further work the two ports of the
ATM media card cannot be used simultaneously to connect two GRFs with a
© Copyright IBM Corp. 1998
209
Download from Www.Somanuals.com. All Manuals Search And Download.
fast network. (The spare second port might be in use for example for a
connection to an ATM attached device that needs a fast path to the SP Switch
network).
To make use of both ports, four possible solutions come to mind:
1. Break up your already existing network topology (nodes of the SP) and
create different subnets, which in turn can be routed over the different
ATM subnets on the two ports.
We do not expect that any customer will do this, but the solution is
mentioned here for the sake of completeness.
2. Configure the two ATM ports on different networks and route one half of
the nodes over the first ATM port and route the other half over the second
ATM port. It will be a pain, though, to maintain the different routes, so
solution 4. below might be considered.
3. Configure the two ATM ports for transparent bridging and use the two
ports simultaneously with just one IP address assigned to them. This
solution sounds promising and will be covered in Section 7.1.2, “ATM
OC-3c Backbone - Using Two Ports” on page 215. Be prepared for a
surprise.
4. Configure dynamic routing and use the open shortest path first (OSPF)
protocol to use the two ATM ports equivalently. They have to be in
different subnets, but gated will take care of that. Configuring gated is said
to be a non-trivial task and once running, it might well turn a stable system
into a true binary specimen.
7.1.1 ATM OC-3c Backbone - Using One Port
Now, let us consider the setup of the one-port ATM backbone connection with
two GRF routers.
Configuration assumptions:
• The SP Switch Router ATM media card has been installed according to
and works properly.
• The SP Switch Router Adapter card has been installed according to
GRF routers and works properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet on the respective SP.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP addresses (strongly recommended!).
210
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
In this scenario, we have the SP Switch of SP21 connected to the GRF 1600.
The GRF 1600 has its ATM OC-3c media card’s port 00 connected to the
GRF 400 ATM OC-3c media card’s port 00. The GRF 400 in turn is attached
The netmask for all interfaces is 255.255.255.0.
SP
Net 192.168.13.0
Net 10.1.1.2
Switch
Router 1
IP 192.168.13.4
SP processor node
ATM OC-3c
Adapter card
SP Switch 1
SP2
SP Switch Router
Adapter card 1
Mask 255.255.255.0
SP processor node
GRF 400
Mask 255.255.255.0
Net 192.168.14.0
SP
Net 10.1.1.1
Switch
Router 2
IP 192.168.14.4
SP processor node
SP processor node
ATM OC-3c
Adapter card
SP Switch 2
SP21
SP Switch Router
Adapter card 2
Mask 255.255.255.0
GRF 1600
Figure 68. SP Switch - ATM - SP Switch Connection
Multiple RS/6000 SPs and Multiple GRFs
211
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 23. Configuration of SP Switch - ATM - SP Switch
Adapter
IP Address
192.168.13.4
SP Switch Router Adapter card 1
SP Switch Router ATM media card 10.1.1.2
(port 00) in GRF 400
SP Switch Router ATM media card 10.1.1.1
(port 00) in GRF 1600
SP Switch Router Adapter card 2
192.168.14.4
To successfully run this configuration, a route to the distant SP Switch
network has to be set on every SP Switch Router. On the nodes of SP2 and
SP21, respectively, routes to the nodes of the distant SP have to be set.
The media card on the GRF routers should already be up and running
settings are repeated here, nevertheless, to be on the safe side:
• On GRF 1600 (ATM card in slot 1, SP Switch card in slot 3):
1. The file /etc/gratm.conf needs the configuration statements for the port
used:
Traffic_Shape name=high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
Interface ga010 traffic_shape=high_speed_high_quality
PVC ga010 0/132 proto=ip traffic_shape=high_speed_high_quality
2. The file /etc/grifconfig.conf has the following entries:
gt030 192.168.14.4
ga010 10.1.1.1
255.255.255.0 -
255.255.255.0 -
mtu 65520
mtu 9180
3. The file /etc/grroute.conf has the following line:
192.168.13.0 255.255.255.0 10.1.1.2
This will set the correct route to the other SP Switch network over the
ATM interface automatically; of course, this route could also be set
manually every time the GRF is rebooted.
4. The SP Switch Router Adapter card is connected to the SP Switch and
configured, too. Check with SDRGetObjects switch_respondson the CWS
and use Eunfenceif needed.
• On GRF 400 (ATM card in slot 2, SP Switch card in slot 1):
212
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
1. The file /etc/gratm.conf needs the configuration statements for the port
used:
Traffic_Shape name=high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
Interface ga020 traffic_shape=high_speed_high_quality
PVC ga020 0/132 proto=ip traffic_shape=high_speed_high_quality
2. The file /etc/grifconfig.conf has the following entries:
gt010 192.168.13.4
ga020 10.1.1.2
255.255.255.0 -
255.255.255.0 -
mtu 65520
mtu 9180
3. The file /etc/grroute.conf has the following line:
192.168.14.0 255.255.255.0 10.1.1.1
This will set the correct route to the other SP Switch network over the
ATM interface automatically; of course, this route could also be set
manually every time the GRF is rebooted.
4. The SP Switch Router Adapter card is connected to the SP Switch and
configured, too. Check with SDRGetObjects switch_respondson the CWS
and use Eunfenceif needed.
5. On the nodes in SP21, the following route needs to be set:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 9180 \
192.168.14.4
6. On the nodes in SP2, the following route needs to be set:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 9180 \
192.168.13.4
The above command should be given in one line, the \ is to show where
the line break occurred during printing. To avoid any pitfalls, set the MTU
size explicitly to the size of the ATM adapter.
Hint: You must not use SMIT to set this route and put it into the ODM. The
SP Switch is not operational at the time, this route would be set during
boot time. Therefore this route would be put onto another, already
available network interface, for example the Control Ethernet, and this is
definitely not what you want to happen.
Use a separate /etc/rc.routes shell script that is run only after an Estartor
an Eunfencewas issued, or use some other mechanisms to have this route
set only after the css0 interfaces on the SP nodes are up and running.
Setup is done now, and every node in SP2 should now be able to pingevery
node in SP21 and vice versa.
5. Check for correct routing entries on all nodes in SP21:
Multiple RS/6000 SPs and Multiple GRFs
213
Download from Www.Somanuals.com. All Manuals Search And Download.
root@sp21cw0:/ dsh -a netstat -rn | grep 192.168.13
sp21n01: 192.168.13/24
sp21n05: 192.168.13/24
sp21n06: 192.168.13/24
sp21n07: 192.168.13/24
sp21n08: 192.168.13/24
sp21n09: 192.168.13/24
sp21n10: 192.168.13/24
sp21n11: 192.168.13/24
sp21n13: 192.168.13/24
sp21n15: 192.168.13/24
root@sp21cw0:/
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
UG
UG
UG
UG
UG
UG
UG
UG
UGc
UGc
1 1020086 css0 9180 -
0
0
12456 css0 9180 -
15 css0 9180 -
0 731470 css0 9180 -
0 css0 9180 -
0
0 1533484 css0 9180 -
0 643863 css0 9180 -
0 1460254 css0 9180 -
0
0
0 css0 9180 -
0 css0 9180 -
6. Check for correct routing entries on all nodes in SP2:
root@sp2cw0:/ dsh -a netstat -rn | grep 192.168.14
sp2n01: 192.168.14/24
sp2n05: 192.168.14/24
sp2n06: 192.168.14/24
sp2n07: 192.168.14/24
sp2n08: 192.168.14/24
sp2n09: 192.168.14/24
sp2n10: 192.168.14/24
sp2n11: 192.168.14/24
sp2n12: 192.168.14/24
sp2n13: 192.168.14/24
sp2n14: 192.168.14/24
sp2n15: 192.168.14/24
root@sp2cw0:/
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
0 1099887 css0 9180 -
0 1299484 css0 9180 -
0 352513 css0 9180 -
0 999147 css0 9180 -
0 146784 css0 9180 -
0 355671 css0 9180 -
0 879759 css0 9180 -
0 1026871 css0 9180 -
0 1099748 css0 9180 -
0 340367 css0 9180 -
0 347869 css0 9180 -
0 351648 css0 9180 -
7. Issue some pingcommands to check the connection:
On the SP21 nodes, pingthe SP Switch interface of nodes in SP; on
nodes in SP2, pingthe SP Switch interface of nodes in SP21.
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router ATM media card or the SP
Switch Router Adapter card to find the failing part:
ping 192.168.14.4 (on SP21 processor nodes)
ping 192.168.13.4 (on SP2 processor nodes)
ping 10.1.1.1and ping 10.1.1.2 (on both GRF 400 and GRF 1600)
If any errors occur, check cabling, the configuration of the SP Switch
Router media cards (see Section 3.7, “Step-by-Step Media Card
page 110) and the Switch adapters in the SP nodes.
214
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Performance:
To get a rough overview of the data transfer rates that can be achieved in this
scenario, the following test was performed:
1. We used ftp to conduct several file transfers of a 300 MB file from the
nodes in SP2 to one chosen node in SP21, and at the same time used ftp
to conduct several file transfers of a 300 MB file from the nodes in SP21 to
a chosen node in SP2. We sent the files to /dev/null on the receiving node
to eliminate any hard disk influence.
We saw up to about 14.5 MB/s with just one side sending data; with all
nodes sending and receiving, we achieved an duplex throughput of about
24 MB/s on the ATM port.
So it just might turn out that one 155 Mbit ATM port is not enough for a
performance connection between two SP systems.
7.1.2 ATM OC-3c Backbone - Using Two Ports
This setup is basically the same as using just one port of the ATM card, as
210. To avoid the difficulties regarding the need of different (sub)networks on
the two ports, we decided to use the GRF bridging implementation as
SP
Net 192.168.13.0
Net 10.1.1.2
Switch
Router 1
IP 192.168.13.4
SP processor node
ATM OC-3c
Adapter card
SP Switch 1
SP2
SP Switch Router
Adapter card 1
Mask 255.255.255.0
SP processor node
GRF 400
Mask 255.255.255.0
Net 192.168.14.0
SP
Net 10.1.1.1
Switch
Router 2
IP 192.168.14.4
SP processor node
SP processor node
ATM OC-3c
Adapter card
SP Switch 2
SP21
SP Switch Router
Adapter card 2
Mask 255.255.255.0
GRF 1600
Figure 69. SP Switch - ATM Bridged - SP Switch Connection
Multiple RS/6000 SPs and Multiple GRFs
215
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 24. Configuration of SP Switch - ATM Bridged - SP Switch
Adapter
IP Address
SP Switch Router Adapter card 1
192.168.13.4
SP Switch Router ATM media card 10.1.1.2
(port 00) in GRF 400
SP Switch Router ATM media card 10.1.1.1
(port 00) in GRF 1600
SP Switch Router Adapter card 2
192.168.14.4
Basically, the two ports are managed by a specific daemon and packets are
sent over the participating ports depending on parameters in the
configuration file /etc/bridged.conf.
A "wrapped" version of vi, called bredit, is available on the GRF router, and
you should use it to create or edit the configuration file by running the
command bredit.
First, we look at the GRF 1600:
1. The following screen shot gives you the minimum required data to be put
into /etc/bridged.conf:
# bredit
# bridge_group bg0 {
# port gf000 gf001 gf002 gf003;
# };
bridge_group bg1 (
port ga010 ga0180;
};
:wq!
/tmp/bridged.conf.6117: 7 lines, 101 characters.
Update /etc/bridged.conf with these changes? [y/n] [n] y
Parsed file "/tmp/bridged.conf.6117" successfully.
Changes committed.
Use grwrite(8) to make the changes permanent.
Signal bridged to effect changes now? [y/n] [n] n
Use ’brsig hup’ to force bridged(8) to re-read the configuration file.
#
out for our convenience. Up to 16 bridge groups are allowed on a GRF
216
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Router, so there was no technical reason for it. The :wq!in the screen shot
is there to remind you how to end editing of the file, and we give an n(no)
as answer to the last question, as the ATM ports are already in use and
therefore cannot be modified. So we need to rebootthe GRF, and this
takes care of the relevant settings anyway. See Section 4.6.9.3, “Editing
Utility – Bredit” on page 146 for more information.
2. The following changes need to be applied to /etc/grifconfig.conf:
#ga010 10.1.1.1
#ga0180 10.1.2.1
255.255.255.0 -
255.255.255.0 -
255.255.255.0 -
mtu 9180
mtu 9180
mtu 9180
bg1
10.1.1.1
Remarks: The data for the ga010 and ga0180interface needs to be
commented out and the line for bg1bridge_group needs to be put in. The
netmask entry is mandatory for bridge_groupentries.
3. Finally, bridging requires these entries in the /etc/gratm.conf file:
Traffic_Shape name=high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
This entry remains the same as in the basic configuration:
Interface ga010 traffic_shape=high_speed_high_quality \
bridge_method=llc_encapsulated
Interface ga0180 traffic_shape=high_speed_high_quality \
bridge_method=llc_encapsulated
These two lines were changed from the basic configuration!
PVC ga010 0/132 proto=llc,bridging
PVC ga0180 0/134 proto=llc,bridging
Because the GRF supports inverse ATM ARP (InATMARP), there is no need
to put any entries in /etc/grarp.conf.
The file /etc/grroute.conf also remains unchanged:
192.168.13.0
255.255.255.0 10.1.1.2
If there is no such entry in /etc/grroute.conf, the following command must be
run on GRF 1600:
route add -net 192.168.13.0 -netmask 255.255.255.0 -mtu 9180 10.1.1.2
Multiple RS/6000 SPs and Multiple GRFs
217
Download from Www.Somanuals.com. All Manuals Search And Download.
Now it is time to look at the GRF 400:
1. The following screen shot gives you the minimum required data to be put
into /etc/bridged.conf:
# bredit
Could not find default config file ’/etc/bridged.conf’.
This seems to be the first time bridged is being configured.
Do you want to use the template configuration file ? [y/n] [n] y
G
bridge_group bg1 (
port ga020 ga0280;
};
:wq!
/tmp/bridged.conf.2371: 7 lines, 101 characters.
Update /etc/bridged.conf with these changes? [y/n] [n] y
Parsed file "/tmp/bridged.conf.2371" successfully.
Changes committed.
Use grwrite(8) to make the changes permanent.
Signal bridged to effect changes now? [y/n] [n] n
Use ’brsig hup’ to force bridged(8) to re-read the configuration file.
#
Remarks: On the GRF 400, bridging was never defined before, so we
have to create /etc/bridged.conf from a template. Just jump to the end of
the template and type in the data. The :wq!is there to remind you how to
end editing of the file. We give an n(no) as answer to the last question, as
the ATM ports are already in use and therefore cannot be modified. So we
need to rebootthe GRF, and takes care of the relevant settings anyway.
2. The following changes need to be applied to /etc/grifconfig.conf:
#ga020 10.1.1.2
#ga0280 10.1.2.2
255.255.255.0 -
255.255.255.0 10.1.2.1
255.255.255.0 -
mtu 9180
bg1
10.1.1.2
mtu 9180
Remarks: The data for the ga020 and ga0280interface needs to be
commented out and the line for bg1bridge_group needs to be put in. The
netmask entry is mandatory for bridge_groupentries.
3. Finally, bridging requires four entries in the /etc/gratm.conf file:
Traffic_Shape name=high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
This following entries remains the same as is the basic configuration:
Interface ga020 traffic_shape=high_speed_high_quality \
bridge_method=llc_encapsulated
Interface ga0280 traffic_shape=high_speed_high_quality \
bridge_method=llc_encapsulated
218
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
This two lines were changed from the basic configuration!
PVC ga020 0/132 proto=llc,bridging
PVC ga0280 0/134 proto=llc,bridging
Because the GRF supports InATMARP, there is no need to have any entries
in /etc/grarp.conf.
The file /etc/grroute.conf also remains unchanged:
192.168.14.0
255.255.255.0 10.1.1.1
If there is no such entry in /etc/grroute.conf, the following command must be
run on GRF 400 after every reboot:
route add -net 192.168.14.0 -netmask 255.255.255.0 -mtu 9180 10.1.1.1
We chose to have the IP address of the bg1 "interface" the same as the
respective ATM port0 interface before, so there was no need to change any
routes on the SP2 or SP21 nodes.
Check for correct routing entry on all nodes in SP21:
root@sp21cw0:/ dsh -a netstat -rn | grep 192.168.13
sp21n01: 192.168.13/24
sp21n05: 192.168.13/24
sp21n06: 192.168.13/24
sp21n07: 192.168.13/24
sp21n08: 192.168.13/24
sp21n09: 192.168.13/24
sp21n10: 192.168.13/24
sp21n11: 192.168.13/24
sp21n13: 192.168.13/24
sp21n15: 192.168.13/24
root@sp21cw0:/
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
UG
UG
UG
UG
UG
UG
UG
UG
UGc
UGc
1 1020086 css0 9180 -
0
0
12456 css0 9180 -
15 css0 9180 -
0 731470 css0 9180 -
0 css0 9180 -
0
0 1533484 css0 9180 -
0 643863 css0 9180 -
0 1460254 css0 9180 -
0
0
0 css0 9180 -
0 css0 9180 -
Check for correct routing entry on all nodes in SP2:
Multiple RS/6000 SPs and Multiple GRFs
219
Download from Www.Somanuals.com. All Manuals Search And Download.
root@sp2cw0:/ dsh -a netstat -rn | grep 192.168.14
sp2n01: 192.168.14/24
sp2n05: 192.168.14/24
sp2n06: 192.168.14/24
sp2n07: 192.168.14/24
sp2n08: 192.168.14/24
sp2n09: 192.168.14/24
sp2n10: 192.168.14/24
sp2n11: 192.168.14/24
sp2n12: 192.168.14/24
sp2n13: 192.168.14/24
sp2n14: 192.168.14/24
sp2n15: 192.168.14/24
root@sp2cw0:/
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
0 1099887 css0 9180 -
0 1299484 css0 9180 -
0 352513 css0 9180 -
0 999147 css0 9180 -
0 146784 css0 9180 -
0 355671 css0 9180 -
0 879759 css0 9180 -
0 1026871 css0 9180 -
0 1099748 css0 9180 -
0 340367 css0 9180 -
0 347869 css0 9180 -
0 351648 css0 9180 -
Issue some pingcommands to check the connection:
On the SP21 nodes, pingthe SP Switch interface of nodes in SP2, on
nodes in SP2 pingthe SP Switch interface of nodes in SP21.
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router ATM media card or the SP
Switch Router Adapter card, to find the failing part:
ping 192.168.14.4 (on SP21 processor nodes)
ping 192.168.13.4 (on SP2 processor nodes)
ping 10.1.1.1and ping 10.1.1.2 (on both GRF 400 and GRF 1600)
If any errors occur, check cabling, the configuration of the SP Switch
Router media cards (see Section 3.7, “Step-by-Step Media Card
page 110) and the Switch adapters in the SP nodes.
Performance:
To get a rough overview of the data transfer rates that can be achieved in this
scenario, the following test was performed:
1. We used ftp to conduct several file transfers of a 300 MB file from the
nodes in SP2 to one chosen node in SP21, and at the same time used ftp
to conduct several file transfers of a 300 MB file from the nodes in SP21 to
a chosen node in SP2. We sent the files to /dev/null on the receiving
nodes to eliminate any hard disk influence:
We saw up to about 14.5 MB/s with just one side sending data; with all
nodes sending and receiving, we achieved a duplex throughput of no more
than 24 MB/s over the ATM ports.
220
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
So what happened to the expected doubling of the aggregate throughput?
As it turns out, even with bridging activated, only one ATM port is allowed
to send and receive data. The second port is blocked, as can be seen in
the following screen shot:
grf16:/root brstat
Bridge Group bg1
Spanning Tree: Enabled
Designated Root: 32768 00:c0:80:84:8c:eb
Bridge ID:
32768 00:c0:80:96:38:68
Root Port: ga010, Root Path Cost: 10
Topology Change Detected: No
Root Max Age: 20, Hello Time: 2, Forward Delay: 15
Bridge Max Age: 20, Hello Time: 2, Forward Delay: 15, Hold Time: 1
Path Desig Desig
Cost Cost Bridge
Desig
Port
Interface Port ID Con State
--------- ------- --- ---------- ----- ----- ----------------------- -------
*ga010
ga0180
128 1 Yes Forwarding 10
128 2 Yes Blocking 10
0
0
32768 00:c0:80:84:8c:eb 128 1
32768 00:c0:80:84:8c:eb 128 2
Dump snapshot finished at Mon Jun 15 20:01:33 1998
This bridging environment is useful, nevertheless. It provides an inherent
redundancy and failover mechanism. When port 0 is unplugged, the
up-to-then blocked port 1 gets activated and takes over the traffic with a
delay of about 10 seconds. With port 1 having the same IP address, this is
fully transparent to any clients.
See the next screen shot for details:
Multiple RS/6000 SPs and Multiple GRFs
221
Download from Www.Somanuals.com. All Manuals Search And Download.
grf16:/root brstat
Bridge Group bg1
Spanning Tree: Enabled
Designated Root: 32768 00:c0:80:84:8c:eb
Bridge ID:
32768 00:c0:80:96:38:68
Root Port: ga0180, Root Path Cost: 10
Topology Change Detected: No
Root Max Age: 20, Hello Time: 2, Forward Delay: 15
Bridge Max Age: 20, Hello Time: 2, Forward Delay: 15, Hold Time: 1
Path Desig Desig
Cost Cost Bridge
Desig
Port
Interface Port ID Con State
--------- ------- --- ---------- ----- ----- ----------------------- -------
ga010
*ga0180
128 1 No Disabled 10
128 2 Yes Forwarding 10
0
32768 00:c0:80:84:8c:eb 128 2
Dump snapshot finished at Mon Jun 15 20:01:51 1998
When port 0 is plugged in again, it resumes and takes over port 1, which in
turn falls back to the blocked state, as the following screen shot proves:
grf16:/root brstat
Bridge Group bg1
Spanning Tree: Enabled
Designated Root: 32768 00:c0:80:84:8c:eb
Bridge ID:
32768 00:c0:80:96:38:68
Root Port: ga010, Root Path Cost: 10
Topology Change Detected: No
Root Max Age: 20, Hello Time: 2, Forward Delay: 15
Bridge Max Age: 20, Hello Time: 2, Forward Delay: 15, Hold Time: 1
Path Desig Desig
Cost Cost Bridge
Desig
Port
Interface Port ID Con State
--------- ------- --- ---------- ----- ----- ----------------------- -------
*ga010
ga0180
128 1 Yes Forwarding 10
128 2 Yes Blocking 10
0
0
32768 00:c0:80:84:8c:eb 128 1
32768 00:c0:80:84:8c:eb 128 2
Dump snapshot finished at Mon Jun 15 20:02:03 1998
We expect this to happen also with the one-ported ATM OC-12c 622Mbit
media card, so if two of them are put into a bridging group, this could be a
fault tolerant scenario.
7.2 ATM OC-12c Backbone - One Port
This setup is basically the same as using just one port of an ATM OC-3c card,
as described in Section 7.1.1, “ATM OC-3c Backbone - Using One Port” on
222
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Configuration assumptions:
• The SP Switch Router ATM media card has been installed according to
routers and works properly.
• The SP Switch Router Adapter card has been installed according to
GRF routers and works properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet on the respective SP.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Configuration:
In this scenario, we have the SP Switch of SP21 connected to the GRF 1600.
The GRF 1600 has its ATM OC-12c media card’s port 00 connected to the
GRF 400 ATM OC-12c media card’s port 00. The GRF 400 in turn is attached
The netmask on all interfaces is 255.255.255.0.
SP
Net 192.168.13.0
Net 10.50.1.2
Switch
Router 1
IP 192.168.13.4
SP processor node
SP processor node
ATM OC-12c
Adapter card
SP Switch 1
SP2
SP Switch Router
Adapter card 1
Mask 255.255.255.0
GRF 400
Mask 255.255.255.0
Net 192.168.14.0
SP
Net 10.50.1.1
Switch
Router 2
IP 192.168.14.4
SP processor node
SP processor node
ATM OC-12c
Adapter card
SP Switch 2
SP21
SP Switch Router
Adapter card 2
Mask 255.255.255.0
GRF 1600
Mask 255.255.255.0
Figure 70. SP Switch - ATM OC-12c - SP Switch Connection
Multiple RS/6000 SPs and Multiple GRFs
223
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 25. Configuration of SP Switch - ATM OC-12c - SP Switch
Adapter
IP Address
SP Switch Router Adapter card 1
192.168.13.4
SP Switch Router ATM media card 10.20.30.2
(port 00) in GRF 400
SP Switch Router ATM media card 10.20.30.1
(port 00) in GRF 1600
SP Switch Router Adapter card 2
192.168.14.4
To successfully run this configuration, on every SP Switch Router a route to
the distant SP Switch network has to be set. On the nodes of SP2 and SP21,
respectively, routes to the nodes of the distant SP have to be set.
The media card adapters on the GRF routers should already be up and
running (according to Section 3.7, “Step-by-Step Media Card Configuration”
important settings are repeated here, nevertheless, to be on the safe side:
• On the GRF 1600 (ATM card in slot 2, SP Switch card in slot 3):
1. The file /etc/gratm.conf needs the configuration statements for the port
used:
Traffic_Shape name=bigg_speed_high_quality \
peak=622000 sustain=622000 burst=2048 qos=high
Interface ga020 traffic_shape=bigg_speed_high_quality
PVC ga020 0/132 proto=ip traffic_shape=bigg_speed_high_quality
2. The file /etc/grifconfig.conf has the following entries:
gt030 192.168.14.4
ga020 10.20.30.1
255.255.255.0 -
255.255.255.0 -
mtu 65520
mtu 9180
3. The file /etc/grroute.conf has the following line:
192.168.13.0 255.255.255.0 10.20.30.2
This sets the correct route to the other SP Switch network over the
ATM OC-12c interface automatically; of course, this route could also be
set manually every time the GRF is rebooted.
4. The SP Switch Router Adapter card is connected to the SP Switch and
configured, too. Check with SDRGetObjects switch_respondson the CWS
and use Eunfenceif needed.
• On the GRF 400 (ATM card in slot 1, SP Switch card in slot 3):
224
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
1. The file /etc/gratm.conf needs the configuration statements for the port
used:
Traffic_Shape name=bigg_speed_high_quality \
peak=622000 sustain=622000 burst=2048 qos=high
Interface ga010 traffic_shape=bigg_speed_high_quality
PVC ga010 0/132 proto=ip traffic_shape=bigg_speed_high_quality
2. The file /etc/grifconfig.conf has the following entries:
gt030 192.168.13.4
ga010 10.20.30.2
255.255.255.0 -
255.255.255.0 -
mtu 65520
mtu 9180
3. The file /etc/grroute.conf has the following line:
192.168.14.0 255.255.255.0 10.20.30.1
This sets the correct route to the other SP Switch network over the
ATM OC-12c interface automatically; of course, this route could also be
set manually every time the GRF is rebooted.
4. The SP Switch Router Adapter card is connected to the SP Switch and
configured, too. Check with SDRGetObjects switch_respondson the CWS
and use Eunfenceif needed.
2. On the nodes in SP21 the following route needs to be set:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 9180 \
192.168.14.4
3. On the nodes in SP2 the following route needs to be set:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 9180 \
192.168.13.4
To avoid any pitfalls, set the MTU size explicitly to the size of the ATM
adapter.
Hint: You must not use SMIT to set this route and put it into the ODM. The
SP Switch is not operational at the time this route would be set during boot
time. Therefore, this route would be put onto another, already available
network interface, for example, the Control Ethernet, and this is definitely
not what you want to happen.
Use a separate /etc/rc.routes shell script that is run only after an Estartor
an Eunfencewas issued, or use some other mechanisms to have this route
set only after the css0 interfaces on the SP nodes are up and running.
Setup is done now, and every node in SP2 should now be able to pingevery
node in SP21 and vice versa.
4. Check for correct routing entries on all nodes in SP21:
Multiple RS/6000 SPs and Multiple GRFs
225
Download from Www.Somanuals.com. All Manuals Search And Download.
root@sp21cw0:/ dsh -a netstat -rn | grep 192.168.13
sp21n01: 192.168.13/24
sp21n05: 192.168.13/24
sp21n06: 192.168.13/24
sp21n07: 192.168.13/24
sp21n08: 192.168.13/24
sp21n09: 192.168.13/24
sp21n10: 192.168.13/24
sp21n11: 192.168.13/24
sp21n13: 192.168.13/24
sp21n15: 192.168.13/24
root@sp21cw0:/
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
UG
UG
UG
UG
UG
UG
UG
UG
UGc
UGc
1 1020086 css0 9180 -
0
0
12456 css0 9180 -
15 css0 9180 -
0 731470 css0 9180 -
0 css0 9180 -
0
0 1533484 css0 9180 -
0 643863 css0 9180 -
0 1460254 css0 9180 -
0
0
0 css0 9180 -
0 css0 9180 -
5. Check for correct routing entries on all nodes in SP2:
root@sp2cw0:/ dsh -a netstat -rn | grep 192.168.14
sp2n01: 192.168.14/24
sp2n05: 192.168.14/24
sp2n06: 192.168.14/24
sp2n07: 192.168.14/24
sp2n08: 192.168.14/24
sp2n09: 192.168.14/24
sp2n10: 192.168.14/24
sp2n11: 192.168.14/24
sp2n12: 192.168.14/24
sp2n13: 192.168.14/24
sp2n14: 192.168.14/24
sp2n15: 192.168.14/24
root@sp2cw0:/
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
0 1099887 css0 9180 -
0 1299484 css0 9180 -
0 352513 css0 9180 -
0 999147 css0 9180 -
0 146784 css0 9180 -
0 355671 css0 9180 -
0 879759 css0 9180 -
0 1026871 css0 9180 -
0 1099748 css0 9180 -
0 340367 css0 9180 -
0 347869 css0 9180 -
0 351648 css0 9180 -
6. Issue some pingcommands to check the connection:
On the SP21 nodes, pingthe SP Switch interface of nodes in SP2; on
nodes in SP2 pingthe SP Switch interface of nodes in SP21.
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router ATM media card or the SP
Switch Router Adapter card to find the failing part:
ping 192.168.14.4 (on SP21 processor nodes)
ping 192.168.13.4 (on SP2 processor nodes)
ping 10.20.30.1and ping 10.20.30.2 (on both GRF 400 and GRF 1600)
If any errors occur, check cabling, the configuration of the SP Switch
Router media cards (see Section 3.7, “Step-by-Step Media Card
226
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Performance:
To get a rough overview of the data transfer rates that can be achieved in this
scenario, the following test was performed:
1. We used ftp to conduct several file transfers of a 300 MB file from the
nodes in SP2 to one chosen node in SP21, and at the same time used ftp
to conduct several file transfers of a 300 MB file from the nodes in SP21 to
a chosen node in SP2. We sent the files to /dev/null on the receiving node
to eliminate any hard disk influence.
We saw up to about 44.5 MB/s with just one side sending data. With all
nodes sending and receiving, we achieved a duplex throughput of about
64 MB/s on the ATM port.
Although this is far from the theoretical maximum throughput an ATM
OC-12c adapter can provide, it turns out that a 622 Mb/s link between two
SP Switch Routers will be a viable solution.
7.3 HIPPI Backbone Connection
With a nominal speed of 100 MB/s duplex, connecting two GRFs with HIPPI
media cards seems the fastest choice. But then, HIPPI cables are limited to a
length of 25m (Ascend provides and guarantees 50m cables).
Below are the steps to set up the HIPPI backbone connection with two GRF
routers.
Configuration assumptions:
• The SP Switch Router HIPPI media card has been installed according to
Section 4.5, “HIPPI Configuration” on page 133 on both GRF routers, and
works properly.
• The SP Switch Router Adapter card has been installed according to
GRF routers, and works properly.
• The SP Switch Router Adapter card and SP processor node Switch
adapters are in the same IP subnet on the respective SP.
• ARP should be enabled on the SP Switch network to provide the most
flexibility in assigning IP addresses (strongly recommended!).
• If ARP is disabled on the SP Switch network, the IP addresses assigned to
the nodes must be determined by the switch node numbers.
Note: The SP Switch Router Adapter card will not properly forward IP data to
nodes assigned with an IP address that is in another subnet.
Multiple RS/6000 SPs and Multiple GRFs
227
Download from Www.Somanuals.com. All Manuals Search And Download.
Configuration:
In this scenario, we have the SP Switch of SP21 connected to an GRF 1600.
The GRF 1600 has its HIPPI media card’s ports cross-connected to the GRF
400 HIPPI media card’s ports. That means that on both sides DESTINATION
is cabled to SOURCE. The GRF 400 in turn is attached to the SP Switch of
255.255.255.0.
SP
Net 192.168.13.0
Net 10.50.1.2
Switch
Router 1
IP 192.168.13.4
SP processor node
SP processor node
HIPPI
SP Switch 1
SP2
Adapter card
SP Switch Router
Adapter card 1
Mask 255.255.255.0
GRF 400
Mask 255.255.255.0
Net 192.168.14.0
SP
Net 10.50.1.1
Switch
Router 2
IP 192.168.14.4
SP processor node
SP processor node
HIPPI
SP Switch 2
SP21
Adapter card
SP Switch Router
Adapter card 2
Mask 255.255.255.0
GRF 1600
Mask 255.255.255.0
Figure 71. SP Switch - HIPPI - SP Switch Connection
Table 26. Configuration of SP Switch - HIPPI - SP Switch
Adapter
IP Address
SP Switch Router Adapter card 1
192.168.13.4
SP Switch Router HIPPI media card 10.50.1.2
in GRF 400
SP Switch Router HIPPI media card 10.50.1.1
in GRF 1600
SP Switch Router Adapter card 2
192.168.14.4
To successfully run this configuration, a route to the distant SP Switch
network has to be set on every SP Switch Router. On the nodes of SP2 and
SP21, respectively, routes to the nodes of the distant SP have to be set.
228
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
The media card adapters on the GRF routers should already be up and
running (according to Section 3.7, “Step-by-Step Media Card Configuration”
settings are repeated here, nevertheless, to be on the safe side:
On the GRF 1600 (HIPPI card is in slot 8, SP Switch card is in slot 3) check
for the following:
1. The file /etc/grifconfig.conf has the following entries:
gh080 10.50.1.1
gt030 192.168.14.4
255.255.255.0 -
255.255.255.0 -
mtu 65280
mtu 65520
2. The file /etc/grlamap.conf has the following entry (note that ’0x40’ is a
hexadecimal value equivalent to 64 decimal):
*
0xfc0 0x40
3. The file /etc/grarp.conf has the following entry (I-field):
gh080 10.50.1.2 0x03666fc0
4. The file /etc/grroute.conf has the following line:
192.168.13.0 255.255.255.0 10.50.1.2
# default mapping of IP for all portcards
This sets the correct route to the other SP Switch network over the HIPPI
interface automatically; of course, this route can also be set manually
every time the GRF is rebooted.
5. The SP Switch Router Adapter card is connected to the SP Switch and
configured. Check with SDRGetObjects switch_respondson the CWS and
use Eunfenceif needed.
On the GRF 400 (HIPPI card is in slot 0, SP Switch card is in slot 1):
1. The file /etc/grifconfig.conf has the following entries:
gh000 10.50.1.2
gt010 192.168.13.4
255.255.255.0 -
255.255.255.0 -
mtu 65280
mtu 65520
2. The file /etc/grlamap.conf has the following entry:
0xfc0 0x40 # default mapping of IP for all portcards
3. The file /etc/grarp.conf has the following entry (I-field):
gh000 10.50.1.1 0x03555fc0
4. The file /etc/grroute.conf has the following line:
192.168.14.0 255.255.255.0 10.50.1.1
*
This sets the correct route to the other SP Switch network over the ATM
interface automatically; of course, this route could also be set manually
every time the GRF is rebooted.
Multiple RS/6000 SPs and Multiple GRFs
229
Download from Www.Somanuals.com. All Manuals Search And Download.
5. The SP Switch Router Adapter card is connected to the SP Switch and
configured. Check with SDRGetObjects switch_respondson the CWS and
use Eunfenceif needed.
The following tasks are performed on the respective SP nodes:
1. On the nodes in SP21, the following route needs to be set:
route add -net 192.168.13 -netmask 255.255.255.0 -mtu 65280 10.50.1.2
2. On the nodes in SP2, the following route needs to be set:
route add -net 192.168.14 -netmask 255.255.255.0 -mtu 65280 10.50.1.1
Hint: You must not use SMIT to set the routes and put them into the ODM.
The SP Switch is not working when these routes are set, at boot time.
Therefore, they are put onto another, already available network interface, for
example the control Ethernet, and this is definitely not what you expect to
happen.
Use a separate /etc/rc.routes shell script that is run only after an Estartor an
Eunfencewas issued, or use some other mechanisms to have this route set
only after the css0 interfaces on the SP nodes are up and running.
Setup is done now, and every node in SP2 should now be able to pingevery
node in SP21 and vice versa.
3. Check for correct routing entries on all nodes in SP21:
root@sp21en0:/ dsh netstat -rn | grep 192.168.13
sp21n01: 192.168.13/24
sp21n05: 192.168.13/24
sp21n06: 192.168.13/24
sp21n07: 192.168.13/24
sp21n08: 192.168.13/24
sp21n09: 192.168.13/24
sp21n10: 192.168.13/24
sp21n11: 192.168.13/24
sp21n13: 192.168.13/24
sp21n15: 192.168.13/24
root@sp21en0:/
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
192.168.14.4
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
5 717959 css0 65280 -
2 172837 css0 65280 -
0
0 css0 65280 -
2 141374 css0 65280 -
0
2
0
0 css0 65280 -
78305 css0 65280 -
0 css0 65280 -
0 202459 css0 65280 -
5 543370 css0 65280 -
1 384799 css0 65280 -
4. Check for correct routing entries on all nodes in SP2:
230
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
root@sp2en0:/ dsh netstat -rn | grep 192.168.14
sp2n01: 192.168.14/24
sp2n05: 192.168.14/24
sp2n06: 192.168.14/24
sp2n07: 192.168.14/24
sp2n08: 192.168.14/24
sp2n09: 192.168.14/24
sp2n10: 192.168.14/24
sp2n11: 192.168.14/24
sp2n12: 192.168.14/24
sp2n13: 192.168.14/24
sp2n14: 192.168.14/24
sp2n15: 192.168.14/24
root@sp2en0:/
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
192.168.13.4
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
UG
6 575324 css0 65280 -
1 316772 css0 65280 -
0
77420 css0 65280 -
3 272682 css0 65280 -
1 237408 css0 65280 -
2 298665 css0 65280 -
4 367369 css0 65280 -
4 667058 css0 65280 -
1 423454 css0 65280 -
1 566707 css0 65280 -
0 196490 css0 65280 -
0
2 css0 65280 -
5. Issue some pingcommands to check the connection:
On the SP21 nodes, pingthe SP Switch interface of nodes in SP2; on
nodes in SP2 pingthe SP Switch interface of nodes in SP21.
If these pingcommands fail, check routing settings again. If everything is
as it should be, try to pingthe SP Switch Router HIPPI media card or the
SP Switch Router Adapter card to find the failing part:
ping 192.168.14.4 (on SP21 processor nodes)
ping 192.168.13.4 (on SP2 processor nodes)
ping 10.50.1.1and ping 10.50.1.2 (on both the GRF 400 and the GRF
1600)
If any errors occur, check cabling, the configuration of SP Switch Router
media cards (see Section 3.7, “Step-by-Step Media Card Configuration”
Switch adapters in the SP nodes.
Note: Both HIPPI media cards must be online and cabled, or pingwill not
succeed.
Performance:
To get a rough overview of the data transfer rates that can be achieved in this
scenario, the following test was performed:
1. We used ftp to conduct several file transfers of a 300 MB file from the
nodes in SP2 to one chosen node in SP21, and at the same time used ftp
to conduct several file transfers of a 300 MB file from the nodes in SP21 to
a chosen node in SP2. We sent the files to /dev/null on the receiving node
to eliminate any hard disk influence.
Multiple RS/6000 SPs and Multiple GRFs
231
Download from Www.Somanuals.com. All Manuals Search And Download.
We saw up to about 48 MB/s with just one side sending data. With all
nodes sending and receiving, we achieved a duplex throughput of about
54 MB/s on the HIPPI port.
232
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix A. Laboratory Hardware and Software Configuration
This appendix contains a detailed description of the hardware and software
configuration used to test scenarios described in the second part of this
redbook. All hostnames, IP addresses, adapters and other configuration
information mentioned there refer to the following section if no other
information is given.
A.1 Node and Control Workstation Configuration
This section describes the basic node and CWS configuration used to
establish the scenarios.
nodes in both RS/6000 SPs, their types, hostnames, built-in adapters, and
generally used IP addresses. If other IP addresses were applied in some
scenarios, they are mentioned in the corresponding chapter.
© Copyright IBM Corp. 1998
233
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 27. Configuration of SP 21
Node
Node Type
Hostname
Adapter
IP address
Node0 (CWS)
RS/6000 570
sp21en0
ent0
tok0
192.168.4.137
9.12.1.137
Node 1
Node 5
Node 6
Node 7
Node 8
Node 9
Node 10
Node 11
Node 13
Node 15
high node
112 Mhz,6 proc.
sp21n01
sp21n05
sp21n06
sp21n07
sp21n08
sp21n09
sp21n10
sp21n11
sp21n13
sp21n15
ent0
css0
192.168.4.1
192.168.14.1
thin node
66 MHz
ent0
css0
192.168.4.5
192.168.14.5
thin node
66 MHz
ent0
css0
192.168.4.6
192.168.14.6
thin node
66 MHz
ent0
css0
192.168.4.7
192.168.14.7
thin node
66 MHz
ent0
css0
192.168.4.8
192.168.14.8
thin node
66 MHz
ent0
css0
192.168.4.9
192.168.14.9
thin node
66 MHz
ent0
css0
192.168.4.10
192.168.14.10
wide node
66 MHz
ent0
css0
192.168.4.11
192.168.14.11
wide node
66 MHz
ent0
css0
192.168.4.13
192.168.14.13
wide node
66 MHz
ent0
css0
192.168.4.15
192.168.14.15
234
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 28. Configuration of SP 2
Node
Node Type
Hostname
Adapter
IP address
Node0 (CWS)
RS/6000 590
sp2en0
ent0
tok0
192.168.3.37
9.12.1.37
Node1
Node5
Node6
Node7
Node8
Node9
Node10
Node11
Node12
Node13
Node14
Node15
high node
112 MHz, 8 proc.
sp2n01
sp2n05
sp2n06
sp2n07
sp2n08
sp2n09
sp2n10
sp2n11
sp2n12
sp2n13
sp2n14
sp2n15
ent0
css0
192.168.3.1
192.168.13.1
thin node
66 MHz
ent0
css0
192.168.3.5
192.168.13.5
thin node
66 MHz
ent0
css0
192.168.3.6
192.168.13.6
thin node
66 MHz
ent0
css0
192.168.3.7
192.168.13.7
thin node
66 MHz
ent0
css0
192.168.3.8
192.168.13.8
thin node
66 MHz
ent0
css0
192.168.3.9
192.168.13.9
thin node
66MHz
ent0
css0
192.168.3.10
192.168.13.10
thin node
66MHz
ent0
css0
192.168.3.11
192.168.13.11
thin node
66 MHz
ent0
css0
192.168.3.12
192.168.13.12
thin node
66 MHz
ent0
css0
192.168.3.13
192.168.13.13
thin node
66 MHz
ent0
css0
192.168.3.14
192.168.13.14
wide node
66 MHz
ent0
css0
192.168.3.15
192.168.13.15
A.1.1 Hard Disks
All nodes and CWSs include one or more internal SCSI disks. Some nodes
on page 185) some 9333 Serial Optical Link disks were connected to
Laboratory Hardware and Software Configuration
235
Download from Www.Somanuals.com. All Manuals Search And Download.
selected nodes. Refer to the scenario descriptions in second part of this
redbook to see which disks were used.
Table 29. Hard Disk Equipment of SP 21
Node
Disks
Description
Node (CWS)
hdisk0
hdisk1
hdisk2
hdisk3
1.2 GB SCSI Disk Drive
1.2 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
Node 1
hdisk0
hdisk1
2.2 GB SCSI Disk Drive
2.2 GB SCSI Disk Drive
Node 5
Node 6
Node 7
Node 8
Node 9
hdisk0
hdisk0
hdisk0
hdisk0
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
Node 10
Node 11
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0GB Serial-Link Disk Drive
2.0GB Serial-Link Disk Drive
2.0GB Serial-Link Disk Drive
Node 13
Node 15
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0GB Serial-Link Disk Drive
2.0GB Serial-Link Disk Drive
2.0GB Serial-Link Disk Drive
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0GB Serial-Link Disk Drive
2.0GB Serial-Link Disk Drive
2.0GB Serial-Link Disk Drive
236
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 30. Hard Disk Equipment of SP 2 Part 1 of 2
Node
Disks
Description
Node0 (CWS)
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
Node1
Node5
hdisk0
hdisk1
2.2 GB SCSI Disk Drive
2.2 GB SCSI Disk Drive
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
hdisk6
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
7135 Disk Array Device
7135 Disk Array Device
7135 Disk Array Device
7135 Disk Array Device
7135 Disk Array Device
Node6
Node7
Node8
Node9
Node10
Node11
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
Node12
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
Laboratory Hardware and Software Configuration
237
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 31. Hard Disk Equipment of SP 2 Part 2 of 2
Node
Disks
Description
Node13
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
Node14
hdisk0
hdisk1
hdisk2
hdisk3
hdisk4
hdisk5
hdisk6
hdisk7
hdisk8
1.0 GB SCSI Disk Drive
1.0 GB SCSI Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
SSA Logical Disk Drive
Node15
hdisk0
hdisk1
hdisk2
hdisk3
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
2.0 GB SCSI Disk Drive
238
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
A.1.2 Software Configuration
Both CWSs and every SP node are installed with AIX 4.3.1, including all fixes
Table 32. Software Levels on CWS and All Nodes Part 1 of 14
Fileset
Level
Description
Java.rte.bin
1.1.4.0
1.1.4.0
1.1.4.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
Java Runtime Environment Executables
Java Runtime Environment Classes
Java Runtime Environment Libraries
AIX CDE ToolTalk Support
Java.rte.classes
Java.rte.lib
X11.Dt.ToolTalk
X11.Dt.bitmaps
X11.Dt.helpmin
X11.Dt.helprun
X11.Dt.lib
AIX CDE Bitmaps
AIX CDE Minimum Help Files
AIX CDE Runtime Help
AIX CDE Runtime Libraries
X11.Dt.rte
AIX Common Desktop Environment (CDE) 1.0
AIXwindows aixterm Application
X11.apps.aixterm
X11.apps.clients
X11.apps.custom
X11.apps.msmit
X11.apps.pm
AIXwindows Client Applications
AIXwindows Customizing Tool
AIXwindows msmit Application
AIXwindows Power Management GUI Utility
AIXwindows Runtime Configuration Applications
AIXwindows Utility Applications
X11.apps.rte
X11.apps.util
X11.apps.xterm
X11.base.common
X11.base.lib
AIXwindows xterm Application
AIXwindows Runtime Common Directories
AIXwindows Runtime Libraries
X11.base.rte
AIXwindows Runtime Environment
AIXwindows Runtime Shared Memory Transport
AIXwindows Motif 1.2 Compatibility Development Toolkit
AIXwindows PC850 Fonts Compatibility
AIXwindows Motif 1.0 Libraries Compatibility
AIXwindows Motif 1.1.4 Libraries Compatibility
X11.base.smt
X11.compat.adt.Motif12
X11.compat.fnt.pc
X11.compat.lib.Motif10
X11.compat.lib.Motif114
Laboratory Hardware and Software Configuration
239
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 33. Software Levels on CWS and All Nodes Part 2 of 14
Fileset
Level
Description
X11.compat.lib.X11R3
X11.compat.lib.X11R4
X11.compat.lib.X11R5
X11.fnt.coreX
4.3.0.0
4.3.0.0
4.3.1.1
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.1
4.3.1.0
AIXwindows X11R3 Libraries Compatibility
AIXwindows X11R4 Libraries Compatibility
AIXwindows X11R5 Compatibility Libraries
AIXwindows X Consortium Fonts
X11.fnt.defaultFonts
X11.fnt.iso1
AIXwindows Default Fonts
AIXwindows Latin 1 Fonts
X11.fnt.iso_T1
AIXwindows Latin Type1 Fonts
X11.loc.en_US.Dt.rte
X11.loc.en_US.base.lib
X11.loc.en_US.base.rte
X11.motif.lib
AIX CDE Locale Configuration - U.S. English
AIXwindows Client Locale Config - U.S. English
AIXwindows Locale Configuration - U.S. English
AIXwindows Motif Libraries
X11.motif.mwm
AIXwindows Motif Window Manager
X11.msg.en_US.Dt.helpmin
X11.msg.en_US.Dt.rte
X11.msg.en_US.apps.aixterm
X11.msg.en_US.apps.clients
X11.msg.en_US.apps.custom
X11.msg.en_US.apps.pm
X11.msg.en_US.apps.rte
X11.msg.en_US.base.common
X11.msg.en_US.base.rte
X11.msg.en_US.motif.lib
X11.msg.en_US.motif.mwm
X11.vsm.lib
AIX CDE Minimum Help Files - U.S. English
AIX CDE Messages - U.S. English
AIXwindows aixterm Messages - U.S. English
AIXwindows Client Application Msgs - U.S. English
AIXwindows Customizing Tool Msgs - U.S. English
AIXwindows Power Mgmt GUI Msgs - U.S. English
AIXwindows Runtime Config Messages - U.S. English
AIXwindows Common Messages - U.S. English
AIXwindows Runtime Env. Messages - U.S. English
AIXwindows Motif Libraries Messages - U.S. English
AIXwindows Motif Window Mgr Msgs - U.S. English
Visual System Management Library
bos.acct
Accounting Services
bos.acct
Accounting Services
bos.adt.base
Base Application Development Toolkit
Base Application Development Toolkit Data
Base Application Development Debuggers
Base Application Development Graphics Include Files
bos.adt.data
bos.adt.debug
bos.adt.graphics
240
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 34. Software Levels on CWS and All Nodes Part 3 of 14
Fileset
Level
Description
bos.adt.include
bos.adt.lib
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.1
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.0.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
Base Application Development Include Files
Base Application Development Libraries
Base Application Development Math Library
Base Profiling Support
bos.adt.libm
bos.adt.prof
bos.adt.prt_tools
bos.adt.samples
bos.adt.sccs
Printer Support Development Toolkit
Base Operating System Samples
SCCS Application Development Toolkit
System Calls Application Development Toolkit
Base Application Development Utilities - lex and yacc
Alternate Disk Installation Disk Boot Images
Alternate Disk Installation Runtime
ATM LAN Emulation Client Support
AIX 3.2 Compatibility Commands
AIX 3.2 to 4 Compatibility Links
bos.adt.syscalls
bos.adt.utils
bos.alt_disk_install.boot_images
bos.alt_disk_install.rte
bos.atm.atmle
bos.compat.cmds
bos.compat.links
bos.content_list
bos.data
AIX Release Content List
Base Operating System Data
bos.diag.com
Common Hardware Diagnostics
Hardware Diagnostics
bos.diag.rte
bos.diag.util
Hardware Diagnostics Utilities
bos.dlc.8023
IEEE Ethernet (802.3) Data Link Control
Common Data Link Control Files
Common Ethernet Data Link Files
Standard Ethernet Data Link Control
Token-Ring Data Link Control
bos.dlc.com
bos.dlc.com_enet
bos.dlc.ether
bos.dlc.token
bos.dosutil
DOS Utilities
bos.help.msg.en_US.com
bos.help.msg.en_US.smit
bos.html.en_US.topnav.navigate
bos.iconv.com
WebSM/SMIT Context Helps - U.S. English
SMIT Context Helps - U.S. English
Top Level Navigation - U. S. English
Common Language to Language Converters
Unicode Base Converters for AIX Code Sets/Fonts
bos.iconv.ucs.com
Laboratory Hardware and Software Configuration
241
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 35. Software Levels on CWS and All Nodes Part 4 of 14
Fileset
Level
Description
bos.loc.iso.en_US
4.3.1.0
4.3.1.1
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
Base System Locale ISO Code Set - U.S. English
Mail Handler
bos.mh
bos.msg.en_US.alt_disk_install.rte
bos.msg.en_US.diag.rte
bos.msg.en_US.net.tcp.client
bos.msg.en_US.rte
Alternate Disk Install Msgs - U.S. English
Hardware Diagnostics Messages - U.S. English
TCP/IP Messages - U.S. English
Base Operating System Runtime Msgs - U.S. English
NIM GUI Messages - U.S. English
bos.msg.en_US.sysmgt.nim.master_
gui
bos.msg.en_US.txt.tfs
bos.net.ate
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
Text Formatting Services Messages - U.S. English
Asynchronous Terminal Emulator
Network Computing System 1.5.1
Network File System Development Toolkit
CacheFS File System
bos.net.ncs
bos.net.nfs.adt
bos.net.nfs.cachefs
bos.net.nfs.client
bos.net.nfs.server
bos.net.ppp
Network File System Client
Network File System Server
Async Point to Point Protocol
TCP/IP Application Toolkit
bos.net.tcp.adt
bos.net.tcp.client
bos.net.tcp.server
bos.net.tcp.smit
bos.perf.diag_tool
bos.perf.pmr
TCP/IP Client Support
TCP/IP Server
TCP/IP SMIT Support
Performance Diagnostic Tool
Performance PMR Data Collection Tool
Power Management Runtime Software
Base Operating System Runtime
Desktop Integrator
bos.powermgt.rte
bos.rte
bos.rte.Dt
bos.rte.ILS
International Language Support
System Resource Controller
AIXwindows Device Support
Asynchronous I/O Extension
Archive Commands
bos.rte.SRC
bos.rte.X11
bos.rte.aio
bos.rte.archive
242
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 36. Software Levels on CWS and All Nodes Part 5 of 14
Fileset
Level
Description
bos.rte.bind_cmds
bos.rte.boot
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.1
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.1
4.3.1.0
Binder and Loader Commands
Boot Commands
Base OS Install Commands
Commands
bos.rte.bosinst
bos.rte.commands
bos.rte.compare
bos.rte.console
bos.rte.control
bos.rte.cron
File Compare Commands
Console
System Control Commands
Batch Operations
Date Control Commands
Base Device Drivers
Device Driver Messages
Diagnostics
bos.rte.date
bos.rte.devices
bos.rte.devices_msg
bos.rte.diag
bos.rte.edit
Editors
bos.rte.filesystem
bos.rte.iconv
bos.rte.ifor_ls
bos.rte.im
Filesystem Administration
Language Converters
iFOR/LS Libraries
Input Methods
bos.rte.install
bos.rte.jfscomp
bos.rte.libc
LPP Install Commands
JFS Compression
libc Library
bos.rte.libcfg
bos.rte.libcur
bos.rte.libdbm
bos.rte.libnetsvc
bos.rte.libpthreads
bos.rte.libqb
libcfg Library
libcurses Library
libdbm Library
Network Services Libraries
libpthreads Library
libqb Library
bos.rte.libs
libs Library
bos.rte.loc
Base Locale Support
Logical Volume Manager
Man Commands
bos.rte.lvm
bos.rte.man
Laboratory Hardware and Software Configuration
243
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 37. Software Levels on CWS and All Nodes Part 6 of 14
Fileset
Level
Description
bos.rte.methods
bos.rte.misc_cmds
bos.rte.net
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
Device Config Methods
Miscellaneous Commands
Network
bos.rte.odm
Object Data Manager
bos.rte.printers
Front End Printer Support
bos.rte.security
Base Security Function
bos.rte.serv_aid
bos.rte.shell
Error Log Service Aids
Shells (bsh, ksh, csh)
bos.rte.streams
bos.rte.tty
Streams Libraries
Base TTY Support and Commands
License Management
bos.sysmgt.loginlic
bos.sysmgt.nim.client
bos.sysmgt.nim.master
bos.sysmgt.nim.master_gui
bos.sysmgt.nim.spot
bos.sysmgt.quota
bos.sysmgt.serv_aid
bos.sysmgt.smit
bos.sysmgt.sysbr
bos.sysmgt.trace
bos.terminfo.com.data
bos.terminfo.dec.data
bos.terminfo.ibm.data
bos.terminfo.pc.data
bos.terminfo.rte
bos.terminfo.wyse.data
bos.twintail
Network Install Manager - Client Tools
Network Install Manager - Master Tools
Network Install Manager - GUI
Network Install Manager - SPOT
Filesystem Quota Commands
Software Error Logging and Dump Service Aids
System Management Interface Tool (SMIT)
System Backup and BOS Install Utilities
Software Trace Service Aids
Common Terminal Definitions
Digital Equipment Corp. Terminal Definitions
IBM Terminal Definitions
Personal Computer Terminal Definitions
Run-time Environment for AIX Terminals
Wyse Terminal Definitions
Twintail SCSI Software Support
Bibliography Support
bos.txt.bib
bos.txt.bib.data
Bibliography Support Data
bos.txt.hplj.fnt
Fonts for Hewlett Packard Laser Jet Printers
244
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 38. Software Levels on CWS and All Nodes Part 7 of 14
Fileset
Level
Description
bos.txt.spell
4.3.1.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
Writer’s Tools Commands
bos.txt.spell.data
Writer’s Tools Data
bos.txt.tfs
Text Formatting Services Commands
Text Formatting Services Data
bos.txt.tfs.data
bos.txt.ts
TranScript Tools
bos.up
Base Operating System Uniprocessor Runtime
Base System Diagnostics
devices.base.diag
devices.base.rte
RISC System 6000 Base Device Software
RISC CHRP Base System Device Diagnostics
RISC PC Base System Device Software (CHRP)
Common Serial Adapter Diagnostics
Common ATM Software
devices.chrp.base.diag
devices.chrp.base.rte
devices.common.IBM.async.diag
devices.common.IBM.atm.rte
devices.common.IBM.bbl.diag
devices.common.IBM.cx.rte
devices.common.IBM.disk.rte
devices.common.IBM.ethernet.rte
devices.common.IBM.fda.diag
devices.common.IBM.fda.rte
devices.common.IBM.fddi.rte
devices.common.IBM.ktm_std.diag
devices.common.IBM.ktm_std.rte
devices.common.IBM.modemcfg.data
devices.common.IBM.pmmd_chrp.rte
devices.common.IBM.ppa.diag
devices.common.IBM.ppa.rte
devices.common.IBM.scsi.rte
devices.common.IBM.ssa.diag
devices.common.IBM.ssa.rte
devices.common.IBM.tokenring.rte
devices.common.base.diag
Common Graphics Adapter Diagnostics
CX Common Adapter Software
Common IBM Disk Software
Common Ethernet Software
Common Diskette Adapter and Device Diagnostics
Common Diskette Device Software
Common FDDI Software
Common Keyboard, Mouse, and Tablet Device Diagnostics
Common Keyboard, Tablet, and Mouse Software
Sample Service Processor Modem Configuration Files
CHRP Power Management Software
Common Parallel Printer Adapter Diagnostics
Common Parallel Printer Adapter Software
Common SCSI I/O Controller Software
SSA Common Adapter Diagnostics
Common SSA Adapter Software
Common Token Ring Software
Common Base System Diagnostics
Laboratory Hardware and Software Configuration
245
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 39. Software Levels on CWS and All Nodes Part 8 of 14
Fileset
Level
Description
devices.common.rspcbase.rte
devices.graphics.com
devices.mca.0200.diag
devices.mca.0200.rte
devices.mca.61fd.diag
devices.mca.61fd.rte
devices.mca.8d77.diag
devices.mca.8d77.rte
devices.mca.8d77.ucode
devices.mca.8ee4.X11
devices.mca.8ee4.diag
devices.mca.8ee4.rte
devices.mca.8ef2.com
devices.mca.8ef2.diag
devices.mca.8ef2.diag.com
devices.mca.8ef2.rte
devices.mca.8ef3.diag
devices.mca.8ef3.rte
devices.mca.8ef4.diag
devices.mca.8ef4.rte
devices.mca.8ef4.ucode
devices.mca.8ef5.diag
devices.mca.8ef5.rte
devices.mca.8efc.com
devices.mca.8efc.diag
devices.mca.8efc.rte
devices.mca.8f62.diag
devices.mca.8f62.rte
devices.mca.8f64.diag
devices.mca.8f64.rte
4.3.1.0
4.3.1.1
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
RISC PC Common Base System Device Software
Graphics Adapter Common Software
Wide SCSI Adapter Diagnostics
Wide SCSI Adapter
64-Port Asynchronous Adapter Diagnostics
64-Port Asynchronous Adapter Software
8-bit SCSI I/O Controller Diagnostics
8-bit SCSI I/O Controller Software
8-bit SCSI I/O Controller Microcode
AIXwindows Color Graphics Display Adapter Software
Color Graphics Display Adapter Diagnostics
Color Graphics Display Adapter Software
Common Integrated Ethernet Software
Integrated Ethernet Adapter (8ef2) Diagnostics
Common Integrated Ethernet Diagnostics
Integrated Ethernet Adapter (8ef2) Software
Integrated Ethernet Adapter (8ef3) Diagnostics
Integrated Ethernet Adapter (8ef3) Software
FDDI Adapter (8ef4) Diagnostics
FDDI Adapter (8ef4) Software
FDDI Adapter (8ef4) Microcode
Ethernet High-Performance LAN Adapter (8ef5) Diagnostics
Ethernet High-Performance LAN Adapter (8ef5) Software
Common 16-bit SCSI I/O Controller Software
16-bit SCSI I/O Controller Diagnostics
16-bit SCSI I/O Controller Software
IBM MCA 10/100 Mb Ethernet Adapter (8f62) Diagnostics
IBM MCA 10/100 Mb Ethernet Adapter Software (8f62)
155 Mbps ATM Adapter (8f64) Diagnostics
Turboways 155 MCA ATM Adapter (8f64) Software
246
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 40. Software Levels on CWS and All Nodes Part 9 of 14
Fileset
Level
Description
devices.mca.8f67.com
devices.mca.8f67.diag
devices.mca.8f67.diag.com
devices.mca.8f67.rte
devices.mca.8f67.ucode
devices.mca.8f7f.diag
devices.mca.8f7f.rte
devices.mca.8f7f.ucode
devices.mca.8f95.diag
devices.mca.8f95.rte
devices.mca.8f97.com
devices.mca.8f97.diag
devices.mca.8f97.rte
devices.mca.8f98.diag
devices.mca.8f98.rte
devices.mca.8f9d.diag
devices.mca.8f9d.rte
devices.mca.8fa2.diag
devices.mca.8fa2.rte
devices.mca.8fc8.diag
devices.mca.8fc8.rte
devices.mca.8fc8.ucode
devices.mca.dee6.rte
devices.mca.df5f.com
devices.mca.edd0.com
devices.mca.edd0.diag
devices.mca.edd0.rte
devices.mca.ffe1.diag
devices.mca.ffe1.rte
devices.mca.ffe1.ucode
4.3.1.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.0.0
Common Turboways ATM Software
155 Mbps ATM Adapter (8f67) Diagnostics
Common ATM 155 Mbps ATM Adapter Diagnostics
Turboways 155 MCA ATM Adapter (8f67) Software
Turboways 155 MCA ATM Adapter (8f67) Microcode
100 Mbps ATM Adapter (8f7f) Diagnostics
Turboways 100 MCA ATM Adapter (8f7f) Software
Turboways 100 MCA ATM Adapter (8f7f) Microcode
Ethernet High-Performance LAN Adapter (8f95) Diagnostics
Ethernet High-Performance LAN Adapter (8f95) Software
Common SSA Adapter (8f97) Software
SSA Adapter (8f97) Diagnostics
SSA Adapter (8f97) Software
Integrated Ethernet Adapter (8f98) Diagnostics
Integrated Ethernet Adapter (8f98) Software
LAN SCSI Adapter Diagnostics
LAN SCSI Adapter
Token Ring High-Performance Adapter (8fa2) Diagnostics
Token Ring High-Performance Adapter (8fa2) Software
Token Ring High-Performance Adapter (8fc8) Diagnostics
Token Ring High-Performance Adapter (8fc8) Software
Token Ring High-Performance Adapter (8fc8) Microcode
Standard I/O (dee6) Adapter Software
Standard I/O Adapter Common Software
Common Async Adapter Support
8-Port Asynchronous Adapter EIA-232 Diagnostics
8-Port Asynchronous Adapter EIA-232 Software
128-Port Asynchronous Adapter Diagnostics
128-Port Asynchronous Adapter Software
128-Port Asynchronous Adapter Microcode
Laboratory Hardware and Software Configuration
247
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 41. Software Levels on CWS and All Nodes Part 10 of14
Fileset
Level
Description
devices.msg.en_US.base.com
devices.msg.en_US.diag.rte
devices.msg.en_US.rspc.base.com
devices.msg.en_US.sys.mca.rte
devices.rs6ksmp.base.rte
devices.rspc.base.diag
devices.rspc.base.rte
devices.scsi.disk.diag.com
devices.scsi.disk.diag.rte
devices.scsi.disk.rspc
devices.scsi.disk.rte
4.3.0.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.1.0
4.3.1.0
Base System Device Software Msgs - U.S. English
Device Diagnostics Messages - U.S. English
RISC PC Software Messages - U.S. English
Micro Channel Bus Software Messages - U.S. English
Multiprocessor Base System Device Software
RISC PC Base System Device Diagnostics
RISC PC Base System Device Software
Common Disk Diagnostic Service Aid
SCSI CD_ROM, Disk Device Diagnostics
RISC PC SCSI CD-ROM, Disk, Read/Write Optical Software
SCSI CD-ROM, Disk, Read/Write Optical Device Software
RAIDiant Array DA Device Diagnostics
7135 RAIDiant Array DA Device Software Support
SCSI Enclosure Services Device Diagnostics
SCSI Enclosure Device Software
devices.scsi.scarray.diag
devices.scsi.scarray.rte
devices.scsi.ses.diag
devices.scsi.ses.rte
devices.scsi.tape.diag
devices.scsi.tape.rspc
devices.scsi.tape.rte
devices.scsi.tm.rte
SCSI Tape Device Diagnostics
RISC PC SCSI Tape Device Software
SCSI Tape Device Software
SCSI Target Mode Software
devices.sio.fda.diag
Diskette Adapter and Device Diagnostics
Diskette Adapter Software
devices.sio.fda.rte
devices.sio.ktma.diag
devices.sio.ktma.rte
Keyboard Tablet & Mouse Device and Adapter Diagnostics
Keyboard Tablet & Mouse Device and Adapter Software
Parallel Printer Adapter Diagnostics
devices.sio.ppa.diag
devices.sio.ppa.rte
Parallel Printer Adapter Software
devices.sio.sa.diag
Built-in Serial Adapter Diagnostics
devices.sio.sa.rte
Built-in Serial Adapter Software
X11.compat.lib.Motif114
devices.ssa.IBM_raid.rte
devices.ssa.disk.rte
AIXwindows Motif 1.1.4 Libraries Compatibility
SSA Raid Manager Software
SSA DASD Software
248
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 42. Software Levels on CWS and All Nodes Part 11 of 14
Fileset
Level
Description
devices.ssa.network_agent.rte
devices.ssa.tm.rte
devices.sys.mca.rte
devices.sys.slc.diag
devices.sys.slc.rte
devices.tty.rte
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
2.2.0.0
2.2.0.0
2.2.31.1
2.2.31.0
4.3.1.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
SSA Network Agent Support
Target Mode SSA Support
Micro Channel Bus Software
Serial Optical Link Diagnostics
Serial Optical Link Software
TTY Device Driver Support Software
License Use Management Runtime Code
License Use Management Runtime GUI
License Use Management Compatibility Code
License Use Management Compatibility GUI
LUM Runtime Code Messages - U.S. English
LUM Runtime GUI Messages - U.S. English
LUM Compatibility Code Messages - U.S. English
LUM Compatibility GUI Messages - U.S. English
IPF Messages - U.S. English
ifor_ls.base.cli
ifor_ls.base.gui
ifor_ls.compat.cli
ifor_ls.compat.gui
ifor_ls.msg.en_US.base.cli
ifor_ls.msg.en_US.base.gui
ifor_ls.msg.en_US.compat.cli
ifor_ls.msg.en_US.compat.gui
ipfx.msg.en_US.rte
ipfx.rte
Information Presentation Facility Runtime
Performance Agent Daemons & Utilities
Local Performance Analysis & Control Commands
Package Installation Database for Current Media
Hewlett-Packard JetDirect Network Printer Attachment
Hewlett-Packard LaserJet II
perfagent.server
perfagent.tools
pkg_gd
printers.hpJetDirect.attach
printers.hplj-2.rte
printers.hplj-3.rte
Hewlett-Packard LaserJet III
printers.hplj-3si.rte
printers.hplj-4+.rte
printers.hplj-4.rte
Hewlett-Packard LaserJet IIISi
Hewlett-Packard LaserJet 4 Plus
Hewlett-Packard LaserJet 4
printers.hplj-4000.rte
printers.hplj-4si.rte
printers.hplj-4v.rte
printers.hplj-5si.rte
printers.hplj-5siMopier.rte
Hewlett-Packard LaserJet 4000
Hewlett-Packard LaserJet 4si
Hewlett-Packard LaserJet 4V
Hewlett-Packard LaserJet 5si
Hewlett-Packard LaserJet 5si Mopier
Laboratory Hardware and Software Configuration
249
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 43. Software Levels on CWS and All Nodes Part 12 of 14
Fileset
Level
Description
printers.hplj-c.rte
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.2.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
Hewlett-Packard LaserJet Color
IBM 2380 Personal Printer II
IBM 2381 Personal Printer II
IBM 2390 Personal Printer II
IBM 3130 LaserPrinter
printers.ibm2380.rte
printers.ibm2381.rte
printers.ibm2390.rte
printers.ibm3130.rte
printers.ibm4019.rte
printers.ibm4029.rte
printers.ibm4039.rte
printers.ibm4079.rte
printers.ibm4201-2.rte
printers.ibm4201-3.rte
printers.ibm4202-2.rte
printers.ibm4303.rte
printers.ibm4312.rte
printers.ibm4317.rte
printers.ibm4320.rte
printers.ibm4324.rte
printers.ibmNetColor.attach
printers.ibmNetPrinter.attach
printers.lex2380-3.rte
printers.lex2381-3.rte
printers.lex2390-3.rte
printers.lex2391-3.rte
printers.lex4039+.rte
printers.lex4049.rte
printers.lex4076-2c.rte
printers.lex4079+.rte
printers.lex4227.rte
printers.lexOptra+.rte
printers.lexOptraC.rte
IBM 4019 LaserPrinter
IBM 4029 LaserPrinter
IBM 4039 LaserPrinter
IBM 4079 Color Jetprinter PS
IBM 4201 Model 2 Proprinter II
IBM 4201 Model 3 Proprinter III
IBM 4202 Model 2 Proprinter II XL
IBM Network Color Printer
IBM Network Printer 12
IBM Network Printer 17
IBM InfoPrint 20
IBM Network Printer 24
IBM Network Color Printer Attachment
IBM Network Printer Attachment
Lexmark 2380 Plus printer (Model 3)
Lexmark 2381 Plus printer (Model 3)
Lexmark 2390 Plus printer (Model 3)
Lexmark 2391 Plus printer (Model 3)
Lexmark 4039 plus LaserPrinter
Lexmark Optra LaserPrinter
Lexmark ExecJet IIc
Lexmark 4079 Color Jetprinter Plus
Lexmark Forms Printer 4227
Lexmark Optra Plus Laser Printer
Lexmark Optra C Color Laser Printer
250
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 44. Software Levels on CWS and All Nodes Part 13 of 14
Fileset
Level
Description
printers.lexOptraE.rte
printers.lexOptraEp.rte
printers.lexOptraN.rte
printers.lexOptraS.rte
printers.lexOptraSC.rte
printers.msg.en_US.rte
printers.qms100.rte
printers.rte
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.0.0
4.3.1.0
4.3.0.0
4.3.1.0
2.4.0.0
2.4.0.1
2.4.0.1
2.4.0.1
2.4.0.0
2.4.0.1
2.4.0.1
2.4.0.0
2.4.0.0
2.4.0.0
2.4.0.0
2.4.0.0
2.4.0.0
2.4.0.1
2.4.0.0
2.4.0.0
2.4.0.1
2.4.0.0
2.4.0.1
2.4.0.0
4.3.1.0
4.3.1.0
Lexmark Optra E Laser Printer
Lexmark Optra Ep Laser Printer
Lexmark Optra N Laser Printer
Lexmark Optra S Laser Printer
Lexmark Optra SC Color Laser Printer
Printer Backend Messages - U.S. English
QMS ColorScript 100, Model 20
Printer Backend
ssp.authent
SP Authentication Server
ssp.basic
SP System Support Package
SP Authenticated Client Commands
SP Communication Subsystem Package
SP man and info files
ssp.clients
ssp.css
ssp.docs
ssp.gui
SP System Monitor Graphical User Interface
SP High Availability Services
ssp.ha
ssp.ha_clients
ssp.jm
SP High Availability Services (Client)
SP Job Manager Package
ssp.perlpkg
SP PERL Distribution Package
SP Problem Management
ssp.pman
ssp.public
Public Code Compressed Tar Files
SP Extension Node SNMP Manager
Switch Table API Package
ssp.spmgr
ssp.st
ssp.sysctl
SP Sysctl Package
ssp.sysman
Optional System Management Programs
SP Communication Subsystem Topology Package
SP System Partitioning Aid
ssp.top
ssp.top.gui
ssp.topsvcs
SP Topology Services/ES
ssp.ucode
SP Supervisor Microcode Package
WebSM Context Helps - U.S. English
WebSM Client Apps. Messages - U.S. English
sysmgt.help.msg.en_US.websm
sysmgt.msg.en_US.websm.apps
Laboratory Hardware and Software Configuration
251
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 45. Software Levels on CWS and All Nodes Part 14 of 14
Fileset
Level
Description
sysmgt.sguide.rte
sysmgt.websm.apps
sysmgt.websm.framework
sysmgt.websm.icons
sysmgt.websm.rte
sysmgt.websm.ucf
sysmgt.websm.widgets
xlC.cpp
4.3.1.0
4.3.1.1
4.3.1.1
4.3.1.0
4.3.1.1
4.3.1.0
4.3.1.0
3.1.4.4
3.1.4.2
3.6.3.0
3.6.3.0
TaskGuide Runtime Environment
Web-based System Manager Applications
Web-based System Manager Client/Server Support
Web-based System Manager Icons
Web-based System Manager Runtime Environment
Web-based System Manager Container Framework
Web-based System Manager Base Widgets
C for AIX Preprocessor
xlC.msg.en_US.cpp
xlC.msg.en_US.rte
xlC.rte
C for AIX Preprocessor Messages en_US
C Set ++ for AIX Application Runtime Messages en_US
C Set ++ for AIX Application Runtime
252
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
A.1.3 Network Options and Tuning
nodes. Options can be changed with the /etc/nocommand.
Table 46. Network Options of CWS and All Nodes Part 1 of 3
Parameters
Value
thewall
16384
sockthresh
sb_max
85
1310720
somaxconn
1024
0
clean_partial_conns
net_malloc_police
rto_low
0
1
rto_high
64
7
rto_limit
rto_length
inet_stack_size
arptab_bsiz
arptab_nb
tcp_ndebug
ifsize
13
16
7
25
100
8
arpqsize
1
ndpqsize
50
0
route_expire
strmsgsz
0
strctlsz
1024
8
nstrpush
strthresh
85
20
20
15
12288
90
psetimers
psebufcalls
strturncnt
pseintrstack
lowthresh
Laboratory Hardware and Software Configuration
253
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 47. Network Options of CWS and All Nodes Part 2 of 3
Parameters
Value
medthresh
95
psecache
1
subnetsarelocal
maxttl
1
255
ipfragttl
60
ipsendredirects
ipforwarding
udp_ttl
1
1
30
tcp_ttl
60
arpt_killc
20
tcp_sendspace
tcp_recvspace
udp_sendspace
udp_recvspace
rfc1122addrchk
nonlocsrcroute
tcp_keepintvl
tcp_keepidle
bcastping
327680
327680
65536
655360
0
1
150
14400
1
udpcksum
1
tcp_mssdflt
1448
0
icmpaddressmask
tcp_keepinit
150
1
rfc1323
pmtu_default_age
pmtu_rediscover_interval
udp_pmtu_discover
tcp_pmtu_discover
ipqmaxlen
10
30
0
0
100
1
directed_broadcast
254
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 48. Network Options of CWS and All Nodes Part 3 of 3
Parameters
Value
ipignoreredirects
ipsrcroutesend
ipsrcrouterecv
0
1
1
ipsrcrouteforward
1
ip6srcrouteforward
ip6_defttl
1
64
ndpt_keep
120
ndpt_reachable
ndpt_retrans
30
1
ndpt_probe
5
ndpt_down
3
3
ndp_umaxtries
ndp_mmaxtries
ip6_prune
3
2
tcp_timewait
1
tcp_ephemeral_low
tcp_ephemeral_high
udp_ephemeral_low
udp_ephemeral_high
32768
65535
32768
65535
A.2 SP Switch Pool Size Settings
Two important switch parameters were changed from their default values:
rpoolsize and spoolsize. To check the current values, enter:
lsattr -El css0
and look for rpoolsize and spoolsize. 512 KB per processor is the default
size. Both values should be increased if high I/O traffic is expected on the SP
Switch. To increase both values to 5 MB (as applied in all of our scenarios),
enter:
/usr/lib/methods/chgcss -l css0 -a spoolsize=5242880 -a rpoolsize=5242880
For the changes to take effect, the SP nodes must be rebooted.
Laboratory Hardware and Software Configuration
255
Download from Www.Somanuals.com. All Manuals Search And Download.
A.3 7025-F50 Configuration
A 7025-F50 was used for some ATM and Ethernet network tests. This
machine is equipped with two166 MHz-604e processors, three 4500 MB
16-bit SCSI Disk Drives, an IBM 155 Mbps ATM PCI Adapter and an IBM
100/10 Mbps Ethernet PCI Adapter. AIX 4.3.1.1 is installed. For detailed
the sample configurations.
Table 49. Network Options of 7025-F50 Part 1 of 3
Parameter
Value
16384
thewall
sockthresh
sb_max
85
1310720
somaxconn
1024
0
clean_partial_conns
net_malloc_police
rto_low
0
1
rto_high
64
7
rto_limit
rto_length
inet_stack_size
arptab_bsiz
arptab_nb
tcp_ndebug
ifsize
13
16
7
25
100
8
arpqsize
1
ndpqsize
50
0
route_expire
strmsgsz
0
strctlsz
1024
8
nstrpush
strthresh
85
20
psetimers
256
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 50. Network Options of 7025-F50 Part 2 of 3
Parameter
Value
psebufcalls
strturncnt
20
15
pseintrstack
lowthresh
medthresh
psecache
12288
90
95
1
subnetsarelocal
maxttl
1
255
60
ipfragttl
ipsendredirects
ipforwarding
udp_ttl
1
0
30
tcp_ttl
60
arpt_killc
20
tcp_sendspace
tcp_recvspace
udp_sendspace
rfc1122addrchk
nonlocsrcroute
tcp_keepintvl
tcp_keepidle
bcastping
327680
327680
9216
0
0
150
14400
0
udpcksum
1
tcp_mssdflt
512
0
icmpaddressmask
tcp_keepinit
rfc1323
150
1
pmtu_default_age
pmtu_rediscover_interval
udp_pmtu_discover
10
30
0
Laboratory Hardware and Software Configuration
257
Download from Www.Somanuals.com. All Manuals Search And Download.
Table 51. Network Options of 7025-F50 Part 3 of 3
Parameter
Value
ipqmaxlen
100
directed_broadcast
ipignoreredirects
ipsrcroutesend
ipsrcrouterecv
ipsrcrouteforward
ip6srcrouteforward
ip6_defttl
1
0
1
0
1
1
64
ndpt_keep
120
ndpt_reachable
ndpt_retrans
30
1
ndpt_probe
5
ndpt_down
3
3
ndp_umaxtries
ndp_mmaxtries
ip6_prune
3
2
tcp_timewait
1
tcp_ephemeral_low
tcp_ephemeral_high
udp_ephemeral_low
udp_ephemeral_high
32768
65535
32768
65535
A.4 SP IP Switch Router Configuration
A GRF 400 and/or a GRF 1600 SP Switch Router were used for all the tests.
Ascend Embedded/OS 1.4.6.4 was installed.
Both Switch Routers contain at least one SP Switch Router Adapter card.
Depending on the scenario, several SP Switch Router media cards were
installed, including Ethernet, FDDI, ATM OC-3c, ATM OC-12c and HIPPI
media cards.
258
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
The applied IP addresses vary with the scenario and are specified in the
corresponding chapter. For specific SP Switch Router sample configuration
files, refer to Appendix B, “GRF Configuration Files” on page 261.
Laboratory Hardware and Software Configuration
259
Download from Www.Somanuals.com. All Manuals Search And Download.
260
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix B. GRF Configuration Files
This appendix contains relevant SP Switch Router configuration files.
Some of them are here just for information, some of them were worked out
manually during setup of hardware or software and some of them were
created using Ascend-supplied tools. If you need up to date information about
these files, look on the SP Switch Router in directory /etc. Most of the files
mentioned in this appendix come as a *.conf.template file that you can use as
a starting point for further investigation and as a skeleton for your own files.
Also, make use of the man utility that is running on the SP Switch Router.
Although there are no entries for basic operating system commands, the GRF
specific commands, which can be recognized by their gr- prefix, do.
B.1 /root/.profile
The following is a modified /root/.profile for the root user. We liked to have a
system prompt that shows the hostname and path. We also preferred emacs
command line settings over the vi settings.
We further commented out the call to the command line interface (CLI),
because most of the time when you log onto the GRF, you will be working
with a shell anyway. CLI can be started with the command ncli.
Because the file /root/.profile is not located in the /etc directory, it must be
saved with the grsitecommand to be there after system reboot. If you want
the modified file to survive even software updates, use grsite --perm instead.
#
# NOTE: This file is considered part of the Ascend GRF software,
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
and may be overwritten by future releases of Ascend GRF
software. IF YOU EDIT THIS FILE DIRECTLY, YOUR CHANGES
MAY BE LOST WHEN YOU UPGRADE SOFTWARE.
To allow local customization of the root login, this
script will look for the file .profile.local and source
it in if it exists. This should allow you to set
environment variables or execute initialization commands
in that file.
If you find that you cannot adequately customize your
root account’s login process through .profile.local, and
must customize it by editing this file directly, then of
course, do what you need to make things work.
If you find that you must edit this .profile script directly,
please inform the High Performance Networking Division of
Ascend Communications (through normal support channels) of
the change you needed to make to this .profile script, so
© Copyright IBM Corp. 1998
261
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
that we may consider enhancing it in future software
releases to support your needs.
#
# GRF systems require /usr/nbin in the path.
#
PATH=/sbin:/usr/sbin:/bin:/usr/bin:/usr/local/bin:/usr/contrib/bin:/usr/nbin
export PATH
echo ’erase ^H, kill ^U, intr ^C status ^T’
stty crt erase kill - intr status
umask 022
HOME=/root
export HOME
BLOCKSIZE=1k
export BLOCKSIZE
#
# Enable "vi"-style ksh editing, to be consistent with GR 4.x releases.
#
FCEDIT=vi
# We prefer VISUAL=emacs /*UU*/
VISUAL=emacs
export FCEDIT
export VISUAL
# gimme a meaningful prompt /*UU*/
export PS1
# take care, the following is a command substitution,
# so this ‘ is a backtick. (that ’ is a aingle quote)
host=‘hostname -s‘
export host
PS1="$host:\$PWD "
#
# Look for a local .profile.local file, owned by root,
# and source it in if such a thing exists.
#
# NOTE: Do NOT put an "exit" statement in the .profile.local file.
#
#
#
Because we source it in, an "exit" statement in
.profile.local will cause the login shell to terminate.
# ... and if you use this .profile.local, dont forget to
# grsite --perm it ... /*UU*/
LOCAL=./.profile.local
if [ -s ${LOCAL} ]
then
if [ X‘find ${LOCAL} -user root -print‘ = X ]
then
OWNER=‘ls -l ${LOCAL} | awk ’{print $3}’‘
echo "’${LOCAL}’ owned by ’${OWNER}’, not ’root’; skipping sourc
ing it." >&2
else
. ${LOCAL}
fi
fi
unset LOCAL
#
# Check to see if this is an interactive session.
262
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#
if [ -t 0 ]
then
#
# Ask for a terminal type; the default is the canonical "vt100".
#
if [ X${TERM} = X ]
then
TERM=vt100
fi
eval ‘tset -s -m ?$TERM‘
export TERM
#
# It’s interactive, so exec the new CLI shell for the GRF.
#
# from here on commented out the next 15 lines as 99.99999% of work
# is from the shell prompt, and if you exit from CLI, .profile is not
# obeyed, sigh.
# you have to call ncli, to get the super> prompt.
# to see the information that pops up, if you start sh within super>,
# cat /etc/motd, or run sh within super> /*UU*/
# NCLI=/usr/nbin/ncli
# if [ -x ${NCLI} ]
# then
# ${NCLI}
# STATUS=$?
# if [ ${STATUS} -eq 0 ]; then
# exit 0
# fi
# echo "’${NCLI}’ exited status ${STATUS}; continuing with ’${SHELL}’."
# elif [ -f ${NCLI} ]
# then
# echo "Warning: ’${NCLI}’ is not executable; using ’${SHELL}’.
"
# else
# echo "Warning: ’${NCLI}’ is not found; using ’${SHELL}’."
# fi
# end of commenting out the call to CLI. /*UU*/
fi
B.2 /etc/Release
This file’s only content is the actual version of the operating system the GRF
is running. If in doubt, look here and also have a look at Appendix B.15,
A_1_4_6_ibm,default
GRF Configuration Files
263
Download from Www.Somanuals.com. All Manuals Search And Download.
B.3 /etc/bridged.conf
This file holds the configuration data for transparent bridging. It is created
using the utility bredit.
#
#
#
NetStar $Id$
# Configuration file for Bridge Daemon (bridged).
#
#
#
#
#
#
#
Note: bridged will not start if it finds an error while
trying to parse this file. Use the "-d" option on the
command line with bridged to find proximity of the offending
line.
#bridge_group bg0 {
#
# The main reason we need to configure this stuff is to
# specify which ports are part of a bridge group.
#
# Declare all the ports in this group on a single
# line if you don’t need to set anything special
# for the port. This is the normal case.
#
# multiple lines are ok.
#
# eg. port gf070;
#
#
#
port gf041 gf072;
port gf080;
#port gf000 gf001 gf002 gf003;
#
# If you need to set specific values for a port in this
# bridge group, then use the structure below..
#
# port gf040 {
#
#
# priority : port priority allows the network manager to
#
#
#
#
#
influence the choice of port when a bridge has
two ports connected in a loop.
priority 5;
# valid states are : blocking, listening, learning and
#
#
#
#
forwarding. the default start state is blocking.
state blocking;
# root_path_cost : This is the cost to be added to the root
#
#
#
#
#
#
#
path cost field in a configuration message received
on this port in order to determine the cost of the
path to the root through this port. This value is
individually settable on each port.
Setting this value to be large on a particular port
makes the LAN reached through that port more likely
264
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
#
#
#
#
#
#
to be a lead or at least low in the spanning tree.
The closer a LAN is to being a leaf in the tree, the
less through traffic it will be asked to carry. A
LAN would be a candidate for having a large path
cost if it has a lower bandwidth ot if someone wants
to minimize unnecessary traffic on it. A better
description is possibly link cost or port cost.
root_path_cost 5;
# Forward packets with the following destination addresses
# through this port.
#
#
#
forward 00:a0:24:2a:50:e6 00:a0:24:2a:50:e7;
# };
# port gf041 {
#
#
#
state blocking;
root_path_cost 6;
priority 5;
# };
#
# Configuration and tuning parameters that
# govern how a bridge group functions.
#
# priority: This is used to create the bridge ID for
#
#
#
#
#
#
#
#
#
#
#
this bridge. this along with the MAC address of one
of the ports in the group is used to set the bridge ID.
This value allows the network manager to influence the
choice of root bridge and the designated bridge. It is
appended as the most significant portion of a bridge ID.
A lower numerical value for bridge priority makes the
bridge more likely to become the root.
priority 128;
# hello_time: Interval between the transmission of configuration
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
BPDUs by a bridge that is attempting to become the root
bridge or is root bridge.
It is the timer that elapses
between generation of configuration messages by a bridge
that assumes itself to be the root.
Shortening this time will make the protocol more
robust, in case the probability of loss of configuration
messages is high. Lengthening the timer lowers the overhead
of the alogorithm (because the interval between transmission
of configuration messages will be larger).
The recommended time is 2 sec.
hello_time 2 seconds;
# forward_delay: The time value advertised by this
#
#
#
#
#
#
#
#
bridge for deciding the time delay that a port must
spend in the listening and learning states.
This parameter temporarily prevents a bridge from starting
to forward data packets to and from a link until news of
a topology change has spread to all parts of a bridged
network. This shousl give all links that need to be turned
off in the new topology time to do so before new links are
GRF Configuration Files
265
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
#
#
#
#
turned on.
Setting the forward delay too small would result in temporary
loops as the spannign tree algorithm converges. Setting this
value too large results in longer partitions after the
spanning tree reconfigures.
The recommended value is 15 sec.
#forward_delay 15 seconds;
#
# maximum_age: This is the time value advertised by this bridge for
#
#
#
#
#
#
#
#
#
#
#
#
#
#
deciding whether to discard spanning tree frames based on
message age.
If the selected max_age value is too small, then occasionally,
the spanning tree will reconfigure unnecessarily, possibly
causing temporary loss of connectivity in the network. If the
selected value is too large, the network will take longer then
necessary to adjust to a new spanning tree after a topological
event such as restarting or crashing of a bridge or link.
The recommended value is 20 sec.
maximum_age 20 seconds;
# route_maximum_age: This parameter determines how often routes
#
#
#
#
#
#
will be aged out of the learnt route table.
Default : 300 seconds
route_maximum_age 300 seconds;
# And last, hardwiring the forwarding table with
# specific MAC addresses to discard.
#
# Sink packets with the following destination
# addresses.
#
#
#
discard 00:40:0b:0c:95:60 00:40:0b:0c:95:6a;
# Disable Spanning Tree for the group
#
#
spanning_tree disabled
#} ;
#
# To create a bridge group with no ports in it, use the
# following NULL declaration:
#
#
#
#
#
#
bridge_group bg0 {
;
};
#debug_level
debug_level
5;
7 ;
# traces all events of level NOTICE and above
# For FDDI Backbone test Chap.5.1.4.2
bridge_group bg0 {
port gf000 gf001 gf002 gf003;
};
266
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
# For ATM - ATM two ports Test Chap.7.1.2
bridge_group bg1 {
port ga010 ga0180;
#
spanning_tree disabled;
};
B.4 /etc/fstab
This file holds the filesystem mount information. This file was changed when
adding a PCMCIA hard disk in Section 3.3.2, “Installing the PCMCIA Spinning
#
# Filesystem mount table information. See the fstab(5) man page
# and the /etc/fstab.sample file for more information and examples.
#
# Each line is of the form:
#
# device
#
mount_point
type
flags
dump
fsck_pass
# Note that multiple flags (when used) are specified as a
# comma separated list without spaces.
#
# Blank lines and lines beginning with ‘#’ are comments.
#
/dev/rd0a
/dev/wd3a
#
/
ufs
ufs
rw
rw
0 0
0 2
/var/log
# Example line to mount a remote file system in such a way that it
# will try to mount for a long time and is interuptable:
#nfs_host:/home /home
nfs
rw,bg,intr
0 0
B.5 /etc/grarp.conf
This file holds ARP information for distant interfaces that do not support
inverse ARP.
# NetStar $Id: grarp.conf,v 1.2.22.1 1997/05/09 17:35:00 jim Exp $
#
# Template grarp.conf file.
#
#
#
#
#
#
This file contains any hardwired IP-address-to-hardware-address
mappings you may want for GigaRouter interfaces. It is especially
important for HIPPI, whose ARP tables are manually configured.
For HIPPI, this file replaces the "gria" command.
# File syntax:
#
# [ifname]
host
hwaddr [temp] [pub] [trail]
#
# [ifname]
#
If given, this is the interface name as you would find it
in column 1 of the netstat -i output.
#
# host
#
Hostname or IP address of the remote system
# hwaddr
Hardware address of the remobe system in one of the
GRF Configuration Files
267
Download from Www.Somanuals.com. All Manuals Search And Download.
#
following formats:
#
#
48-bit MAC address for
#
Ethernet or FDDI: xx:xx:xx:xx:xx:xx
#
where ’xx’ are hexadecimal digits.
#
#
32-bit I-field for
#
#
HIPPI: C-language syntax for a 32-bit constant
Example: 0x03000555 for logical address x’555.
#
#
20-byte NSAP address or VPI/VCI for
#
ATM:
vp/vc
VPI/VCI for PVCs
#
where ’vp’ and ’vc’ are decimal integers
#
#
xx.xx.xx. . .xx.xx.xx.xx
where ’xx’ are hexadecimal digits.
Add your configuration following this line ############
NSAP address
#############
# This is the HIPPI Interface from Chap.7.3
# This is on grf16, 10.50.1.2 and 0x03666fc0 is grf4
# grf4 has a line like gh0?0
10.50.1.1
0x03555fc0
gh080
10.50.1.2
0x03666fc0
B.6 /etc/gratm.conf
In this file, the parameters for ATM (OC-3c or OC-12c) are set.
#
# NetStar $Id$
#
# gratm.conf - GigaRouter ATM Configuration File
#
#
# This file is used to configure GigaRouter ATM interfaces.
# Statements in this file are used to configure ATM PVC’s,
# signalling protocols, arp services, and traffic shapes.
#
# gratm(8) uses this file as input when it is run by grinchd(8)
# whenever an ATM media card boots to configure the card.
#
#
# gratm.conf is divided into five sections:
#
# The Service section is where ATM ARP services are defined (entries
#
#
#
defined in this section are referenced from the Interface section
of this file to define which ARP service an interface should use).
# The Traffic Shaping section is where traffic shapes are defined (entries
#
#
#
#
defined in this section are referenced from the Interface and PVC
sections of this file to define which traffic shapes interfaces
and PVC’s should use).
# The Signalling section is where the signalling protocol to be used
#
#
#
by a physical interface to establish Switched Virtual Circuits
is specified.
# The Interfaces section is where per-logical-interface parameters
#
#
#
such as ARP services and Traffic shapes are bound to specific
logical interfaces.
268
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
# The PVC section is where Permanent Virtual Circuits are defined,
#
#
#
using traffic shapes defined in the Traffic Shaping section,
along with other parameters specific to PVC configuration.
#
# Notes on the format of this file:
#
# Comments follow the Bourne Shell style (all characters following a #
# on a line are ignored).
#
# Statements in this file are separated by newlines. A statement may
# span multiple lines by ending each incomplete line of the statement
# with a ’\’ character. Example:
#
# Traffic_Shape name=high_speed peak=15000
#
# this is a statement
# Service name=bc0 type=bcast addr=198.174.11.1 \ # this is also a statement
#
#
addr=198.176.11.1
# User-defined names for Traffic_Shape’s and Service’s must be defined
# before they are used, ie., the definition of a traffic shape must
# precede its use in an Interface or PVC specification, and the
# definition of a Service must precede the use of the name of that
# service in defining any Interfaces.
#
#
# ARP Service info
#
# Lines beginning with the keyword "Service" define virtual "services" which
# may or may not be present on an ATM network attached to a GigaRouter.
#
# Each Service entry the ATM configuration file has the following format:
#
# Service name=value
#
type=arp|bcast addr=value [addr=value ...]
# The "name" field is a unique name to identify this ATM service
#
# The "type" field specifies the type of ATM service being configured.
# and how the address argument(s) which follow are interpreted.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
type=arp
Indicates an ARP service. One to three "addr" field(s)
must follow, defining NSAP addresses of ARP
services on the attached ATM network.
A logical interface using this "Service" entry
will connect to one of the addresses defined for
this service to get ATM address information for
any IP addresses it does not already know the
ATM address of.
type=bcast
Defines a "broadcast service". The "addr" field(s)
contain IP addresses of hosts on a given logical
logical ATM network to which copies of any broadcast
packets will be sent, allowing the GigaRouter, when
so configured, to simulate broadcast over a logical
IP network.
#Service name=arp0 type=arp addr=47000580ffe1000000f21513eb0020481513eb00
GRF Configuration Files
269
Download from Www.Somanuals.com. All Manuals Search And Download.
#Service name=bc0 type=bcast addr=198.174.20.1 addr=198.174.22.1 \
#
addr=198.174.21.1
#
# Traffic shaping parameters
#
# Lines beginning with the keyword "Traffic_Shape" define
# traffic shapes which may be used to configure the performance
# characteristics of ATM Virtual Circuits.
#
# The Traffic_Shape’s defined here are to be referenced by name when
# to assign traffic shapes to PVC’s or Interfaces later in this
# configuration file. (See Examples in the PVC or Interface section
# of this file for examples on how to reference traffic shapes defined here.)
#
# Each Traffic_Shape entry the ATM configuration file has the following format:
#
# Traffic_Shape name=value peak=bps [sustain=bps burst=cells] [qos=high|low]
#
# The "name" field is a unique name to identify this ATM service, so we
# can refer to the collection of peak, [sustain, burst], [qos] parameters
# as a group when configuring PVC’s or Interfaces later in this file.
#
# The ’peak’, ’sustain’, and ’burst’ fields specify, respectively,
# the peak cell rate, the sustained cell rate, and the burst rate.
# The values for ’peak’ and ’sustain’ are in kilobits per second (maximum
# of 155000), and the value for ’burst’ is in cells (maximum of 2048).
#
# The ’qos’ (Quality of Service) field specifies which rate queues
# to use. A value of ’high’ corresponds to high priority service
# which uses the high-priority rate queues, and a value of ’low’ corresponds
# to low priority service which uses the low-priority rate queues.
#
# The peak rate is the only parameter which is mandatory. If ommitted,
# the sustain and burst rates are set to match the peak rate. If qos
# is not specified, it defaults to "high".
#
#Traffic_Shape name=high_speed_high_quality \
#
peak=155000 sustain=155000 burst=2048 qos=high
#Traffic_Shape name=medium_speed_low_quality \
peak=75000 qos=low
#
#Traffic_Shape name=low_speed_high_quality \
#
peak=15000 qos=high
#
# Signalling parameters
#
# Lines beginning with the keyword "Signalling" define
# the signalling protocol which will be used on a physical
# ATM interface to establish Switched Virtual Circuits for
# any logical interfaces on the named physical interface.
#
# Physical interfaces on GigaRouter ATM cards are identified by
# the slot number of the interface card in the GigaRouter chassis
# in hex notation (0-f) plus the location of the physical interface
# on the card (either the top connector, or the bottom connector on the card).
#
# Each Signalling entry the ATM configuration file has the following format:
#
# Signalling card=hex connector=top|bottom [protocol=UNI3.0|UNI3.1|NONE] \
270
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
[mode=SDH|SONET] [clock=Ext|Int]
# The ’card’ and ’connector’ specification are mandatory.
#
# The card should be identified by a hexidecimal digit representing
# the slot number of the card in the GigaRouter chassis.
#
# The connector should be either ’top’ or ’bottom’.
#
# The ’protocol’ parameter defines the signalling protocol to be used
# in the setup of Switched Virtual Circuits (SVC’s) on this physical
# interface. This parameter is optional. If left unspecified, the
# ATM card uses UNI3.0 signalling by default.
#
# Valid values for the signalling parameter include:
#
#
#
#
#
#
#
UNI3.0
UNI3.1
NONE
- for the UNI 3.0 signalling protocol.
- for the UNI 3.1 signalling protocol.
- for no signalling protocol.
# The ’mode’ specification is optional. It can be either SDH or SONET.
# By default, it uses SONET.
#
# The ’clock’ specification is also optional. It can be either Ext(ernal)
# or Int(ernal). The default setting is Internal clocking.
#
#Signalling card=9 connector=top protocol=UNI3.1
#Signalling card=9 connector=bottom protocol=NONE
#Signalling card=a connector=top protocol=UNI3.1
#Signalling card=a connector=bottom protocol=NONE
#
# Interfaces
#
# Lines beginning with the keyword "Interface" define
# GigaRouter logical ATM interfaces.
#
# The format of a logical interface definition is:
#
# Interface ifname [service=service_name] [traffic_shape=shape_name] \
#
#
[bridge_method=method[,restriction]]
# The optional ’service’ parameter allows an ATM service to be
# be defined for this logical interface (the ’service_name’ must be
# a name defined in the Services section above).
#
# The optional ’traffic_shape’ parameter allows a traffic shape
# to be defined for this logical interface (the ’shape_name’ must be
# a named defined as a Traffic_Shape above).
#
# If no traffic shape is specified, a default shape of 155Kbps, high
# quality of service is used.
#
# The optional ’bridge_method’ parameter allows an interface to be used
# for RFC 1483 bridging. The valid values for the bridge_method parameter
# are:
#
#
#
llc_encapsulated
- Use a single PVC, with each frame
encapsulated with an LLC header to identify
GRF Configuration Files
271
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
the protocol
vc_multiplexed
- Use a separate PVC for each protocol
# LLC encapsulated bridging allows any LAN frame type to be transmitted, and
# also allows IP datagrams to be sent directly on the VC. The optional
# ’restriction’ parameter can limit how IP datagrams are routed to the
# interface, and on what kind of LAN frames are transmitted on it.
# The valid values for the restrction parameter are:
#
#
#
#
#
#
#
#
#
#
#
#
#
broute_to_ether
- Transmit all routed IP datagrams as
Ethernet frames
ether_only
- Transmit all frames (routed IP datagrams and
all bridged LAN frames) as Ethernet frames
broute_to_fddi
fddi_only
- Transmit all routed IP datagrams as
FDDI frames
- Transmit all frames (routed IP datagrams and
all bridged LAN frames) as FDDI frames
# Note that unless a restriction or an ARP service is specified, an
# LLC-encapsulated bridging interface will only be able to route to the host at
# the other end of the ATM PVC.
#Interface ga090 service=arp0 traffic_shape=high_speed_high_quality
#Interface ga0980 service=net20 traffic_shape=low_speed_high_quality
#Interface ga0a0 service=arp0 traffic_shape=high_speed_high_quality
#Interface ga0a80 service=net20 traffic_shape=low_speed_high_quality
#Interface ga091 traffic_shape=high_speed_high_quality \
#
bridge_method=llc_encapsulated,broute_to_ether
#
# PVC’s
#
# Lines beginning with the keyword "PVC" define
# Permanent virtual circuits.
#
# The format of a PVC definition is:
#
# PVC ifname VPI/VCI \
#
#
#
#
proto=ip|raw|vc|ipnllc|isis|llc[,bridging]|vcmux_bridge,bpro|vc_atmp \
[input_aal=3|5|NONE] [traffic_shape=shape] \
[dest_if=logical_if [dest_vc=VPI/VCI]]
# The first three parameters (ifname, VPI/VCI, and proto) are mandatory.
#
# ’ifname’ specifies the GigaRouter ATM logical interface in the usual
# format (e.g., ga030, ga0e80).
#
# ’VPI/VCI’ specifies the (decimal) Virtual Path Identifier and
# Virtual Circuit Identifier of the PVC, separated by a slash (/).
#
# ’proto’ specifies the protocol to be supported on this PVC. Legal
# values are:
#
#
#
#
’ip’
Internet Protocol (with LLC/SNAP headers)
raw adaptation layer (AAL-5 or AAL-3/4) packets
IS-IS packets
’raw’
’isis’
’llc’
any LLC-encapsulated protocol supported by the GRF
272
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
(except for RFC 1483 bridging)
’llc,bridging’ any LLC-encapsulated protocol, including RFC 1483
bridging [This is the PVC type for an interface using
bridge_method=llc_encapsulated.]
’vcmux_bridge’ bridged packets [This is a PVC type for an interface
using bridge_method=vc_multiplexed.] An additional
parameter specifies the protocol carried on the VC:
ether_fcs
ether_nofcs
fddi_fcs
fddi_nofcs
bpdu
Ethernet frames, with Frame Check Sequence
Ethernet frames, without Frame Check Sequence
FDDI frames, with Frame Check Sequence
FDDI frames, without Frame Check Sequence
802.1D Bridging Protocol Data Units
’vc’
IP datagrams [This is a PVC type for an interface using
bridge_method=vc_multiplexed.]
atmp home network connections using VC Based
Multiplexing.
’vc_atmp’
# If the ’proto’ specified is ’raw’, the ’dest_if’ parameter specifies the
# GigaRouter interface of the destination for this raw adaptation layer
# connection (specified in the same GigaRouter interface format
# described above), and (optionally) the dest_vc parameter specifies
# the destination VPI/VCI.
#
# the ’input_aal’ parameter may be used to specify the adaptation layer,
#
#
#
input_aal=3
input_aal=5
specifies AAL-3/4
specifies AAL-5
# The optional ’traffic_shape’ parameter allows a traffic shape
# to be defined for this logical interface (the ’shape_name’ must be
# a named defined as a Traffic_Shape above).
#
# If no traffic shape is specified, a default shape of 155Kbps, high
# quality of service is used.
#
#PVC ga090 0/32 proto=ip traffic_shape=high_speed_high_quality
#PVC ga0980 0/32 proto=ip traffic_shape=high_speed_high_quality
#PVC ga0a0 1/33 proto=raw traffic_shape=high_speed_high_quality \
#
dest_if=ga090 dest_vc=5/50 input_aal=5
#PVC ga0a80 1/33 proto=ip traffic_shape=high_speed_high_quality
#PVC ga0b0 0/40 proto=isis traffic_shape=high_speed_high_quality
#PVC ga0c0 0/41 proto=isis_ip traffic_shape=high_speed_high_quality
#PVC ga030 0/32 proto=llc,bridging
#PVC ga031 0/32 proto=vcmux_bridge,ether_nofcs
#PVC ga031 0/33 proto=vcmux_bridge,fddi_nofcs
#PVC ga031 0/34 proto=vcmux_bridge,bpdu
# Here come the different settings we used during
# the residency.
Traffic_Shape name=high_speed_high_quality \
peak=155000 sustain=155000 burst=2048 qos=high
Traffic_Shape name=low_speed_high_quality \
peak=15500 qos=high
# ATM OC-12c
Traffic_Shape name=bigg_speed_high_quality \
GRF Configuration Files
273
Download from Www.Somanuals.com. All Manuals Search And Download.
peak=622000 sustain=622000 burst=2048 qos=high
Signalling card=1 connector=top protocol=NONE
Signalling card=1 connector=bottom protocol=NONE
# Interface ga010 traffic_shape=high_speed_high_quality
# PVC ga010 0/132 proto=ip traffic_shape=high_speed_high_quality
Interface ga010 traffic_shape=high_speed_high_quality \
bridge_method=vc_multiplexed
PVC ga010 0/132 proto=vcmux_bridge,bpdu
PVC ga010 0/133 proto=vcmux_bridge,ether_nofcs
PVC ga010 0/134 proto=vcmux_bridge,fddi_nofcs
PVC ga010 0/135 proto=llc
#PVC ga010 0/136 proto=ip
# Interface ga0180 traffic_shape=high_speed_high_quality
# PVC ga0180 0/134 proto=ip traffic_shape=high_speed_high_quality
Interface ga0180 traffic_shape=high_speed_high_quality \
bridge_method=vc_multiplexed
PVC ga0180 0/144 proto=vcmux_bridge,bpdu
PVC ga0180 0/145 proto=vcmux_bridge,ether_nofcs
PVC ga0180 0/146 proto=vcmux_bridge,fddi_nofcs
PVC ga0180 0/147 proto=llc
#PVC ga0180 0/148 proto=ip
# OC-12c
Interface ga020 traffic_shape=bigg_speed_high_quality
PVC ga020 0/132 proto=ip traffic_shape=bigg_speed_high_quality
B.7 /etc/grclean.conf
This file contains the specifications on how to handle log file management on
the GRF. It includes the file /etc/grclean.logs.conf, where the location of the
log files in the filesystem is handled. See Appendix B.8,
“/etc/grclean.logs.conf” on page 275 for further reference.
################################################################################
#
NetStar $Id: grclean.conf,v 1.10.4.1 1997/08/27 19:58:30 jeanne Exp $
################################################################################
# grclean configuration file
# keyword
# -------
# include
# hold
# local
# remove
# arch
# ash
default
-----------------
<filename>
7
no
no
none
none
what is it
-------------------------------
name of another config file to act on
# of archives kept
keep arch in same place as logfile
remove logfile after archival
archive name
command to execute after archival
command to execute before archival
command to execute if no archival
zip/compress command
# bsh
# nsh
# zip
none
none
/usr/contrib/bin/gzip
# zap
/usr/contrib/bin/gunzip unzip/uncompress command
# ext
auto
extension to use for compressed archive
# dir
# tmp
/var/log
/tmp
where to put the archives if ! local
tmp directory
# date
# size
‘/bin/date +%x %r"‘
0
date command for msgs
size threshold for archive or not
274
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
# DEFAULTS
sets all of the above keywords to the above defaults.
################################################################################
################################################################################
# log files.
################################################################################
include=/etc/grclean.logs.conf
################################################################################
################################################################################
################################################################################
# port card dump files.
################################################################################
hold=4
size=1
remove=y
local=y
logfile=/var/portcards/grdump.*
################################################################################
B.8 /etc/grclean.logs.conf
This file is included from /etc/grclean.conf. Take special care of the location of
the log files once a PCMCIA hard disk is installed. Refer to Section 3.3.2,
#
$
NetStar $Id: grclean.logs.conf,v 1.9.2.3 1997/08/27 19:58:31 jeanne Exp
################################################################################
# grclean configuration file
# keyword
# -------
# include
# hold
# local
# remove
# arch
# ash
default
-----------------
<filename>
7
no
no
none
none
what is it
-------------------------------
name of another config file to act on
# of archives kept
keep arch in same place as logfile
remove logfile after archival
archive name
command to execute after archival
command to execute before archival
command to execute if no archival
zip/compress command
# bsh
# nsh
# zip
none
none
/usr/contrib/bin/gzip
# zap
/usr/contrib/bin/gunzip unzip/uncompress command
# ext
auto
extension to use for compressed archive
# dir
# tmp
/var/log
/tmp
where to put the archives if ! local
tmp directory
# date
# size
‘/bin/date +%x %r"‘
0
date command for msgs
size threshold for archive or not
################################################################################
################################################################################
# of some interest.
# console log file.
# boot log file.
################################################################################
ash=kill -1 ‘cat /var/run/syslog.pid‘
hold=2
local=y
################################################################################
GRF Configuration Files
275
Download from Www.Somanuals.com. All Manuals Search And Download.
# Log files that used to be archived by the /etc/{daily|weekly|monthly}
# scripts.
################################################################################
size=10000
logfile=/var/account/acct
size=10000
logfile=/var/log/maillog
size=150000
logfile=/var/log/messages
size=10000
logfile=/var/log/daemon.log
size=10000
logfile=/var/log/cron
size=10000
logfile=/var/log/xferlog
size=10000
logfile=/var/log/httpd/access_log
size=10000
logfile=/var/log/httpd/error_log
size=10000
logfile=/var/log/ftp.log
size=10000
logfile=/var/log/kerberos.log
size=10000
logfile=/var/log/lpd-errs
size=10000
logfile=/var/log/gritd.packets
size=150000
logfile=/var/log/gr.console
size=11000
logfile=/var/log/gr.boot
size=150000
logfile=/var/log/grinchd.log
size=10000
logfile=/var/log/gr.conferrs
size=25000
logfile=/var/log/mib2d.log
size=25000
logfile=/var/tmp/gated.rip
size=25000
logfile=/var/log/mibmgrd.log
size=25000
logfile=/var/log/cli.log
size=75000
logfile=/var/log/fred.log
################################################################################
# Process site specific clean conf, if any.
################################################################################
include=/etc/grclean.site.conf
################################################################################
# /var/tmp/core.* file cleanup.
################################################################################
hold=1
remove=y
size=1024
logfile=/var/tmp/core.*
################################################################################
# cleanup our own log file, if necessary.
################################################################################
276
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
DEFAULTS
hold=2
local=y
size=10000
logfile=/var/log/grclean.log
B.9 /etc/grdev1.conf
This file normally gets updated automatically by the SNMP daemon running
on the CWS.
############################################################################
# DEV1 Configuration
###########################################################################
#
# There are several variables that an SP Adapter card needs at start up.
# These are handled by a set of GRINCHES whose descriptors are indexed
# by card number and interface number as follows:
#
#
#
2.21.{CARD+1}.1.{INTERFACE+1}
# This template specifies the start-up values for all potential cards
# in a 16-card GRF router. Initially these are default values indicating
# that the card needs to be configured.
#
# The descriptors are grouped by card and interface so that a particular
# interface can be easily configured.
#
############################################################################
#
# CARD 0 Interface 0
#
2.21.1.1.1.1
2.21.1.1.1.2
2.21.1.1.1.3
2.21.1.1.1.4
2.21.1.1.1.5
2.21.1.1.1.6
2.21.1.1.1.7
2.21.1.1.1.8
2.21.1.1.1.9
2.21.1.1.1.10
2.21.1.1.1.11
2.21.1.1.1.12
2.21.1.1.1.13
2.21.1.1.1.14
2.21.1.1.1.15
#
"00"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 1 Interface 0
#
2.21.2.1.1.1
2.21.2.1.1.2
2.21.2.1.1.3
2.21.2.1.1.4
2.21.2.1.1.5
2.21.2.1.1.6
2.21.2.1.1.7
2.21.2.1.1.8
2.21.2.1.1.9
2.21.2.1.1.10
2.21.2.1.1.11
"01"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
1
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
"no name"
GRF Configuration Files
277
Download from Www.Somanuals.com. All Manuals Search And Download.
2.21.2.1.1.12
2.21.2.1.1.13
2.21.2.1.1.14
2.21.2.1.1.15
#
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 2 Interface 0
#
2.21.3.1.1.1
2.21.3.1.1.2
2.21.3.1.1.3
2.21.3.1.1.4
2.21.3.1.1.5
2.21.3.1.1.6
2.21.3.1.1.7
2.21.3.1.1.8
2.21.3.1.1.9
2.21.3.1.1.10
2.21.3.1.1.11
2.21.3.1.1.12
2.21.3.1.1.13
2.21.3.1.1.14
2.21.3.1.1.15
#
"02"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 3 Interface 0
#
2.21.4.1.1.1
2.21.4.1.1.2
2.21.4.1.1.3
2.21.4.1.1.4
2.21.4.1.1.5
2.21.4.1.1.6
2.21.4.1.1.7
2.21.4.1.1.8
2.21.4.1.1.9
2.21.4.1.1.10
2.21.4.1.1.11
2.21.4.1.1.12
2.21.4.1.1.13
2.21.4.1.1.14
2.21.4.1.1.15
#
"03"
4
# Node Name
# Node Number
"00:00:00:01:00:00:00:06:00:01" # Switch Token
2
3
x192.168.14.4
x255.255.255.128
1024
1
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
# Node State
# Switch Chip Link
# Node Delay
1
sp21en0
2
1
31
1
# Admin Status
# CARD 4 Interface 0
#
2.21.5.1.1.1
2.21.5.1.1.2
2.21.5.1.1.3
2.21.5.1.1.4
2.21.5.1.1.5
2.21.5.1.1.6
2.21.5.1.1.7
2.21.5.1.1.8
2.21.5.1.1.9
2.21.5.1.1.10
2.21.5.1.1.11
2.21.5.1.1.12
2.21.5.1.1.13
2.21.5.1.1.14
2.21.5.1.1.15
#
"04"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 5 Interface 0
#
2.21.6.1.1.1
2.21.6.1.1.2
"05"
16
# Node Name
# Node Number
278
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.21.6.1.1.3
2.21.6.1.1.4
2.21.6.1.1.5
2.21.6.1.1.6
2.21.6.1.1.7
2.21.6.1.1.8
2.21.6.1.1.9
2.21.6.1.1.10
2.21.6.1.1.11
2.21.6.1.1.12
2.21.6.1.1.13
2.21.6.1.1.14
2.21.6.1.1.15
#
"00:00:00:01:00:00:00:07:00:03" # Switch Token
2
# Switch ARP
15
# Switch Node Number
# IP Address
x192.168.14.129
x255.255.255.128
# Net Mask
1024
-14
1
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
sp21en0
2
# Node State
3
31
1
# Switch Chip Link
# Node Delay
# Admin Status
# CARD 6 Interface 0
#
2.21.7.1.1.1
2.21.7.1.1.2
2.21.7.1.1.3
2.21.7.1.1.4
2.21.7.1.1.5
2.21.7.1.1.6
2.21.7.1.1.7
2.21.7.1.1.8
2.21.7.1.1.9
2.21.7.1.1.10
2.21.7.1.1.11
2.21.7.1.1.12
2.21.7.1.1.13
2.21.7.1.1.14
2.21.7.1.1.15
#
"06"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 7 Interface 0
#
2.21.8.1.1.1
2.21.8.1.1.2
2.21.8.1.1.3
2.21.8.1.1.4
2.21.8.1.1.5
2.21.8.1.1.6
2.21.8.1.1.7
2.21.8.1.1.8
2.21.8.1.1.9
2.21.8.1.1.10
2.21.8.1.1.11
2.21.8.1.1.12
2.21.8.1.1.13
2.21.8.1.1.14
2.21.8.1.1.15
#
"07"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 8 Interface 0
#
2.21.9.1.1.1
2.21.9.1.1.2
2.21.9.1.1.3
2.21.9.1.1.4
2.21.9.1.1.5
2.21.9.1.1.6
2.21.9.1.1.7
2.21.9.1.1.8
2.21.9.1.1.9
2.21.9.1.1.10
2.21.9.1.1.11
"08"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
1
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
"no name"
GRF Configuration Files
279
Download from Www.Somanuals.com. All Manuals Search And Download.
2.21.9.1.1.12
2.21.9.1.1.13
2.21.9.1.1.14
2.21.9.1.1.15
#
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 9 Interface 0
#
2.21.10.1.1.1
2.21.10.1.1.2
2.21.10.1.1.3
2.21.10.1.1.4
2.21.10.1.1.5
2.21.10.1.1.6
2.21.10.1.1.7
2.21.10.1.1.8
2.21.10.1.1.9
2.21.10.1.1.10
2.21.10.1.1.11
2.21.10.1.1.12
2.21.10.1.1.13
2.21.10.1.1.14
2.21.10.1.1.15
#
"09"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 10 Interface 0
#
2.21.11.1.1.1
2.21.11.1.1.2
2.21.11.1.1.3
2.21.11.1.1.4
2.21.11.1.1.5
2.21.11.1.1.6
2.21.11.1.1.7
2.21.11.1.1.8
2.21.11.1.1.9
2.21.11.1.1.10
2.21.11.1.1.11
2.21.11.1.1.12
2.21.11.1.1.13
2.21.11.1.1.14
2.21.11.1.1.15
#
"10"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 11 Interface 0
#
2.21.12.1.1.1
2.21.12.1.1.2
2.21.12.1.1.3
2.21.12.1.1.4
2.21.12.1.1.5
2.21.12.1.1.6
2.21.12.1.1.7
2.21.12.1.1.8
2.21.12.1.1.9
2.21.12.1.1.10
2.21.12.1.1.11
2.21.12.1.1.12
2.21.12.1.1.13
2.21.12.1.1.14
2.21.12.1.1.15
#
"11"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 12 Interface 0
#
2.21.13.1.1.1
2.21.13.1.1.2
"12"
-1
# Node Name
# Node Number
280
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
2.21.13.1.1.3
2.21.13.1.1.4
2.21.13.1.1.5
2.21.13.1.1.6
2.21.13.1.1.7
2.21.13.1.1.8
2.21.13.1.1.9
2.21.13.1.1.10
2.21.13.1.1.11
2.21.13.1.1.12
2.21.13.1.1.13
2.21.13.1.1.14
2.21.13.1.1.15
#
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
# Switch ARP
0
# Switch Node Number
# IP Address
x0.0.0.0
x0.0.0.0
# Net Mask
1024
0
1
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
"no name"
2
# Node State
0
64
1
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 13 Interface 0
#
2.21.14.1.1.1
2.21.14.1.1.2
2.21.14.1.1.3
2.21.14.1.1.4
2.21.14.1.1.5
2.21.14.1.1.6
2.21.14.1.1.7
2.21.14.1.1.8
2.21.14.1.1.9
2.21.14.1.1.10
2.21.14.1.1.11
2.21.14.1.1.12
2.21.14.1.1.13
2.21.14.1.1.14
2.21.14.1.1.15
#
"13"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 14 Interface 0
#
2.21.15.1.1.1
2.21.15.1.1.2
2.21.15.1.1.3
2.21.15.1.1.4
2.21.15.1.1.5
2.21.15.1.1.6
2.21.15.1.1.7
2.21.15.1.1.8
2.21.15.1.1.9
2.21.15.1.1.10
2.21.15.1.1.11
2.21.15.1.1.12
2.21.15.1.1.13
2.21.15.1.1.14
2.21.15.1.1.15
#
"14"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
1
"no name"
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
# CARD 15 Interface 0
#
2.21.16.1.1.1
2.21.16.1.1.2
2.21.16.1.1.3
2.21.16.1.1.4
2.21.16.1.1.5
2.21.16.1.1.6
2.21.16.1.1.7
2.21.16.1.1.8
2.21.16.1.1.9
2.21.16.1.1.10
2.21.16.1.1.11
"15"
-1
# Node Name
# Node Number
"00:00:00:00:00:00:00:00:00:00" # Switch Token
2
0
x0.0.0.0
x0.0.0.0
1024
0
1
# Switch ARP
# Switch Node Number
# IP Address
# Net Mask
# Max Link Pckt Len.(bytes)
# IP Host Offset
# Configuration State
# System Name
"no name"
GRF Configuration Files
281
Download from Www.Somanuals.com. All Manuals Search And Download.
2.21.16.1.1.12
2.21.16.1.1.13
2.21.16.1.1.14
2.21.16.1.1.15
2
0
64
1
# Node State
# Switch Chip Link
# Node Delay (cycles)
# Admin Status
B.10 /etc/grifconfig.conf
This file documents the correlation of the logical interfaces on the media
cards in the GRF to IP addresses, together with some other information, like
MTU.
#
#
NetStar $Id: grifconfig.conf,v 1.10.2.3 1997/08/01 17:24:04 pargal Exp $
# Configuration file for GigaRouter/GRF interfaces.
#
# The contents of this file specify the IP addressing information for
# the networks attached to the system’s interfaces. This includes
# interfaces on media cards as well as directly attached interfaces
# such as de0 or ef0 (maintenance Ethernet) or lo0 (software loopback).
#
# The addresses of directly attached interfaces are configured
# directly from this file by the /etc/netstart calling the grifconfig(8)
# script.
#
# The addresses of the interface(s) on a given media card are
# configured into the BSD/OS kernel when the media card boots and
# comes on line.
#
# Each entry in this file has the following format:
#
# name address
#
netmask
broad_dest
arguments
# The name of a GigaRouter interface encodes the hardware type,
# GigaRouter cage number, slot number, and interface number.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
-- The first character must be ’g’ (to specify a GigaRouter
interface).
-- The second character is the hardware type of the
interface: ’a’ for ATM, ’e’ for ETHERNET,
’f’ for FDDI, ’h’ for HIPPI, ’p’ for PPP, ’s’ for HSSI.
(’l’ is also used, for GigaRouter software loopback.)
-- The third character is the number of the GigaRouter cage.
(Currently this must be ’0’, as multiple GigaRouter cages
are not yet supported.)
-- The fourth character is the hex digit (0 through f) of
the slot number within the GigaRouter cage.
-- The fifth (and sixth) characters specify the number of the
LOGICAL interface on the card:
For ATM cards, the fifth and sixth characters are
the hex digits of the logical interface. Logical
interfaces numbered from 0 to 7f are on the top
physical connector on the ATM card, and logical
interfaces numbered 80 to ff are on the bottom
physical connector. NOTE: These logical interface
numbers are NOT the same as the VPI/VCI numbers
of a PVC (see /etc/grpvc.conf for that).
For FDDI cards, the fifth character will be 0, 1,
2, or 3 to specify the logical interface on the
282
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
FDDI card. NOTE: The logical interface number
may be different from the physical interface on
the card, depending on the single- or dual-
attachedness of the various interfaces. Examples:
"gf073" specifies the bottom-most connector
on the FDDI card in slot 7; "gf020" specifies
top-most connector on the FDDI card in slot 2,
or the top TWO connectors on that card if they’re
configured dual-attached.
For ETHERNET cards, the fifth character will be 0, 1,
2, 3, 4, 5, 6 or 7 to specify the physical interface
on the ETHERNET card.
Examples:
"ge067" specifies the 8th physical connector
on the ETHERNET card in slot 6.
"ge000" specifies the first (top) connector on the
ETHERNET card in slot 0.
"ge0f7" specifies the last (bottom) connector on the
ETHERNET card in slot 15.
For HIPPI cards, which only have one interface,
the fifth character is always 0. Example:
"gh0f0" specifies the interface for a HIPPI card
in slot 15.
# The IP "address", "netmask" (optional), and "broad_dest" (optional)
# address fields must be specified in canonical IP dotted-quad notation.
# An entry of "-" (a single hyphen) may be specified for any of these
# fields as a place-holder. This may be useful, e.g., if no netmask
# is desired but a broadcast or destination address must be specified
# in the next field.
#
#
# For this release, the "broad_dest" field specifies the broadcast
# IP address for Ethernet & FDDI interfaces, and the destination of
# a point-to-point ATM or HIPPI interface.
#
# The "arguments" field is for any additional arguments to be supplied
# to the underlying ifconfig(8) command that will be executed by
# grifconfig(8). The most useful purpose would be to specify an
# MTU value for the interface using the "mtu" keyword of ifconfig(8).
# The keyword "iso" can also be specified here which designates the current
# line as an iso address entry.
# See the example entry below, and the man page for ifconfig(8).
#
#
# NOTE: All interface names are case sensitive ! Always use lower case letters
#
#
#
when defining interface names.
# name address
#
netmask
broad_dest
arguments
mtu 1024
#de0
lo0
192.0.2.1
127.0.0.1
255.255.255.0
255.0.0.0
192.0.2.255
#gl000 127.0.1.1
# configuration for iso addresses
#gf0xx 49.0000.80.3260.3260.3260.00 49.0000.80 -
iso
# All possible IOSTB3 Interfaces
# gt000 0.0.0.0
#gt010 0.0.0.0
255.255.255.0
255.255.255.0
-
-
mtu 65520
mtu 65520
GRF Configuration Files
283
Download from Www.Somanuals.com. All Manuals Search And Download.
#gt020 0.0.0.0
gt030 192.168.14.4
#gt040 192.168.14.129 255.255.255.128 -
gt050 192.168.14.129 255.255.255.128 -
255.255.255.0
255.255.255.128 -
-
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
mtu 65520
#gt060 0.0.0.0
#gt070 0.0.0.0
#gt080 0.0.0.0
#gt090 0.0.0.0
#gt0a0 0.0.0.0
#gt0b0 0.0.0.0
#gt0c0 0.0.0.0
#gt0d0 192.168.13.16
#gt0e0 0.0.0.0
#gt0f0 0.0.0.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
255.255.255.0
-
-
-
-
-
-
-
-
-
-
de0
192.168.4.4
#ga010 10.1.1.1
#ga0180 10.1.2.1
-
mtu 9180
10.1.2.1
bg1
10.1.1.1
-
-
-
-
-
-
mtu 9180
mtu 4352
mtu 4352
mtu 4352
mtu 4352
mtu 4352
# gf000 10.2.1.15
# gf001 10.3.1.16
# gf002 10.4.1.17
# gf003 10.5.1.18
bg0
10.10.1.13
ge070 10.20.30.1
-
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
mtu 1500
#ge071 192.168.4.71
#ge072 192.168.4.72
#ge073 192.168.4.73
#ge074 192.168.4.74
#ge075 192.168.4.75
#ge076 192.168.4.76
#ge077 192.168.4.77
-
-
-
-
-
-
-
-
gh080
10.50.1.1
mtu 65280
mtu 9180
# ga020
10.20.30.1
-
B.11 /etc/grlamap.conf
This file contains the mapping of logical addresses to destination portcards
for HIPPI interfaces. We used this file in Section 7.3, “HIPPI Backbone
#
#
NetStar $Id: grlamap.conf,v 1.1 1995/02/09 20:29:27 scotth Exp $
################################################################################
#
# There are 3 fields to determine how logical addresses get
# mapped to destination portcards for each hippi portcard.
#
# The first field is a comma separated list which determines the
# portcard on which the other 2 fields will apply.
# Ranges are allowed. i.e., 1,4-9,15 == 1,4,5,6,7,8,9,15
# ’*’ signifies that all portcards get this mapping.
#
# The second field is a comma separated list of logical addresses being mapped.
# Ranges are allowed.
# values: 0 - 4095 inclusive, ’*’ signifies the default (i.e., all LAs).
#
# The third field is a comma separated list of destination port cards.
# Only the first 4 values will be taken as valid, any extra values will
# be flagged invalid, a message will be printed to gr.console & grlamap will
284
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
# exit.
# If only 3 values are designated, the first value will be repeated as the
# fourth value.
#
# No whitespace is allowed in any field.
#
# ie.
# 5,6
997,998
1,0x4,8,15
# When port cards 5 & 6 receive an IP packet with a logical address of 997 or
# 998, it will then attempt to randomly forward the packet to one of the
# mapped ports 1, 4, 8 or 15.
#
################################################################################
#*
#5
*
*
5
6
# default mapping for all LAs for all portcards.
# default mapping for LAs for portcard 5.
################################################################################
# IP mapping.
################################################################################
# *
*
0xfc0
0xfc0
3,8
0x40
# default mapping of IP for all portcards.
# default mapping of IP for all portcards.
################################################################################
#5
#5
1-100
100-200 4
9
# default mapping for LAs for portcard 5.
# default mapping for LAs for portcard 5.
B.12 /etc/grroute.conf
Contains the default route one is supposed to provide during first installation.
Routes to selected networks may also be entered here and will be brought up
automatically during start up procedures of the SP Switch Router.
#
#
NetStar $Id: grroute.conf,v 1.3 1995/03/15 22:09:07 knight Exp $
# grroute.conf -- configuration file for GigaRouter static remote routes
#
# This file should only contain routes to remote networks and
# hosts--i.e., networks and hosts not directly attached to a
# GigaRouter interface. Routes for networks directly attached
# to the GigaRouter are created as part of configuring the
# GigaRouter interfaces; see /etc/grifconfig.conf.
#
# NOTE: THIS FILE SHOULD NOT BE USED ON SYSTEMS WITH DYNAMIC
# ROUTING (gated). If you are running gated, then you should specify
# static routes using the "static" statement in /etc/gated.conf.
#
# Whenever a port card boots, comes on line and has its interface(s)
# configured, the routes specified in this file that are for gateways
# on the network(s) directly attached to those interface(s) are
# configured into the BSD/386 kernel.
#
# The format of each line is follows:
#
#
#
destination
netmask
gateway/next hop
# The destination is the IP address of the remote host or network.
# For the default route, specify a destination of "0.0.0.0" or the
# word "default".
#
GRF Configuration Files
285
Download from Www.Somanuals.com. All Manuals Search And Download.
# A netmask is required for all entries in this configuration file.
#
# The netmask is normally the mask of the remote network:
#
#
#
192.0.2.0
255.255.255.0
123.45.67.89
# For remote host routes, specify a netmask of 255.255.255.255:
#
#
#
192.0.2.1
255.255.255.255 123.45.67.89
# In the case of the default route (0.0.0.0), the netmask is ignored,
# but some value must be present for the file to parse correctly.
#
#
#
# destination
netmask
gateway/next hop
#
#0.0.0.0
default
#
0.0.0.0
0.0.0.0
192.0.2.2
192.168.4.137
# preferred routes during the residency
#
# 192.168.13.0
192.168.13.0
255.255.255.0
255.255.255.0
10.1.1.2
10.20.30.2
B.13 /etc/hosts
The file /etc/hosts contains the correlation between IP addresses and
hostnames.
#
# Host Database
# This file should contain the addresses and aliases
# for local hosts that share this file.
# It is used only for "ifconfig" and other operations
# before the nameserver is started.
#
#
127.1
#
localhost
# 10.200.160.3 install1.venus
192.168.4.4
192.168.4.137
grf16.msc.itso.ibm.com grf16
sp21en0.msc.itso.ibm.com sp21cw0
B.14 /etc/inetd.conf
This file is included here just for curiosity in case you are interested in why
ftp to the GRF is not possible.
#
#
BSDI
$Id: inetd.conf,v 2.2 1996/01/02 19:55:38 polk Exp $
#
@(#)inetd.conf 8.2 (Berkeley) 3/18/94
#
#ftp
stream tcp
nowait root
nowait root
nowait root
nowait root
nowait root
/usr/libexec/tcpd
/usr/libexec/tcpd
/usr/libexec/tcpd
/usr/libexec/tcpd
/usr/libexec/tcpd
ftpd -l -A
telnetd
rshd
rlogind -a
rexecd
telnet stream tcp
#shell stream tcp
#login stream tcp
#exec
stream tcp
286
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#uucpd stream tcp
#finger stream tcp
nowait root
nowait nobody /usr/libexec/tcpd
/usr/libexec/tcpd
uucpd
fingerd
tftpd
comsat
ntalkd
popper
identd -l
bootpd -t 1
#tftp
#comsat dgram
#ntalk dgram
dgram
udp
udp
udp
wait
wait
wait
nobody /usr/libexec/tcpd
root
root
/usr/libexec/tcpd
/usr/libexec/tcpd
/usr/libexec/tcpd
/usr/libexec/identd
/usr/libexec/tcpd
internal
#pop
#ident stream tcp
#bootp dgram udp
stream tcp
stream tcp
stream tcp
nowait root
nowait sys
wait
nowait root
nowait root
root
#echo
#discard
#chargen
#daytime
internal
internal
internal
stream tcp
stream tcp
nowait root
nowait root
#tcpmux stream tcp
nowait root
nowait root
internal
internal
internal
root
root
root
#time
stream tcp
#echo
dgram
udp
wait
udp
root
wait
wait
wait
root
wait
#discard
#chargen
#daytime
#time
dgram
dgram
dgram
udp
internal
internal
internal
udp
udp
dgram
wait
udp
internal
# amanda
amandad
dgram
operator /usr/contrib/lib/amanda/amandad
# Kerberos authenticated services
#klogin stream tcp nowait root
stream tcp nowait root
/usr/libexec/rlogind
rlogind -k
rlogind
#eklogin
/usr/libexec/rlogind
-k -x
#kshell stream tcp
nowait root
/usr/libexec/rshd
rshd -k
# Services run ONLY on the Kerberos server
#krbupdate stream tcp nowait root /usr/libexec/registerd registerd
#kpasswd stream tcp nowait root /usr/libexec/kpasswdd kpasswdd
B.15 /etc/motd
This file contains the greeting message of the system and within this
information about the running version and release of the operating system.
Complement this information with the content of Appendix B.2, “/etc/Release”
Ascend Embedded/OS GR TA1.4.6.4 Kernel #0 (nit): Wed Mar 4 10:10:10 CST 1998
Ascend Embedded/OS 1.4.6
Copyright 1992,1993,1994,1995,1996,1997,1998 Ascend Communications, Inc.
IMPORTANT: By use of this software you become subject to the terms and
conditions of the license agreement on file /etc/license and any other
license agreements previously provided to you by Ascend Communications.
B.16 /etc/rc.local
Use this file for local modifications that are to be carried out during boot.
#c
#c
#c
#c
#c
#c
(c) COPYRIGHT 1992-1996, NetStar, Inc.
All rights reserved
NetStar, Inc. CONFIDENTIAL AND PROPRIETARY
This product is the sole property of NetStar, Inc.
GRF Configuration Files
287
Download from Www.Somanuals.com. All Manuals Search And Download.
#c
#c
#c
#c
#c
#c
#c
#c
#c
#c
#c
#
and is protected by U.S. and other copyright laws and the
laws protecting trade secret and confidential information.
This product contains trade secret and confidential
information, and its unauthorized disclosure is prohibited.
Reproduction, utilization and transfer of rights to this
product, whether in source or binary form, is permitted
only pursuant to a written agreement signed by an authorized
officer of NetStar, Inc.
#
NetStar $Id: rc.local,v 1.1 1997/01/15 07:45:03 stuarts Exp $
#
# Site-specific script for local startup daemons and other actions.
# By default on a GRF, there, there are no local daemons, so
# just comment out the echo-logic that BSD/OS uses at startup.
# Uncomment this line (and the trailing "echo ’.’" line below)
# if you really do have local daemons and want this sort of
# pretty-printing at system startup.
#echo -n ’starting local daemons:’
# An example of starting a local daemon.
# Uncomment, replicate, and modify as needed for your local daemons.
#echo -n " local_daemon";
/usr/local/bin/local_daemon
# Terminate the pretty-print list of local daemons we started.
#echo ’.’
# Put other local customizations here...
# Have to put this in here on demand. See README for 1.4.6.4
sysctl -w net.inet.ip.fwdirbcast=1
# Exit with a successful return status.
exit 0
B.17 /etc/snmpd.conf
This file is used to specify access and control permissions to the snmp
demon. During first installation, the CWS has to be added to this file; see
#
# Default Agent Configuration File
#
#
#
#
#
#
#
#
This file allows MANAGERS to be specified. This is used to
specify which managers will be receiving which traps.
Also, COMMUNITYs can be specified. This allows that agent to
be configured such that it will only except requests from
certain managers and with certain community strings.
# GRAMMAR:
#
#
#
#
INITIAL
<name> <String>
<name>
TRANSPORT
288
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
[SNMP | SMUX]
[OVER [UNIX | UDP | TCP] [SOCKET | TLI]]
[AT <addr>]
MANAGER
<addr> [ON TRANSPORT <name>]
[SEND [ALL | NO | traplist] TRAPS
[TO PORT <#> ]
[WITH COMMUNITY <name>]]
COMMUNITY
<name>
ALLOW op [,op]* [OPERATIONS]
[AS <name>]
[USE encrypt ENCRYPTION]
[MEMBERS
<addr> [,<addr>] ]
ALLOW <subagentId> [ON <hostSpec>]
WITH <passwordSpec> [<entitySpec>] [<timeout>]
DENY <subagentId> ON <hostSpec> WITH <passwordSpec>
ENTITY <EntityName> DESCRIPTION <String>
LOCAL CONTEXT <contextName> [USES] VIEW <viewName>
REFERS TO ENTITY <entityName> AS <oid>
PROXY CONTEXT <oid> [USES] SOURCE PARTY <oid>
DESTINATION PARTY <oid>
[AND] CONTEXT <oid>
VIEW <viewName> [[INCLUDE | EXCLUDE] SUBTREE <oid> [MASK <bitmask>]]+
ALLOW op [,op]* [OPERATIONS] <sugar> SOURCE PARTY <partyName>
DESTINATION PARTY <partyName>
[AND] CONTEXT <contextName>
<partyDefinition> ::= [LOCAL] PARTY <name> ON TRANSPORT <transport>
AT <addr> <AuthPriv>
AS <oid>
<transport> ::= [ snmpUDPDomain | snmpCLNSDomain | snmpCONSDomain |
snmpDDPDomain | snmpIPXDomain | rfc1157Domain
<transport> ::=
<AuthPriv> ::= <noAuth> <noPriv> |
<md5Auth> <noPriv> |
<md5Auth> <desPriv>
<noAuth> ::= <sugar> NO AUTHENTICATION
<sugar> ::= [AND] [[WITH | USING]]
<noPriv> ::= <sugar> NO ENCRYPTION
<md5Auth> ::= <sugar> MD5 AUTHENTICATION <key>
<key> ::= <sugar> <string> AS KEY
<desPriv> ::= <sugar> DES ENCRYPTION <key>
<subagentId> ::= SUBAGENT <oid> |
SMUX SUBAGENT <oid> |
GRF Configuration Files
289
Download from Www.Somanuals.com. All Manuals Search And Download.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
UNSPECIFIED SUBAGENTS
<hostSpec> ::= HOST <hostid> |
UNSPECIFIED HOST[S]
<passwordSpec> ::= PASSWORD <string>
UNSPECIFIED PASSWORDS
entitySpec ::= AS ENTITY <entityName>
<timeout> ::= USING <specificTimeout> TIMEOUT
<specificTimeout> ::= <number> SECOND[S] |
NO
addr ::=
ip-kind ::=
<ip-kind> | <rfc1449addr> | <full-ip>
<hostid> |
<hostid> <portid> |
<portid>
hostid ::=
portid ::=
<hostname> | <ip>
where: hostname is defined in /etc/host
[PORT | : ] <#>
full-ip ::=
ip ::=
traplist ::=
trap ::=
<ip>:<#>
<#>.<#>.<#>.<#>
trap [, trap]*
<trap_name>
op ::=
encrypt ::=
ALL | GET | SET | TRAP
NO | <name>
rfc1449addr ::= tcp_ip_addr | osi_addr
tcp_ip_addr ::= <ip>/<#>
osi_addr ::= <nsap>/<tsel>
nsap ::= hexes
tsel ::= hexes
hexes = hexbyte[: hexbyte]*
ALLOW
SUBAGENT 1.3.6.1.4.1.1080.1.1.1
WITH OTHER PASSWORD
USE 15 SECOND TIMEOUT
COMMUNITY
MANAGER
public
ALLOW GET OPERATIONS
USE NO ENCRYPTION
192.168.4.137
SEND ALL TRAPS
TO PORT 162
WITH COMMUNITY spenmgmt
MANAGER
192.168.3.37
SEND ALL TRAPS
TO PORT 162
WITH COMMUNITY spenmgmt
COMMUNITY
spenmgmt
ALLOW ALL OPERATIONS
USE NO ENCRYPTION
290
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
B.18 /etc/syslog.conf
Use this file to specify the type of logging in the system. If no additional hard
disk is installed, logging can be directed to another network attached system;
otherwise local files are used.
Stanzas in /etc/grcons.log.conf determine how large the respective files may
grow and how many versions are kept of them.
#
#
NetStar $Id: syslog.conf,v 1.6.4.3 1997/09/15 14:57:37 pargal Exp $
# To enable the logging of system messages on GRF systems, edit the
# entries in the "Log messages to Network" section below.
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
#
- uncomment the lines in the "Log messages to Network" section
of this config file (just below these instructions)
- change "server.domain.com" to the name of a syslog server on
your local network to which this router may send log messages
- add the IP address of the log server and its host name to /etc/hosts
on this GRF system.
- run "grwrite -v" command to save your changes to
/etc/syslog.conf and /etc/hosts
- add the following lines to the /etc/syslog.conf on your log server:
(do not include the # from the first column of this file, ie.,
add "local5.* /var/log/mib2d.log" not "#local5.* /var/...")
#
# Syslog configuration for syslog server systems
# GRF-specific log files (from GRF systems over the network)
#
local0.info
local1.info
local2.*
/var/log/gritd.packets
/var/log/gr.console
/var/log/gr.boot
local3.*
local4.*
local5.*
local6.*
/var/log/grinchd.log
/var/log/gr.conferrs
/var/log/mib2d.log
/var/log/fred.log
- kill and restart syslogd on your syslog server machine after making
sure that the log files added to the config file exist
on the log server machine exist (use the "touch" command
to create them if they do not exist, then restart syslogd
on the syslog server machine).
- kill and restart syslogd on the GRF router (or reboot the GRF).
# To uncomment, remove "#net# from each line in this section
#
#
# Log messages to Network
#
#net#*.err;kern.debug;auth.notice;mail.crit
@server.domain.com
#net#*.notice;kern.debug;lpr,auth.info;mail.crit
@server.domain.com
GRF Configuration Files
291
Download from Www.Somanuals.com. All Manuals Search And Download.
#net#cron.info
#net#local0.info
#net#local1.info
#net#local2.*
#net#local3.*
#net#local4.*
#net#local5.*
#net#local6.*
@server.domain.com
@server.domain.com
@server.domain.com
@server.domain.com
@server.domain.com
@server.domain.com
@server.domain.com
@server.domain.com
# ---------------------------------------------------------------------------
# GRF users need not read further - the following configuration examples
# are for IRMS systems
# ---------------------------------------------------------------------------
#
# On IRMS systems (which have sufficient disk space) system messages
# may be logged to disk by uncommenting the lines in the "Log messages to disk"
# section below, touching the log file names to make sure they exist and
# then restarting syslogd.
#
# Note: using the "Log messages to Disk" entries to log messages to
# the RAM-based file system on a GRF system is strongly discouraged
# because the log files can easily fill the RAM file system.
#
# To uncomment, remove "#disk# from each line in this section
#
#
# Log messages to Disk
#
*.err;*.notice;kern.debug;lpr,auth.info;mail.crit
/var/log/messages
cron.info
local0.info
local1.info
local2.*
/var/log/cron
/var/log/gritd.packets
/var/log/gr.console
/var/log/gr.boot
local3.*
local4.*
local5.*
local6.*
/var/log/grinchd.log
/var/log/gr.conferrs
/var/log/mib2d.log
/var/log/fred.log
*.notice;auth.debug
*.emerg
root
*
B.19 /etc/ttys
In this file, the terminals for direct attachment to the GRF as well as the
pseudo terminal for network access (Telnet) are defined. If they are marked
as secure, they are active and can be used.
#
#
#
#
BSDI ttys,v 2.1 1995/02/03 05:54:46 polk Exp
@(#)ttys 5.2 (Berkeley) 6/10/93
# Removing the secure flag from console will also cause init to require
# the root password before going into single-user mode. You can disable
# this by recompiling init without the -DSECURE option.
#
# Use ‘‘kill -HUP 1’’ to make init(8) re-read this file when changes are made.
#
292
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
# Changes to the order of entries or number of ttys should only be made in
# single-user mode.
#
# name getty
type
status
comments
#
console "/usr/libexec/getty pccons"
vt100
on secure
# Virtual consoles (named after the function key used to reach them)
ttyc2
ttyc3
ttyc4
ttyc5
ttyc6
ttyc7
ttyc8
"/usr/libexec/getty pccons"
"/usr/libexec/getty pccons"
"/usr/libexec/getty pccons"
"/usr/libexec/getty pccons"
"/usr/libexec/getty pccons"
"/usr/libexec/getty pccons"
"/usr/libexec/getty pccons"
ibmpc3 off secure
ibmpc3 off secure
ibmpc3 off secure
ibmpc3 off secure
ibmpc3 off secure
ibmpc3 off secure
ibmpc3 off secure
# PC COM ports (tty00 is DOS COM1)
tty00
tty01
tty02
tty03
"/usr/libexec/getty std.9600"
"/usr/libexec/getty std.9600"
"/usr/libexec/getty t9600-hf"
"/usr/libexec/getty std.9600"
unknown off
unknown off
vt100
on secure
unknown off
ttyp0
ttyp1
ttyp2
ttyp3
ttyp4
ttyp5
ttyp6
ttyp7
ttyp8
ttyp9
ttypa
ttypb
ttypc
ttypd
ttype
ttypf
ttyq0
ttyq1
ttyq2
ttyq3
ttyq4
ttyq5
ttyq6
ttyq7
ttyq8
ttyq9
ttyqa
ttyqb
ttyqc
ttyqd
ttyqe
ttyqf
ttyr0
ttyr1
ttyr2
ttyr3
ttyr4
ttyr5
ttyr6
ttyr7
ttyr8
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
network secure
network secure
network secure
network secure
network secure
network secure
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
GRF Configuration Files
293
Download from Www.Somanuals.com. All Manuals Search And Download.
ttyr9
ttyra
ttyrb
ttyrc
ttyrd
ttyre
ttyrf
ttys0
ttys1
ttys2
ttys3
ttys4
ttys5
ttys6
ttys7
ttys8
ttys9
ttysa
ttysb
ttysc
ttysd
ttyse
ttysf
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
none
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
network
294
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix C. Hardware and Software Information
Appendix C gives an overview of the LEDs on the front panel of the SP
Switch Adapter card and shows tables with the meaning of the LEDs’ blinking
patterns.
A diagram of the chip interconnections on a TBS Switch Board is provided for
a quick reference in helping you to find out the correct Switch port numbers or
the correct jack.
You will get some information about updating the IBM 9077 software and how
to get updates.
C.1 The Front Panel of the SP Switch Router Adapter Card - Operational
Figure 72 shows the front panel of the Switch Router adapter:
.
PWR ON
3V
RX HB
RX ST0
RX ST1
RX ERR
MD RCV
SW XMIT
TX HB
TX ST0
TX ST1
TX ERR
MD XMIT
SW RCV
Figure 72. Front Panel of the SP Switch Router Adapter Card with LEDs
© Copyright IBM Corp. 1998
295
Download from Www.Somanuals.com. All Manuals Search And Download.
C.2 SP Switch Router Adapter Media Card LEDs
Table 52. SP Switch Router Adapter Media Card LEDs
LED
Description
PWR ON
3V
This green LED is on when 5 volts are present.
This green LED is on when 3 volts are present.
This green LED blinks to show the heartbeat pattern for the receiver.
RX HB
MD RCV
This amber LED turns on when data is received from its media port
(RS/6000 SP Switch).
SW XMIT This amber LED turns on when data is sent to the crosspoint switch
(through the serial daughter board).
TX HB
This green LED blinks to show the heartbeat pattern for the transmit side
CPU.
MD XMIT This amber LED turns on when data is transmitted from its media port
(RS/6000 SP Switch).
SW RCV
This amber LED turns on when data is received from the crosspoint switch
(through the serial daughter card).
Table 53. SP Switch Router Adapter Media Card LEDs - RX/TX
RX/TX ST0 RX/TX ST1 RX/TX ERR Description
(green)
(amber)
(amber)
on
on
on
STATE_0 for hardware initialization.
off
on
on
STATE_1 for software initialization. Port
waiting for configuration parameters.
on
off
off
off
on
on
STATE_2 for configuration parameters in
place. Port waiting to be connected.
STATE_3 for port is connected and link is
good. The media adapter is ready to be
online.
on
off
on
STATE_4 for port is online and
running/routing.
296
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
C.3 SP Switch Router Adapter Media Card - Bootup
during bootup.
Table 54. SP Switch Router Adapter Media Card LEDs During Bootup
RX/TX HB RX/TX ST0 RX/TX ST1 RX/TX ERR Description
(green)
on
(green)
on
(amber)
on
(amber)
on
All LEDs are lit for 0.5
seconds during reset as
part of onboard diagnostics.
off
off
off
on
Error condition: checksum
error is detected in flash
memory.
on
on
off
off
on
off
off
on
Error condition: SRAM fails
memory test.
During loading, HB and ST1
flash as each section of the
code loads.
Hardware and Software Information
297
Download from Www.Somanuals.com. All Manuals Search And Download.
C.4 Connectors and Receptacles for Different Media
connectors for the various media cards.
Table 55. Media Card Cables and Connectors
Media Card
Cable Support
Connector
SP Switch
SP Switch Cable (5, 10, 15, 20m) 2-row, 50-pin
shielded tab
connector
ATM OC-3c Multimode
ATM OC-3c Singlemode
ATM OC-12c Multimode
ATM OC-12c Singlemode
10/100 Base T
Two 62.5/125 micron
fiber optic cables
Duplex SC
transceiver
Two 9/125 micron
fiber optic cables
Duplex SC
transceiver
One 62.5/125 micron
fiber optic cables
Duplex SC
transceiver
One 9/125 micron
fiber optic cables
Duplex SC
transceiver
Four/Eight STP or UTP CAT5 RJ 45
cable
FDDI
Four mulitmode cables
(62.5/125 micron)
MIC transceiver
HIPPI
HSSI
Two twisted-pair copper cable
Two 25 twisted-pair
shielded coax cables
2-row, 50-pin
receptacle
heads
IP/SONET OC-3c Singlemode One 9/125 micron
fiber optic cables
Duplex SC
transceiver
IP/SONET OC-3c Multimode
One 62.5/125 micron
fiber optic cables
Duplex SC
transceiver
C.5 Chip Interconnection on the TBS Board
Determination Guide, SG24-4778-00, Chapter 4.2 "Reviewing Switch
Boards".
298
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Figure 73. The SP Switch Board
C.6 Updating Router Software
This part provides general information about obtaining and installing new
operating software (hereafter referred to as machine code) for the SP Switch
Router.
C.6.1 The SP Switch Router as an IBM Product
As is noted in this Redbook, the SP Switch Router is based on a product from
Ascend Communications, Inc. IBM customers order and receive the SP
Switch Router from IBM. IBM provides all support for this product for IBM
customers.
Hardware and Software Information
299
Download from Www.Somanuals.com. All Manuals Search And Download.
SP Switch Routers are delivered with the current level of machine code
already installed. Customers who wish to upgrade to new releases of the
machine code should contact their IBM representative.
C.6.2 Obtaining New Machine Code
New releases of the machine code must be obtained from the IBM FTP
server: service2.boulder.ibm.com.
You are prompted for the SP Switch Router customer ID and password when
you ftp to this server.
Although a new release of the machine code will correspond to an Ascend
release of GRF code, only the IBM version of the code will work on the SP
Switch Router. Do not try to use GRF code releases obtained from the
Ascend FTP site on the SP Switch Router.
Instructions on how to download new releases from the FTP site and install
them are included in the Release Notes provided with each release. Be sure
to use the "binary" file transfer type.
C.6.3 Support for Code Installation
The Release Notes are posted on the SP Service and Support web site when
a new release becomes available. As this is written, the starting page for the
SP Service and Support web site is:
http://www.rs6000.ibm.com/support/sp. Look for 9077 “SP Switch Router”
information in the “Service status” pages.
C.6.4 Sample Steps to Upgrade the System Software
Follow the steps below:
1. Log on as root and start the UNIX shell:
super> sh
#
2. Change directory:
# cd /usr/nbin
Then, ftpto the IBM server:
# ftp service2.boulder.ibm.com
ftp>
Enter the SP Switch Router customer ID and password as requested.
3. Now change to the /releases directory:
300
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
ftp> cd releases
ftp> cd A1_4_6
ftp> cd patches
ftp> cd A1_4_6_4
4. Set the file format and download the files:
ftp> bin
ftp> get grf_update
ftp> get RN1_4_6_4.pdf
ftp> get RN1_4_6_4.txt
ftp> quit
#
5. Change the script permissions:
# chmod 755 grf_update
6. Install the script:
# grsite --perm grf_update
7. Read the documentation with an appropriate reader command (acroread
for the pdf file, moreor vifor the txt file)!
8. Run the script:
# ./grf_update
For a sample execution of the grf_update script, refer to Appendix C.6.5,
9. To verify the installation, use this command to check that the directed
broadcast setting is now one of the sysctlexecutables:
# sysctl net.inet.ip.fwdirbcast
net.inet.ip.fwdirbcast = 1
Note: Be prepared, that grf_updatewill rebootthe SP Switch Router!
C.6.5 Sample Execution of grf_update Script
Here is a sample transcript of a session in which the grf_updatescript first
upgrades a GRF 1600 system (testbox.site.com) that is currently running
A_1_4_6,boston to 1.4.6.ibm,default, and then installs the 1.4.6.4.ibm patch
release file on the system.
# grf_update
IBM GRF upgrade: testbox.site.com - Thu Apr 16 09:28:14 CDT 1998
This script will upgrade the router system software to
release 1.4.6.ibm and/or install the 1.4.6.4 directed
partial release patch file on testbox.site.com.
This script REQUIRES:
Hardware and Software Information
301
Download from Www.Somanuals.com. All Manuals Search And Download.
- Ftp access to service2.boulder.ibm.com from testbox.site.com.
- Approximately 30MB of disk space in /flash.
Please be aware:
- testbox.site.com will be REBOOTED if any system software and/or patch
file is installed.
Please take a moment to ensure that the requirement(s) are met and that
testbox.site.com can be rebooted at this time.
Continue with upgrade? (y/n): y
Release: currently running A_1_4_6,boston. Upgrade needed to
1.4.6.ibm,default
Checking for 1.4.6.ibm system release files.
testbox.site.com will be upgraded using files from
service2.boulder.ibm.com.
Loading 1.4.6.ibm release files.
Connected to service2.boulder.ibm.com.
get 1.4.6.ibm.TAR.gz
local: 1.4.6.ibm.TAR.gz remote: 1.4.6.ibm.TAR.gz
226 Transfer complete.
get 1.4.6.ibm.root.gz
local: 1.4.6.ibm.root.gz remote: 1.4.6.ibm.root.gz
226 Transfer complete.
get Addendum.pdf
local: Addendum.pdf remote: Addendum.pdf
226 Transfer complete.
get RN1_4_6.pdf
local: RN1_4_6.pdf remote: RN1_4_6.pdf
226 Transfer complete.
Performing grwrite. Please wait.
Performing grfins. Please wait.
Device /dev/wd0a mounted on /flash.
Device /dev/wd0a unmounted.
The next release version after rebooting will be: 1.4.6.ibm,default
Patch File: currently running 1.4.6. Need 1.4.6.4 Patch File.
Loading 1.4.6.ibm patch release file.
Connected to service2.boulder.ibm.com.
get 1.4.6.ibm,default.site.TAR.gz 1.4.6.ibm,default.site.TAR.gz
local: 1.4.6.ibm,default.site.TAR.gz remote:
1.4.6.ibm,default.site.TAR.gz
226 Transfer complete.
get RN1_4_6_4.pdf
local: RN1_4_6_4.pdf remote: RN1_4_6_4.pdf
226 Transfer complete.
Loading 1.4.6.4 bsd kernel from 1.4.6.ibm,default.site.TAR.gz.bsd
tar: ustar vol 1, 20 files, 3645440 bytes read.
Directed Broadcast: already enabled. No need to modify
/flash/etc.1.4.6.ibm,default/rc.local.
The directed bcast will be enabled after the GRF reboots.
302
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
To temporarily disable the directed bcast setting later on, use:
# sysctl -w net.inet.ip.fwdirbcast=0
To verify that the bcast setting is one of the sysctl executables, use:
# sysctl net.inet.ip.fwdirbcast
IBM GRF upgrade: testbox.site.com is up-to-date
testbox.site.com will be upgraded to:
Next Revision: 1.4.6.ibm Version: default
Patch Revision: 1.4.6.4.ibm
WARNING: testbox.site.com will now be REBOOTED to complete the upgrade.
10 9 8 7 6 5 4 3 2 1
Continuing ...
Shutdown NOW!
shutdown: [pid 5524]
*** FINAL System shutdown message from [email protected] ***
System going down IMMEDIATELY
System shutdown time has arrived
Connection closed by foreign host.
Hardware and Software Information
303
Download from Www.Somanuals.com. All Manuals Search And Download.
304
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix D. Special Notices
This publication is intended to help IBM customers, Business Partners, IBM
System Engineers and other RS/6000 SP specialist who are involved in SP
Switch Router (IBM 9077) projects, including education of RS/6000 SP
professionals responsible for installing, configuring, and administering SP
Switch Router. The information in this publication is not intended as the
specification of any programming interfaces that are provided by
POWERparallel System Support Programs. See the PUBLICATIONS section
of the IBM Programming Announcement for POWERparallel System Support
Programs for more information about what publications are considered to be
product documentation.
References in this publication to IBM products, programs or services do not
imply that IBM intends to make these available in all countries in which IBM
operates. Any reference to an IBM product, program, or service is not
intended to state or imply that only IBM’s product, program, or service may be
used. Any functionally equivalent program that does not infringe any of IBM’s
intellectual property rights may be used instead of the IBM product, program
or service.
Information in this book was developed in conjunction with use of the
equipment specified, and is limited in application to those specific hardware
and software products and levels.
IBM may have patents or pending patent applications covering subject matter
in this document. The furnishing of this document does not give you any
license to these patents. You can send license inquiries, in writing, to the IBM
Director of Licensing, IBM Corporation, 500 Columbus Avenue, Thornwood,
NY 10594 USA.
Licensees of this program who wish to have information about it for the
purpose of enabling: (i) the exchange of information between independently
created programs and other programs (including this one) and (ii) the mutual
use of the information which has been exchanged, should contact IBM
Corporation, Dept. 600A, Mail Drop 1329, Somers, NY 10589 USA.
Such information may be available, subject to appropriate terms and
conditions, including in some cases, payment of a fee.
The information contained in this document has not been submitted to any
formal IBM test and is distributed AS IS. The information about non-IBM
("vendor") products in this manual has been supplied by the vendor and IBM
assumes no responsibility for its accuracy or completeness. The use of this
© Copyright IBM Corp. 1998
305
Download from Www.Somanuals.com. All Manuals Search And Download.
information or the implementation of any of these techniques is a customer
responsibility and depends on the customer’s ability to evaluate and integrate
them into the customer’s operational environment. While each item may have
been reviewed by IBM for accuracy in a specific situation, there is no
guarantee that the same or similar results will be obtained elsewhere.
Customers attempting to adapt these techniques to their own environments
do so at their own risk.
Any pointers in this publication to external Web sites are provided for
convenience only and do not in any manner serve as an endorsement of
these Web sites.
Any performance data contained in this document was determined in a
controlled environment, and therefore, the results that may be obtained in
other operating environments may vary significantly. Users of this document
should verify the applicable data for their specific environment.
Reference to PTF numbers that have not been released through the normal
distribution process does not imply general availability. The purpose of
including these reference numbers is to alert IBM customers to specific
information relative to the implementation of the PTF when it becomes
available to each customer according to the normal IBM PTF distribution
process.
The following terms are trademarks of the International Business Machines
Corporation in the United States and/or other countries:
400
AIX
AT
BookManager
IBM
PROFS
SP
Current
POWERparallel
RS/6000
SP2
The following terms are trademarks of other companies:
C-bus is a trademark of Corollary, Inc.
Java and HotJava are trademarks of Sun Microsystems, Incorporated.
Microsoft, Windows, Windows NT, and the Windows 95 logo are trademarks
or registered trademarks of Microsoft Corporation.
PC Direct is a trademark of Ziff Communications Company and is used
by IBM Corporation under license.
306
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Pentium, MMX, ProShare, LANDesk, and ActionMedia are trademarks or
registered trademarks of Intel Corporation in the U.S. and other
countries.
UNIX is a registered trademark in the United States and other
countries licensed exclusively through X/Open Company Limited.
Other company, product, and service names may be trademarks or
service marks of others.
Special Notices
307
Download from Www.Somanuals.com. All Manuals Search And Download.
308
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Appendix E. Related Publications
The publications listed in this section are considered particularly suitable for a
more detailed discussion of the topics covered in this redbook.
E.1 International Technical Support Organization Publications
For information on ordering these ITSO publications see “How to Get ITSO
• Technical Presentation for PSSP 2.3, SG24-2080
• Technical Presentation for PSSP 2.4, SG24-5173
• RS/6000 SP: Problem Determination Guide, SG24-4778
E.2 Redbooks on CD-ROMs
Redbooks are also available on CD-ROMs. Order a subscription and
receive updates 2-4 times a year at significant savings.
CD-ROM Title
Subscription
Number
Collection Kit
Number
System/390 Redbooks Collection
SBOF-7201
SBOF-7370
SBOF-7240
SBOF-6899
SBOF-6898
SBOF-7270
SBOF-7230
SBOF-7205
SBOF-8700
SBOF-7290
SK2T-2177
SK2T-6022
SK2T-8038
SK2T-8039
SK2T-8044
SK2T-2849
SK2T-8040
SK2T-8041
SK2T-8043
SK2T-8037
Networking and Systems Management Redbooks Collection
Transaction Processing and Data Management Redbook
Lotus Redbooks Collection
Tivoli Redbooks Collection
AS/400 Redbooks Collection
RS/6000 Redbooks Collection (HTML, BkMgr)
RS/6000 Redbooks Collection (PostScript)
RS/6000 Redbooks Collection (PDF Format)
Application Development Redbooks Collection
E.3 Other Publications
These publications are also relevant as further information sources:
• RS/6000 SP: Administration Guide Version 2 Release 4, GC23-3897
• RS/6000 SP: Installation and Migration Guide Version 2 Release 4,
GC23-3898
© Copyright IBM Corp. 1998
309
Download from Www.Somanuals.com. All Manuals Search And Download.
• RS/6000 SP: Diagnosis and Messages Guide Version 2 Release 4,
GC23-3899
• RS/6000 SP: Command and Technical Reference Version 2 Release 4,
GC23-3900
• RS/6000 SP: Maintenance Information Volume 1
Installation and Customer Engineer Operations, GC23-3903
• RS/6000 SP: Maintenance Information Volume 2, Volume 3, GC23-3904
• RS/6000 SP Planning Volume 1
Hardware and Physical Environment, GA22-7280
• GRF Configuration Guide 1.4, GA22-7366
• GRF Reference Guide 1.4, GA22-7367
• GRF 400/1600 Getting started 1.4, GA22-7368
• SP Switch Router Adapter Guide for 1.4, GA22-7310
310
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
How to Get ITSO Redbooks
This section explains how both customers and IBM employees can find out about ITSO redbooks,
CD-ROMs, workshops, and residencies. A form for ordering books and CD-ROMs is also provided.
This information was current at the time of publication, but is continually subject to change. The latest
information may be found at http://www.redbooks.ibm.com/.
How IBM Employees Can Get ITSO Redbooks
Employees may request ITSO deliverables (redbooks, BookManager BOOKs, and CD-ROMs) and
information about redbooks, workshops, and residencies in the following ways:
• Redbooks Web Site on the World Wide Web
http://w3.itso.ibm.com/
• PUBORDER – to order hardcopies in the United States
• Tools Disks
To get LIST3820s of redbooks, type one of the following commands:
TOOLCAT REDPRINT
TOOLS SENDTO EHONE4 TOOLS2 REDPRINT GET SG24xxxx PACKAGE
TOOLS SENDTO CANVM2 TOOLS REDPRINT GET SG24xxxx PACKAGE (Canadian users only)
To get BookManager BOOKs of redbooks, type the following command:
TOOLCAT REDBOOKS
To get lists of redbooks, type the following command:
TOOLS SENDTO USDIST MKTTOOLS MKTTOOLS GET ITSOCAT TXT
To register for information on workshops, residencies, and redbooks, type the following command:
TOOLS SENDTO WTSCPOK TOOLS ZDISK GET ITSOREGI 1998
• REDBOOKS Category on INEWS
• Online – send orders to: USIB6FPL at IBMMAIL or DKIBMBSH at IBMMAIL
Redpieces
For information so current it is still in the process of being written, look at "Redpieces" on the
Redbooks Web Site (http://www.redbooks.ibm.com/redpieces.html). Redpieces are redbooks in
progress; not all redbooks become redpieces, and sometimes just a few chapters will be published
this way. The intent is to get the information out much quicker than the formal publishing process
allows.
© Copyright IBM Corp. 1998
311
Download from Www.Somanuals.com. All Manuals Search And Download.
How Customers Can Get ITSO Redbooks
Customers may request ITSO deliverables (redbooks, BookManager BOOKs, and CD-ROMs) and
information about redbooks, workshops, and residencies in the following ways:
• Online Orders – send orders to:
IBMMAIL
Internet
In United States
In Canada
Outside North America
usib6fpl at ibmmail
caibmbkz at ibmmail
dkibmbsh at ibmmail
• Telephone Orders
United States (toll free)
Canada (toll free)
1-800-879-2755
1-800-IBM-4YOU
Outside North America
(long distance charges apply)
(+45) 4810-1020 - German
(+45) 4810-1620 - Italian
(+45) 4810-1270 - Norwegian
(+45) 4810-1120 - Spanish
(+45) 4810-1170 - Swedish
(+45) 4810-1320 - Danish
(+45) 4810-1420 - Dutch
(+45) 4810-1540 - English
(+45) 4810-1670 - Finnish
(+45) 4810-1220 - French
• Mail Orders – send orders to:
IBM Publications
Publications Customer Support 144-4th Avenue, S.W.
P.O. Box 29570
Raleigh, NC 27626-0570
USA
IBM Publications
IBM Direct Services
Sortemosevej 21
DK-3450 Allerød
Denmark
Calgary, Alberta T2P 3N5
Canada
• Fax – send orders to:
United States (toll free)
Canada
Outside North America
1-800-445-9269
1-800-267-4455
(+45) 48 14 2207 (long distance charge)
• 1-800-IBM-4FAX (United States) or (+1) 408 256 5422 (Outside USA) – ask for:
Index # 4421 Abstracts of new redbooks
Index # 4422 IBM redbooks
Index # 4420 Redbooks for last six months
• On the World Wide Web
Redbooks Web Site
IBM Direct Publications Catalog
http://www.redbooks.ibm.com
http://www.elink.ibmlink.ibm.com/pbl/pbl
Redpieces
For information so current it is still in the process of being written, look at "Redpieces" on the
Redbooks Web Site (http://www.redbooks.ibm.com/redpieces.html). Redpieces are redbooks in
progress; not all redbooks become redpieces, and sometimes just a few chapters will be published
this way. The intent is to get the information out much quicker than the formal publishing process
allows.
312
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
IBM Redbook Order Form
Please send me the following:
Title
Order Number
Quantity
First name
Company
Address
City
Last name
Postal code
Country
Telephone number
Telefax number
VAT number
Invoice to customer number
Credit card number
Credit card expiration date
Card issued to
Signature
We accept American Express, Diners, Eurocard, Master Card, and Visa. Payment by credit card not
available in all countries. Signature mandatory for credit card payment.
313
Download from Www.Somanuals.com. All Manuals Search And Download.
314
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
EPROM
Erasable
Programmable
Read-Only Memory
List of Abbreviations
ACL
AIX
Access Control List
FIFO
GB
First-In First-Out
Gigabytes
Advanced Interactive
Executive
GL
Group Leader
AMG
ANS
Adapter Membership
Group
GPFS
General Purposes File
System
Abstract Notation
Syntax
GRF
GS
Goes Real Fast
Group Services
APA
API
All Points Addressable
Application
Programming Interface
GSAPI
Group Services
Application
Programming Interface
ARP
ATM
Address Resolution
Protocol
GVG
hb
Global Volume Group
heart beat
Asynchronous Transfer
Mode
HiPS
High Performance
Switch
BIS
Boot/Install Server
BSD
Berkeley Software
Distribution
HPGN
High Performance
Gateway Node
BUMP
Bring-Up
Microprocessor
hrd
host respond daemon
Hashed Shared Disk
HSD
IBM
CP
Crown Prince
International Business
Machines Corporation
CPU
CSS
Central Processing Unit
Communication
Subsystem
ICMP
Internet Control
Message Protocol
CW
Control Workstation
Dual Attach Station
Database
IP
Internet Protocol
DAS
DB
ISB
Intermediate Switch
Board
ISC
Intermediate Switch
Chip
FDDI
Fiber Distributed Data
Interface
ISO
International Standards
Organization
EM
Event Management
EMAPI
Event Management
Application
Programming Interface
ITSO
International Technical
Support Organization
JFS
LAN
LCD
LED
Journaled File System
Local Area Network
Liquid Crystal Display
Light Emitting Diode
EMCDB
EMD
Event Management
Configuration Database
Event Manager
Daemon
315
Download from Www.Somanuals.com. All Manuals Search And Download.
LRU
LSC
LVM
Last Recently Used
Link Switch Chip
RCP
RM
Remote Copy Protocol
Resource Monitor
Logical Volume
Manager
RMAPI
Resource Monitor
Application
Programming Interface
MB
Megabytes
RPQ
RSI
Request for Product
Quotation
MIB
Management
Information Base
Remote Statistics
Interface
MPI
Message Passing
Interface
RVSD
Recoverable Virtual
Shared Disk
MPL
MPP
MTU
NBMA
NIM
Message Passing
Library
SAS
SBS
SDR
Single Attach Station
Structure Byte String
Massive Parallel
Processors
Maximum Transmission
Unit
System Data
Repository
Non Broadcast, Multi
Access
SNMP
Simple Network
Management Protocol
Network Installation
Manager
SP
RS/6000 SP
SVC
VC
Switched Virtual Circuit
Virtual Circuit
NSB
NSC
OID
Node Switch Board
Node Switch Chip
Object ID
VCI
VP
Virtual Circuit Identifier
Virtual Path
ODM
PE
Object Data Manager
Parallel Environment
Process ID
VPI
Virtual Path Identifier
PID
PROFS
Professional Office
System
PSSP
Parallel System
Support Programs
PTC
Prepare To Commit
PTPE
Performance Toolbox
Parallel Extensions
PTX/6000
PVC
Performance
Toolbox/6000
Permanent Virtual
Circuit
RAM
Random Access
Memory
316
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
Index
Symbols
ATE 31
/etc/grifconf.conf 130
OC-12 225
port 110
autosense 105
B
/etc/inetd.conf 286
/etc/motd 287
/etc/rc.boot 77
bandwidth
/etc/rc.local 287
/etc/rc.routes 213
/etc/SP/expected.top.annotated 91
/root/.profile 261
/var/log 78
/var/log/gr.console 79
C
Numerics
10Base2 34
10Base5 34
CLNP 23
command
A
bredit 146
brinfo 147
© Copyright IBM Corp. 1998317
Download from Www.Somanuals.com. All Manuals Search And Download.
config_netstat 70
csconfig 79
Eannotator 82
Eprimary 51
grconslog 79
grf_update 301
grifconfig 94
grrmb 109
grsite 96
grsnapshot 97
iflash 77
lsdev 168
no 95
pax 77
reboot 79
D
datagrams 19
architecture
downtime
dual-speed 105
E
route 10
smit
hostname 81
annotator 91
delete_extadapter 47
delete_extnode 46
list_extadapter 50
list_extnode 49
sphardware 52
splstdata 84
adapter
spmon
sysctl 301
318
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
IPI-3 138
MTU 138
F
fault service daemon
concentrator 126
hop 14
hostname 65
hot plug capabilities
hot-pluggable 16
hot-swappable 15
interface 128
HPGN
hw-table 107
FIFO 38
fragmented 95
I
IDRP 23
IGMP 23
InATMARP 219
G
gr.conferrs 76
gr.console 76
grinchd.log 76
gritd.packets 76
H
IP gateway
ARP 138
Backbone 227
backbone 227
device 135
H0 138
interface 139
319
Download from Www.Somanuals.com. All Manuals Search And Download.
MIB 59
mib2d 100
microchannel
mrouted 22
default 94
J
jack 82
discovery 95
K
keep-count 107
N
L
log files
network traffic
gr.boot 76
gr.conferr 76
gr.console 76
grinchd.log 76
nonbroadcast 94
gritd.packets 76
mib2d.log 76
O
ODM 225
M
optical-bypass 124
out.top 85
maint 109
P
PCMCIA
card 34
device 72
320
IBM 9077 SP Switch Router: Get Connected to the SP Switch
Download from Www.Somanuals.com. All Manuals Search And Download.
interface 79
metrics 21
slot 34
performance limitation
point-to-multipoint 119
Routing Information Protocol
routing protocol
S
segments 20
Q
service daemon
service packets
COMMUNITY 88
community 64
R
rejoin 51
manage 100
MANAGER 88
protocol 58
snmpd 89
traps 24
1583 22
routed 21
Routers
routing
SP
configuration
321
Download from Www.Somanuals.com. All Manuals Search And Download.
ITSO Redbook Evaluation
IBM 9077 SP Switch Router: Get Connected to the SP Switch
SG24-5157-00
Your feedback is very important to help us maintain the quality of ITSO redbooks. Please complete
this questionnaire and return it using one of the following methods:
• Use the online evaluation form found at http://www.redbooks.ibm.com
• Fax this form to: USA International Access Code + 1 914 432 8264
Which of the following best describes you?
_ Customer _ Business Partner
_ None of the above
_ Solution Developer
_ IBM employee
Please rate your overall satisfaction with this book using the scale:
(1 = very good, 2 = good, 3 = average, 4 = poor, 5 = very poor)
Overall Satisfaction
__________
Please answer the following questions:
Was this redbook published in time for your needs?
If no, please explain:
Yes___ No___
What other redbooks would you like to see published?
Comments/Suggestions:
(THANK YOU FOR YOUR FEEDBACK!)
© Copyright IBM Corp. 1998
323
Download from Www.Somanuals.com. All Manuals Search And Download.
IBM 9077 SP Switch Router: Get Connected to the SP Switch
SG24-5157-00
Download from Www.Somanuals.com. All Manuals Search And Download.
|