Recently we discussed some of the benefits of running Linux on an IBM System p. Now we'll take a look at how the latest Red Hat Enterprise Linux (RHEL) version does on Power. In this tip, Ken Milberg discusses the reliability, scalability and performance values of running RHEL 5 on IBM System p, plus how to install the right tools prior to putting any system into production (such as the service and productivity tools available for your Red Hat distribution).
RHEL on IBM System p
Red Hat Enterprise Linux has been available on Power since November of 2003 with the release of RHEL AS 3 for Power. It was updated to provide support for the p5, OpenPower and JS20 Blades in August of 2004 with Update 3. As of Feb 2005, RHEL4 AS 4 became available. This was based on the 2.6 kernel and included support for large page support and the preemptive kernel. Red Hat Enterprise Linux 5 support on Powerbecame available shortly after its official release in 2007. Where running Red Hat on the System p really shows value is where we can micro-partition Linux hosts such as web servers, firewalls or DNS servers to provide partitions as small as 1/10 of a processor (in increments as small as 1/100th of a processor). Linux can actually be run on logical partitions on IBM's System p servers without the use of AIX at all. Consolidating many larger servers into one physical box saves on space, lowers total cost of ownership (TCO) by reducing reduced footprint in the datacenter and is one of the primary business drivers for migrating to the IBM System p. It's worth noting that on the IBM system p, all partitioning (and by this we don't mean disk partitioning, we're referring to creating our logical host partitions) is done via the hypervisor using either the Hardware Management Console (HMC) or the Integrated Virtualization Manager (IVM). The HMC is typically used in larger shops, because a separate management workstation is necessary for the console. There are also extra capabilities with the HMC that IVM does not yet provide. IVM's primary benefit is that it is software-based and does not require a separate management console.
Figure 1- HMC
We won't discuss logical partitioning in detail here, though you should understand that your Red Hat host system has been installed and configured using either an HMC or IVM. Red Hat provides an excellent guide on installing RHEL5 on IBM's Power that is a must-read for administrators that are ready to install RHEL5 on Power. As illustrated in Figure 1, the System p server you are using supports multiple partitioned environments (Linux-only, if you are using an older OpenPower system). Note that you will also need either the HMC or IVM to configure virtual resources, such as Virtual SCSI or shared Ethernet, which are a part of IBM's Advanced Power Virtualization (APV).
Let's start working on our system. First we'll verify our system:
[root --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> --> _29_138_39 ~]# cd /proc [root@172_29_138_39 proc]# more cpuinfo processor : 0 cpu : POWER5 (gr) clock : 1654.344000MHz revision : 2.1 (pvr 003a 0201) processor : 1 cpu : POWER5 (gr) clock : 1654.344000MHz revision : 2.1 (pvr 003a 0201) timebase : 207050000 platform : pSeries machine : CHRP IBM,9117-570 [root@172_29_138_39 proc]# [root@172_29_138_39 etc]# more redhat-release Red Hat Enterprise Linux Server release 5 (Tikanga) [root@172_29_138_39 etc]#
The commands above tell us that we are in fact running RHEL5 on a POWER5.
There are some important LoP tools that you'll need to download in order to make use of the architecture's full capabilities. It's important to note that there is also an installation sequence that you should follow when installing these tools. For example, before installing the hardware inventory packages, the platform enablement library package must be installed. This package provides the libraries that allow functionality to be provided by the Power architectures' firmware. The second package that should be installed prior to the hardware inventory package is the services aids package. These aids enable the reliability, availability and serviceability (RAS) features of the Power platform. The utilities include bootlist, which provides information on the order of boot devices, and the snap command, which is a critical aid used for capturing data to aid analyses of errors.
I like bootlist, because it provides a quick snapshot into the boot sequencing order.
[root@172_29_138_39 ~]# bootlist -m normal -o sda [root@172_29_138_39 ~]#
And you can also make changes to your boot sequencing order using bootlist.
Running snap provides you with more information. It is the primary tool that you should use when sending information to IBM support. As you can see below, snap creates a directory called ibmsupt, where it puts all its output. When you are having problems, you can send the entire zip file right to IBM. We unzipped it and tar'd it, so that you can view the output of snap's output directory structure. The output files generated are invaluable and I also like to also use these as documentation aids. For example, if your network goes south on you, you'll have a copy of what the ifconfig used to look like.
[root@172_29_138_39 ibmsupt]# snap output written to snap.tar.gz [root@172_29_138_39 ibmsupt]# [root@172_29_138_39 tmp]# ls ibmsupt [root@172_29_138_39 tmp]# [root@172_29_138_39 ibmsupt]# ls -ltr total 24 drw-r--r-- 3 root root 4096 Nov 26 05:41 var drw-r--r-- 9 root root 4096 Nov 26 05:41 proc drw-r--r-- 2 root root 4096 Nov 26 05:41 dev drw-r--r-- 2 root root 4096 Nov 26 05:42 etc -rw-r--r-- 1 root root 2287 Nov 26 06:14 lscfg.out -rw-r--r-- 1 root root 1746 Nov 26 06:14 ifconfig.out [root@172_29_138_39 ibmsupt]# cd etc [root@172_29_138_39 etc]# ls fstab redhat-release yaboot.conf
All these utilities (installed as standard rpms) are available through the service and productivity tools that we talked about earlier. After you've downloaded the first two packages, it's now time to look at the Hardware Inventory Package which contains the lsvpd, lscfg and lsmcode commands. These commands provide the hardware inventory systems. You'll find that running an lscfg will be much more helpful than anything you have been accustomed to on Linux (e.g., dmesg). This command provides server-specific information, as well as a much more readable version of lsvpd. Lsvpd provides vital product data (VPD) about all the hardware components.
First we need to install the package:
[root@172_29_138_39 u0004773]# rpm -i lsvpd-0.15.1-1.ppc.rpm [root@172_29_138_39 u0004773]#
Let's run lscfg:
[root@172_29_138_39 u0004773]# lscfg /sbin/lscfg: Error - database not initialised "/var/lib/lsvpd/db". Please run "/sbin/update-lsvpd-db".
The error indicates that we must run the boot-time scanning script to initialize the database. Let's do that.
[root@172_29_138_39 u0004773]# update-lsvpd-db [root@172_29_138_39 u0004773]#
One more time…
[root@172_29_138_39 u0004773]# lscfg INSTALLED RESOURCE LIST The following resources are installed on the machine. +/- = Added or deleted from Resource List. * = Diagnostic support not available. Model Architecture: chrp Model Implementation: Multiple Processor, PCI Bus + sys0 System Object + sysplanar0 System Planar + eth0 U9117.570.109BBFD-V12-C5-T1 Interpartition Logical LAN + eth1 U9117.570.109BBFD-V12-C6-T1 Interpartition Logical LAN + scsi0 U9117.570.109BBFD-V12-C3-T1 Virtual SCSI I/O Controller + sda U9117.570.109BBFD-V12-C3-T1-L1-L0 Virtual SCSI Disk Drive (21400 MB) + scsi1 U9117.570.109BBFD-V12-C4-T1 Virtual SCSI I/O Controller + sdb U9117.570.109BBFD-V12-C4-T1-L1-L0 Virtual SCSI Disk Drive (21400 MB) + mem0 Memory + proc0 Processor
AIX administrators will recognize the output above, which is extremely useful in clearly detailing the systems hardware resources including Virtual I/O resources. Another useful command that is part of this package is lsmcode, which provides micro-code and firmware level information.
One final package worth a mention is the System Resource Controller (SRC) utilities. This productivity tool helps manage the daemons on the system and is a standard command interface for defining, starting and stopping daemons.
The SRC package was already preinstalled on our system. To view the status of all subsystems using SRC, we can run the command using the –a flag.
[root@172_29_138_39 ~]# lssrc -a Subsystem Group PID Status ctrmc rsct 3489 active IBM.ERRM rsct_rm 3522 active IBM.CSMAgentRM rsct_rm 3536 active IBM.DRM rsct_rm 3546 active IBM.AuditRM rsct_rm 3560 active IBM.LPRM rsct_rm 3768 active IBM.HostRM rsct_rm 3788 active ctcas rsct inoperative IBM.SensorRM rsct_rm inoperative
We can start and stop subsystems by using the startrc and stopsrc commands, respectively. Most AIX administrators will recall these commands fondly, as they used on AIX partitions. I would do a man on these commands to help you understand all its capabilities. While you can manage partitions just fine without this package, I personally like them much better then the Linux utilities for managing services (i.e., chkconfig).
It's important to note that running RHEL5 on Power is just like running it on an x86 box but with added value and capabilities. All your standard Red Hat Linux commands are there but there are packages available that you should install which can enhance your RHEL5/Power experience. While some of these commands are recycled AIX utilities and not necessarily requisites for anything, others are critical in helping you take full advantage of the true capabilities of the Power. For more information, check out IBM's LoP site, a must-see for any LoP aficionado.
About the author: Ken Milberg is a systems consultant with two decades of experience working with Unix and Linux systems. He is a SearchEnterpriseLinux.com Ask the Experts advisor and columnist.