Installing SUSE LPARs in a Power architecture over a network is much easier than any other installation options. It is wise to set up a PXE environment when installing more than two SUSE LPARs. Refer to a number of articles on my blog detailing network deployment for various flavors of linuxes.
Once the installation is complete, maintenance of software updates becomes important, and an RMT server is definitely the easiest solution for this. It creates a network isolation for patch delivery still allowing access to the latest available subscription software. RMT can also be used directly during the installation phase, installing the latest packages right away.
The RMT server lacks a full PXE stack, in this article we will add it and configure all the missing components to get it. We will install it (RMT + PXE) directly on the Power equipment in order to eliminate doubts about the violation of the purchased licenses.
Like VmWare, for which it is not enough just to turn on the ESX server to get a complete infrastructure, the IBM Power infrastructure has some prerequisites and settings.
A virtual machine in the IBM world is called LPAR (Logical PARtition) because this machine is not virtual, but real hardware is allocated to create a virtual server. This process is managed by the HMC (Hardware Management Console), which plays a role similar to the VC in VmWare world. An HMC can be a physical or virtual server running on any other existing virtualization platform at the customer's site. The HMC is connected to a public network and managed through a web interface. The HMC is also connected to a dedicated private network. This dedicated network is not routable and is directly connected to all IBM Power servers, to the HMC interface. To achieve redundancy, it is common to install two vHMCs and two separate private networks.
The very first LPAR installed is usually the VIOS (Virtual Input Output Server). The VIOS helps share I/O cards (such as FC storage access cards and network cards) between payload LPARs. Again, two VIOS are usually installed to make the solution redundant.
Create on one VIOS "Virtual Library" and create there a virtual disk for new LPAR (usually 120g is enough).
Upload or transfer a desired ISO file (for example SLE-15-SP3-Full-ppc64le-QU2-Media1.iso) to "Virtual library". Configure vitual optical device and load an ISO to it. Pay attention that downlaoded iso is for ppc64le architecture, other versions will not suits.
Create an LPAR with two small (0.1) virtual CPU and 4G RAM. Create virtual SCSI adapter and mark it "required". Connect to it both virtual disk and virtual DVD. An HMC CLI commands could be helpfull:
lsrep mkrep --sp rootvg --size 20G mkvopt -name SLE15SP3.iso --file /home/padmin/SLE-15-SP3-Full-ppc64le-QU2-Media1.iso lsmap --all mkvdev -fbo -vadapter vhost2 loadopt -disk SLE15SP3.iso -vtd vtopt0 unloadopt -vtd vtopt0
The boot will occur from ISO, and a default GRUB menu appear.
You have to enable module Server Applications (usually enabled) to be able install:
# zypper in rmt-server
Next is about configure an RMT according to the SUSE documentation.
After setting up your company credentials, you can add some channels for mirroring. Use the same official guide for this and "enable" the products you want. Synchronization will take time, mainly depending on the network bandwidth.
The DVD media used during installation can be used as the installation source for the rest of the LPARS. First of all, let's mount it permanently into the appropriate location.
The installation source should be available via HTTP to be used during installation. The RMT software uses an NGINX HTTP server as its engine. As a result of installing RMT, there are two virtual hosts configured at NGINX. The /etc/nginx/nginx.conf file configures localhost then it cannot be used to serve external requests. The file /etc/nginx/vhosts.d/rmt-server-http.conf describes the virtual host for $HOSTNAME (we sat it as rmt earlier). This site serves files related to the path /usr/share/rmt/public. We will utilize this configuration and mount our installation disk below this location:
# ll /dev/disk/by-label/ total 0 lrwxrwxrwx 1 root root 9 May 8 12:14 SLE-15-SP3-Full-ppc64le201.3.001 -> ../../sr0 # mkdir /usr/share/rmt/public/install # echo "/dev/disk/by-label/SLE-15-SP3-Full-ppc64le201.3.001 /usr/share/rmt/public/install auto defaults 0 0" >> /etc/fstab # mount /usr/share/rmt/public/install mount: /usr/share/rmt/public/install: WARNING: device write-protected, mounted read-only. # mkdir /usr/share/rmt/public/autoyast
The /usr/share/rmt/public/autoyast directory will contain your custom autoyast.xml files later.
Install TFTP server and client.
# zypper in tftp
In addition to the TFTP server itself, you need to populate the TFTP root directory with the necessary files. This could be done in two ways. You can copy /usr/share/rmt/public/install/boot from an already mounted DVD or install the package like:
# zypper in tftpboot-installation-SLE-15-SP3-ppc64le
This package unpacks some files into the /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le/ directory. Let's set this directory as the TFTPROOT directory by editing the file:
# egrep -v "^$|^#" /etc/sysconfig/tftp TFTP_USER="tftp" TFTP_OPTIONS="-v" TFTP_DIRECTORY="/usr/share/tftpboot-installation/SLE-15-SP3-ppc64le"
Enable TFTP server and add it to firewall rules if firewall still in use:
# systemctl enable --now tftp.socket Created symlink /etc/systemd/system/sockets.target.wants/tftp.socket → /usr/lib/systemd/system/tftp.socket. # firewall-cmd --add-service=tftp --add-service=dhcp --permanent # firewall-cmd --reload # firewall-cmd --list-all public target: default icmp-block-inversion: no interfaces: sources: services: ssh dhcpv6-client http https tftp dhcp ports: 10050/tcp protocols: masquerade: no forward-ports: source-ports: icmp-blocks: rich rules:
Test TFTP server:
# cd /tmp /tmp # tftp localhost tftp> get README tftp> quit /tmp # ll README -rw-r--r-- 1 root root 442 May 8 12:35 README
PXE uses a specially configured DHCP server, but usually another DHCP server already exists in the organization. Therefore, we will configure our own DHCP server to serves only certain clients and does not interfere with other DHCP requests.
Install the package:
# zypper in dhcp-server
Put the following configuration file:
# /etc/dhcpd.conf
allow booting;
allow bootp;
ddns-update-style none;
default-lease-time 14400;
deny unknown-clients;
# Send hostname over DHCP
get-lease-hostnames true;
use-host-decl-names on;
subnet 192.168.0.0 netmask 255.255.255.0 {
option domain-name "localdomain";
#option domain-name-servers 192.168.0.1;
option routers 192.168.0.1;
#option ntp-servers 192.168.0.1;
option subnet-mask 255.255.255.0;
# This is an IP of RMT server, where TFTP server run and serve "filename" below.
next-server 192.168.0.20;
# This is GRUB2 for ppc64le architecture. This option could be used on host level too.
filename "/boot/ppc64le/grub2-ieee1275/core.elf";
host ppc {
hardware ethernet 52:54:00:53:36:51;
fixed-address 192.168.0.11;
}
}
Define DHCPD_INTERFACE at /etc/sysconfig/dhcpd file, then start DHCP server:
# systemctl enable --now dhcpd
There are two grub.cfg files that you can find in the /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le directory we deployed. The effective file is /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le/boot/ppc64le/grub2-ieee1275/grub.cfg, while the second one exists for an unknown reason. The best way is to create a hard link between them:
# rm /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le/boot/grub2/grub.cfg # ln /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le/boot/ppc64le/grub2-ieee1275/grub.cfg /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le/boot/grub2/grub.cfg
Then you can edit anyone of them to make similar:
# cat /usr/share/tftpboot-installation/SLE-15-SP3-ppc64le/boot/grub2/grub.cfg
with_gfx=0
gfxmode=auto
locale_dir=$prefix/locale
lang=en_US
set default='local'
insmod gettext
if sleep --interruptible 0 ; then
timeout=60
fi
insmod http
# in our case http server is the same as tftp server, otherwice it should be root=(http,IP-of-HTTP-server)
set root=(http)
# Once root set, the kernel and initrd files are related to it. Rest are kernel option, then should include full url
menuentry 'Installation' --class opensuse --class gnu-linux --class gnu --class os {
echo 'Loading kernel ...'
linux /install/boot/ppc64le/linux install=http://192.168.0.20/install autoyast=http://192.168.0.20/autoyast/sap.xml
echo 'Loading initial ramdisk ...'
initrd /install/boot/ppc64le/initrd
}
menuentry 'VNC Installation' --class opensuse --class gnu-linux --class gnu --class os {
echo 'Loading kernel ...'
linux /install/boot/ppc64le/linux install=http://192.168.0.20/install vnc=1 vncpassword=P@ssw0rd
echo 'Loading initial ramdisk ...'
initrd /install/boot/ppc64le/initrd
}
menuentry 'Rescue System' $arch --class opensuse --class gnu-linux --class gnu {
echo 'Loading kernel ...'
linux /install/boot/ppc64le/linux rescue=1
echo 'Loading initial ramdisk ...'
initrd /install/boot/ppc64le/initrd
}
menuentry 'Upgrade' $arch --class opensuse --class gnu-linux --class gnu {
echo 'Loading kernel ...'
linux /install/boot/ppc64le/linux upgrade=1
echo 'Loading initial ramdisk ...'
initrd /install/boot/ppc64le/initrd
}
menuentry 'local' {
exit
}
submenu 'Other options...' {
menuentry 'Reboot' {
reboot
}
menuentry 'Exit to Open Firmware' {
exit
}
}
The autoyast/sap.xml above should exist. Take one of mine.
You have to fix the RMT server name/IP and its certificate SHA1 SSL fingerprint. To get correct fingerprint, do this on any linux:
# wget http://rmt.2cloud4.com/rmt.crt # openssl x509 -fingerprint -noout -sha1 -in rmt.crt
Update the /etc/dhcpd.conf with real MAC address for your new LPAR and set relevant IP address. The restart of DHCP server required to make changes active. When powering on the LPAR using HMC, select network boot and explicitely define your RMT as boot server. Take a look on console to see installation progress.