X-Git-Url: https://gerrit.fd.io/r/gitweb?p=csit.git;a=blobdiff_plain;f=resources%2Ftools%2Ftestbed-setup%2FREADME.md;h=4550f5c59a0a7bbdf522412bcc8a48dd9c003f96;hp=0a423cc953a4a15a3b2e468a729991a0c0242ba0;hb=29f64252f7ad9d51e83cfbcd848d08848c5dc688;hpb=4da982b7a0f5872de2cc721607894216a20892f4 diff --git a/resources/tools/testbed-setup/README.md b/resources/tools/testbed-setup/README.md index 0a423cc953..4550f5c59a 100644 --- a/resources/tools/testbed-setup/README.md +++ b/resources/tools/testbed-setup/README.md @@ -24,50 +24,34 @@ The process below assumes that there is a host used for boostrapping (referred to as "PXE boostrap server" below), and that the directory containig this README is available on the PXE bootstrap server in ~testuser/host-setup. -### Prepare the PXE bootstrap server (one-time) +### Prepare the PXE bootstrap server when there is no http server AMD64 - `sudo apt-get install isc-dhcp-server tftpd-hpa nginx-light ansible` - edit dhcpd.conf and place it to /etc/dhcp/ - `sudo cp dhcpd.cfg /etc/dhcp/` - `sudo service isc-dhcp-server restart` - `cd ~testuser/host-setup` - - `wget 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-server-amd64.iso'` - `sudo mkdir /mnt/cdrom` - - `sudo mount -o loop ubuntu-16.04.2-server-amd64.iso /mnt/cdrom/` + - Ubuntu Xenial + - `wget 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-server-amd64.iso'` + - `sudo mount -o loop ubuntu-16.04.2-server-amd64.iso /mnt/cdrom/` + - Ubuntu Bionic + - `wget 'http://cdimage.ubuntu.com/ubuntu/releases/18.04/release/ubuntu-18.04-server-amd64.iso'` + - `sudo mount -o loop ubuntu-18.04-server-amd64.iso /mnt/cdrom/` - `sudo cp -r /mnt/cdrom/install/netboot/* /var/lib/tftpboot/` - - `sudo mkdir /usr/share/nginx/html/ubuntu` - - `sudo cp -r /mnt/cdrom/* /usr/share/nginx/html/ubuntu/` - - `sudo cp /mnt/cdrom/ubuntu/isolinux/ldlinux.c32 /var/lib/tftpboot` - - `sudo cp /mnt/cdrom/ubuntu/isolinux/libcom32.c32 /var/lib/tftpboot` - - `sudo cp /mnt/cdrom/ubuntu/isolinux/libutil.c32 /var/lib/tftpboot` - - `sudo cp /mnt/cdrom/ubuntu/isolinux/chain.c32 /var/lib/tftpboot` - - `sudo umount /mnt/cdrom` - - edit ks.cfg and replace IP address with that of your PXE bootstrap server - - `sudo cp ks.cfg /usr/share/nginx/html/ks.cfg` - - edit boot-screens_txt.cfg and replace IP address with that of your PXE bootstrap server - - `sudo cp boot-screens_txt.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/txt.cfg` - - `sudo cp syslinux.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/syslinux.cfg` - -### Prepare the PXE bootstrap server (alternative way without NGINX) - - - `sudo apt-get install isc-dhcp-server tftpd-hpa ansible` - - edit dhcpd.conf and place it to /etc/dhcp/ - - `sudo cp dhcpd.cfg /etc/dhcp/` - - `sudo service isc-dhcp-server restart` - - `cd ~testuser/host-setup` - - `wget 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-server-amd64.iso'` - - `sudo mkdir /mnt/cdrom` - - `sudo mount -o loop ubuntu-16.04.1-server-amd64.iso /mnt/cdrom/` - - `sudo cp -r /mnt/cdrom/install/netboot/* /var/lib/tftpboot/` - - `sudo mkdir /var/www/download/ubuntu` - - `sudo cp -r /mnt/cdrom/* /var/www/download/ubuntu/` + - figure out where nginx will look for files on the filesystem when + responding to HTTP requests. The configuration is in one of the + files in /etc/nginx/conf.d/, /etc/nginx/sites-enabled/ or in + /etc/nginx/nginx.conf under section server/root. Save the path to WWW_ROOT + - `sudo mkdir -p ${WWW_ROOT}/download/ubuntu` + - `sudo cp -r /mnt/cdrom/* ${WWW_ROOT}/download/ubuntu/` - `sudo cp /mnt/cdrom/ubuntu/isolinux/ldlinux.c32 /var/lib/tftpboot` - `sudo cp /mnt/cdrom/ubuntu/isolinux/libcom32.c32 /var/lib/tftpboot` - `sudo cp /mnt/cdrom/ubuntu/isolinux/libutil.c32 /var/lib/tftpboot` - `sudo cp /mnt/cdrom/ubuntu/isolinux/chain.c32 /var/lib/tftpboot` - `sudo umount /mnt/cdrom` - edit ks.cfg and replace IP address with that of your PXE bootstrap server and subdir in /var/www (in this case /download) - - `sudo cp ks.cfg /var/www/download/ks.cfg` + - `sudo cp ks.cfg ${WWW_ROOT}/download/ks.cfg` - edit boot-screens_txt.cfg and replace IP address with that of your PXE bootstrap server and subdir in /var/www (in this case /download) - `sudo cp boot-screens_txt.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/txt.cfg` - `sudo cp syslinux.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/syslinux.cfg` @@ -76,12 +60,14 @@ is available on the PXE bootstrap server in ~testuser/host-setup. - set CIMC address - set CIMC username, password and hostname +- set IPMI address +- set IPMI username, password and hostname ### Bootstrap the host -From PXE boostrap server: +Optional: From PXE boostrap server in case of installing Haswell - - `cd ~testuser/host-setup/cimc` + - `cd resources/tools/testbed-setup/cimc` - Initialize args.ip: Power-Off, reset BIOS defaults, Enable console redir, get LOM MAC addr - `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -i` - Adjust BIOS settings @@ -90,7 +76,7 @@ From PXE boostrap server: - Reboot server with boot from PXE (restart immediately) - `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -pxe` -While Ubuntu install is running: +Optional: If RAID is not created on Haswells. Execute while Ubuntu install is running - create RAID array. Reboot if needed. - `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d --wipe` @@ -100,35 +86,24 @@ While Ubuntu install is running: - Set the next boot from HDD (without restart) - `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -hdd` -When installation is finished: - - - `ssh-copy-id <>` - - `cd ~testuser/host-setup/playbooks` - - edit /etc/ansible/hosts; add the hosts you are installing. *REMOVE ANY HOSTS YOU ARE NOT CURRENTLY INSTALLING*. - - Example for physical testbed hosts: - ~~~ - [tg] - 10.30.51.24 hostname=t3-tg1 isolcpus="1-17,19-35" ansible_python_interpreter=/usr/bin/python2.7 +Optional: If installing Skylake machine - [sut] - 10.30.51.25 hostname=t3-sut1 isolcpus="1-17,19-35" ansible_python_interpreter=/usr/bin/python2.7 - 10.30.51.26 hostname=t3-sut2 isolcpus="1-17,19-35" ansible_python_interpreter=/usr/bin/python2.7 - ~~~ + - Get MAC address of LAN0 + - `ipmitool -U ADMIN -H $HOST_ADDRESS raw 0x30 0x21 | tail -c 18` + - Reboot into PXE for next boot only + - `ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN chassis bootdev pxe` - Example for VIRL hosts -- use the "virl" tag and specify the flat network start and end addresses: - ~~~ - [virl] - 10.30.51.28 hostname=t4-virl1 virl_l2_start=10.30.51.31 virl_l2_end=10.30.51.105 - ~~~ +When installation is finished: - - `ansible-playbook --ask-sudo-pass 01-host-setup.yaml` - - `ansible-playbook reboot.yaml` + - Copy ssh keys for no pass access: `ssh-copy-id 10.30.51.x` + - Clone CSIT actual repo: `git clone https://gerrit.fd.io/r/csit` + - Go to ansible directory: `cd csit/resources/tools/testbed-setup/ansible` + - Edit production file and uncomment servers that are supposed to be installed: `ansible-playbook --ask-become-pass --inventory production site.yaml --list-hosts` + - Run ansible on selected hosts: `ansible-playbook --ask-become-pass --inventory production site.yaml` For non-VIRL hosts, stop here. - ### VIRL installation After the host has rebooted: