to as "PXE boostrap server" below), and that the directory containig this README
is available on the PXE bootstrap server in ~testuser/host-setup.
-### Prepare the PXE bootstrap server (one-time)
+### Prepare the PXE bootstrap server when there is no http server AMD64
- `sudo apt-get install isc-dhcp-server tftpd-hpa nginx-light ansible`
- edit dhcpd.conf and place it to /etc/dhcp/
- `sudo cp dhcpd.cfg /etc/dhcp/`
- `sudo service isc-dhcp-server restart`
- `cd ~testuser/host-setup`
- - `wget 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-server-amd64.iso'`
- `sudo mkdir /mnt/cdrom`
- - `sudo mount -o loop ubuntu-16.04.2-server-amd64.iso /mnt/cdrom/`
+ - Ubuntu Bionic
+ - `wget 'http://cdimage.ubuntu.com/ubuntu/releases/18.04/release/ubuntu-18.04-server-amd64.iso'`
+ - `sudo mount -o loop ubuntu-18.04-server-amd64.iso /mnt/cdrom/`
- `sudo cp -r /mnt/cdrom/install/netboot/* /var/lib/tftpboot/`
- - `sudo mkdir /usr/share/nginx/html/ubuntu`
- - `sudo cp -r /mnt/cdrom/* /usr/share/nginx/html/ubuntu/`
- - `sudo cp /mnt/cdrom/ubuntu/isolinux/ldlinux.c32 /var/lib/tftpboot`
- - `sudo cp /mnt/cdrom/ubuntu/isolinux/libcom32.c32 /var/lib/tftpboot`
- - `sudo cp /mnt/cdrom/ubuntu/isolinux/libutil.c32 /var/lib/tftpboot`
- - `sudo cp /mnt/cdrom/ubuntu/isolinux/chain.c32 /var/lib/tftpboot`
- - `sudo umount /mnt/cdrom`
- - edit ks.cfg and replace IP address with that of your PXE bootstrap server
- - `sudo cp ks.cfg /usr/share/nginx/html/ks.cfg`
- - edit boot-screens_txt.cfg and replace IP address with that of your PXE bootstrap server
- - `sudo cp boot-screens_txt.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/txt.cfg`
- - `sudo cp syslinux.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/syslinux.cfg`
-
-### PREFERED: Prepare the PXE bootstrap server (alternative way without NGINX)
-
- - `sudo apt-get install isc-dhcp-server tftpd-hpa ansible`
- - edit dhcpd.conf and place it to /etc/dhcp/
- - `sudo cp dhcpd.cfg /etc/dhcp/`
- - `sudo service isc-dhcp-server restart`
- - `cd ~testuser/host-setup`
- - `wget 'http://releases.ubuntu.com/16.04.2/ubuntu-16.04.2-server-amd64.iso'`
- - `sudo mkdir /mnt/cdrom`
- - `sudo mount -o loop ubuntu-16.04.1-server-amd64.iso /mnt/cdrom/`
- - `sudo cp -r /mnt/cdrom/install/netboot/* /var/lib/tftpboot/`
- - `sudo mkdir /var/www/download/ubuntu`
- - `sudo cp -r /mnt/cdrom/* /var/www/download/ubuntu/`
+ - figure out where nginx will look for files on the filesystem when
+ responding to HTTP requests. The configuration is in one of the
+ files in /etc/nginx/conf.d/, /etc/nginx/sites-enabled/ or in
+ /etc/nginx/nginx.conf under section server/root. Save the path to WWW_ROOT
+ - `sudo mkdir -p ${WWW_ROOT}/download/ubuntu`
+ - `sudo cp -r /mnt/cdrom/* ${WWW_ROOT}/download/ubuntu/`
- `sudo cp /mnt/cdrom/ubuntu/isolinux/ldlinux.c32 /var/lib/tftpboot`
- `sudo cp /mnt/cdrom/ubuntu/isolinux/libcom32.c32 /var/lib/tftpboot`
- `sudo cp /mnt/cdrom/ubuntu/isolinux/libutil.c32 /var/lib/tftpboot`
- `sudo cp /mnt/cdrom/ubuntu/isolinux/chain.c32 /var/lib/tftpboot`
- `sudo umount /mnt/cdrom`
- edit ks.cfg and replace IP address with that of your PXE bootstrap server and subdir in /var/www (in this case /download)
- - `sudo cp ks.cfg /var/www/download/ks.cfg`
+ - `sudo cp ks.cfg ${WWW_ROOT}/download/ks.cfg`
- edit boot-screens_txt.cfg and replace IP address with that of your PXE bootstrap server and subdir in /var/www (in this case /download)
- `sudo cp boot-screens_txt.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/txt.cfg`
- `sudo cp syslinux.cfg /var/lib/tftpboot/ubuntu-installer/amd64/boot-screens/syslinux.cfg`
- set CIMC address
- set CIMC username, password and hostname
+- set IPMI address
+- set IPMI username, password and hostname
### Bootstrap the host
-From PXE boostrap server:
+Convenient way to re-stage host via script:
+
+ `sudo ./bootstrap_setup_testbed.sh <linux_ip> <mgmt_ip> <username> <pass>`
+
+Optional: CIMC - From PXE boostrap server
- - `cd ~testuser/host-setup/cimc`
- Initialize args.ip: Power-Off, reset BIOS defaults, Enable console redir, get LOM MAC addr
- `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -i`
- Adjust BIOS settings
- `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -s '<biosVfIntelHyperThreadingTech rn="Intel-HyperThreading-Tech" vpIntelHyperThreadingTech="disabled" />' -s '<biosVfEnhancedIntelSpeedStepTech rn="Enhanced-Intel-SpeedStep-Tech" vpEnhancedIntelSpeedStepTech="disabled" />' -s '<biosVfIntelTurboBoostTech rn="Intel-Turbo-Boost-Tech" vpIntelTurboBoostTech="disabled" />'`
- - add MAC address to DHCP (/etc/dhcp/dhcpd.conf)
- - Reboot server with boot from PXE (restart immediately)
- - `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -pxe`
-
-While Ubuntu install is running:
-
- - create RAID array. Reboot if needed.
+ - Add MAC address to DHCP (/etc/dhcp/dhcpd.conf)
+ - If RAID is not created in CIMC. Create RAID array. Reboot.
- `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d --wipe`
- `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -r -rl 1 -rs <disk size> -rd '[1,2]'`
Alternatively, create the RAID array manually.
-
- - Set the next boot from HDD (without restart)
+ - Reboot server with boot from PXE (restart immediately)
+ - `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -pxe`
+ - Set the next boot from HDD (without restart) Execute while Ubuntu install is running.
- `./cimc.py -u admin -p Cisco1234 $CIMC_ADDRESS -d -hdd`
-When installation is finished:
-
- - `ssh-copy-id 10.30.51.x`
- - `cd ~testuser/host-setup/playbooks`
- - edit /etc/ansible/hosts; add the hosts you are installing. *REMOVE ANY HOSTS YOU ARE NOT CURRENTLY INSTALLING*.
-
- Example for physical testbed hosts:
- ~~~
- [tg]
- 10.30.51.24 hostname=t3-tg1 isolcpus="1-17,19-35" ansible_python_interpreter=/usr/bin/python2.7
+Optional: IPMI - From PXE boostrap server
- [sut]
- 10.30.51.25 hostname=t3-sut1 isolcpus="1-17,19-35" ansible_python_interpreter=/usr/bin/python2.7
- 10.30.51.26 hostname=t3-sut2 isolcpus="1-17,19-35" ansible_python_interpreter=/usr/bin/python2.7
- ~~~
+ - Get MAC address of LAN0
+ - `ipmitool -U ADMIN -H $HOST_ADDRESS raw 0x30 0x21 | tail -c 18`
+ - Add MAC address to DHCP (/etc/dhcp/dhcpd.conf)
+ - Reboot into PXE for next boot only
+ - `ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN chassis bootdev pxe`
+ - `ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN power reset`
+ - For live watching SOL (Serial-over-LAN console)
+ - `ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN sol activate`
+ - `ipmitool -I lanplus -H $HOST_ADDRESS -U ADMIN sol deactivate`
- Example for VIRL hosts -- use the "virl" tag and specify the flat network start and end addresses:
-
- ~~~
- [virl]
- 10.30.51.28 hostname=t4-virl1 virl_l2_start=10.30.52.2 virl_l2_end=10.30.52.253 virl_l2_network=10.30.52.0/24 virl_l2_ip=10.30.52.254 virl_public_port=eth0
- ~~~
+When installation is finished:
- - `ansible-playbook --ask-sudo-pass 01-host-setup.yaml`
- - `ansible-playbook reboot.yaml`
+ - Copy ssh keys for no pass access: `ssh-copy-id 10.30.51.x`
+ - Clone CSIT actual repo: `git clone https://gerrit.fd.io/r/csit`
+ - Go to ansible directory: `cd csit/resources/tools/testbed-setup/ansible`
+ - Edit production file and uncomment servers that are supposed to be
+ installed.
+ - Run ansible on selected hosts:
+ `ansible-playbook --vault-id vault_pass --extra-vars '@vault.yml' --inventory production site.yaml`
For non-VIRL hosts, stop here.
-
### VIRL installation
After the host has rebooted:
`ansible-playbook 04-disk-image.yaml`
-The VIRL host should now be operational. Test, and when ready, create a ~jenkins-in/status file with the appropriate status.
+The VIRL host should now be operational. Test, and when ready, create a
+~jenkins-in/status file with the appropriate status.
Code Review / csit.git / blobdiff