Here is a brief outline on the steps needed to set up a Red Hat Enterprise Linux 6 server to boot diskless clients using nfs.
To do this, you need to set up dhcp, tftp (to pxe boot) and a nfs server to serve the rootfs.
To keep it simple, I did everything on the same server, but you can easily have multiple servers for each service. The same steps should also work on Fedora (14+) with minor changes.
Requirements :
Server : Fresh RHEL 6 installation (registered to RHN)
IP = 192.168.50.254/24 (static) on eth0 # vi /etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE=eth0 HWADDR=52:54:00:73:1C:CB ONBOOT=yes BOOTPROTO=static IPADDR=192.168.50.254 NETMASK=255.255.255.0
Client : A box on the same network (with support for PXE)
MAC = 52:54:00:ca:c6:71
On the Server :
Step 1 – Create the nfs rootfs first :
# mkdir /netboot/ # rsync -av --progress --exclude=/proc --exclude=/sys --exclude=/netboot / /netboot/
(i.e. copy everything from root to the /netboot folder)
# mkdir /netboot/{proc,sys} # vi /netboot/etc/fstab 192.168.50.254:/netboot/ / nfs defaults 1 1 tmpfs /dev/shm tmpfs defaults 0 0 devpts /dev/pts devpts gid=5,mode=620 0 0 sysfs /sys sysfs defaults 0 0 proc /proc proc defaults 0 0 # vi /netboot/etc/sysconfig/network-scripts/ifcfg-eth0 DEVICE="eth0" HWADDR="52:54:00:ca:c6:71" ONBOOT="no" BOOTPROTO="dhcp"
Step 2 – Install and enable services :
# yum install dhcp syslinux tftp-server -y # yum groupinstall "NFS file server" -y # chkconfig dhcpd on # chkconfig tftp on # chkconfig nfs on # service xinetd start
For simplicity, disable the firewall :
# iptables -F # service iptables save
Step 3 – configure dhcp server :
# vi /etc/dhcp/dhcpd.conf # DHCP Server Configuration file. # see /usr/share/doc/dhcp*/dhcpd.conf.sample # see 'man 5 dhcpd.conf' # ddns-update-style interim; ignore client-updates; allow booting; allow bootp; subnet 192.168.50.0 netmask 255.255.255.0 { option routers 192.168.50.254; option subnet-mask 255.255.255.0; option nis-domain "domain.org"; option domain-name "domain.org"; option domain-name-servers 192.168.50.254; option time-offset -18000; # Eastern Standard Time default-lease-time 21600; max-lease-time 43200; host netboot6 { next-server 192.168.50.254; hardware ethernet 52:54:00:ca:c6:71; fixed-address 192.168.50.100; filename "pxelinux.0"; } } # service dhcpd start
Tip : To restrict dhcpd to a particular interface edit the following file :
# cat /etc/sysconfig/dhcpd # Command line options here DHCPDARGS="eth1"
Step 4 – enable nfs and export the rootfs :
# vi /etc/exports /netboot/ *(rw,async,no_root_squash) # service rpcbind start # service nfs start
Step 5 – generate the initramfs file (here is where the magic happens):
# yum install dracut-network -y # dracut -f /boot/netboot6.img `uname -r` root=dhcp root-path=nfs:192.168.50.254:/netboot/
Step 6 – configure pxe :
# mkdir /var/lib/tftpboot/pxelinux.cfg/ # cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot/ # cp /boot/netboot6.img /var/lib/tftpboot/ # cp /boot/vmlinuz-2.6.32-71.el6.x86_64 /var/lib/tftpboot # vi /var/lib/tftpboot/pxelinux.cfg/default default netboot6 timeout 30 prompt 1 label netboot6 kernel /vmlinuz-2.6.32-71.el6.x86_64 append initrd=/netboot6.img rw root=nfs:192.168.50.254:/netboot/ selinux=0 enforcing=0
On the Client :
Configure the client to boot over the network (i.e. enable pxe boot) and start the system.
The following screen shots show a successful boot :
If more than one node is used what will happen to the /var/log/messages if each node?
Well, messages from every node will be dumped there, and likely cause file corruption. You can use a local mount (if local storage is available) for the logs and other node specific data. Or, I would recommend sending your logs to a central log aggregator such as splunk or logstash.
Would you expect to get any boot issues using this method? i.e. need to have a 60 second delay between each node starting?
Depends on the number of nodes you have. You could measure the bandwidth required by each node during and after boot, and allocate that much network bandwidth. Also, if all nodes come up at the same time, a constant delay will not help, instead add a random delay, that way all nodes won’t hit the nfs server at the same time. You’ll have to experiment and do some testing to figure out what works best for you.
Sure, you can use %s in case of grub to replace string with client ip and creating separate root fs of each client.
I have tried with two nodes booting at the same time. rpcbind appears to fail on the nodes.
As rpcbind fails, ssh fails and needs restarting. Waiting for each node to boot there is not an issue.
I’m wondering if it because they are sharing the same root filesystem and if having snapshots for each node might cure it. Not really sure how you do snapshots in Centos6 was easy in centos5!
It could be due to nfs locking. Check the logs on the nfs server. Honestly, I haven’t tried this on two hosts; additional config changes may be required to get two plus nodes up and running.
all working fine but diskless client getting stuck in below line
Starting statd
“Starting NFS statd: nfs: server not responding, still trying”
you know about this issue
thaks in advanced
its working fine with yum groupinstall base –installroot=/netboot
Great! Thanks for sharing the solution. That message usually indicates a problem with the nfs server, just like the message states.
i think your client is not having ip while trying to connect nfs sderver, try providing static ip in “/netboot/etc/sysconfig/network-scripts/ifcfg-eth0” instead dhcp.
Hi there, I desire to subscribe for this website to obtain most up-to-date updates, thus where can i do itt please help out.
Hi! I’m trying to diskless boot Centos 7. Server is based on Centos 7 too (Used guide is https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/diskless-nfs-config.html).
I’ve already configured DHCP, TFTP and PXE servers successfully, so my client machine successfully boot kickstart installation or Centos-7-LiveCD image via PXE, but I have trouble trying to boot it with root on my server. I configured my server in such way:
1. the root directory:
yum groupinstall Base –installroot=/netboot
2. Configured the exported file system’s /etc/fstab to contain the following configuration:
none /tmp tmpfs defaults 0 0
tmpfs /dev/shm tmpfs defaults 0 0
sysfs /sys sysfs defaults 0 0
proc /proc proc defaults 0 0
3. Installed dracut-network package with `yum install dracut-network` and created the initrd with command:
dracut -a network initramfs.img \`uname -r\`
and added this image and proper vmlinuz to tftp-boot directory
4. Edited the /var/lib/tftpboot/pxelinux.cfg/default :
label centos7
kernel vmlinuz
append initrd=initramfs.img root=nfs:192.168.1.1:/netboot rw selinux=0 nomodeset
But when I try to boot my client machine I receive:
dracut-initqueue[249]: Warning: Could not boot.
(in journalctl it seems that it tries to boot from my harddrive, not via nfs)
I’m noob with linux and I’ve already tried a lot of guides to fix this. Does anyone know where the problem can be?
I just found the documentation of dracut lacking and the cli in this article is not what centos7 and Fedora v21 expect.
dracut -m “nfs network base” –kernel-cmdline “root=nfs4:10.1blah blah blah…” initramfs-nfs.img was what it took to complete the initramfs img file.
Hi,
Thanks for the steps.
I followed exact steps as written. At last while creating initramfs.img image I am failing.
here is the error.
dracut -f /boot/netboot6.img `uname -r` root=dhcp root-path=nfs:192.168.50.254:/netboot/
Unknown arguments: root=dhcp root-path=nfs:192.168.50.254:/netboot/
Usage: /usr/sbin/dracut [OPTION]… [ []]
Version: 033-241.el7_1.5
Creates initial ramdisk images for preloading modules
-h, –help Display all options
If a [LIST] has multiple arguments, then you have to put these in quotes.
For example:
# dracut –add-drivers “module1 module2” …
But this works dracut -m “nfs network base” initramfs-nfs2-only.img fine. But using this image to boot it cannot see nfs path. Its fails as mount.nfs no such device
Pingback: RHEL nfs boot error « news-Knowlage FeeD
If some one desires to be updated with most recent technologies afterward he must be pay a quick visit this website and be up to date every day.