Using nfsroot to boot diskless clients on RHEL

Here is a brief outline on the steps needed to set up a Red Hat Enterprise Linux 6 server to boot diskless clients using nfs.

To do this, you need to set up dhcp, tftp (to pxe boot) and a nfs server to serve the rootfs.

To keep it simple, I did everything on the same server, but you can easily have multiple servers for each service. The same steps should also work on Fedora (14+) with minor changes.

Requirements :

Server : Fresh RHEL 6 installation (registered to RHN)

IP = 192.168.50.254/24 (static) on eth0

# vi /etc/sysconfig/network-scripts/ifcfg-eth0
DEVICE=eth0
HWADDR=52:54:00:73:1C:CB
ONBOOT=yes
BOOTPROTO=static
IPADDR=192.168.50.254
NETMASK=255.255.255.0

Client : A box on the same network (with support for PXE)

MAC = 52:54:00:ca:c6:71

On the Server :

Step 1 – Create the nfs rootfs first :

# mkdir /netboot/
# rsync -av --progress --exclude=/proc --exclude=/sys --exclude=/netboot / /netboot/

(i.e. copy everything from root to the /netboot folder)

# mkdir /netboot/{proc,sys}
# vi /netboot/etc/fstab

192.168.50.254:/netboot/  /                      nfs     defaults       1 1
tmpfs                   /dev/shm                tmpfs   defaults        0 0
devpts                  /dev/pts                devpts  gid=5,mode=620  0 0
sysfs                   /sys                    sysfs   defaults        0 0
proc                    /proc                   proc    defaults        0 0

# vi /netboot/etc/sysconfig/network-scripts/ifcfg-eth0

DEVICE="eth0"
HWADDR="52:54:00:ca:c6:71"
ONBOOT="no"
BOOTPROTO="dhcp"

Step 2 – Install and enable services :

# yum install dhcp syslinux tftp-server -y
# yum groupinstall "NFS file server" -y

# chkconfig dhcpd on
# chkconfig tftp on
# chkconfig nfs on
# service xinetd start

For simplicity, disable the firewall :

# iptables -F
# service iptables save

Step 3 – configure dhcp server :

# vi /etc/dhcp/dhcpd.conf

# DHCP Server Configuration file.
#   see /usr/share/doc/dhcp*/dhcpd.conf.sample
#   see 'man 5 dhcpd.conf'
#
ddns-update-style interim;
ignore client-updates;
allow booting;
allow bootp;
subnet 192.168.50.0 netmask 255.255.255.0 {
option routers      192.168.50.254;
option subnet-mask   255.255.255.0;
option nis-domain   "domain.org";
option domain-name  "domain.org";
option domain-name-servers  192.168.50.254;
option time-offset    -18000; # Eastern Standard Time
default-lease-time 21600;
max-lease-time 43200;

host netboot6 {
next-server 192.168.50.254;
hardware ethernet 52:54:00:ca:c6:71;
fixed-address 192.168.50.100;
filename "pxelinux.0";
}
}

# service dhcpd start

Tip : To restrict dhcpd to a particular interface edit the following file :

# cat /etc/sysconfig/dhcpd
# Command line options here
DHCPDARGS="eth1"

Step 4 – enable nfs and export the rootfs :

# vi /etc/exports
 /netboot/            *(rw,async,no_root_squash)

# service rpcbind start
# service nfs start

Step 5 – generate the initramfs file (here is where the magic happens):

# yum install dracut-network -y
# dracut -f /boot/netboot6.img `uname -r` root=dhcp root-path=nfs:192.168.50.254:/netboot/

Step 6 – configure pxe :

# mkdir /var/lib/tftpboot/pxelinux.cfg/
# cp /usr/share/syslinux/pxelinux.0 /var/lib/tftpboot/
# cp /boot/netboot6.img /var/lib/tftpboot/
# cp /boot/vmlinuz-2.6.32-71.el6.x86_64 /var/lib/tftpboot

# vi /var/lib/tftpboot/pxelinux.cfg/default

 default netboot6
 timeout 30
 prompt 1

 label netboot6
 kernel /vmlinuz-2.6.32-71.el6.x86_64
 append initrd=/netboot6.img rw root=nfs:192.168.50.254:/netboot/ selinux=0 enforcing=0

On the Client :

Configure the client to boot over the network (i.e. enable pxe boot) and start the system.

The following screen shots show a successful boot :

17 thoughts on “Using nfsroot to boot diskless clients on RHEL

  1. Well, messages from every node will be dumped there, and likely cause file corruption. You can use a local mount (if local storage is available) for the logs and other node specific data. Or, I would recommend sending your logs to a central log aggregator such as splunk or logstash.

      • Depends on the number of nodes you have. You could measure the bandwidth required by each node during and after boot, and allocate that much network bandwidth. Also, if all nodes come up at the same time, a constant delay will not help, instead add a random delay, that way all nodes won’t hit the nfs server at the same time. You’ll have to experiment and do some testing to figure out what works best for you.

    • Sure, you can use %s in case of grub to replace string with client ip and creating separate root fs of each client.

  2. I have tried with two nodes booting at the same time. rpcbind appears to fail on the nodes.
    As rpcbind fails, ssh fails and needs restarting. Waiting for each node to boot there is not an issue.
    I’m wondering if it because they are sharing the same root filesystem and if having snapshots for each node might cure it. Not really sure how you do snapshots in Centos6 was easy in centos5!

  3. all working fine but diskless client getting stuck in below line
    Starting statd
    “Starting NFS statd: nfs: server not responding, still trying”
    you know about this issue
    thaks in advanced

  4. Hi! I’m trying to diskless boot Centos 7. Server is based on Centos 7 too (Used guide is https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Storage_Administration_Guide/diskless-nfs-config.html).

    I’ve already configured DHCP, TFTP and PXE servers successfully, so my client machine successfully boot kickstart installation or Centos-7-LiveCD image via PXE, but I have trouble trying to boot it with root on my server. I configured my server in such way:

    1. the root directory:

    yum groupinstall Base –installroot=/netboot

    2. Configured the exported file system’s /etc/fstab to contain the following configuration:

    none /tmp tmpfs defaults 0 0

    tmpfs /dev/shm tmpfs defaults 0 0

    sysfs /sys sysfs defaults 0 0

    proc /proc proc defaults 0 0

    3. Installed dracut-network package with `yum install dracut-network` and created the initrd with command:

    dracut -a network initramfs.img \`uname -r\`

    and added this image and proper vmlinuz to tftp-boot directory

    4. Edited the /var/lib/tftpboot/pxelinux.cfg/default :

    label centos7
    kernel vmlinuz
    append initrd=initramfs.img root=nfs:192.168.1.1:/netboot rw selinux=0 nomodeset

    But when I try to boot my client machine I receive:
    dracut-initqueue[249]: Warning: Could not boot.
    (in journalctl it seems that it tries to boot from my harddrive, not via nfs)
    I’m noob with linux and I’ve already tried a lot of guides to fix this. Does anyone know where the problem can be?

  5. I just found the documentation of dracut lacking and the cli in this article is not what centos7 and Fedora v21 expect.
    dracut -m “nfs network base” –kernel-cmdline “root=nfs4:10.1blah blah blah…” initramfs-nfs.img was what it took to complete the initramfs img file.

  6. Hi,

    Thanks for the steps.

    I followed exact steps as written. At last while creating initramfs.img image I am failing.
    here is the error.

    dracut -f /boot/netboot6.img `uname -r` root=dhcp root-path=nfs:192.168.50.254:/netboot/

    Unknown arguments: root=dhcp root-path=nfs:192.168.50.254:/netboot/

    Usage: /usr/sbin/dracut [OPTION]… [ []]

    Version: 033-241.el7_1.5

    Creates initial ramdisk images for preloading modules

    -h, –help Display all options

    If a [LIST] has multiple arguments, then you have to put these in quotes.

    For example:

    # dracut –add-drivers “module1 module2” …

    But this works dracut -m “nfs network base” initramfs-nfs2-only.img fine. But using this image to boot it cannot see nfs path. Its fails as mount.nfs no such device

  7. Pingback: RHEL nfs boot error « news-Knowlage FeeD

Leave a comment