Adventure in PXE Land

Pixies – An alternative rock group

PXE – Intel’s Pre-Boot Execution Environment

Not to be confused!

But, we can use PXE to play Pixies.

Haifux Lecture by: Ami Chayun
Date: 6/12/2004

Introduction – The Players: PXE

The PXE Protocol is overviewed at: http://www.ietf.org/proceedings/99mar/slides/remboot-agenda-99mar/sld009.htm

The protocol was presented by Intel in 1999 and since then became the de-facto industry standard for network boot.

The target of the protocol is to support:
Remote new system setup – When no OS is installed, assist the installation.
Remote emergency boot – In case of hardware failure, supply an alternative
Remote network boot – Where no local storage is available (or needed) load the entire OS from the network

The protocol uses only industry open standards, namely TFTP and DHCP

The solution is a hardware implementation and it requires support both in the machine BIOS and NIC

Introduction – The Players: DHCP

DHCP is defined at: RFC 2131 - Dynamic Host Configuration Protocol and
RFC 2132 - DHCP Options and BOOTP Vendor Extensions

The BOOTSTRAP PROTOCOL is an IP/UDP protocol which allows a diskless client machine to discover its own IP address, the address of a server host, and the name of a file to be loaded into memory and executed (usually via TFTP server).
The DHCP protocol which inherited BOOTP and extended it.

The designated DHCP server hosts allocate network addresses and deliver configuration parameters to dynamically configured hosts.
The basic phases of the protocol are:
1. DHCPDISCOVER - Client broadcast to locate available servers.
2. DHCPOFFER - Server replies to client response to DHCPDISCOVER with offer of configuration parameters (Available network IP address etc.).
3. DHCPRELEASE - Client to server relinquishing network address and cancelling remaining lease.

Introduction – The Players: TFTP

TFTP is defined at: RFC 1350 - The TFTP Protocol

Trivial File Transfer Protocol is a very simple protocol used to transfer files.

TFTP is usually implemented over UDP. One side of the transfer simply breaks a file into packets of fixed size (normally 512 bytes) and sends them to the client. Packets are either ACKed or retransmitted in case of timeout.

Several extensions to TFTP allows it to be used for PXE boot sequence.

PXE Basic Requirements

Client - PXE requires the client to implement a BIOS support for DHCP client and TFTP client.
Both are usually hardware implemented in the NIC. The BIOS simply switch them on.

Server - The server is required to run a DHCP server that provides a location of a TFTP server and an executable file (compiled to the client’s architecture) to act as a boot loader.

Our purpose is to load a Linux system, so let’s prepare a server!

PXE Server - dhcpd

dhcpd is available for all major distributions and is also available in sources

DHCP Configuration file: The dhcpd.conf file is structured as follow:

# LAN dynamic addresses

option domain-name “example.com";

subnet 192.168.1.0 netmask 255.255.255.0 {
# Server and filename where to get boot loader

next-server 192.168.1.12;

filename "pxelinux.0";
# Classic DHCP definitions – address range and restrictions

range dynamic-bootp 192.168.1.200 192.168.1.250;

allow unknown-clients;

}

More DHCP 3 specific options in relation to PXELINUXcan be found at: http://syslinux.zytor.com/pxe.php#special

For the lazy of us, use the webmin-dhcp utility.

PXE Server – TFTP Server

The TFTP server used for PXE requires some extensions to the original TFTP.

We will use atftpd which is available in Debian, but can also be compiled from sources. For basic PXE operation, any tftp server will do.

By default it is started by inetd on most sytems, but may run as a stand alone daemon. This server is multi‐threaded and supports all options described in RFC2347 (option extension), RFC2349 (tsize and timeout) and RFC2090 (multicast option). It also supports mtftp as defined in the PXE specification.

The server also support RFC 2348 (blksize) which allows transfer of files larger than the original TFTP, by changing the size of each data packet to values larger than 512.
This is important, since there is almost no limit to the filesize we can send to the client...

PXE Server – ATFTPD Configuration

Installation: Once atftpd is installed, inetd.conf needs to be edited, and the following entry added:
tftp dgram udp wait nobody
/usr/sbin/tcpd /usr/sbin/in.tftpd \
--tftpd-timeout 300 --retry-timeout 5 \
--mcast-port 1758 --mcast-addr 192.168.1.254 \
--maxthread 100 --verbose=5 \
/usr/local/PXE-Linux/tftp-root
Parameters and meaning:

mcast-addr 192.168.1.254 - If multicast is required, direct to the multicast server.

usr/local/PXE-Linux/tftp-root – Point to the tftp root directory.

PXE Server – network boot loader

In order to boot the client we need a Linux network boot loader. Just like everything else in Linux there is always more than one option:

Etherboot – Support both network and ROM loading

PXELINUX – PXE specific boot loader

GRUB – All purpose boot loader, with PXE support

We will demonstrate the use of PXELINUX since it is simple and specific. The following files will be placed in the tftp-root dir:

pxelinux.0 – The boot loader executable, complied for the target architecture (available for i386, arm and more)

pxelinux.cfg/default – the default configuration file is placed in the pxelinux.cfg dir.

Configuration file syntax: The syntax is very similar to LILO, as the example shows.

PXE Server – PXELINUX config file

# display will print the content of the greeting file.

display welcome.msg

#With or without prompt? 1 – with prompt

prompt 1

# Labels – Each label is an option that can be selected at the prompt.

label localboot

LOCALBOOT 0

label NETBOOT

# Kernel image

kernel vmlinuz

# Parameters to pass to the kernel

append init=/linuxrc initrd=initrd.img lang= devfs=nomount ramdisk_size=14336 vga=788

# After timeout is over or no option is entered,

# the loader will default to the specified label

timeout 100

default localboot

PXE Server – Contents of tftp-root

For now the contents of the tftp-root directory are:

-rw-r--r-- 1 root staff 11K 2004-06-03 13:29 pxelinux.0

drwxr-sr-x 2 root staff 4.0K 2004-12-06 12:38 pxelinux.cfg/

-rw-r--r-- 1 root staff 208 2004-12-06 12:38 pxelinux.cfg/default

-rwxr-xr-x 1 root root 1.4M 2004-11-22 16:26 vmlinuz*

-rw-r--r-- 1 root staff 205 2004-12-06 12:38 welcome.msg

Let's take a peek at the kernel...

BOOT Kernel

The linux kernel can be compiled to fit in an HDD MBR, so it's not a problem to load it from network.

In the example, the kernel is Linux 2.6.5, but any kernel will do. No special requirements are needed. The kernel is loaded by the boot loader exactly like it would from the hard drive.

How to install the kernel to the tftp root?
Simplest way, copy the kernel image:
arch/i386/boot/bzImage
to your tftp-root dir as vmlinuz or whatever name is defined in pxelinux.cfg and make it world readable.

But wait, what about kernel modules?
What happens if I want to use kernel modules for my system?
The answer is of course: initrd...

The initrd

What is initrd? (Taken from man 4 initrd)

The special file /dev/initrd is a read-only block device. Device dev/initrd is a RAM disk that is initialized (e.g. loaded) by the boot loader before the kernel is started. The kernel then can use the the block device /dev/initrd's contents for a two phased system boot-up.

In the first boot-up phase, the kernel starts up and mounts an initial root file-system from the contents of /dev/initrd (e.g. RAM disk initialized by the boot loader). In the second phase, additional drivers or other modules are loaded from the initial root device's contents. After loading the additional modules, a new root file system (i.e. the normal root file system) is mounted from a different device.

In the configuration file we saw that we are passing the kernel the following parameters, what do they mean?

init=/linuxrc – Once the kernel is loaded, it will look for this executable (in the initrd) and run it. This is the default for initrd.

ramdisk_size=14336 – The boot loader will conserve 14MB of RAM, to load the initrd, and will behave as a disk for all our purposes.

initrd=initrd.img – This image will be downloaded via tftp, uncompressed and mounted on the ramdisk created. It will then be the new root of our system.

The initrd

In the configuration file we saw that we are passing the kernel the following parameters, what do they mean?

init=/linuxrc – Once the kernel is loaded, it will look for this executable (in the initrd) and run it. This is the default for initrd.

ramdisk_size=14336 – The boot loader will conserve 14MB of RAM, to load the initrd, and will behave as a disk for all our purposes.

initrd=initrd.img – This image will be downloaded via tftp, uncompressed and mounted on the ramdisk created. It will then be the new root of our system.

The initrd.img is a file, containing a skeleton filesystem for the kernel. So, is it a file or a file system? For the matter of fact, it's both!

In this example, we will build the initrd as agzipped ext2 filesystem. But wait, can a file be a file system?!
Why not?
Here's a recipe:

Linux From Scratch (well, almost)

First thing to do is locate some space on the hard drive for the would be file system, so in a designated dir run:

dd if=/dev/zero of=ramdisk_img bs=14000k count=1

Now create an ext2 filesystem:

mke2fs -m 0 -b 1024 ramdisk_img

mke2fs will warn you that the file is just a file, and not a real block device, assure mke2fs you know what you're doing and type 'y' when prompted:

ramdisk_img is not a block special device.

Proceed anyway? (y,n)

Lastly Mount the new file on the target dir, create an initrd-full dir and run:

mount -o loop ramdisk_img initrd-full

Putting some meat on those bones

The initrd is a skeleton, but it does need some meet in it. So let's put in some basic directories:

/dev – Lazy people just copy from an existing system

/proc – Empty dir for proc

/tmp – always good to have

Next up: Kernel modules. The Linux kernel is always glad to put it's modules anywhere you like. Just run:

make INSTALL_MOD_PATH=path_to_your_initrd_image_root modules_install

This will create:

/lib/modules/kernel.version – as usual...

What else? How about a shell?

Busybox – The little binary that could

Kernel is fine enough, but who cares? I want a command prompt you say.

No problem! Busybox got the answer.

First up, fetch a fresh copy of busybox from busybox.org

Untar, run make menuconfig, select any options you like and shove it into your initrd:

make PREFIX=/path_to_image/initrd-full/ install

Note that you will need to compile it statically if you don't plan to put in any libc (regular or uClibc)

To use busybox as init, make a symbolic link in the initrd:

ln -s bin/busybox linuxrc

Busybox provides beside shell, some very initeresting userspace utilities.

Putting it all together

Build the initrd file:

umount initrd-full

gzip -c ramdisk_img > initrd.img

chmod +x initrd.img

And copy it into your tftp-root dir.

Test run –
Reboot the client, and watch the kernel loads.
Finally, busybox should load and (after some warning messeges for not having proper rcS init scripts) you should get a shell!

Next step – Userspace Applications

Userspace applications usualy require shared libraries, namely glibc is a must.
If size is important, try looking into the uClibc project. uClibc is a compact

Shared Libraries – uClibc

Next step – Userspace Applications

Userspace applications are rearly complied statically, and usualy require shared libraries. Namely, glibc is a must.
If size is important, try looking into the uClibc project. uClibc is a compact version of the full Libc, with several limitations. In the words of the developrs:
“Why should I use uClibc?
I don't know if you should use uClibc or not. It depends on your needs. If you are building an embedded Linux system and you are tight on space, then using uClibc instead if glibc may be a very good idea.
If you are building an embedded Linux system and you find that glibc is eating up too much space, you should consider using uClibc. If you are building a huge fileserver with 12 Terabytes of storage, then using glibc may make more sense. Unless, for example, that 12 Terabytes will be Network Attached Storage and you plan to burn Linux into the system's firmware... “
For our purposes, this is completely sufficient.

Note: Just remember to compile all the applications with uClibc. Even busybox, which makes it smaller than compiled statically.

Shared Libraries – Really?!

Ok, I complied everything with my cool new uClic and I want to install libraries on my PXE image. Care to explain how?

To install shared libraries we need to do two things:

Copy the libraries to the tftp-root dir. For example, if we want libxmlparse (required for 'discover') we need to copy:
cp libxmlparse.so.1.1 /path-to-initrd-full/usr/lib/

Create symbolic link (libxmlparse.so.1) and ldcache. Fortunately the command ldconfig can run (like everything we've seen so far) on any directory, and do all the work for us. So after you got all your nifty shared libraries, run: ldconfig -r /full-path-to-initrd-full/
This tells ldconfig to chroot to full-path-to-initrd-full. Is this cool or what?!

Note: ldconfig expects /etc/ld.so.conf file, which tells it which directories to search for extra shared libraries (besides /lib).
If no other libraies exist (/usr/lib for example) simply touch etc/ld.so.conf in your initrd-full root. Edit the file to add /usr/lib if required.

Even More Stuff

So far we got almost nothing in our initrd file. What happens when we need a lot more data? Luckily we are connected to the network!

Tftp – Crude but simple:
Our first option is to use tftp, which we already used to load the base syste. Tftp support can be compiled in busybox, and all we need to do is to put stuff in the tftp root on our PXE server.

HTTP / FTP:
Wget and FTP can also be compiled in busybox, so all we need is a web / ftp server.

Networked Filesystems: NFS
NFS is only one of the options which we can use to load a remote filesystem. Pxelinux boot loader can even load the initrd root system from NFS.

Ideas for PXE loaded machine:

Installation parties: With a mirror of a Linux distribution, a network install can be performed completely from the network! No CD at all.

Almost embedded machines: Let's say you have a computer at home, and you want to listen to music in the bathroom or use MythTV to watch movies in the living room.
Get an old PC, build an initrd with mplayer, and in no time you have a diskless MP3 player!

Now you got a working diskless Linux distribution built from almost nothing!
This was just a taste...
Thank you!
Questions?

Adventure in PXE Land

The Monster Redux

2004-12-22T15:02:51