Stochastic Ideations

I've been working on bringing up a new Soekris net5501 box on FreeBSD 8.2. Since the Soekris boot ROM doesn't understand locally attached CD-ROM drives, you have to do this over the network.

Follow up:

There's an excellent discussion of how to do that here but I found there were still a couple of pieces I had to figure out by myself. Part of the trick is that I'm running m0n0wall as my firewall and DHCP server, which doesn't allow you to configure the root-path DHCP option. But it turns out you can do without that.

Configuration

I'm running a Soekris net5501 box with an internal laptop hard disk.  The system is primarily a web and DNS server, although it has some other minor services.  It runs in a dedicated DMZ, using a m0n0wall firewall/router running on another Soekris box.  DHCP runs on m0n0wall.  I have an amd64 running FreeBSD 8.0 from which I'm serving the distribution.  I have a large scratch filesystem called /l1 that will be used to store the data.

Getting Started

The instructions at Installing FreeBSD 8.x via serial console and PXE by Jeremy Chadwick is a good place to start.  In fact, I'll try to make sure these instructions mimic that document.  I'll reference that document as "JC" here.

JC has DHCP and TFTP running on the same server.  That isn't possible when using m0n0wall, but this is easy to fix, as will be described below.

Setting Up DHCP (JC Section 3.1)

Since we are using m0n0wall for the DHCP server, it isn't possible to set up option root-path.  As it turns out, if that isn't set pxeboot(8) defaults to using "/pxeroot", which we will use below.  In this configuration, next-server is required; it should be the address of the TFTP/NFS server.  There may be a way to separate those as well, but it wasn't something I was worried about.

Configuring TFTP (JC Section 3.2)

I made an attempt to just serve directly off the CD-ROM rather than copying it onto the server local disk, using unionfs(8) to cover it with a writable layer.  That didn't work; in particular, NFS doesn't seem to be able to export a unionfs mount point.  I resorted to copying the entire distribution medium onto local disk, as recommended by JC.  In my case I used /l1/freebsd8.2, which was also the root for tftpd in inetd.conf that is, I used:

tftp    dgram   udp     wait    root    /usr/libexec/tftpd      tftpd -l -s /l1/freebsd8.2

The -l flag turns on logging, which uses the LOG_FTP facility.  As it turns out that wasn't going anywhere, so I had to tweak /etc/syslog.conf.  Logging is very useful for debugging your setup.

Configuring NFS (JC Section 3.3)

The JC instructions for NFS are fine, but I also recommend adding -l to mountd_flags in /etc/rc.conf to enable logging.  The mountd(8) man page doesn't bother to tell you that it logs as LOG_DAEMON, so be sure you're getting this somewhere useful.  In particular, if showmount -e doesn't show anything exported, look at your log to find out why.

Recompiling pxeboot (JC Section 3.5)

For some bizarre reason, pxeboot assumes that ANSI escape sequences to position the cursor before every character that it outputs is a good idea.  The actual result is that everything it outputs is immediately overwritten.  You can get around this by running the connection to the Soekris serial port in script(1) and then inspect the typescript file with strings(1).  But your life will be easier if you do as JC suggests.  I left the port speed at 9600 though, since that's the default for Soekris.

Fixing File Permissions (JC Section 3.7)

JC talks about fixing file permissions in section 3.7.  Yes indeed, this is essential.  Annoying and a pain, but essential.

Doing the PXE Boot (JC Section 3.8)

Assuming everything goes well everything should just come up.  Ha!  I had it wait for an extra prompt for me, something about manually overriding the root, but an empty input was good enough.

Eric Allman

June 2011