Note: this wiki is now retired and will no longer be updated!
The static final versions of the pages are left as a convenience for readers. Note that meta-pages such as "discussion," "history," etc., will not work.
IPMI and SOL on an Intel S3200SHV
This page documents the steps needed to get IPMI 2.0 serial-over-LAN (SOL) working on a Intel S3200SHV-based system running Debian GNU/Linux. As of March 2009, Newegg sells this motherboard for a reasonable price[1]. The board includes a decent, if a bit slow, BMC and IPMI 2.0 implementation. This particular model doesn't support SSH or telnet into the BMC like Sun's Sun Fire systems do (e.g., the x2200 M2), but if have another system on your network capable of running an IPMI 2.0 client, you can still get access to the machine's serial console from a remote host.
IPMI 2.0 SOL is encrypted and, if configured properly, requires a password for authentication. However, since password authentication is susceptible to dictionary attacks, and because anecdotal evidence suggests that many IPMI implementations have security holes, I don't recommend that you run IPMI SOL on a public-facing network. The BMC IP address, or better yet, the entire machine, should be firewalled from any untrusted networks.
Note that these steps are from memory, so there may be a few bits missing.
I can't provide any support for people attempting do this on their own systems beyond what I've documented here. However, if you find any mistakes here or have useful tips to add, please contact me and I'll update the page.
Upgrade to latest firmware
Intel is fairly good about making regular updates to its firmware, including fixes for IPMI and BMC bugs. In my case, upgrading to the February 2009 firmware also sped up the boot process and made it more reliable: previously the system would occasionally hang on boot, waiting for removable storage to spin up that wasn't even attached.
Intel provides a flash update tool for RedHat and SUSE GNU/Linux[2], but I had a hard time getting it to work on a Debian sid x86-64 system. It's packaged as an i386 RPM and needs to load a "binary blob" kernel module that gets built by an init script (!). alien can convert RPMs to debs, but the i386/x86-64 architecture mismatch complicates things, and I didn't want to go through the hassle of converting the package to a .tgz file and installing from that.
Mercifully, all Intel firmware updates for this board are packaged with a turnkey EFI update script. Simply unzip the firmware update onto a FAT32-formatted USB stick, insert the stick into a USB port on the system, reboot and hit F2 when prompted by the BIOS to enter the BIOS setup. Switch to the Boot Manager BIOS screen and select the EFI Shell boot option. You should be immediately prompted by the update script on the USB stick whether you want to proceed with the update. This process isn't documented here, but it's straightforward if you simply read the prompts and choose the default options. (Note that the February 2009 firmware update even allows you to specify which fans are attached to the motherboard system fan headers in your particular system, so that the IPMI sdr command will return proper fan speed data.)
Install ipmitool
You'll need ipmitool to configure IPMI.
# apt-get install ipmitool
We'll do the initial IPMI configuration on the system itself. Once it's set up, you can optionally configure it from remote machines capable of running an IPMI 2.0 client (e.g., other GNU/Linux systems).
Install IPMI kernel modules
Add these lines to your /etc/modules.conf to ensure they're loaded at boot time:
ipmi_poweroff ipmi_si ipmi_msghandler ipmi_devintf ipmi_watchdog
You may need to insmod these manually just this once before proceeding with IPMI configuration, but subsequent reboots will load them automatically.
Some of these modules, e.g., the watchdog module, aren't covered here, but I include them for completeness in case you want to use these features in the future.
User setup
IPMI provides user accounts with variable names, privileges and authentication methods. The accounts are numbered, starting from 1 and going up to 15 on the S3200SHV. Account 1 is special and has something to do with default access. I won't cover its setup here as I don't use it. If you're paranoid about users on your system's network, you should probably read the IPMI documentation and figure out how to disable it completely. (It appears to be configured by default to be relatively harmless/unprivileged on the S3200SHV, but I haven't tested this extensively.)
Enabling/disabling and passwords
You can get a list of users like so:
# ipmitool user list 1
Here the "1" refers to IPMI channel 1; don't worry about what it means, just know that we'll always use channel 1 here.
You should see output that looks something like this:
ID Name Callin Link Auth IPMI Msg Channel Priv Limit 1 true false true ADMINISTRATOR 2 root false true true ADMINISTRATOR 3 test1 false false true ADMINISTRATOR 4 test2 false false true ADMINISTRATOR 5 test3 false false true ADMINISTRATOR 6 false false true ADMINISTRATOR 7 false false true ADMINISTRATOR 8 false false true ADMINISTRATOR 9 false false true ADMINISTRATOR 10 false false true ADMINISTRATOR 11 false false true ADMINISTRATOR 12 false false true ADMINISTRATOR 13 false false true ADMINISTRATOR 14 false false true ADMINISTRATOR 15 false false true ADMINISTRATOR
As you can see, my S3200SHV came pre-configured with 3 "test" users and one "root" user. Remember that the user with ID 1 is anonymous and is used for default access (see above), so we leave that ID alone.
I decided to use ID 2 (root) and ID 3, which I renamed with my GNU/Linux userid, and disabled the others. It's entirely possible just to use the root account and disable everything else, but I wanted to give myself the option of using finer-grained privileges with a non-root account in the future.
First let's disable the two unused test accounts. I believe they're disabled by default anyway, but let's be safe. Some ipmitool commands identify the user by ID, and others by username. This is annoying, but there's not much to be done about it.
# ipmitool user disable 4
# ipmitool user disable 5
If these commands are successful, you'll get no output. (This is true in general of IPMI commands: silence means success.)
Now let's enable the root user, rename and enable the other user, and assign passwords to both. Replace dhess here and throughout the rest of the instructions with the name you want to use for the other user.
# ipmitool user enable 2
# ipmitool user enable 3
# ipmitool user set name 3 dhess
# ipmitool user set password 2
# ipmitool user set password 3
Each of the set password commands will prompt you for a password twice (2nd time to verify).
We could give the non-root user lower privileges here (the default is ADMINISTRATOR), but I haven't bothered with that yet.
Run a quick test to see if the passwords are working:
# ipmitool user test 20 2
# ipmitool user test 20 3
Each command will prompt for that user's password and output Success if everything is working.
Setting access
Enable access to all features on channel 1 for the non-root account. (I can't change settings for the root account, for some reason, but the defaults work fine for our purposes.)
# ipmitool channel setaccess 1 3 callin=on ipmi=on link=on
Verify:
# ipmitool channel getaccess 1 3 Maximum User IDs : 15 Enabled User IDs : 0 User ID : 3 User Name : dhess Fixed Name : No Access Available : call-in / callback Link Authentication : enabled IPMI Messaging : enabled Privilege Level : ADMINISTRATOR
Enable network access to BMC
Now we need to enable network access to the BMC so that we can access the serial console from remote hosts. The S3200SHV provides two Gigabit Ethernet NICs, one with PCI address 0000:06:02.0 and the other with PCI address 00:19.0. The former is an Intel 82541GI connected to the system via a dedicated 32-bit/33MHz PCI bus, and the latter is an Intel 82566DM-2 connected to the system via a high-speed serial link. The 82541GI uses the e1000 driver on my Debian GNU/Linux system, and the 82566DM-2 uses the e1000e driver. The 82541GI physical port is on top of the two USB ports on the back of the system, and the 82566DM-2 port sits next to them.
The BMC controller shares the 82541GI with the operating system by creating a 2nd MAC address on the same physical port. The OS doesn't even see this address (neither the MAC address nor the IP address we're about to assign to it), and both the OS and the BMC can use the same physical interface simultaneously. (Think of it as a small 2-port hub with a single "uplink" jack: any machines connected to that uplink jack see multiple MAC addresses transmitting and receiving over one physical cable.) However, I use the 82566DM-2 for the system NIC and the 82541GI just for IPMI access. I believe that the 82566DM-2 is a faster/more efficient interface (citation needed) and I have plenty of ports in my switch to accomodate both, but if you're short on cabling or ports, you can certainly use just the 82541GI.
Use the following command to see the IPMI network configuration:
# ipmitool lan print
Assign an IP address
The default configuration isn't useful; the BMC NIC has no IP address, for starters. We need to give it one before we can communicate with it over the network. I have a DHCP server that permits static IP assignments, so I configured the BMC to use DHCP and told the DHCP server to give a static IP and internal hostname to the BMC MAC address, but you should use a static assignment if you don't have a similar DHCP setup. (Static assignment isn't covered here, but it's easy to figure out.) You can see your BMC NIC's MAC address using the ipmitool lan print command.
# ipmitool lan set 1 ipsrc dhcp
This command will take a few seconds the first time you run it. If the 82541GI NIC is attached to your network, when you run the ipmitool lan print command you should now see an IP address, broadcast mask and gateway, all assigned by the DHCP server.
There's no need to change anything in your Debian networking configuration. As I mentioned above, the Linux kernel doesn't even see the BMC's MAC address, so it couldn't manage it even if you wanted it to. The BMC firmware is responsible for configuring and maintaining its own IP address. The settings are persistent across system reboots, BMC resets and complete power loss. Basically, once you set them you should never have to touch them again, unless you decide to change IPs, switch from DHCP to static assignment, etc.
Note that if you try to ping this IP address, it won't respond. Apparently some vendors disable ICMP responses on their BMCs, and Intel appears to be one of them. There are other ways to test whether the BMC is responding, however, and I'll get to those in a bit.
Enable LAN access
Now enable access to IPMI channel 1 via the network:
# ipmitool lan set 1 access on
Set SNMP public community string
If you use SNMP on your network, set your public community string for the BMC interface; otherwise, skip this step. In my case, it's "public".
# ipmitool lan set 1 snmp public
Force authentication for all privileges
By default, the S3200SHV IPMI comes configured to allow bypassing authentication, although in practice I haven't seen this actually work, perhaps because of other settings. Just to be safe, I configured IPMI on my system to require MD5 or password auth for network connections at all privilege levels:
# ipmitool lan set 1 auth callback md5,password
# ipmitool lan set 1 auth user md5,password
# ipmitool lan set 1 auth operator md5,password
# ipmitool lan set 1 auth admin md5,password
In practice, so far I'm only using one of these privilege levels (admin), but this should close any holes or services that I haven't yet probed.
Note that the ipmitool lan print command on this board shows an "OEM" privilege level, but there's no way to change its authentication setting, and it's not clear whether it's actually a usable level or if it's just a placeholder for unimplemented functionality.
Test network settings
At this point, assuming your BMC NIC is connected to the network and has an IP address, you can test your access to the system's IPMI layer from a remote host. If you have another machine on the network capable of running ipmitool (e.g., another GNU/Linux machine), run these commands from that machine to ensure that everything is working, but if you don't, you can run the tests on the same system, too. (Note that Mac OS X 10.5.6 provides ipmitool, but it seems to be lacking support for the lanplus interface that we'll need to connect to IPMI 2.0 serial-over-LAN, so it doesn't actually work with a remote S3200SHV system.)
This command will connect to our S3200SHV system via the network, using IPMI over UDP, and print the S3200SHV system's sensor data. Replace the example IP address and username shown here with appropriate values for your network: the IP address should be the address you assigned to the BMC NIC and the username should be the name you gave to user ID 3.
# ipmitool -I lanplus -H 192.168.1.6 -U dhess sdr
You should be prompted for a password (type your IPMI password, not your UNIX password), and after a short delay you should see output something like the following (the exact values and sensors shown will vary a bit from system to system, depending on your configuration and operating environment):
BB +1.8V SM | 1.82 Volts | ok BB +3.3V | 3.32 Volts | ok BB +3.3V STBY | 3.22 Volts | ok BB +5.0V | 5.11 Volts | ok Processor Vcc | 1.02 Volts | ok BB Ambient Temp | 45 degrees C | ok Chassis Fan 1 | 1890 RPM | ok Chassis Fan 2 | 1190 RPM | ok Chassis Fan 4 | 1190 RPM | ok Chassis Fan 5 | 1190 RPM | ok P1 Therm Margin | -61 degrees C | ok Power Unit | 0x00 | ok IPMI Watchdog | 0x00 | ok Processor Status | 0x00 | ok
If you don't get similar results, try the sdr command directly on the S3200SHV system, by running ipmitool sdr. If that works, check your network, authentication and password settings, and then try the lanplus version of the command again. If the local command doesn't work, you might have forgotten to load one or more IPMI kernel modules. Adding the -vv verbosity flag to the ipmitool command will produce lots of debugging information, most of which will probably be indecipherable, but some of it may be useful in determining the problem.
Another useful command is the power command. ipmitool -I lanplus -H 192.168.1.6 -U dhess power status returns the power state of your system (off or on). The power command also allows you to soft- or hard-reset the machine, power-cycle it, or turn it on or off remotely; as long as power is being supplied to the motherboard, the BMC is always operational and reachable over the network via the IPMI interface, even if the host system is off or hung.
At this point, you might also want to test one or more of these commands using the lanplus interface and the root user. It might also be prudent to make sure the test2 and test3 users are unable to run any IPMI commands over the network. Also, omitting the -U option will cause ipmitool to use the default account (user ID 1). If you've configured your system as specified in this HOWTO, none of these commands should work when -U is omitted. (If you do this, you should still be prompted for a password, but because we haven't assigned one to the default account, and also because we've disabled it, no password that you supply should allow the command to succeed.)
Remember that commands you run directly on the hardware (i.e., without specifying -I lanplus) bypass the IPMI user authentication layer and assume that if you've got sufficient privileges on the hardware to run the ipmitool command, you're privileged enough to do whatever you like with IPMI. The -U option is only applicable when using -I lanplus (or -I lan, which is like lanplus except with no encryption).
IPMI 2.0 SOL
Now it's time to configure IPMI serial-over-LAN so that we can access the machine's serial console remotely. Because we can't SSH or telnet to the S3200SHV BMC directly, the only way to view the machine's serial console over the network is via ipmitool itself (or possibly with other IPMI tools such as various Microsoft or Intel proprietary management applications, which aren't covered here).
This part took quite awhile to figure out, and the SOL support on this system is quite particular (as of the February 2009 firmware update, anyway), so I suggest that you set things up on your S3200SHV system exactly as I've shown here the first time. Once you've got it working, you can fine tune it for your particular tastes or requirements.
Testing SOL also requires another physical machine capable of running ipmitool, since the whole point is to be able to observe and control the system from power-on all the way to GNU/Linux login. From this point forward, whenever I show an ipmitool command that doesn't use the -I lanplus option, you should run that command on the S3200SHV system whose serial console you're trying to activate, and whenever I show an ipmitool command that does use the -I lanplus option, you should run that command on the machine from which you're trying to observe and control the S3200SHV system's serial console.
Which serial port to use?
The S3200SHV provides two serial ports. In BIOS terms these are called "serial port A" and "serial port B". In GNU/Linux they're named /dev/ttyS0 and /dev/ttyS1, respectively. One of these serial ports has a physical connector on the back of the machine, just above the VGA connector. I don't have the motherboard documentation handy as I write this, so I don't know which serial port that is, but it doesn't matter, as we're not going to use it. The S3200SHV BMC is capable of capturing both the system's BIOS-initiated VGA output and keyboard input, and serial port A's input and output, and redirecting those to IPMI channel 1 as IPMI 2.0 serial-over-LAN (SOL) protocol traffic. Any tool capable of handling that protocol, such as ipmitool, can then interact with the system.
The S3200SHV BIOS allows you to redirect the "console" (i.e., VGA output generated via BIOS calls, and keyboard input read via BIOS calls) to either serial port A or B. When the BIOS is set to redirect the VGA+keyboard console to either serial port A or B, the BMC also captures the VGA+keyboard console output and makes it available via IPMI SOL, regardless of which serial port you've chosen for redirection of the VGA+keyboard console. As long as you configure the BIOS to redirect the console to a serial port, you can observe all BIOS VGA output and control all BIOS keyboard input via IPMI SOL, which means that you can see the BIOS POST output, control the BIOS setup process and even observe and control GRUB from a remote machine running IPMI SOL. However, as soon as the Linux kernel begins to boot, the kernel stops using the BIOS for input and output, and you'll see nothing from that point on unless you tell the Linux kernel to use the serial console.
Here's where things get confusing. On the S3200SHV, in order to see serial console output from Linux over IPMI SOL, or to send serial console input to Linux over IPMI SOL, you must tell Linux to use the first serial port, i.e., the port that Linux calls ttyS0. If you try to use ttyS1, you will see nothing once Linux begins to boot and you'll be unable to send characters to your remote GNU/Linux system, even if you've configured the S3200SHV BIOS to use serial port B (i.e., ttyS1) for console redirection. This caused me no end of confusion and trying to change serial port bit rates, hardware handshakes and so on, as originally I had configured ttyS1 as my serial console in GNU/Linux and told the BIOS to use serial port B, and everything appeared to be working fine until Linux began to boot, at which point the SOL output mysteriously stopped.
As far as I can tell, the BMC will redirect BIOS VGA output and BIOS keyboard input to the IPMI SOL interface whenever you configure the BIOS to redirect the system console to either serial port; but it can only redirect traffic generated at the actual serial port to IPMI SOL when the operating system is using serial port A. In other words, there should probably be a separate BIOS setting that tells the BMC whether or not to redirect the system console to IPMI SOL, that is completely orthogonal to redirecting the system console output to either of the serial ports, since it appears to be a completely different mechanism. There does not appear to be a path from serial port B to IPMI SOL, and the fact that configuring the BIOS to redirect the system console to serial port B also happens to activate system console redirection to IPMI SOL is a red herring.
To make a long story short, save yourself the hassle and frustration and configure the BIOS to redirect the system console to serial port A, and configure GNU/Linux to run a serial console on ttyS0. Ignore serial port B/ttyS1, it's useless from an IPMI SOL standpoint.
Note that everything I've said here about using serial port A and not serial port B applies only to the Intel S3200SHV motherboard (and, strictly speaking, only the S3200SHV with the February 2009 firmware update, as far as I know, because that's the only configuration I've used as I write this). Other Intel S3200-series boards may have the same quirky behavior, but I can't verify that. I do know from anecdotal evidence that other Intel-based OEM systems sometimes require that you use one particular serial port for IPMI SOL, though it's not always port A. Anyway, the point is, whatever I've said here about this nasty bit of confusion, don't assume any of it applies to your motherboard unless it's an S3200SHV.
Choosing a bitrate
The S3200SHV console redirection is pretty slow compared to other systems I've used, e.g., the Sun Sun Fire X2200 M2, but it appears to throttle the output when it's having problems keeping up; I have yet to receive any messages about dropped characters. To make things tolerable, I've cranked up the bitrates on my system to the max, 115200bps. I suggest you start with this setting, and if you have problems, drop down to 19200bps and work your way back up to a tolerable bitrate. (See the problems section below.)
Technically, I believe that you can choose different bitrates for console redirection and Linux serial console support (I think I had that working at one point), but there's really no reason to do so and it only complicates matters, so in this HOWTO I assume you're using the same rate for everything.
Setting the IPMI SOL bitrate
On the S3200SHV system, set the IPMI SOL bitrates. There are two: one for the BIOS console redirection (the "volatile bit rate") and one for the OS serial port redirection (the "non-volatile bit rate").
# ipmitool sol set volatile-bit-rate 115.2
# ipmitool sol set non-volatile-bit-rate 115.2
(If you run into problems and need to change the bitrate later, the other supported settings on the S3200SHV are 9.6, 19.2, 38.4 and 57.6. There's also a "serial" setting that appears to use whatever the BIOS setting or OS uses, but I haven't verified that.)
Force authentication and encryption
Require that authentication be used with SOL, so that random users can't control your machine.
# ipmitool sol set force-authentication true
Also, unless your network is completely trusted, it's probably a good idea to force encryption for the SOL traffic so that, for example, nobody can easily snoop your GNU/Linux password if you log in via the serial console:
# ipmitool sol set force-encryption true
Strictly speaking, I believe that IPMI 2.0 SOL requires (or at least defaults to) encryption, but this will eliminate any risk of a misconfigured or buggy client implementation.
Set the privilege level to "admin"
Require administrator-level privilege for all IPMI user IDs that attempt to use IPMI SOL:
# ipmitool sol set privilege-level admin
Since we already gave our root and user accounts admin privileges, this will still allow us to connect with either of these accounts.
Enable the SOL payload for both of our users
Tell the BMC to send data on this channel for our two users (IDs 2 and 3, i.e., root and the user account).
# ipmitool sol payload enable 1 2
# ipmitool sol payload enable 1 3
Enable SOL
Now that it's configured, enable SOL support:
# ipmitool sol set enabled true
Test SOL
Make sure your settings are correct. When you run the following command now, you should see similar results on your own system:
# ipmitool sol info Set in progress : set-complete Enabled : true Force Encryption : true Force Authentication : true Privilege Level : ADMINISTRATOR Character Accumulate Level (ms) : 60 (*) Character Send Threshold : 96 (*) Retry Count : 7 (*) Retry Interval (ms) : 500 (*) Volatile Bit Rate (kbps) : 115.2 Non-Volatile Bit Rate (kbps) : 115.2 Payload Channel : 1 (0x01) Payload Port : 623
The outputs marked with a * may be slightly different on your system, but as long as they're close to these values they should be fine; I've never had to change them from their default settings. All of the other settings should look identical to the ones shown here. If that's the case, test your SOL connection with a loopback test (do this from the remote system running the ipmitool client):
# ipmitool -I lanplus -H 192.168.1.6 -U dhess sol looptest 20
You'll be prompted for your password. If you entered it correctly and your SOL support is configured properly, you'll see 20 messages like this:
remain loop test counter: 20 [SOL Session operational. Use ~? for help] remain loop test counter: 19 [SOL Session operational. Use ~? for help] ... remain loop test counter: 1 [SOL Session operational. Use ~? for help]
That's about all you can do with SOL right now because we haven't enabled serial port redirection yet, but the IPMI part is ready to go.
Configure /etc/inittab
We want to run a getty (or agetty, or mgetty, or whatever flavor of getty you use on your system) on ttyS0 so that we can log into the system once serial port redirection is working. On Debian systems, this is configured in /etc/inittab. Ubuntu systems use a different mechanism, and other GNU/Linux distros might use something else altogether. I'm only showing the Debian configuration here. If you're running something else, Google for "serial console xxx" where "xxx" is your distribution of choice and follow the directions for making *getty work with a serial console on ttyS0 on your distro.
On Debian systems, there are two commented-out lines in /etc/inittab that refer to serial consoles. One uses ttyS0 and the other uses ttyS1. Uncomment the one that uses ttyS0. It should be the right thing for whichever *getty your system is using. Here's what that line looks like (uncommented) on my system:
T0:23:respawn:/sbin/getty -h -L ttyS0 115200 vt100
It's a bit different than the original, commented-out line. Specifically, the data rate has been changed to 115200 and the -h flag tells getty to use hardware handshaking, which we use to make things a bit more controllable. I believe the flag is the same for agetty, but I'm not sure about other *gettys; see the man page.
Optional: add ttyS0 to /etc/securetty
If you want to login on the SOL serial console as root, make sure ttyS0 appears somewhere in /etc/securetty. This is not required by or specific to SOL, it's just a useful setting for serial consoles in general.
Configure GRUB and the Linux kernel
There are two more configuration steps in GNU/Linux before we turn to the BIOS settings. We need to tell the Linux kernel to use a serial console, and we possibly need to tell GRUB to give us a little more time to override the default boot sequence, since the Debian default of 3 seconds is a little too fast for the slow S3200SHV console redirection to keep up.
Both of these settings are made in the GRUB /boot/grub/menu.lst configuration file. The directions I'm giving here are for Debian distros. It'll probably work as-is on Ubuntu distros, as well. For other distros, things may be a bit different, but if you've gotten this far you can probably figure it out for yourself.
(If you're using a different bootloader, e.g., LILO, sorry, but you're on your own.)
In /boot/grub/menu.lst, change the timeout parameter to something reasonable, if it's not already. I set mine to 10 (seconds):
timeout 10
Next, find the line in the file that looks something like this:
# kopt=root=/dev/something ro ...
Yes, it's commented out (starts with a # character), and it's meant to be, as the Debian GRUB system uses it as a pre-processor marker to fill in the command line for all of the kernel targets automatically. When you modify it, make sure it stays commented out with a single # character.
All we're going to do is add console=tty0 console=ttyS0,115200n81r to the end of this line. Keep everything else on the line the same, unless you're already using a serial console setting, in which case you should remove it and replace it with this one. After making the change, my line looks like this, but yours will undoubtedly look slightly different (probably the root device part):
# kopt=root=/dev/mapper/main-sid--root ro console=tty0 console=ttyS0,115200n81r
Finally, if you have any splash screens configured, either for GRUB or for the kernel, remove them. The serial console only works with text output and splash screens may (will, in the case of GRUB) squash the serial output. I don't use any of these, so I can't tell you exactly what to change, but the GRUB manual and your distro's documentation are your friend.
That's it. Save the file and run Debian's update-grub tool to auto-generate the kernel command lines. If everything went well, you'll see something like this:
Searching for GRUB installation directory ... found: /boot/grub Searching for default file ... found: /boot/grub/default Testing for an existing GRUB menu.lst file ... found: /boot/grub/menu.lst Searching for splash image ... none found, skipping ... Found kernel: /vmlinuz-2.6.28-1-amd64 Found kernel: /vmlinuz-2.6.26-1-amd64 Found kernel: /memtest86+.bin Updating /boot/grub/menu.lst ... done
You may be wondering why we didn't tell GRUB to use a serial console, like we did with the Linux kernel. Theoretically, it should work: I have a Sun X2200 M2 system whose GRUB config is set up to take over the serial console from the BIOS during the boot process, and it works well. However, my experience so far with the S3200SHV is that GRUB and the system's BIOS do not handle the "change-over" correctly. When I configure GRUB on my S3200SHV system to use serial port A (GRUB calls it "unit 0"), I get a few lines of output from GRUB and then nothing. Apparently, this problem is not unique to the S3200SHV, as there are other reports of this same issue with different IPMI implementations.
Unlike the Linux kernel, though, GRUB uses the BIOS for input and output when it's not told to use a serial console instead, so the BIOS console redirection handles GRUB's input and output over SOL just fine, and there's no need for GRUB to use the serial console at all. It might be interesting to figure out if GRUB can be made to work properly on an S3200SHV with its own serial console support, but there's no compelling reason to do so, and I don't particularly care since the BIOS console redirection works fine as-is. (The corollary here is that if you currently have GRUB configured to use a serial console, disable that.)
Configure the BIOS console redirection
Now reboot your S3200SHV system, hit F2 during POST to enter the BIOS setup, and configure serial port console redirection as follows:
- Serial port A
- Flow control: RTS/CTS
- Rate: 115k
- Terminal type: VT100
- Legacy OS redirection: disabled
Test
You're ready to test things now. On the remote/client machine, run the following IPMI command, which, after prompting for your password, should connect you to the S3200SHV system's SOL port without any complaints (again, substituting your username and host IP/name where appropriate):
# ipmitool -I lanplus -H 192.168.1.6 -U dhess sol activate
Note that you will not yet be able to see or control the S3200SHV system's BIOS setup, but that'll change in just a few seconds.
On the S3200SHV system again, save the BIOS settings and reboot the machine. If everything is working properly, you'll see the BIOS and peripheral POST messages show up after a short pause in your IPMI SOL client session. During the POST process, in the IPMI SOL client session, type ESC followed by 2 (which emulates the F2 keypress over a serial port) to enter the BIOS setup screen. You should be able to use the arrow keys, Enter, +/-, etc. to navigate the BIOS setup and manipulate settings. Use the BIOS setup to reboot and let it proceed through POST uninterrupted this time. Soon you'll see your GRUB menu. Hit ESC again and make sure you can navigate in GRUB with the arrow keys, Enter, etc. Boot your Linux kernel of choice and you should see the Linux boot messages fly by, eventually terminating with a login prompt. You should be able to log into the machine from this prompt.
If all of that worked, you're good to go. Type ~. to leave your IPMI SOL client session. You might also want to make sure that you can boot single-user or run a memtest from GRUB, but all of those things should work fine if you've gotten this far. Otherwise, keep reading.
If you have problems
- If the output sort-of works but looks garbled, or lines run over top of each other, make sure the dimensions of the window in which you're running the IPMI SOL client are 80x24 or 80x25. Some terminal programs can't deal with the console-imposed dimensions when they're different than the window's. Non-VT100 special characters in the BIOS setup, e.g., dithered and solid blocks, won't show up properly with plain VT100 emulation, but it's just cosmetic; everything still works fine. You can try other emulations in the BIOS console redirection settings if you're feeling adventurous.
- If everything works properly but you don't eventually get a login prompt from GNU/Linux, you probably forgot to run *getty on ttyS0; or you might have misconfigured either the *getty bitrate or the kernel serial console bitrate, in which case you'll probably see messages in your syslog saying something about *getty dying too quickly.
- If you see BIOS/GRUB output, but only a few lines of Linux kernel boot messages before everything stops, you're probably using serial port B/ttyS1. Use serial port A/ttyS0. Alternatively, you might have forgotten to put the Linux serial console settings in your GRUB config.
- If GRUB prints only a few lines and then appears to hang, you've probably got GRUB configured to output to the serial port. Disable this in GRUB (but leave the kernel command-line serial port config) and let the BIOS console redirection take care of it.
- If you get BIOS output but no GRUB output, and then eventually see Linux boot messages, you're probably using a GRUB splash-screen. Disable it and use the text menus.
- If you only begin to see output once the Linux kernel starts booting, but no BIOS or GRUB output, you probably aren't redirecting the console in the BIOS, or you might have set the BIOS console redirection bitrate to a different speed than the IPMI volatile bitrate.
- If you can't login as root on the serial console, you forgot to add ttyS0 to /etc/securetty.
- I've seen some reports (not specific to the S3200SHV) that IPMI SOL connections which share a physical port with an Ethernet interface used by the operating system will sometimes experience dropouts due to the BMC/chipset prioritizing system traffic at the expense of SOL traffic. In my case the SOL port is dedicated to IPMI, and no OS traffic goes over that wire, so I haven't experienced this problem, but if you do you might try reducing the bitrate to 56kbps or even 19.2kbps. If you change the bitrate setting, make sure you do it in all four places: IPMI (both volatile and non-volatile bitrate settings), BIOS console redirection, kernel command line and *getty in /etc/inittab (or wherever your distro puts that stuff).
- Occasionally while testing IPMI SOL support I've managed to get the connection wedged and unresponsive. A combination of CTRL-C and ~. eventually works and causes ipmitool to drop the connection, but subsequent connections exhibit the same problem. In these cases, I simply issue an ipmitool bmc reset cold command to hard-reset the BMC. While the BMC is rebooting, you'll be unable to communicate with it via IPMI (whether it's SOL, dumping sensor data or whatever), but it should become responsive again after 10-15s.
Final words
This tutorial from IBM was particularly helpful to me while I was configuring my S3200SHV system. The instructions given there are somewhat particular to IBM's own IPMI implementations, but most of the information is useful in general.
