[Kernel, courtesy IowaFarmer.com CornCam]

CS 261 Research Topics in Operating Systems, Fall 2011

Tools

You'll use two sets of tools in this class: an x86 emulator, QEMU, for running your kernel; and a compiler toolchain, including assembler, linker, C compiler, and debugger, for compiling and testing your kernel. Here's the information you'll need to download and install your own copies. This class assumes familiarity with Unix commands throughout.

Compiler Toolchain

A "compiler toolchain" is the set of programs, including C and C++ compilers, assemblers, and linkers, that turn code into executable binaries. You'll need a compiler toolchain that generates code for 32-bit Intel architectures ("x86" architectures) in the ELF binary format. We recommend you configure one of your machines (such as a laptop) with a working toolchain, but if you cannot do this, a class compute server could be made available shortly.

Test Your Compiler Toolchain

Modern Linux and BSD UNIX distributions already provide a toolchain suitable for CS 261 Research Topics in test your distribution, try the following commands:

% objdump -i

The second line should say elf32-i386.

% c++ -m32 -print-libgcc-file-name

The command should print something like /usr/lib/gcc/i486-linux-gnu/version/libgcc.a or /usr/lib/gcc/x86_64-linux-gnu/version/32/libgcc.a.

If both these commands succeed, you're all set, and don't need to compile your own toolchain.

If the c++ command fails, you may need to install a development environment. On Ubuntu Linux, try this:

% sudo apt-get install build-essential

On 64-bit machines, you may need to install a 32-bit support library. The symptom is that linking fails with error messages like "__udivdi3 not found" and "__muldi3 not found". On Ubuntu Linux, try this to fix the problem:

% sudo apt-get install gcc-multilib

Using a Virtual Machine

Otherwise, the easiest way to get a compatible toolchain is to install a modern Linux distribution on your computer. With platform virtualization, Linux can cohabitate with your normal computing environment. Installing a Linux virtual machine is a two step process. First, you download the virtualization platform.

VirtualBox is a little slower and less flexible, but free!

Once the virtualization platform is installed, download a boot disk image for the Linux distribution of your choice.

This will download a file named something like ubuntu-10.04.1-desktop-i386.iso. Start up your virtualization platform and create a new (32-bit) virtual machine. Use the downloaded Ubuntu image as a boot disk; the procedure differs among VMs but is pretty simple. Type objdump -i, as above, to verify that your toolchain is now set up. You will do your work inside the VM.

Building Your Own Toolchain

This will take longer to set up, but give slightly better performance than a virtual machine, and lets you work in your own familiar environment. (Mac/Unix Only)

We assume that you are installing the toolchain into /usr/local. You will need a fair amount of disk space to compile the tools (around 1GiB). If you don't have that much space, delete each directory after its make install step.

Download the following packages:

(You may also use newer versions of these packages.) Unpack and build the packages. The green bold text shows you how to install into /usr/local, which is what we recommend. To install into a different directory, $PFX, click herenote the differences in lighter type (hide). If you have problems, see below.

export PATH=$PFX/bin:$PATH
export LD_LIBRARY_PATH=$PFX/lib:$LD_LIBRARY_PATH      # On Mac OS X, use DYLD_LIBRARY_PATH


tar xjf gmp-5.0.2.tar.bz2
cd gmp-5.0.2
./configure --prefix=/usr/local--prefix=$PFX
make
make install             # This step may require privilege (sudo make install)
cd ..

tar xjf mpfr-3.0.1.tar.bz2
cd mpfr-3.0.1
./configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX
make
make install             # This step may require privilege (sudo make install)
cd ..

tar xzf mpc-0.9.tar.gz
cd mpc-0.9
./configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX
make
make install             # This step may require privilege (sudo make install)
cd ..


tar xjf binutils-2.21.1.tar.bz2
cd binutils-2.21.1
./configure --prefix=/usr/local--prefix=$PFX --target=i386-jos-elf --disable-werror
make
make install             # This step may require privilege (sudo make install)
cd ..

i386-jos-elf-objdump -i
# Should produce output like:
# BFD header file version (GNU Binutils) 2.21.1
# elf32-i386
#  (header little endian, data little endian)
#   i386...


tar xjf gcc-core-4.6.1.tar.bz2
tar xjf gcc-g++-4.6.1.tar.bz2
cd gcc-4.6.1
mkdir build              # GCC will not compile correctly unless you build in a separate directory
cd build
../configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX --with-mpc=$PFX \
    --target=i386-jos-elf --disable-werror \
    --disable-libssp --disable-libmudflap --with-newlib \
    --without-headers --enable-languages=c,c++

MAC OS X 10.7 "LION" NOTE: The default clang compiler on Mac OS X 10.7
cannot build a working version of GCC. Use the following configure
line to work around the problem:
../configure --prefix=/usr/local--prefix=$PFX --with-gmp=$PFX --with-mpfr=$PFX --with-mpc=$PFX \
    --target=i386-jos-elf --disable-werror \
    --disable-libssp --disable-libmudflap --with-newlib \
    --without-headers --enable-languages=c,c++ \
    CC=/usr/bin/gcc-4.2 CPP=/usr/bin/cpp-4.2 \
    CXX=/usr/bin/g++-4.2 LD=/usr/bin/gcc-4.2

make all-gcc
make install-gcc         # This step may require privilege (sudo make install-gcc)
make all-target-libgcc
make install-target-libgcc     # This step may require privilege (sudo make install-target-libgcc)
cd ../..

i386-jos-elf-gcc -v
# Should produce output like:
# Using built-in specs.
# COLLECT_GCC=i386-jos-elf-gcc
# COLLECT_LTO_WRAPPER=/usr/local/libexec/gcc/i386-jos-elf/4.6.1/lto-wrapper
# Target: i386-jos-elf


tar xjf gdb-7.3.1.tar.bz2
cd gdb-7.3.1
./configure --prefix=/usr/local--prefix=$PFX --target=i386-jos-elf --program-prefix=i386-jos-elf- \
    --disable-werror
make all
make install             # This step may require privilege (sudo make install)
cd ..

Troubleshooting

Q. I can't run make install because I don't have root permission on this machine.
A. Our instructions assume you are installing into the /usr/local directory. However, this may not be allowed in your environment. If you can only install code into your home directory, that's OK. In the instructions above, replace --prefix=/usr/local with --prefix=$HOME (and click here to update the instructions further). You will also need to change your PATH and LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (Mac OS X) environment variables, to inform your shell where to find the tools. For example: 
export PATH=$HOME/bin:$PATH
export LD_LIBRARY_PATH=$HOME/lib:$LD_LIBRARY_PATH
Enter these lines in your ~/.bashrc file so you don't need to type them every time you log in.
Q. My build fails with an inscrutable message about "library not found".
A. You need to set your LD_LIBRARY_PATH (Linux) or DYLD_LIBRARY_PATH (Mac OS X) environment variable to include the libraries you built; see above. The environment variable must include the PREFIX/lib directory (for instance, /usr/local/lib).

QEMU Emulator

We provide a patched version of the QEMU emulator with improved support for debugging. (Credit to Kohler and Austin Clements and others at MIT 6.828.) We recommend you compile QEMU from this source; if you use the version that came installed on your Linux distribution, you will not be able to complete all the debugging exercises.

First, download the source:

On a Linux machine, you will need to install the SDL graphics library to build QEMU. On Ubuntu or Debian:

sudo aptitude install libsdl1.2-dev

Then unpack, compile, and install QEMU as follows.

tar xjf qemu-0.15.0-jos.tar.bz2
cd qemu-0.15.0-jos

To configure on Mac OS X:
   ./configure --prefix=/usr/local --disable-sdl --enable-cocoa \
      --disable-docs --target-list="i386-softmmu x86_64-softmmu"
   
MAC OS X 10.7 "LION" NOTE: The default clang compiler on Mac OS X 10.7
cannot build a working version of QEMU. Use the following configure
line to work around the problem:
   ./configure --prefix=/usr/local --disable-sdl --enable-cocoa \
      --disable-docs --target-list="i386-softmmu x86_64-softmmu" \
      --cc=/usr/bin/gcc-4.2 --host-cc=/usr/bin/gcc-4.2

To configure on Linux:
   ./configure --prefix=/usr/local --target-list="i386-softmmu x86_64-softmmu"

make
make install             # This step may require privilege (sudo make install)

The QEMU monitor is accessible by pressing Ctrl-Alt-2 inside the QEMU window. Return to normal display with Ctrl-Alt-1. QEMU tends to take control of your mouse. If you can't find a mouse pointer, check the QEMU title bar for text like "Press Ctrl-Alt to exit grab" and, if so, press Ctrl-Alt to regain control.

Source Code Control

You will do your work using a version control system. This will help you maintain two parallel branches—one with your changes, and one with the labs as we release them. We will support the Git distributed version control system, which may be familiar to you from other classes or Github. You don't need to compile CVS or Git yourself. The packaged version that comes with your OS will be good enough. You just need to install it.

  • Ubuntu/Debian Linux: Run sudo aptitude install git-core
  • Mac OS X: Install MacPorts, then run sudo port install git-core
  • Windows: Versions are available; contact the instructor if you need help

Using Cygwin on a Windows Machine

If you have a Windows machine, but cannot run a Linux virtual machine or install a dual-boot configuration, the easiest way to work on labs from home will be to use a class compute server coupled with VNC remote display. Let us know as soon as possible if you want this configuration to work.

Nevertheless, it should be possible to execute many of the above steps and get a functional build environment running on Windows using Cygwin. Here's how to install Cygwin on your Windows machine.

  1. Install Cygwin/X. Follow this procedure. Make sure you select the "xorg-x11-base" package and the "openssh" package. Also select the "flex" and "bison" packages from the development header. This sets up a Unix-like environment on your Windows machine, including an X server.
  2. Start the Cygwin X server by running startxwin.bat. "Run /usr/X11R6/bin/startxwin.bat by double-clicking it in Windows Explorer." http://x.cygwin.com/docs/ug/using.html

You should also be able to compile the class tools and the labs on your Windows machine, once Cygwin has been installed. Let us know if you have success with this.

Back to CS 261 Research Topics in Operating Systems