This minilab is a tiny operating system that will demonstrate the concepts behind virtual memory.
weensyos3.tar.gz |
Source code for WeensyOS 3.0, the Memory OS |
You will electronically hand in code and a small writeup.
The problem set code, weensyos3.tar.gz
, unpacks into a
directory called weensyos3
.
You'll modify the code in this directory, and add a text file with your
answers to the numbered questions.
When you're done, run the command gmake tarball
.
This should create a file named
weensyos3-yourusername.tar.gz
, which you will submit to
CourseWeb.
Answers to the numbered questions should be in a file named
answers.txt
, answers.html
, or
answers.pdf
.
Text files are strongly preferred.
To review:
weensyos3.tar.gz
.weensyos3
directory.answers.txt
file (or answers.html
or answers.pdf
) in that weensyos3
directory.gmake tarball
from the weensyos3
directory. This will create a file named weensyos3-yourusername.tar.gz
.weensyos3-yourusername.tar.gz
file to CourseWeb.This course's minilabs are real, tiny operating systems designed to give you hands-on experience with operating system concepts. You could take the disk image file this minilab builds, write it to your Intel-compatible laptop's hard drive, and boot up your operating system directly if you wanted! However, it's much easier to work with a virtual machine or PC emulator.
An emulator mimics, or emulates, the behavior of a full hardware platform. A PC emulator acts like a Pentium-class PC: it emulates the execution of Intel x86 instructions, and the behavior of other PC hardware. For example, it can treat a normal file in your home directory as an emulated hard disk; when the program inside the emulator reads a sector from the disk, the emulator reads 512 bytes from the file. PC emulators are much slower than real hardware, since they do all of the regular CPU's job in software -- not to mention the disk controller's job, the console's job, and so forth. However, debugging with an emulator is a lot easier, and you can't screw up your machine!
We've used two PC emulators. The Bochs emulator has pretty nice debugging support. The QEMU package is fast and sleek, but it might be too fast for some of our purposes. If you work on your own machine, try QEMU. If you're interested in working from home, you can download the source for QEMU and/or Bochs and install your own copy using these instructions. Precompiled binaries for Windows and Mac OS X are available too.
You will also need a copy of GCC that compiles code for an x86 ELF target. ELF, or Executable and Linkable Format, is a particular format for storing machine language programs on disk. Recent Linux PCs have the right compiler already set up. However, if you want to work on other platforms, or on Windows, you'll need a cross-compiler: a version of GCC that runs on your machine, but generates binaries for WeensyOS.
We've set up all the required tools on the machines in the Linux lab, and the Solaris machines on SEASnet. In the Linux lab, no special setup is required. On SEASnet, you need to set your environment to use our tools.
Read the minilab tools page and set up your environment appropriately.
Now that you've got all the software set up (or you've just decided to use the Linux lab), it's time to run MemOS.
Download and unpack the source for weensyos3, then boot your emulated computer running MemOS with the gmake run-memos command.
% gzcat weensyos3.tar.gz | tar xf - % ls weensyos3 COPYRIGHT elf.h memos-3.c memos-kern.h memos.h x86mem.h GNUmakefile lib.c memos-4.c memos-loader.c mergedep.pl x86struct.h bootstart.S lib.h memos-app.h memos-pages.c mkbootdisk.c x86sync.h conf memos-1.c memos-boot.c memos-trap.S types.h console.c memos-2.c memos-kern.c memos-x86.c x86.h % gmake run-memos + hostcc mkbootdisk.c + as bootstart.S + cc memos-boot.c ... + mk memos.img + bochs memos.img ======================================================================== Bochs x86 Emulator 2.2.1 Build from CVS snapshot on July 8, 2005 ======================================================================== 00000000000i[ ] reading configuration from .bochsrc Next at t=0 (0) [0xfffffff0] f000:fff0 (unk. ctxt): jmp far f000:e05b ; ea5be000f0 <bochs:1>
The Bochs emulator stops in case you want to enter a breakpoint. You shouldn't need any breakpoints for this lab, so at the <bochs:1> prompt, just enter c to continue. In your Bochs window, you should see something like this:
(This image loops forever, but when you run MemOS, the bars will move to the right and stay there.)
If your bochs runs too slowly (the bars of 1-4's move
slowly), edit the memos.h
file and reduce the
ALLOC_SPEED
constant.
Here's what's going on.
The marching rows of 1's, 2's, 3's, and 4's show how fast the heap spaces for processes 1, 2, 3, and 4 are allocated. Here are two labeled memory diagrams, showing what the characters mean and how memory is arranged.
Read and understand the process code in
memos-1.c
. This code is used for all 4 processes.
In the rest of this lab, you will gradually switch the MemOS to use
virtual memory! This requires that we set up different address spaces, one
for each process, and change the page allocation function,
page_alloc
, to allocate a physical page and map it at
the required address, rather than simply allocating the physical page with
the right address. First, we'll simply set up a single virtual address
space that matches physical memory.
start
function in
memos-kern.c
to initialize virtual memory. After the call to
memory_labels_init()
, add the following two lines:
initial_pgdir = pgdir_new(); paged_virtual_memory_init(initial_pgdir);The
pgdir_new()
function creates and returns an initial
address space (page directory) with mappings for all of physical memory.
The paged_virtual_memory_init
function turns on paged virtual
memory. Its argument is installed as the initial page directory.page_alloc
function in memos-pages.c
.page_alloc
function to add a user-accessible page mapping for the physical page it
allocates. You'll need to call the page_map_user
function.If you run gmake run-memos
at this point, it
should work, and produce similar output as it did before. But let's take a
look at each process's virtual address space as well.
trap
function in
memos-kern.c
, after the call to
show_physical_memory()
, add the following line:
show_virtual_memory_flipper();
When you run gmake run-memos
now (and enter
c
at the <bochs:1>
prompt), you
should see something like this.
There are several changes from the initial display.
MEMORY_FLIPPER_SPEED
constant in
memos.h
.)MemOS is now using virtual memory, but we're getting no benefit from it! Next, we'll update the kernel code to grant each process its own address space. This will isolate the processes from one another; no process will be able to alter another process's memory or data.
There are three changes required.
process_load()
.Switching to an address space uses the lcr3()
function, a
simple wrapper for the lcr3
instruction. The 386+'s
cr3
register holds the physical address of the active page
directory; kernel code may change the value of this register, and thus
switch to a new address space, by calling lcr3()
with the
relevant page directory address.
start()
to give each process its own independent
address space. Use the lcr3()
function to switch to each
process's address space before calling process_load()
. Note
that the pgdir_new()
function creates a new, independent
address space and returns the physical address of a page directory.run()
function in memos-kern.c
to load the process's address space
before running that process.Now when you run gmake run-memos
, you should
see something like this. (Note the greater number of "P" pages.)
Now each process's address space only contains that process's
pages. Furthermore, since processes run in user mode (at protection
level 3), the processor will not allow them to execute the
lcr3
and lcr0
instructions that would install a
new address space or turn off virtual memory. This means the processes are
memory isolated: no process can affect another process's code or
data.
MemOS now maintains an isolated address space for each process. However, it still allocates memory based on physical address. Given virtual memory, we can allocate memory far more flexibly. In this section, you will change MemOS to allocate memory independently of physical address, and to give each process a much larger heap space, allowing any process to potentially allocate most available physical memory.
Exercise: Edit the page_alloc
function so that it can use any free page, rather than just the
physical page with the given address. If no free page is available,
page_alloc
should return -1.
Exercise: Initialize each process's stack to start at virtual address 0x300000, the top of MemOS's virtual address space.
Now when you run gmake run-memos
, you should
see something like this.
You have now built a virtual memory system that, in its essentials, is a lot like the virtual memory system in any modern operating system.
Notice that once physical memory is exhausted, Process 4 has used roughly four times as much memory as Process 1. This is because each process's address space is big enough to fit any available physical memory, so physical memory runs out before any process's address space does. Now that we have eliminated the requirement that processes fit in contiguous regions of physical memory, each process can allocate more memory than before!
Question 2: Process 1's code and global data used to be allocated in physical page numbers 0x100 and 0x101 (physical addresses 0x100000 and 0x101000). Which physical page numbers are now used for Process 1's code and global data (i.e., not including its heap or stack)?
Question 3: Why haven't the physical page numbers allocated for kernel code and data changed? Refer to the state of the machine at boot time (while the boot loader, which loads the kernel from disk, is running).
Extra Credit Exercise: It's not necessarily fair that Process 4 gets to use four times as much memory as Process 1 -- especially since Process 1 can get less memory than in the original physically-allocated design. Implement a quota system so that each process is guaranteed that it will be able to allocate at least 1/4 MB (64 pages) of heap space. Any space beyond that 1/4 MB should be allocated first-come, first-served. You will need to keep track of how much physical memory remains available, and how much physical memory each process has allocated.
This completes the minilab. Make sure you have answered all three
numbered questions in your answers.txt
file.