Ranting Jacaré

A 21st century sapien and his random thoughts on binary encoded partial scripts.

© 2021. All rights reserved.

Overview on MIT-6.828 lab JOS

11 Oct 2020
OS
 
lab
 
make
 

JOS is the operating system built in MIT OS-6.828 lab. A quick peek into its internals.

Overview

JOS is a 32-bit educational-purposed Operating System involved in MIT OS-6.828 labs. The course instructor provides the skeleton code and the students finish the implementations. There are a few key differences between JOS and XV6, the other teaching OS involved in the course, which is a ANSI C re-implementation of UNIX V6 for x86 processors. The differences are:

  • XV6 is monolithic while JOS is exokernel;
  • XV6 is modelled on an existing version of UNIX, while JOS is purely for teaching the course;
  • XV6 is used as a reference, and JOS is the lab kit for learning.

The lab with JOS is mainly dependent on QEMU and GNU tools (gcc for compiling, dd for creating the image, make for build process managing, gdb for debuging, etc).

QEMU

QEMU is the hardware emulator. Although the OS image should work with real, compatible hardware, using a software emulator greatly simplifies the running and debugging process, and we can focus on the operating system itself.

QEMU simulates the entire hardware stack, and provides customization on different components:

  • QEMU comes with different binaries, each one simulating an architecture. For example, in this lab, JOS is design for 32-bit x86 systems, therefore using qemu-system-i386 is appropriate;
  • -hda disk-image takes a file to simulate a hard disk;
  • -m memsize sets up the RAM size. If not provided, a default of 128MB RAM is used;
  • --bios rom-image specifies a ROM (that replaces the bundled SeaBIOS ROM).

Above covers the options used in this lab, but a great deal of customizations are possible with QEMU, such as CPU, PCI, GPU, Networking, etc. Other than customizing hardware, QEMU also allows setting up tools using the config options:

  • -serial strictly speaking, it’s also a hardware configuration for setting the serial port. However, when we pass mon:stdio to it, it creates a virtual QEMU monitor that connects to the serial port. After clicking the virtual window, the window traps the focus of our keyboard and mouse, and the key strokes are sent directly to the virtual hardware. To refocus back to the host OS, press Ctrl + Alt;
  • QEMU has built-in support for gdb and it supports remote debugging. To set it up, simply run with -gdb debug-port;
  • -D to specify log file.

Setup

After downloading the lab source files, there are a few steps to set it up:

  1. Download QEMU and dependencies. With APT , install them with 
    sudo apt install qemu-kvm qemu virt-manager virt-viewer libvirt-bin libvirt-daemon-system libvirt-clients bridge-utils virtinst
  2. Set up the correct QEMU binary in conf/env.mk (QEMU=qemu-system-i386);
  3. For 64-bit systems, a 32-bit gcc package gcc-multilib needs to be installed to prevent undefined reference to __udivdi3;
  4. Depending on gcc versions, it might get stuck at “Booting from hard disk” need the a change in the kernel code.
  5. Add working directory to gdb’s safe-path list in ~/.gdbinit: 
    set auto-load safe-path [lab dir]

File Structure

After setting it up, let’s take a look at the file structure of Lab 1 (the following labs inherits the main structure, and only differs in added components).

The artifacts, objects and files generated by build process or runs, includes object files, images and log files:

  • obj/: Build products, including assembly files, object files, binaries and the composed image file to function as a virtual disk for QEMU;
  • *.[log|in|out]: log files, input/output files;
  • .gdbinit: gdb initialization file generated from gdbinit.tmpl.

Source code directories: boot/ (bootloader), inc/ (include), kern/ (kernel), lib/ (library functions).

Others:

  • conf: project config files;
  • Course admin scripts: grade-lab1, gradelib.py, handin-prep, etc;
  • GNUmakefile: the makefile.

Makefile

Lastly, let’s inspect the makefile GNUmakefile. In a previous post, we quickly overviewed some basics for make.

A quick observation we find that the makefile is actually distributed into three files, boot/Makefrag, kern/Makefrag and GNUmakefile, the first two being included into the last one.

The two makefile fragments are simple: boot/Makefrag builds the objects from source files in boot/, and links the objects to a specific start location in obj/boot/boot; similarly, kern/Makefrag builds obj/kern/kernel from the source files, but additionally, it also creates the disk image obj/kern/kernel.img from the binaries and the grub file.

GNUmakefile is the top-level makefile. It does not deal with the meticulous details of compilation, but rather set up the high-level variables and defines the high-level make targets. For example,

  • all is set as default target in it, but later expanded the dependencies in kern/Makefrag;
  • Commands such as clean, qemu[-nox][-gdb] are all defined here. make qemu for example expands to 
    qemu-system-i386 -hda obj/kern/kernel.img -serial mon:stdio -gdb tcp::26000 -D qemu.log
  • option V is defined for verbose. make V=1 prints the commands executed by make.

Related Posts