VSCode, GDB, and Debugging an OS

I've been spending some time going through Philipp Oppermann's amazing blog series, Writing an OS in Rust. Digging into operating systems has been on my todo list for years now and I couldn't pass up the opportunity when I ran across this series on Hacker News.

Phillip has done a wonderful job in his posts and they're extremely easy to follow. Unfortunately for me, though, he's doing his development via Linux so some of the instructions aren't one-to-one. For instance, cross-compiling libcore for x86-64 on Darwin doesn't really work and likely won't for some time. That's ok though, Vagrant to the rescue.

After you have vagrant up and running, you can drop into ssh (vagrant ssh) and follow along with all of the posts.

If you know me, I really like VSCode. I was a huge fan of Sublime Text because of multi-cursors, and VSCode seems better in almost every way. My development environment for the BlogOS is pretty straight-forward.

1. Develop in VSCode on my host machine (MacOS)
2. Build on target machine (Vagrant VM)
3. Run on target machine (Vagrant VM)

The trick to 1 and 2 is to leverage the shared /vagrant directory to share files. You might run into one hiccup with rsync not working so you'll want to specify "virtualbox" as the sync type.

config.vm.synced_folder ".", "/vagrant", type: "virtualbox"

Great, we can write and run code. But what about debugging?

One of the tangential posts, Set Up GDB, goes through the steps of getting GDB setup. That's awesome! What's even more awesome is using VSCode as an IDE to debug - so let's get that setup.

First, as the post suggests, we do need to build our own version of gdb. For the most part the steps for building gdb on MacOS are the same:

git clone git@github.com:phil-opp/binutils-gdb.git
./binutils-gdb/configure  --target=x86_64-pc-linux-gnu \
    --prefix="/some/file/path/rust-os-gdb" \
    --with-python=yes --enable-tui \
    --with-curses --disable-nls \
    --disable-werror --disable-gas \
    --disable-binutils --disable-ld \
    --disable-gprof
mv /some/file/path/rust-os-gdb/bin/x86_64-pc-linux-gnu-gdb /some/file/path/rust-os-gdb/bin/gdb

Now install WebFreak001's amazing gdb/lldb VSCode extension. Reload VSCode and we're ready to set up the launch configuration.

Select GDB as your debug mode and use the following for your configuration:

{
    "version": "0.2.0",
    "configurations": [{
            "name": "Attach to QEMU",
            "type": "gdb",
            "request": "attach",
            "executable": "./build/kernel-x86_64.bin",
            "target": "localhost:1234",
            "remote": true,
            "gdbpath": "../rust-os-gdb/bin/gdb",
            "cwd": "${workspaceRoot}",
            "autorun": [
                "set substitute-path /vagrant ."
            ]
        }
    ]
}

The above is pretty straight-forward. We're defining a launch mode that attaches to a target gdb server at localhost:1234. We specify the executable, the cross-compiled kernel-x86_64.bin file, and we call set substitute-path to let gdb know how to map the remote source paths with our local paths.

Did you catch the mistake above? Our gdb server isn't running locally, it's running in the Vagrant VM. Easy enough, add the port forward in the Vagrantfile.

config.vm.network "forwarded_port", guest: 1234, host: 1234

Note
If you're using the latest version of Vagrant, you might need to specify an additional argument above (host_ip).

And with that, you should be ready to go. If you did everything right you should be able to set a breakpoint, launch the OS with make debug, and then attach to gdb.