blog.golawski.it

Getting Started with User-Mode Linux - Part 1

December 06, 2020

User-Mode Linux is one more way to virtualize enviroment but unlike of other virtualization methods it allows to virtualize only Linux-based environments. You can use UML to create isolated and safe environment using kernel built-in features.

Before we start

Environment

I am working on Debian Buster and all examples are written for Debian-based distribution.

Tools

Make sure that you have installed all required programs:

  • bc,
  • bison,
  • developer’s files for ncurses,
  • flex,
  • gcc,
  • git,
  • make.

You can easily install all of them by executing following line.

kgolawski@~$ sudo apt-get install bc bison build-essential flex git libncurses-dev

Workplace

During this article several directories will be used so you should create some root project directory to keep all files in one place.

kgolawski@~$ mkdir uml-project
kgolawski@~$ cd uml-project/
kgolawski@~/uml-project$

Linux kernel preparation

Source code

The easiest way to download Linux kernel source code is to use git. Make sure that you are downloading the newest version which supports UML.

kgolawski@~/uml-project$ git clone --single-branch --branch v5.9 https://github.com/torvalds/linux.git
Cloning into 'linux'...
remote: Enumerating objects: 666, done.
remote: Counting objects: 100% (666/666), done.
remote: Compressing objects: 100% (666/666), done.
remote: Total 7620987 (delta 0), reused 666 (delta 0), pack-reused 7620321
Receiving objects: 100% (7620987/7620987), 2.86 GiB | 19.79 MiB/s, done.
Resolving deltas: 100% (6322405/6322405), done.
Note: checking out 'bbf5c979011a099af5dc76498918ed7df445635b'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

Checking out files: 100% (70010/70010), done.
kgolawski@~/uml-project$ cd linux/
kgolawski@~/uml-project/linux$

Initial configuration

Linux uses .config file as a configuration file so we need to create the new one.

kgolawski@~/uml-project/linux$ make defconfig ARCH=um
  HOSTCC  scripts/basic/fixdep
  HOSTCC  scripts/kconfig/conf.o
...
  HOSTCC  scripts/kconfig/util.o
  HOSTLD  scripts/kconfig/conf
*** Default configuration is based on 'x86_64_defconfig'
#
# configuration written to .config
#
kgolawski@~/uml-project/linux$ ls .config
.config

Test compilation

Everything is ready to compile kernel. It can take a while so take a breath and go for a tea. After successful compilation you can try to execute linux binary with --help argument.

kgolawski@~/uml-project/linux$ make --jobs=$(nproc) ARCH=um
scripts/kconfig/conf  --syncconfig Kconfig
  WRAP    arch/x86/include/generated/uapi/asm/bpf_perf_event.h
  WRAP    arch/x86/include/generated/uapi/asm/errno.h
...
  LD      vmlinux
  SYSMAP  System.map
  LINK linux
  MODPOST Module.symvers
  CC [M]  arch/um/drivers/hostaudio.mod.o
...
  CC [M]  sound/soundcore.mod.o
  LD [M]  sound/soundcore.ko
kgolawski@~/uml-project/linux$ ls linux
linux
kgolawski@~/uml-project/linux$ ./linux --help
User Mode Linux v5.9.0
        available at http://user-mode-linux.sourceforge.net/

--showconfig
    Prints the config file that this UML binary was generated from.

...

quiet
    Turns off information messages during boot.

hostfs=<root dir>,<flags>,...
    This is used to set hostfs parameters.  The root directory argument
    is used to confine all hostfs mounts to within the specified directory
    tree on the host.  If this isn't specified, then a user inside UML can
    mount anything on the host that's accessible to the user that's running
    it.
    The only flag currently supported is 'append', which specifies that all
    files opened by hostfs will be opened in append mode.

Preparing initrd

Built linux is useless without rootfs. Absolutely minimal rootfs requires dev/console file, init program and also proc/ and sys/ directories.

kgolawski@~/uml-project/linux$ cd ..
kgolawski@~/uml-project$ mkdir initrd
kgolawski@~/uml-project$ cd initrd/
kgolawski@~/uml-project/initrd$ mkdir dev proc sys
kgolawski@~/uml-project/initrd$ sudo mknod dev/console c 5 1
kgolawski@~/uml-project/initrd$ cd ..
kgolawski@~/uml-project$

After all, we need to update .config file to inform UML that there is initrd source directory. It is possible to manually edit .config but the easiest way is to use menuconfig make target.

kgolawski@~/uml-project$ cd linux/
kgolawski@~/uml-project/linux$ make menuconfig ARCH=um
  UPD     scripts/kconfig/mconf-cfg
  HOSTCC  scripts/kconfig/mconf.o
...
  HOSTLD  scripts/kconfig/mconf
scripts/kconfig/mconf  Kconfig

After successful build, you should see next screen.

 .config - Linux/um 5.9.0 Kernel Configuration
 ─────────────────────────────────────────────────────────────────────────────────────────────
  ┌───────────────────────── Linux/um 5.9.0 Kernel Configuration ──────────────────────────┐
  │  Arrow keys navigate the menu.  <Enter> selects submenus ---> (or empty submenus       │
  │  ----).  Highlighted letters are hotkeys.  Pressing <Y> includes, <N> excludes, <M>    │
  │  modularizes features.  Press <Esc><Esc> to exit, <?> for Help, </> for Search.        │
  │  Legend: [*] built-in  [ ] excluded  <M> module  < > module capable                    │
  │ ┌────────────────────────────────────────────────────────────────────────────────────┐ │
  │ │           General setup  --->                                                      │ │
  │ │           UML-specific options  --->                                               │ │
  │ │           UML Character Devices  --->                                              │ │
  │ │           UML Network Devices  --->                                                │ │
  │ │       [ ] UML driver for virtio devices                                            │ │
  │ │           General architecture-dependent options  --->                             │ │
  │ └───────v(+)─────────────────────────────────────────────────────────────────────────┘ │
  ├────────────────────────────────────────────────────────────────────────────────────────┤
  │                <Select>    < Exit >    < Help >    < Save >    < Load >                │
  └────────────────────────────────────────────────────────────────────────────────────────┘

Enter to General Setup and scroll down to see initrd related options.

 .config - Linux/um 5.9.0 Kernel Configuration
 > General setup ─────────────────────────────────────────────────────────────────────────────
  ┌──────────────────────────────────── General setup ─────────────────────────────────────┐
  │  Arrow keys navigate the menu.  <Enter> selects submenus ---> (or empty submenus       │
  │  ----).  Highlighted letters are hotkeys.  Pressing <Y> includes, <N> excludes, <M>    │
  │  modularizes features.  Press <Esc><Esc> to exit, <?> for Help, </> for Search.        │
  │  Legend: [*] built-in  [ ] excluded  <M> module  < > module capable                    │
  │ ┌───────^(-)─────────────────────────────────────────────────────────────────────────┐ │
  │ │       [ ]   Enable deprecated sysfs features by default                            │ │
  │ │       [ ] Kernel->user space relay support (formerly relayfs)                      │ │
  │ │       [ ] Initial RAM filesystem and RAM disk (initramfs/initrd) support           │ │
  │ │       [ ] Boot config support                                                      │ │
  │ │           Compiler optimization level (Optimize for size (-Os))  --->              │ │
  │ │       [ ] Configure standard kernel features (expert users)  ----                  │ │
  │ └───────v(+)─────────────────────────────────────────────────────────────────────────┘ │
  ├────────────────────────────────────────────────────────────────────────────────────────┤
  │                <Select>    < Exit >    < Help >    < Save >    < Load >                │
  └────────────────────────────────────────────────────────────────────────────────────────┘

Enable initrd support by selecting Initial RAM filesystem and RAM disk (initramfs/initrd) support option.

It reveals hidden option Initramfs source file(s) where you can set pathname to the directory with your initrd. You can omit this step and pass initrd by UML argument initrd=... but compiling into UML is easiest for further use. So fill this option with a related path, which is allowed as well as an absolute one, and navigate to ~/uml-project/initrd by writing ../initrd.

 .config - Linux/um 5.9.0 Kernel Configuration
 > General setup ─────────────────────────────────────────────────────────────────────────────
  ┌──────────────────────────────────── General setup ─────────────────────────────────────┐
  │  Arrow keys navigate the menu.  <Enter> selects submenus ---> (or empty submenus       │
  │  ----).  Highlighted letters are hotkeys.  Pressing <Y> includes, <N> excludes, <M>    │
  │  modularizes features.  Press <Esc><Esc> to exit, <?> for Help, </> for Search.        │
  │  Legend: [*] built-in  [ ] excluded  <M> module  < > module capable                    │
  │ ┌───────^(-)─────────────────────────────────────────────────────────────────────────┐ │
  │ │       [ ]   Enable deprecated sysfs features by default                            │ │
  │ │       [ ] Kernel->user space relay support (formerly relayfs)                      │ │
  │ │       [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support           │ │
  │ │       (../initrd) Initramfs source file(s)                                         │ │
  │ │       (0)   User ID to map to 0 (user root) (NEW)                                  │ │
  │ │       (0)   Group ID to map to 0 (group root) (NEW)                                │ │
  │ └───────v(+)─────────────────────────────────────────────────────────────────────────┘ │
  ├────────────────────────────────────────────────────────────────────────────────────────┤
  │                <Select>    < Exit >    < Help >    < Save >    < Load >                │
  └────────────────────────────────────────────────────────────────────────────────────────┘

You can save and exit.

*** End of the configuration.
*** Execute 'make' to start the build or try 'make help'.

kgolawski@~/uml-project/linux$

Preparing init

init program is the last part required to create a minimal UML setup.

kgolawski@~/uml-project/linux$ cd ..
kgolawski@~/uml-project$ touch init.c

Now open init.c with your favorite text editor and fill with the following content.

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/mount.h>
#include <sys/reboot.h>
#include <sys/types.h>
#include <unistd.h>

static int initialize(void) {
    // mount necessary directories
    if (mount("none", "/proc", "proc", MS_MGC_VAL, "") != 0) {
        fprintf(stderr, "error %d: %s\n", errno, strerror(errno));
        return EXIT_FAILURE;
    }
    if (mount("none", "/sys", "sysfs", MS_MGC_VAL, "") != 0) {
        fprintf(stderr, "error %d: %s\n", errno, strerror(errno));
        return EXIT_FAILURE;
    }
    if (mount("none", "/dev", "devtmpfs", MS_MGC_VAL, "") != 0) {
        fprintf(stderr, "error %d: %s\n", errno, strerror(errno));
        return EXIT_FAILURE;
    }
    return EXIT_SUCCESS;
}

int main(int argc, char *argv[]) {
    printf("init\n");
    for (int i = 1; i < argc; i++) {
        printf("\t%s\n", argv[i]);
    }

    if (initialize() != EXIT_SUCCESS) {
        sync();
        reboot(RB_POWER_OFF);

        return EXIT_FAILURE;
    }

    sync();
    reboot(RB_POWER_OFF);

    return EXIT_SUCCESS;
}

Now compile init and kernel.

kgolawski@~/uml-project$ gcc -Wall -Wextra -fPIC -static init.c -oinitrd/init
kgolawski@~/uml-project$ ls initrd/
dev/  init  proc/  sys/
kgolawski@~/uml-project$ cd linux/
kgolawski@~/uml-project/linux$ make --jobs=$(nproc) ARCH=um
  CALL    scripts/checksyscalls.sh
  CALL    scripts/atomic/check-atomics.sh
...
  LD      vmlinux
  SYSMAP  System.map
  LINK linux
  MODPOST Module.symvers

Run

This is the time to run built linux.

kgolawski@~/uml-project/linux$ ./linux arg0 arg1
Core dump limits :
        soft - NONE
        hard - NONE
Checking that ptrace can change system call numbers...OK
Checking syscall emulation patch for ptrace...OK
Checking advanced syscall emulation patch for ptrace...OK
Checking environment variables for a tempdir...none found
Checking if /dev/shm is on tmpfs...OK
Checking PROT_EXEC mmap in /dev/shm...OK
Adding 5169152 bytes to physical memory to account for exec-shield gap
Linux version 5.9.0 (kgolawski@debian) (gcc (Debian 8.3.0-6) 8.3.0, GNU ld (GNU Binutils for Debian) 2.31.1) #3 Sat Dec 5 23:25:00 UTC 2020
Zone ranges:
  Normal   [mem 0x0000000000000000-0x00000000624edfff]
Movable zone start for each node
Early memory node ranges
  node   0: [mem 0x0000000000000000-0x00000000024edfff]
Initmem setup node 0 [mem 0x0000000000000000-0x00000000024edfff]
Built 1 zonelists, mobility grouping on.  Total pages: 9324
Kernel command line: root=98:0
Dentry cache hash table entries: 8192 (order: 4, 65536 bytes, linear)
Inode-cache hash table entries: 4096 (order: 3, 32768 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 26564K/37816K available (3075K kernel code, 1094K rwdata, 992K rodata, 149K init, 164K bss, 11252K reserved, 0K cma-reserved)
NR_IRQS: 16
clocksource: timer: mask: 0xffffffffffffffff max_cycles: 0x1cd42e205, max_idle_ns: 881590404426 ns
Calibrating delay loop... 4390.09 BogoMIPS (lpj=21950464)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 512 (order: 0, 4096 bytes, linear)
Checking that host ptys support output SIGIO...Yes
Checking that host ptys support SIGIO on close...No, enabling workaround
devtmpfs: initialized
random: get_random_u32 called from bucket_table_alloc+0x117/0x140 with crng_init=0
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
futex hash table entries: 256 (order: 0, 6144 bytes, linear)
NET: Registered protocol family 16
clocksource: Switched to clocksource timer
VFS: Disk quotas dquot_6.6.0
VFS: Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 256 (order: 0, 4096 bytes, linear)
TCP established hash table entries: 512 (order: 0, 4096 bytes, linear)
TCP bind hash table entries: 512 (order: 0, 4096 bytes, linear)
TCP: Hash tables configured (established 512 bind 512)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered protocol family 1
printk: console [stderr0] disabled
mconsole (version 2) initialized on /home/kgolawski/.uml/AtsmFz/mconsole
Checking host MADV_REMOVE support...OK
workingset: timestamp_bits=62 max_order=13 bucket_order=0
io scheduler mq-deadline registered
io scheduler kyber registered
NET: Registered protocol family 17
Initialized stdio console driver
Console initialized on /dev/tty0
printk: console [tty0] enabled
Initializing software serial port version 1
printk: console [mc-1] enabled
Failed to initialize ubd device 0 :Couldn't determine size of device's file
This architecture does not have kernel memory protection.
Run /init as init process
init
        arg0
        arg1
reboot: System halted

As you can see kernel started /init program as init process. This program makes nothing else that wrote args into stdout and safely stops. It is a fine base for further work.

Code

Example repository is available here: https://github.com/ventaquil/blog-uml-example.

It is ready to use but read README.md from this repository before you run make.

Next steps

In the next part, we will take a look at putting more programs into UML and creating our own small distribution.

References

  1. The User-mode Linux Kernel Home Page
  2. Debian Wiki - UserModeLinux
  3. Using the initial RAM disk (initrd)
© 2020-2021, Built with Gatsby